So a few weeks ago when I handed in my resignation at work, I was expecting a pretty relaxed wind-down, as the application I had been working on over the past few months was only a couple of weeks off delivery, and after that all I needed to do was hand it over to another developer.
Such are the joys of software development is that it didn’t work out that way. The users had been testing for a couple of weeks, and tried some performance tests – one of the key requirements are some pretty tough loading times – but ended up getting a whole raft of out of memory errors, which became more frequent the more data they had loaded.
It has to be said that .NET Framework applications have a bit of a reputation for consuming vast amounts of memory, for example have a read of this article from SysInternals that compares two versions of Notepad. Having said that, it was pretty unexpected that we would be having problems with just 45 days worth of data. However, after a little investigation we found that it wasn’t simply data volume. We could always quite happily load well in excess of 45 days, however the problems came on a reload. A quick look at the application using the free Process Explorer found that despite regular garbage collections, the memory usage was only ever climbing – sounded like a classic case of memory leaking. Having had a search around for suitable tools, we grabbed a trial copy of the .Net Memory Profiler from SciTech Software. Although most of what the tool does can also be carried out with the Microsoft debuggers, the tool is a lot more straightforward to use.
Taking a look at the objects in memory at various points during the runs of our application, we did spot some areas that we could improve our code, however the big problems were actually somewhat outside of our control, being down to the Infragistics controls we were using for the user interface. Now it has to be said that the controls are not overly popular amongst our developers. Whilst when we kicked off our C# developments back in 2004 they seemed to be the best suite of controls, in use they have caused a good deal of frustration. Amongst a number of gems we’ve found with them, the most annoying is that certain of the controls raise and catch exceptions as part of their normal operation, something which is strongly advised against with .Net development because exceptions are just so slow! We’ve also realised that there are some pretty fundamental things that the chart control cannot do, for example placing an axis at anywhere other than the edge of a chart.
Anyway, the leak problem proved to be that the Infragistics grid control we were using was managing to hold on to references to parts of one of our data objects. Whilst in all the references in our code were correctly being disposed, with the reference from deep inside the grid, the .Net garbage collector was leaving the entire data structure sitting around. Even once we’d managed to code our way around that problem, there was still the issue of how much memory the grid itself was taking, thanks in part to the literally millions of ‘appearance’ objects that were being created.
Appearance objects are attached to all sorts of controls in Infragistics to hold details of the control’s colour, font and so on, however in the case of the grid each cell has one of these objects. More than that, there are appearance objects on each column and row too. You can see from the picture of the application running that this might be a bit of an issue – the potential number of cells on our grid could be as high as 432,000, so we had to find a way to reduce them. If you take a look at the picture, you’ll also notice that most of the cells actually look pretty similar, therefore it is a bit of a waste of time to have multiple appearance objects, all containing the same information. A bit of work by Rob worked out that in fact there were only 23 unique combinations of font and colour in use on the grid. As a result, Rob spent a lot of time this week working around the automatically created appearance objects. Essentially what he is doing is creating the 23 unique combinations as a library of appearance objects and then throwing away the automatically created objects and replacing them with a reference to one of our library of 23. This seems to work pretty well, and we now have just under 2000 appearance objects in total, even during the longest runs. Together with the other tweaks we have made, this has almost halved the memory footprint of our application, and as of Friday, the only problem we have left is that some objects are still being retained by the grid – so we need to do some more leak hunting.
All in all, I’ve been pretty disappointed with Infragistics, and certainly if I were in a situation where I was choosing a set of controls again, I’d probably steer away from them if I could. Whilst on paper the regular releases seem good, implying an active development team behind the suite, what the headline grabbing new features tend to hide is that there are some bugs that don’t seem to have ever got fixed, and there is also real problems like the memory leaks we’ve found over the past few weeks. A month or two ago they sent round a survey, to which I replied that I wished they would spend more time fixing the bugs and getting the existing features working efficiently rather than all these flashy new features. Not surprisingly, the impression I get looking at the examples they produce, that with the grid for example show off how they can hold millions of rows, is that they’ve been very carefully coded to avoid the issues. Maybe I’m being unfair, but when one of my colleagues e-mailed their support with another bug in the grid, the ‘solution’ they e-mailed back didn’t work – in fact it was even worse than the problem we were having.
On the positive side, what I have picked up from this whole experience is a much better and deeper understanding of what .Net is doing internally with my objects, and also a really good tool in the form of the SciTech .Net Memory Profiler.