Friday 29 May 2009

Upgrading a major project to Delphi 2009

Having finished converting a major project that involves a fairly large programming team for several years, to Delphi 2009, I'm now ready to blog about the experience.

If you want to estimate the amount of work involved to convert a project, note that it is not significant, how many lines of code you have. It is more important, what kind of code, and how segmented it is, and how consistent each segment is written. Recent user interface stuff, business logic etc., is very easy to convert. I'd almost say compile & run. Some other parts are definitely not.

Our code was clearly segmented, in the way that each unit belongs to one of these groups:

* Very old code. Some parts really had some ancient stuff in them, especially external components, but this was solved simply by renaming string=>ansistring, char=>ansichar, pchar=>pansichar. Fixing Windows API calls to use Ansi versions. After that, everything runs as before, and it is not unicode enabled.

* 3rd party components. We upgraded most of them, and some old free source code versions were upgraded by ourselves.

* User interface stuff. Besides that we upgraded some component packages, we did not have anything to change here.

* Special home-made components. These sometimes contained some optimizations, that made it necessary to adapt them to Delphi 2009, but generally, they just worked.

* Business logic. Some general things had to be changed, but there were few things and they were easy to spot and fix. It was almost like search & replace.

* Bit-manipulating units. These units need to treat each byte by itself, and the usual remedy was to convert it as if it was really old code, and then sometimes it was fairly easy to unicode-enable it afterwards.

* I/O routines. They basically had to be rewritten. We have switched some of our output text files to utf-8, in order to unicode-enable them, but others we left as they were. The main problem was with code, that tried to do unicode in Delphi 2006/2007, because it stored utf-8 in ansistring. The solution is to remove all utf-8 conversion inside the algorithm, and just apply it at the I/O point.

The hardest part was blobs. Sometimes they contain binary stuff, and sometimes they contain text. Unfortunately, .AsString was suitable for both in Delphi 2007, but now they need to be treated completely separately. The solution was to duplicate a lot of procedures, one for RawByteString and one for String, and then use the appropriate procedure on the appropriate fields.

It was not hard to do the conversion, and now we have unicode enabled a significant part of the system, with more to come, soon. However, it surely takes some refactoring experience to convert source code efficiently - if you are not good at refactoring, it will take significantly more time.

At last, I have a few tips for those, who are about to go through the same process:

* Gradually converting the code is easier than doing it all in one attempt. Make development projects that include more and more of your units, until you have converted everything.

* It is not difficult to make your code able to compile with both Delphi 2006, 2007 and Delphi 2009 at the same time. Do that, so that your team can still be productive until everything works.

* Even when you convert a full unit by making it use ansistring internally, consider to use string for the interface section, so that the conversion is hidden internally in the unit. This keeps the number of special cases lower.

* Get rid of your string warnings by fixing them, not by ignoring them. Most of them are seriously easy to fix.

* Always review your changes before committing them to your source code repository, so that you're sure that you only changed what you meant to change :-)

Thursday 28 May 2009

Deterministic but automatic memory deallocation

Now, that Delphi Prism and Delphi/Win32 have put less focus on source code sharing, it's time to see if memory allocation in Delphi/Win32 can be improved. This is what we often write in our code today:

var o:TMyObject;
FreeAndNil (o);

How can we avoid typing so much? The obvious solutions are:

Garbage Collection: Used in Java and .net, it often makes these consume more RAM than necessary, and is generally not very predictable. Even worse, Delphi code usually does a lot in the destructors, which is not compatible with garbage collection.

C++ objects: This is basically about avoiding the use of pointers. Delphi actually supports these kinds of objects, but then you need to avoid using TObject, and that is not really a good way forward.

There's a third solution: Add a syntax, maybe a keyword, which tells the compiler that the pointer should be freeandnil'ed before the procedure exits. Here is an example where the keyword "local" has been used:

var o:TMyObject;
o:=local TMyObject.Create;

Here, the keyword local forces the compiler to deallocate the object before the function finished. Another example:

(local TMyForm.Create(nil)).ShowModal;

This would create the form, show it modally, and deallocate it again in a deterministic/plannable/non-random way.

Even adapting APIs can be done nicely:

a:=CreateChart (local CustomerStatisticsRetriever.Create (local CustomerRetriever.Create (databaseconnection)));

In this case, the CustomerRetriever provides an API for getting customer data out of the database connection. This is used by the CustomerStatisticsRetriever to provide statistics, which CreateChart() uses to create a chart. After doing this, the pointers are deallocated automatically because of the local keyword.

Possible variations on the topic include:

* Use a different syntax or a different keyword
* Deallocate automatically after the last reference to the pointer, instead of deallocating at the end of the procedure.

Thursday 21 May 2009

Anders Hejlsberg: We have huge amounts of memory and concurrent programming is an exception

See the video here:

It's quite amazing to hear Anders advocating bloat in a world where most of the world still has bad internet connections and hate large downloads, mobile phones prefer 1MB applications over 10MB apps, where RAM can be a bottleneck, where battery life is always too short and where many large organizations struggle with bad performance.

I recently had the chance to see the standard configuration for virtual servers in a large organization. In order to improve network traffic, the standard network adapter is limited to 10Mbit/sec.

Size matters.