Sunday, 22 March 2009

Use RAM disk for many small temporary files

Many systems use small temporary files to exchange information between two applications. There can be many reasons to do so: To support network file shares, to have a queue system that is not tied to any kind of special software, or simply to keep specifications simple. There are many well known examples of this, including mail systems like sendmail and postfix.

The problem is, that most Windows computers use the NTFS file system, which supports journaling. This means that every file that is created, actually activates the physical harddisk. On a system with a load, this can cause serious latency, which slows down any system. Unfortunately, it is not possible to turn off journaling for a single directory.

The solution? Install a RAM disk. It may take a part of your system's RAM, but it's surely extremely fast at creating and deleting files. You can get a RAM disk here. If you want to see performance numbers, see this page (use Google Translate for English version).


phis said...

Now how much cooler would it be if you could create the RAM drive you use from Delphi code?

Unfortunately, though, Delphi isn't exactly the tool of choice for device drivers.

McKeeth said...

We had a legacy system that used an old version of Sybase SQL Anywhere that didn't really support good memory optimization. There was one instance that had a huge database and one of the reports would timeout after running over 24 hours. No matter how much hardware they threw at it the performance was the same. So I suggested a RAM Drive. We set one up and moved the entire database into the RAM drive. The report finished in 15 minutes.