Thursday 15 January 2009

Multi-core now, NUMA next

Quad-core PCs are in the shops, but as Sandia reports, there is a limit to how much we can grow performance by adding more cores.

The next bottleneck is memory - and the simple way to solve that problem is to split the memory, giving each CPU its own memory. This adds a new parameter to allocated memory: Which thread does this memory belong to? Actually, the technology already exists. Windows and Linux both support Non-Uniform Memory Access, NUMA. It is typically used in virtualization hosts in data centers, for instance using VMware ESX Server. If you haven't heard about it, you may want to prepare yourself for the future.

Friday 9 January 2009

Virtualization gives configurable risk profiles

As everybody knows, the risk of failure can be reduced using redundancy, so redundancy must be good. Also, redundancy is expensive, but fewer hardware boxes with lots of redundancy built-in means less costs and less risk at the same time, right? Now, here is the problem: If you virtualize 100 servers on a system, and the system fails, you have 100 virtual servers that fail.

Most organizations can survive the failure of one system. If a postal company cannot bill large customers for packages, but can bill everyone else, they still have cash flow until the problem is solved. But what happens if all systems fail at the same time?

Different organizations need different risk profiles. As software developers, we can use virtualizations to help this out. If our software supports several departments, we can choose how things should fail. If a storage system fails, should all departments lose access to the system (single point of failure = cheap), or just some of them (more expensive)? If the customer wants everything to be as cheap as possible, just put all the software on one physical server using virtualization. If the customer wants the system to be very stable, use several servers in a way that keeps employees productive even when some servers fail.

If it is too expensive to use multiple servers, use virtual servers on different host hardware, maybe even in different hosting centers. You can use 10% of one physical host in 10 different centers, and in the end, the costs are the same as using one physical host in one center.

The Google and Microsoft cloud systems try to solve this problem, too, but until further notice, virtual servers are the most advanced technology in most organizations, to solve this problem. What we can do as software developers, is to design our systems so that they work well, both distributed in several hosting centers, and on one physical server. Also, our systems must work in loosely coupled ways, that make it possible to keep the organization running for a while without a specific subsystem.

Monday 5 January 2009

Simple multithreading using closures

One nice application of closures is to create multithreaded software, suitable for multi-core CPUs. This example can give some inspiration:

procedure FooBar;
var
context:string;
res1,res2:string;
begin
context:='Test';

ParallelExecute (
procedure begin res1:=context+'1'; end,
procedure begin res2:=context+'2'; end);

ShowMessage (res1+slineBreak+res2);
end;


I used the variable name "context", because you would usually have a context in which you parallelize code. A context can be a database connection pool, or simply the raw data on which you want to operate.

ParallelExecute is implemented like this:

type
TParallelFunction=reference to procedure;
TExceptionClassType=class of Exception;
TSimpleThread=
class (TThread)
proc:TParallelFunction;
ExceptionMessage:string;
ExceptionType:string;
procedure Execute; override;
end;

procedure TSimpleThread.Execute;
begin
inherited;
try
proc;
except
on e:Exception do begin
ExceptionMessage:=e.Message;
ExceptionType:=e.ClassName;
end;
end;
end;

procedure ParallelExecute (p1,p2:TParallelFunction);
var
t1,t2:TSimpleThread;
begin
t1:=TSimpleThread.Create(true);
try
t1.proc:=p1;
t1.Resume;
try
t2:=TSimpleThread.Create(true);
try
t2.proc:=p2;
t2.Resume;
t2.WaitFor;
if t2.ExceptionMessage<>'' then
raise Exception.Create(t2.ExceptionType+': '+t2.ExceptionMessage);
finally
FreeAndNil (t2);
end;
finally
t1.WaitFor;
end;
if t1.ExceptionMessage<>'' then
raise Exception.Create(t1.ExceptionType+': '+t1.ExceptionMessage);
finally
FreeAndNil (t1);
end;
end;


The point is, that closures can be used to generate nice code.

Saturday 3 January 2009

Get a degree, but not necessarily a CS degree

Joel Spolsky has started a lot of blog postings in many places about the value of a CS degree. I would like to point out Bill the Lizard's blog, because he has a good point: CS graduates were once without a degree, so they know the difference and the value of the degree.

However, I also recognize some of the problems with CS graduates. Not all CS graduates are better than programmers without degrees, and it is important not to judge people on their degree — you need to evaluate the full person.

That said, however, the entire discussion lacks an important point: There are other degrees than CS out there. Very few businesses can survive with only CS knowledge — usually you need other knowledge too, like chemistry, physics, engineering, biology, mathematics, business, geology etc. The world needs people with those degrees as programmers in order to create great software. Bill Gates and Larry Ellison are not CS graduates.

My advice is: Get a degree, if you can. Respect the knowledge of others, also of those without degrees.