Why not cache lines

From C

$Id: why_not_cache_lines.txt,v 1.3 2009/05/31 23:51:25 db Exp db $

From time to time I will hear someone proclaim that aligning their data on cache lines is a good idea for speed.

This smacks of premature optimisation and the possibility the user really does not know what they are talking about, not to mention the problem of portability.

Those of you who do know what a cache line is and when to worry about it can stop reading now. My concern is with the userland programmer who talks about cache lines. I emphasise, userland.

Simplifying for the rest of you (Those of you who know CPU hardware better than I do have stopped reading already right?) cache is that magic bit of very fast RAM that is inserted between the CPU and the RAM you plug into your machine. It is even faster than the bus you plug your RAM into. The idea is to preload this RAM with instructions and data so the CPU can be kept busier. Of course, this comes with a price, once you change instruction flow your instruction cache is now invalid and has to be reloaded, once you change where your program is reading from, you invalidate your data cache. Keep in mind that in userland, you can be pre-empted at any time to start another process, your cache has to be invalided then. When users are talking about cache lines, they are talking about the size of the really fast memory bits next to the CPU. But keep in mind cache length is not the same on a amd64 vs. a 386 vs. a powerpc, it is not a portable constant you can worry about all the time for speeding up your code. If you are worried about portability, then you are already at a disadvantage. If you are worried about cache lines and alignment without doing benchmarking first to prove it makes a difference, then you are guilty of premature optimisation. Yes, cache lines and alignment concerns are of concern in operating system kernels where an operation is performed many many times, but it also has been benchmarked and proven to improve performance. Normally it is not something you as a userland programmer should be worrying about. If you have done your homework and:

1) Know what cache is.

2) Prove it will make a difference with benchmarks.

3) Understand the portability problems.

Then go worry about cache lines.