Size does matter!

by davidnielsen

Reading the kernel mailing I stumbled upon becnhmark results from Hans Reiser on the Reiser4 compression plugin.

Reading the results it shows that as CPU speed increases adding compression to reduce IO causes a sizable decrease in required time for file operations.

I’ve suspected something like this would turn up eventually, all that’s left to do is to find the break even point and estimate when the majority of consumer PCs are above this break even point.

I remember the good old days using MS-DOS on the family i386 16MHz, we employed STACKER to increase the storage potential of the build in 20MB harddrive. That was my first encounter with this kind of technology, back then the aim was clearly increasing storage. Today however harddrives comes in cheap, a 120 GB addition to ones machine comes at such a low cost that space considerations aren’t that important.

However as every other element of the average PC has increased it’s performance, the harddrive is still hidiously slow. Thus as CPU speed increases we will reach a point where IO time would approach 100% of the operation. Compression therefore make sense again but for the opposite reasons as when I used STACKER.

Compression of course only makes sense for certain files, I doubt we would win much on already compressed files like Ogg Vorbis files. Looking at competing solutions the OpenSolaris ZFS implements block based on the fly compression, where a block is only compressed if the size gain is more than 25% (as opposed to the Reiser4 compression plugin which compresses the entire file). This design seems superior to me, but untill I can actually test the two I’ll refrain from making such bold statements.

Regardless, on the fly compression does seem to carry with it but size and performance gains for standard harddrives when CPU power is plentiful. To good to be true?