Stardate 20020721.2136

(On Screen): Nathan Mates sends me this pointer to something I've been waiting to see for a very long time: a benchmark comparing the latest PCs against the latest Mac to see who is fastest.

An honest benchmark. That means one not done by Steve Jobs.

Let's review, shall we? About 15 months ago, at the time that Motorola finally broke out of the 18 month clock stall, and Apple announced the spiffy new 733 MHz G4, Jobs did a side-by-side comparison test between an engineering prototype of the new machine (real ones didn't ship for another two months) compared to a "1.5 GHz P4" running "Photoshop Filters". The G4 was faster.

Since then, it's been an article of faith amongst the Mac fanatics that clock rates don't matter, and that there is something so deeply right about the PPC architecture that it will always be faster than any x86 irrespective of the clock rates involved.

And for those who despise liars, what set our teeth on edge is that the benchmark was rigged, and not open.

A good benchmark will reveal enough information so that anyone can duplicate it. It's as simple as that. So here are all the things that Apple has never revealed:

They never revealed which filters were used, nor the sequence in which they were run, nor what parameters were given to each. They have never released the raw image which was used.

They never revealed how the PC was configured. They said "1.5 GHz P4" but there's a great deal of range there. They didn't say what OS it used. They didn't say what memory it was configured with, and there's been a strong suspicion that it used PC600 instead of PC800 RAM, which seriously cripples the P4. They never said how much memory it had in it. In fact, they've never revealed anything whatever about the configuration of that machine, except that it had a 1.5 GHz P4 in it.

They did reveal that they were using Photoshop 6.0. Now an interesting thing happened to Photoshop 6.0 on the way to the manufacturing line: someone fucked up the release process for the PC version, and used the wrong flag on the linker and left a whole pile of debug code in place. The result was that compared to Photoshop 5.5 on the PC, Photoshop 6.0 on the PC generally ran most filters anywhere up to five times slower, with three times being typical.

I might mention that this mistake was not made on the Mac version of Photoshop 6.0, and that shortly after Jobs did his demo, Adobe released a point update for the PC where they did the link correctly, with a radical improvement in performance as a result.

Another thing that's interesting about Photoshop 6.0 is that it didn't have SSE2 optimizations in it. SSE2 is the P4 equivalent of what is known as Altivec (or "Velocity Engine") on the G4; it's a set of special instructions capable of doing the same operations simultaneously on several pieces of data at once, which can speed some kinds of operations up quite radically. (Other operations cannot benefit from it. It varies.)

Finally, it is virtually certain that the filters which were run were carefully chosen to be the ones where the Mac performed the best.

I'm deeply suspicious about the entire thing for the simple reason that they've never revealed enough details about that demo to permit anyone else to find out what they actually did. If they had nothing to hide, if they truly believed that the Mac was faster, they would have been glad to reveal the testing protocol so others could prove it. The natural conclusion is that they didn't reveal the details because it would have become blatantly evident that they were cooking the benchmark.

Nonetheless, for the next year one of Apple's advertising mottos was "Power to burn Pentiums". Bah.

Which brings us to the present. An independent group not beholden to Apple, but also not apparently anti-Mac, has used the latest machines and run a serious suite of graphics benchmarks. The Mac was a dual 1 GHz G4. One of the PC's was a dual Athlon running at 1.67 GHz (the so-called "2000+"). The other PC was a single-processor P4 running at 2.53 GHz.

And the result? Sometimes the Athlon was faster, sometimes the P4 was faster. They were never very far apart, and both of them were quite a lot faster than the Mac in every single test.

There are a number of reasons why. First of all, the PCs were configured by PC manufacturers who wanted them to perform well, rather than by the head of Apple who wanted to cripple them. They had adequate RAM and fast enough RAM. The Athlon was using DDR-SDRAM (probably 266 MHz), and the P4 was using the new 533 MHz RDRAM.

On the other hand, the Mac was using 133 MHz SDRAM, and since the tests concentrated primarily on 3D rendering and animations rather than on processing a single image, that meant that the dataset was really quite large, too large to fit in the Mac's L3 cache. So the dual G4's were competing for memory which isn't even fast enough to keep one of them fully fed (which is the only reason why Apple uses L3 cache). It's hardly surprising that the Mac choked.

Another interesting result here was that some kinds of operations do not benefit from the availability of SMP. When the calculation is inherently serial (and many are), then a single processor which is blisteringly fast will do very well compared to two slower processors.

In such cases, the reason that the Athlon box still did very well was b