USS Clueless Stardate 20011124.1345

  USS Clueless

             Voyages of a restless mind

Main:
normal
long
no graphics

Contact
Log archives
Best log entries
Other articles

Site Search

Stardate 20011124.1345 (On Screen): According to this report, there's good reason to believe that with the release of the 1.05GHz UltraSparc III, that to gain the title of "fastest" it's taken to cooking one of the SPEC benchmarks. It's happened before.

Back in the day, when Sun was trying to make Java the primary language for, well, everything there was a legendary tussle with Microsoft. It was not the Java language Microsoft objected to, it was "write once, run anywhere". The big advantage Microsoft worked hard to get and keep was the huge base of software written specifically for Microsoft operating systems. Microsoft was deliberately trying to leverage the "network effect" to add perceived value to their OSs. But if software could be written to be OS-portable, then there would no longer be an app-advantage to running a Microsoft operating system versus one of its competitors. This was a profound danger to Microsoft's business model -- and Sun knew it; it was the main reason they were pushing it. So Microsoft got big into Java for a while and its goal was to make it so that the version of Java running on Windows was better -- and also specialized, with extra and useful capabilities which were only available on the Windows platform. By doing this, their goal was to destroy WORA. And a combination of Microsoft's efforts and Sun incompetence did in fact fail to make WORA a fact of life on the desktop.

But the other thing that happened pretty consistently is that Microsoft's JVM for Windows was consistently much faster than Sun's own JVM for Windows, which earned Sun numerous horselaughs. (They invented Java, after all.) One of the most popular benchmarks for Java was CaffeineMark 2.5. And some frustrated Sun programmer put in a special section on Sun's JIT compiler to recognize one of the specific tests in CM2.5, and made it optimize down to virtually nothing, thus completing in jig time. CM2.5 used an arithmetic mean on all the scores to calculate the overall score, so making one particular test have an immense score made the overall score go way up, even though all the other results were lackluster. It's never been clear just how many people were involved in this, but after the that version of Sun's Java was released, Sun's marketdroids started to trumpet how Sun now had the fastest JVM out there. But it didn't take too much time for others to spot the anomaly, and Pendragon investigated, found what had been done, and then published a scathing denunciation of Sun on their web site. Then they released a new version of CM (CM3.0) and the single most important change they made was to use a geometric average instead of an arithmetic mean, because a geometric average substantially reduces the extent to which one seriously anomalous score can affect the result. Also, they slightly recoded all the tests, and as a result, the Sun JIT didn't recognize the test anymore and did an honest job compiling it -- and Sun's score dropped down to the "second-rate" tier, where it had always been.

It's hardly surprising to learn that the same thing may have happened again. The real problem is that Sun's engineering is competent but not really industry-leading in every regard. You won't go wrong with Sun products, but often there's something else somewhere else which would be cheaper and more powerful. ( (discuss)

Captured by MemoWeb from http://denbeste.nu/entries/00001447.shtml on 9/16/2004