For those folks who aren’t gamers it some times comes as a bit of a surprise just how serious a business video games have become. The article at Wired I linked to in my entry the other day on how new games may require some folks to upgrade their hardware has a small mention of the strong ties between computer hardware makers and software publishers and how a hot game can drive up sales of high-end hardware. The pressure on video card makers to score as high as possible on benchmarks can be overwhelming.
And to make sure this point wasn’t lost on anyone at E3, the trailer for the forthcoming Half-Life 2 was shown not at the booth of the publisher, Vivendi Universal Games, but at the area set aside for graphics hardware maker ATI Technologies.
After all, the last version of Half-Life sold more than 8 million copies (including add-ons and expansion packs). And with gamers frothing at the mouth for the next version, and willing to spend some bucks to upgrade their hardware accordingly, ATI is seeing a lot of gold in this new alien shoot’em-up (as is developer Valve).
High-end $500 video cards often live or die by the benchmark scores they manage to put out as the number of hardcore gamers with that kind of cash to spend is a very limited market. Not to mention that bragging rights on the high-end will often help push sales of the mid-range and low-range cards based on the same chipset which are the real bread and butter makers for these companies. As a result it’s probably only to be expected that companies might sometimes cheat.
The 3DMark series of synthetic benchmarks have long been a favorite yardstick for measuring video card performance by both gamers and journalists evaluating hardware. There’s been much debate over the value of synthetic benchmarks, as opposed to benchmarks using actual games, but it’s generally agreed they have a place. That debate hasn’t stopped many gamers from basing their buying decisions off of how a bit of hardware scores on 3DMark and that fact isn’t lost on hardware makers. When ATI finally managed to best nVidia with the release of their Radeon 9700 Pro the debate on synthetic benchmarks started to heat up. It became near white-hot when nVidia released their new GeForce FX series and failed to reclaim the performance crown from ATI. Since ATI took the top spot nVidia has been trying to downplay the significance of 3DMark03 because it’s a synthetic benchmark even though that never seemed to bother them when they were on top of the heap. Now word comes that nVidia may have cheated with the drivers for their newest iteration of the GeForce FX to artificially inflate their 3DMark03 score:
In our recent GeforceFX 5900 Ultra story, we noticed the Nvidia board was well ahead of ATI’s Radeon 9800 Pro in Futuremark’s 3DMark2003 benchmark, but the two boards were much more evenly matched in our GameGauge application benchmark suite.
We’ve just discovered certain test anomalies that indicate to us that Nvidia may be special-casing 3DMark2003, throwing away work, to attain higher scores.
We had thought, with the GeForceFX 5900 Ultra now out of the bag, that we were in for a bit of a post-launch lull. But as it turns out, things are just starting to get interesting.
nVidia is claiming these anomalies are the result of a bug in the card drivers and not any form of cheating on their part, but it wouldn’t be the first time that a hardware maker has been caught trying to cut corners to improve a benchmark score. ATI themselves got into a big flap awhile back over “optimizations” they made to their drivers to improve their benchmark scores in Quake 3. ATI’s changes, however, could be debated as to whether or not they count as “cheating” because it was for a specific game and did improve performance of that game whereas the changes nVidia is accused of making would only be of benefit for inflating their score in 3DMark03. For a company that claims the 3DMark03 scores are unimportant compared to performance in actual games it would certainly be embarrassing to be caught trying to stack the deck just for a higher score. If nothing else this latest flap illustrates just how cut-throat the high-end video card market can be and the kind of money that’s at stake. It’ll be interesting to see how this all plays out.