Futuremark Backs Down on Nvidia Cheat Claims

Benchmark software company Futuremark has dropped its allegation that graphics card maker Nvidia cheated, ending the dispute between the two firms. "Futuremark now states that Nvidia's driver design is an application-specific optimisation and not a cheat," said Futuremark in a statement.

In the mean time, new ZDNet lab tests show that the scores of Nvidia's latest hardware dropped by between 14 and 22 percent using Futuremark's updated testing software, although at higher image-quality settings the Nvidia chip still matches the performance of ATI's latest Radeon. The update of Futuremark's 3DMark03 eliminates the effects of Nvidia's optimisations.

Nvidia and Futuremark issued a carefully-worded fence-mending joint statement on Tuesday. Explaining why it decided to drop its allegation that Nvidia was fudging results, Futuremark said its normal practice was to frown on optimised code, but said it may change this practice.

"3DMark03 is designed as an un-optimised DirectX test and it provides performance comparisons accordingly. It does not contain manufacturer-specific optimised code paths," the company said. "Because all modifications that change the workload in 3DMark03 are forbidden, we were obliged to update the product to eliminate the effect of optimisations identified in different drivers so that 3DMark03 continued to produce comparable results.

"However, recent developments in the graphics industry and game development suggest that a different approach for game performance benchmarking might be needed, where manufacturer-specific code path optimisation is directly in the code source. Futuremark will consider whether this approach is needed in its future benchmarks."

Last week Futuremark, one of the leading benchmark software providers, alleged that Nvidia tweaked software needed to run its new GeForce FX 5900 processor to distort performance in Futuremark's 3DMark03 testing application.

The company said drivers -- software files that govern how a component interacts with the rest of the PC -- for the new Nvidia chip were altered to detect activity characteristic of a benchmark and adjust performance accordingly.

Benchmarks can be important to the prestige, if not the market share, of graphics chip makers. Nvidia is in the midst of a hard-fought battle with rival ATI Technologies to claim the performance lead in PC graphics processors. After years of Nvidia dominating both in market share and performance, ATI took the speed lead last year with new versions of its Radeon chips.

After settling manufacturing issues that resulted in ongoing delays of product launches, Nvidia hoped to regain the performance lead this year with the 5900.

When ZDNet testers originally ran 3DMark03 tests, using Build 320, Nvidia's 5900 Ultra ran circles around the ATI 9800 Pro 256MB. Under the updated Build 330, the 5900 Ultra's 3DMark03 scores dropped by between 14 and 22 percent, depending on the resolution and the feature settings, testers said.

At both 1,024 by 768 and 1,600 by 1,200 with no advanced features enabled, the Radeon 9800 Pro 256MB is now noticeably faster. However, once anti-aliasing and anisotropic filtering are enabled, the performance of the two cards is almost identical.

Results remained mixed for other tests, as they had been under the original Build 320 results, and testers said that neither card could be said to have taken a decisive lead. However, that is an improvement for Nvidia, whose previous GeForce FX 5800 Ultra did not measure up to the competition.