Re: lies damn lies and benchmarks

Andrew Mobbs <andrewm_at_chiark.greenend.org.uk> wrote in message news:<Efb*2tEnp_at_news.chiark.greenend.org.uk>...
> Mike Ault <mikerault_at_earthlink.net> wrote:
> >
> >Looking at the most current top ten values overall Oracle achieves a
> >TPCC/CPU-MGHTZ rating of 8.11 beaten only by Symphonies 8.24 (whatever
> >the heck Symphony is) SQL2000 comes in 4th place at 3.36. The most
> >bang for the buck comes in with Oracle's 1,019,668.87 per
> >TPCC/CPU-Mghrtz against SQL2000 at 1,592,560.56. UDB comes in at 4.92
> >and a cost of 1,733,687.45. UDB beats SQL2000 in TPCC/CPU-Mghtz but
> >looses in overall price. Oracle beats both handling when you remove
> >the obscuring junk in the numbers.
>
> Who cares about TpmC/MHz ? That's a meaningless metric. CPU performance
> is only loosely correlated with the clock speed, if you look at the SPEC
> CPU benchmarks, you can find a 750MHz PA-RISC outperforming a 1.5GHz
> Pentium 4.
>
> Oracle is a great database, with many impressive and useful
> features. TPC-C shows it scaling to impressive levels, but it (and
> Unix vendors) fall down dramatically on price/performance in the
> mid-range. Unless Oracle and HP/IBM/Sun want to give up this ground to
> MS, they better do something to fix this.
>
> For example, compare these two results:
> http://www.tpc.org/tpcc/results/tpcc_result_detail.asp?id=102031101
> http://www.tpc.org/tpcc/results/tpcc_result_detail.asp?id=102012901
>
> These show two benchmarks both published earlier this year, with an
> IBM/SQL Server system outperforming an Compaq/Oracle system that cost
> twice as much.

Even using these benchmarks, notice that the processors in one case (Microsoft) are 1.6 Ghtz while the ones in the Oracle9i test are 1000 ghtz, scaling Oracle into this added cpacity results in a potential TPCC of over 80,000. The only way to compare against different CPU/MGHTZ settings is to eliminate them from the consideration, if we do this we see an adjusted value of 8.61 for Microsoft while Oracle comes in at 12.53 using your own numbers. Now on the final bang per buck for these two configurations SQL2000 comes out on top at $40,686 while Oracle is at $60,963, but on the same hardware Oracle would have come in at $30,728 and on Oracles hardware SQL2000 would have come in at $88,658 or higher.

This is why I like the PC Week shootout article, same hardware, same disks. Eliminates these issues rather neatly.

Mike Received on Tue May 07 2002 - 13:51:42 CDT