ORacle IO Numbers (ORION)

in the last weeks I was heavily using the ORION test tool available from OTN. I found some rediculous results from a 32 disks array. The array is supposed to physically deliver about 4500 iops.

I believe that the pseudo random number generator (or the way it is used to calculate the block to read) within ORION is not appropriate, as I see cache hits in our SAN storage backend where none are expected from the number of blocks read. But I might be wrong and the effect has a different origin. Look into this test taken, where the latency goes down again after continously increasing:

ran (small): my 1 oth 0 iops 121 size 8 K lat 8.25 ms bw = 0.95 MBps dur 59.99 s READ
ran (small): my 2 oth 0 iops 211 size 8 K lat 9.43 ms bw = 1.66 MBps dur 59.98 s READ
ran (small): my 3 oth 0 iops 299 size 8 K lat 10.03 ms bw = 2.34 MBps dur 59.97 s READ
ran (small): my 4 oth 0 iops 371 size 8 K lat 10.75 ms bw = 2.91 MBps dur 59.96 s READ
ran (small): my 5 oth 0 iops 456 size 8 K lat 10.95 ms bw = 3.57 MBps dur 59.97 s READ
ran (small): my 6 oth 0 iops 511 size 8 K lat 11.74 ms bw = 3.99 MBps dur 59.96 s READ
ran (small): my 7 oth 0 iops 573 size 8 K lat 12.21 ms bw = 4.48 MBps dur 59.95 s READ
ran (small): my 8 oth 0 iops 639 size 8 K lat 12.51 ms bw = 4.99 MBps dur 59.97 s READ
ran (small): my 16 oth 0 iops 1019 size 8 K lat 15.70 ms bw = 7.96 MBps dur 59.96 s READ
ran (small): my 24 oth 0 iops 1294 size 8 K lat 18.53 ms bw = 10.11 MBps dur 59.92 s READ
ran (small): my 32 oth 0 iops 1547 size 8 K lat 20.68 ms bw = 12.09 MBps dur 59.92 s READ
ran (small): my 40 oth 0 iops 1739 size 8 K lat 22.99 ms bw = 13.59 MBps dur 59.90 s READ
ran (small): my 48 oth 0 iops 1936 size 8 K lat 24.78 ms bw = 15.13 MBps dur 59.91 s READ
ran (small): my 56 oth 0 iops 2082 size 8 K lat 26.88 ms bw = 16.27 MBps dur 59.89 s READ
ran (small): my 64 oth 0 iops 2207 size 8 K lat 28.98 ms bw = 17.25 MBps dur 59.87 s READ
ran (small): my 72 oth 0 iops 2324 size 8 K lat 30.96 ms bw = 18.16 MBps dur 59.87 s READ
ran (small): my 80 oth 0 iops 2479 size 8 K lat 32.25 ms bw = 19.37 MBps dur 59.86 s READ
ran (small): my 88 oth 0 iops 2611 size 8 K lat 33.67 ms bw = 20.40 MBps dur 59.84 s READ
ran (small): my 96 oth 0 iops 2655 size 8 K lat 36.13 ms bw = 20.75 MBps dur 59.83 s READ
ran (small): my 104 oth 0 iops 2809 size 8 K lat 36.98 ms bw = 21.95 MBps dur 59.85 s READ
ran (small): my 112 oth 0 iops 2911 size 8 K lat 38.45 ms bw = 22.74 MBps dur 59.82 s READ
ran (small): my 120 oth 0 iops 3007 size 8 K lat 39.88 ms bw = 23.50 MBps dur 59.81 s READ
ran (small): my 128 oth 0 iops 3097 size 8 K lat 41.29 ms bw = 24.20 MBps dur 59.79 s READ
ran (small): my 136 oth 0 iops 4254 size 8 K lat 31.97 ms bw = 33.24 MBps dur 59.77 s READ
ran (small): my 144 oth 0 iops 4752 size 8 K lat 30.30 ms bw = 37.13 MBps dur 59.77 s READ
ran (small): my 152 oth 0 iops 5180 size 8 K lat 29.34 ms bw = 40.47 MBps dur 59.77 s READ
ran (small): my 160 oth 0 iops 5533 size 8 K lat 28.91 ms bw = 43.23 MBps dur 59.77 s READ

The vlun size is 3TB where the cache of the SAN box is 96GB so the cache hit should be minimal (1% of the vlun size was loaded into cache in the end of the test).

The following outcome was taken on the same array with smaller vlun size:

ran (small): my 1 oth 0 iops 2468 size 8 K lat 0.40 ms bw = 19.29 MBps dur 59.97 s READ
ran (small): my 2 oth 0 iops 8482 size 8 K lat 0.24 ms bw = 66.27 MBps dur 60.00 s READ
ran (small): my 3 oth 0 iops 8220 size 8 K lat 0.36 ms bw = 64.22 MBps dur 60.00 s READ
ran (small): my 4 oth 0 iops 8850 size 8 K lat 0.45 ms bw = 69.14 MBps dur 59.99 s READ
ran (small): my 5 oth 0 iops 9264 size 8 K lat 0.54 ms bw = 72.38 MBps dur 59.99 s READ
ran (small): my 6 oth 0 iops 10177 size 8 K lat 0.59 ms bw = 79.52 MBps dur 59.99 s READ
ran (small): my 7 oth 0 iops 11159 size 8 K lat 0.63 ms bw = 87.19 MBps dur 59.99 s READ
ran (small): my 8 oth 0 iops 11321 size 8 K lat 0.71 ms bw = 88.45 MBps dur 59.99 s READ
ran (small): my 16 oth 0 iops 16646 size 8 K lat 0.96 ms bw = 130.05 MBps dur 59.98 s READ
ran (small): my 24 oth 0 iops 22483 size 8 K lat 1.07 ms bw = 175.65 MBps dur 59.97 s READ
ran (small): my 32 oth 0 iops 23312 size 8 K lat 1.37 ms bw = 182.13 MBps dur 59.96 s READ
ran (small): my 40 oth 0 iops 28100 size 8 K lat 1.42 ms bw = 219.54 MBps dur 59.95 s READ
ran (small): my 48 oth 0 iops 29322 size 8 K lat 1.64 ms bw = 229.08 MBps dur 59.95 s READ
ran (small): my 56 oth 0 iops 31635 size 8 K lat 1.77 ms bw = 247.15 MBps dur 59.93 s READ
ran (small): my 64 oth 0 iops 32101 size 8 K lat 1.99 ms bw = 250.79 MBps dur 59.93 s READ
ran (small): my 72 oth 0 iops 33458 size 8 K lat 2.15 ms bw = 261.39 MBps dur 59.92 s READ
ran (small): my 80 oth 0 iops 36226 size 8 K lat 2.21 ms bw = 283.02 MBps dur 59.91 s READ
ran (small): my 88 oth 0 iops 35770 size 8 K lat 2.46 ms bw = 279.46 MBps dur 59.90 s READ
ran (small): my 96 oth 0 iops 37845 size 8 K lat 2.54 ms bw = 295.67 MBps dur 59.89 s READ
ran (small): my 104 oth 0 iops 37771 size 8 K lat 2.75 ms bw = 295.09 MBps dur 59.88 s READ
ran (small): my 112 oth 0 iops 38373 size 8 K lat 2.92 ms bw = 299.79 MBps dur 59.87 s READ
ran (small): my 120 oth 0 iops 38978 size 8 K lat 3.08 ms bw = 304.52 MBps dur 59.86 s READ
ran (small): my 128 oth 0 iops 41127 size 8 K lat 3.11 ms bw = 321.31 MBps dur 59.85 s READ
ran (small): my 136 oth 0 iops 48625 size 8 K lat 2.80 ms bw = 379.89 MBps dur 59.85 s READ
ran (small): my 144 oth 0 iops 51091 size 8 K lat 2.82 ms bw = 399.15 MBps dur 59.84 s READ
ran (small): my 152 oth 0 iops 53775 size 8 K lat 2.83 ms bw = 420.12 MBps dur 59.83 s READ

It reaches up to the limit of the fibre link even though the array of 32 disks is only expected to support about 4500 IO/s. In this second testcase, the proportion of data in the cache is higher (vlun size is 512G) but even if the cache is filled, the expected cache hit ratio is only 96G/512G=19%. Given that a read from cache lasts only 0.3 ms and one from disk lasts 8ms the expected maximum would be somewhere about 8000 or so.

Best regards.

Jan Received on Thu Jan 15 2009 - 12:08:23 CST