Best Oracle configuration for HP DL585, six 6402 controllers and 42 disks?

Case: A new HP DL585 server dedicated to Oracle database use. Equipped with two AMD dual core processors (upgradeable to four CPUs if CPU becomes the bottleneck). Initially 16 gigabytes of RAM, more RAM will be added if required. Red Hat Linux 4 AS. Four internal 15k 146 GB U320 disks for OS and database software (two mirrored pairs) on integrated RAID controller. Six HP 6402 RAID controllers for database disks, each controller has 256 MB of battery backed cache. The controllers have two U320 channels but only one channel per controller is used to maximize controller throughput. Each controller drives seven 15k 146 GB U320 disks. The disks are on MSA30 DB (dual bus) arrays, 14 disks per array so the total number of disks on arrays is 42. A separate storage system is used for RMAN disk-to-disk backups and it is connected via dual 2 Gb FC.

Question: how to configure the controllers and Oracle software to make the best out of the disk subsystem. The server will host several databases, one large and several small ones. The databases are weighted on reads and data warehouse use (multi-user sequential). The system should be up and running most of the time but has no real 24/7 requirements so some downtime can be tolerated.

Assumably we will configure each 7 disk set as RAID 1+0 with hot spare and present each set as one raw device on Red Hat. The Oracle ASM will be used so that it sees six logical raw devices and they will then form a single ASM disk group. All data on all databases (control files, tablespaces, redos, archived redos etc.) will be put on this single ASM disk group so the data and the IO load will be spread equally over all six logical raw devices. Assumably this will give us a maximal IO performance. There is dual striping (stripe on stripe): ASM does striping over the six logical raw devices and under each device there is internal striping done by 6402 controller over the active disks. The controllers would be set to use the cache only for writes and not for reads since Oracle has its own read cache = the buffer cache.

Does this configuration sound reasonable? If ASM does the striping on 1 MB slices what should be the stripe size on 6402 controllers? What is the reliability and failure rate of a 6402 controller - in our configuration all the databases will go down if one controller malfunctions. Received on Tue Apr 25 2006 - 15:04:37 CDT