RE: FW: SAN configuration for Banner

Well, it depends on how you mean a SAN - technically, all of the TPC-C benchmarks use a SAN, as they're using a switched fabric fibre channel architecture. Now, they're not using the "big honking array" type of SAN, but its a meshed Fibre Channel fabric regardless.

In a small SAN, there's no performance advantages to doing direct attached versus switched fabric - as your fabric grows, ISLs (Inter Switch Links) can rapidly become bottlenecks.

As far as the use of RAID-ed disks within the benchmark itself, both the TPC-C and TPC-H have consistency requirements that require you to demonstrate the recoverability of the system through failures of individual disks.

This ties into the difference in the number of disks between the microsoft and oracle benchmarks. The TPC-C benchmarks are structured in such a way that the more throughput you want to demonstrate, the more "warehouses" you have to simulate in the test, which means more disks - since the final result of the TPC-C is a price/performance number, there is a sweet spot for database software and hardware combinations that dictates the hardware sizing that gets used. For microsoft, that was obviously on a much smaller system than the oracle benchmark - I seem to recall the major TPC-C RAC benchmark for Oracle required a 17TB dataset - they RAIDed it because they had to demonstrate recoverability, and adding a tape infrastructure to support 17TB would have added significantly to the cost.

Thanks,
Matt

--
Matthew Zito
GridApp Systems
Email: mzito_at_gridapp.com
Cell: 646-220-3551
Phone: 212-358-8211 x 359
http://www.gridapp.com



> -----Original Message-----

> From: ml-errors_at_fatcity.com [mailto:ml-errors_at_fatcity.com] On 

> Behalf Of Mogens Nørgaard

> Sent: Wednesday, November 19, 2003 11:55 PM

> To: Multiple recipients of list ORACLE-L

> Subject: Re: FW: SAN configuration for Banner

> 

> 

> Oh well, even though we're not talking about RAID-F anymore, 

> let's break 

> the rule now and then :-).

> 

> It's funny to see how the technical guys say the correct things, the 

> vendor guy says some rubbish, and the manager guy decides to 

> do as the 

> vendor says. Maybe that sort of interesting decision process always 

> happens when managers have to make decisions about expensive stuff. 

> Maybe there's a law about big amounts.

> 

> But we should rejoice: We have said the right things, and they didn't 

> listen. We have therefor, without any loss of integrity, created a 

> wonderful area of future work. In these times where companies are 

> looking for RAIA-I and RAIA-C (Redundant Array of Inexpensive 

> Asians - 

> India/China) solutions, anything that creates future performance and 

> availability problems in our home countries should be welcomed.

> 

> Here's a useless fact: When looking yesterday at Full 

> Disclosure Reports 

> for various tpc-c benchmarks at www.tpc.org, I found that all of them 

> use striping, but of course not RAID-5 (hey, benchmarks are for 

> performance).

> 

> And they never use a SAN, of course. Nobody wants all the codepath of 

> 8-9 layers of distraction (or was it abstraction) between the 

> OS and the 

> disk plates. Too much overhead, and it's not needed.

> 

> So I had a chat with a friend of mine who's done real 

> benchmarks. I was 

> commenting on the fact, that for the 1million tpc-c benchmark 

> Oracle did 

> recently, they used 120 73GB disks plus 2100 36GB disks. 

> Microsoft with 

> their 800000 tpc-c benchmark "only" used 1754 disks or so (60 for the 

> log, 2 for the OS, the rest for data).

> 

> My friend then told me that he always believed that you 

> should never use 

> a SAN for a high-performance system. Always direct attach. When doing 

> benchmarks, though, they would run into the problem that with 

> 1000s of 

> disks attached it could take several hours to boot the system 

> (and you 

> need to do that regularly when doing benchmarks!). So in the 

> benchmark 

> world they're moving into RAID-10 now in order to be able to sustain 

> disk losses (they happen frequently when using 1000s of 

> disks) without 

> having to boot the server.

> 

> We also discussed availability of standalone versus clustered 

> nodes. I 

> have, based on the discussion, devised the following simple formula:

> 

>                 A = (100 - Nc)%  where A is Availability and Nc is 

> number of Nodes in a cluster.

> 

> Consolidations mean future work near you! So let's support SAN's, 

> clusters, database consolidation, and all such things. Let's increase 

> chaos. It's our only chance of survival.

> 

> Mogens

> 

> Paul Baumgartel wrote:

> 

> >Oh boy.

> >

> >I'd first challenge the "I disagree..RAID 5 is a proven technology".

> >Ask him for credible research and/or statistics that support his

> >position.  Sure, RAID 5 is a proven technology...so are floppy disks,

> >and so what?

> >

> >Second:  clustered systems with failover mitigate disk array 

> >performance considerations?  Just how does THAT work?

> >

> >Good luck!

> >

> >Paul

> >

> >--- Sam Bootsma <sbootsma_at_gbrownc.on.ca> wrote:

> >  

> >

> >>Hi List,

> >>

> >> 

> >>

> >>We are approaching the cusp of a decision on how to store 

> Oracle data 

> >>files on our SAN.  We don't have the SAN yet, but it is due 

> to arrive 

> >>any week (if not any day).

> >>

> >> 

> >>

> >>I passed Cary's "Is RAID 5 Really a Bargain?" paper to our 

> Sys Admin, 

> >>which he read and succinctly summarized for the Technical Manager 

> >>here. I have also read through a couple of papers referenced in the 

> >>BAARF site.  The Sys Admin comments were:

> >>

> >> 

> >>

> >>Dell would like to know what RAID mode we want configured 

> on the SAN 

> >>for the B80 and 6C4 computers. Sam has told me that, in the Oracle

> >>community, mirroring (RAID1) is preferred over RAID 5 for various

> >>reasons (RAID5 is: more costly for write-intensive applications, 3

> >>times

> >>more likely to incur data loss,  suffers from massive performance

> >>degradation during partial outages). RAID1 will be more costly per

> >>unit

> >>of usable storage. Mirroring seems to be the best choice. 

> Let me know

> >>what you think.

> >>

> >> 

> >>

> >>Here is the Manager's response:

> >>

> >> 

> >>

> >>Any suggestions on how I can counter points 4 and 5 - and the last 

> >>point before his "Thanks" line?  Currently we have two B80's (AIX 

> >>4.3.3) set

> >>up in a HA configuration.  They share an external disk array.  So if

> >>a

> >>hardware component in the primary box fails, then it will

> >>automatically

> >>failover to the secondary box (and at the same time, the secondary

> >>box

> >>takes control of the external disk array).  I think the clustering

> >>term

> >>in point (4) is referring to this setup.

> >>

> >> 

> >>

> >>Thanks for any suggestions.

> >>

> >> 

> >>

> >>Sam.

> >>

> >> 

> >>

> >> 

> >>

> >>Sent: November 18, 2003 5:08 PM

> >>

> >> 

> >>

> >>All the points are valid...however..my thought processes were as

> >>follows:

> >>

> >> 

> >>

> >>1.  The System & Core Application disks are resident on the disks 

> >>within the CPU and Mirrorred (Everyone OK with that I think)

> >>

> >> 

> >>

> >>2.  The Databases are Resident on the SAN

> >>

> >> 

> >>

> >>3.  The SAN disks are RAID 5 as the provide more usable 

> space for the 

> >>cost as compared to mirrorring

> >>

> >> 

> >>

> >>4.  As the IBM Systems (B80's & 6C4's) are clusterd thus 

> effectivley 

> >>Mirrors the RAID 5 Arrays mitigating the issues Sam raises re 

> >>preformance degradation (which will only ever arise in the 

> event of a 

> >>failed disk/automated rebuild which is usually configurable 

> to address

> >>performance degradation)

> >>

> >> 

> >>

> >>5.  Write to Disk/Commit to Database should be a background process 

> >>(although I recognise this is a transaction/write intensive based

> >>system)

> >>

> >> 

> >>

> >>This is a standard model that all servers are being 

> deployed with and 

> >>unless there are any specific technical reasons why this 

> will not work

> >>it is the way I would like to see the systems implemented.  Remeber,

> >>with the SAN...Reconfiguration of Disks is not a large issue anymore

> >>if

> >>required in the future.

> >>

> >> 

> >>

> >>Although not an AIX/Oracle guy...I disagree with the statement that 

> >>RAID5 is 3 times more susceptable to incur Data Loss.  RAID 5 is a 

> >>proven technology

> >>

> >> 

> >>

> >>Thanks.....  Andrew

> >>

> >> 

> >>

> >>	-----Original Message----- 

> >>	From: Carl Nowak 

> >>	Sent: Tue 18/11/2003 2:56 PM 

> >>	To: Andrew Riem 

> >>	 Subject: SAN configuration for Banner

> >>

> >>	 

> >>

> >>	Dell would like to know what RAID mode we want 

> configured on the SAN 

> >>for the B80 and 6C4 computers. Sam has told me that, in the Oracle

> >>community, mirroring (RAID1) is preferred over RAID 5 for various

> >>reasons (RAID5 is: more costly for write-intensive applications, 3

> >>times

> >>more likely to incur data loss,  suffers from massive performance

> >>degradation during partial outages). RAID1 will be more costly per

> >>unit

> >>of usable storage. Mirroring seems to be the best choice. 

> Let me know

> >>what you think.

> >>

> >>	 

> >>

> >>	Carl

> >>

> >>

> >>    

> >>

> >

> >

> >__________________________________

> >Do you Yahoo!?

> >Protect your identity with Yahoo! Mail AddressGuard 

> >http://antispam.yahoo.com/whatsnewfree

> >  

> >

> 

> -- 

> Please see the official ORACLE-L FAQ: http://www.orafaq.net

> -- 

> Author: =?ISO-8859-1?Q?Mogens_N=F8rgaard?=

>   INET: mln_at_miracleas.dk

> 

> Fat City Network Services    -- 858-538-5051 http://www.fatcity.com

> San Diego, California        -- Mailing list and web hosting services

> ---------------------------------------------------------------------

> To REMOVE yourself from this mailing list, send an E-Mail message

> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') 

> and in the message BODY, include a line containing: UNSUB 

> ORACLE-L (or the name of mailing list you want to be removed 

> from).  You may also send the HELP command for other 

> information (like subscribing).

> 


-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.net
-- 
Author: Matthew Zito
  INET: mzito_at_gridapp.com

Fat City Network Services    -- 858-538-5051 http://www.fatcity.com
San Diego, California        -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).