Re: AW: Sizing - RAC, storage subsystem EMC

I can see you point.

Yet an IO is an IO is an IO. As Gaja's excellent presentation at the Hotsos Symposium in Dallas documented, there's about eight levels between the OS and the disk plate.

Yet the disk plate delivers either 50 or about 100 IOPS. And Oracle requests IO's from the OS.
That's it. Oracle is not special, Oracle is a (big) application. Oracle does IO's. Disks deliver IO's.

If the IO is a write, RAID-5 means 4 IOs. If the IO is a read, RAID-5 means 1 IO.

It doesn't really matter whether it's the temp tablespace, the system tablespace, the undo tablespace, index tablespaces or perhaps even now and then the table tablespace that requests IO.

I think I'll write a chapter about this :).

Mogens

Stefan Jahnke wrote:

>Hi
>
>I'm in the same deep hole of being not toooo capable of justifying hardware
>choice issues. All the literature I find deals with the storage technology
>itself, but leaves database specific aspects behind. Everything I read so
>far always leaves me with the "And that means exactly WHAT from an Oracle
>point of view ?" feeling and I still don't know exactly which hardware
>design is more or less suitable for an Oracle server. A book specialized on
>that would be just great.
>
>Stefan
>
>-----Ursprüngliche Nachricht-----
>Von: George.Leonard_at_za.didata.com [mailto:George.Leonard_at_za.didata.com]
>Gesendet: Dienstag, 27. Mai 2003 08:27
>An: Multiple recipients of list ORACLE-L
>Betreff: RE: Sizing - RAC, storage subsystem EMC
>
>
>Hi Dennis
>
>I second that, It would be great to get someone to sit and design/publish a
>book of system design hardware (Sun, HP, Compaq, IBM, DELL etc) and storage
>designs from JBOD to SAN and list the experiences.
>
>We can even have a nice case study section and list designs and
>configuration for companies like Oracle, Dell, Yahoo, French Telecom etc.
>
>It will def go a long way of making designers see what others has done.
>Instead of all of us always starting from over and then trying to get buy in
>from a System Admin because we did the hard drives in the SAN as RAID 10
>instead of his magic always performing cheap RAID 5.
>
>
>
>George
>________________________________________________
>George Leonard
>Oracle Database Administrator
>Dimension Data (Pty) Ltd
>(Reg. No. 1987/006597/07)
>Cell: (+27) 82 655 2466
>Tel: (+27 11) 575 0573
>Fax: (+27 11) 576 0573
>E-mail:george.leonard_at_za.didata.com
>Web: http://www.didata.co.za
>
>You Have The Obligation to Inform One Honestly of the risk, And As a Person
>You Are Committed to Educate Yourself to the Total Risk In Any Activity!
>Once Informed & Totally Aware of the Risk, Every Fool Has the Right to Kill
>or Injure Themselves as They See Fit!
>
>-----Original Message-----
>Sent: 27 May 2003 02:57 AM
>To: Multiple recipients of list ORACLE-L
>
>Mogens - Now you're getting the entrepreneurial spirit! The other idea would
>be to follow in Cary's footsteps and detail your knowledge in a book. That
>automatically increases your credibility. :-)
> Seriously, that has been one of my complaints. We need a Configuring
>Hardware for Oracle book. There have been a few attempts. Alomari is the
>latest one as far as I know. Just a glossary of technical choices and what
>they mean to a DBA would be a good starting point.
>I think there are a couple of pitfalls:
> - Include comparative results. Is RAID5 10 percent slower? 20? Under what
>types of loads?
> - Don't base all results on batch benchmarks but on simulated real loads.
>Remember the days when raw was loudly touted? Couldn't be beaten on batch
>benchmarks. But few production sites found it worth the trouble.
> - Balance performance with vulnerability. If a tradeoff costs a little
>performance but makes failure less likely is good for me. Or makes recovery
>more straightforward. I note the thread on backing up online redo logs has a
>lot to do with making the recovery foolproof. My boss may never notice a 5%
>reduction in performance, but a crash sure gets his attention. And a bungled
>recovery could cause him to pop an artery. Or me.
>Maybe this book isn't practical, but I figure -- wish large, buy what I can
>get.
>
>
>Dennis Williams
>DBA, 80%OCP, 100% DBA
>Lifetouch, Inc.
>dwilliams_at_lifetouch.com
>
>-----Original Message-----
>Sent: Sunday, May 25, 2003 11:57 PM
>To: Multiple recipients of list ORACLE-L
>
>
>I finally see my error. Why do I keep having high blood pressure about
>RAID-5/RAID-4/RAID-53/RAID-S decisions? Doesn't make sense...because:
>
>More RAID-5 -> more performance and availability problems -> more
>performance and availability work -> more money to consultants -> higher
>profit -> more money to company owner -> sex & power to company owner
>
>Best regards,
>
>Mogens Nørgaard
>co-owner of Miracle A/S (a consultancy which specializes in performance and
>availability problems)
>
>DENNIS WILLIAMS wrote:
>
>
>Rachel - You make some excellent points. I am speaking to the situation
>
>where the site doesn't have a performance problem. When there is a problem,
>
>everyone is much more eager to accept suggestions.
>
> As you point out, the DBA usually doesn't buy hardware, so the
>
>hardware people tend to take that as intruding on their responsibilities
>
>with the intent of making them look bad. So they react defensively. And for
>
>every reason you can quote that RAID5 is bad, they will provide a
>
>countervailing reason why your information isn't valid. At least not with
>
>their hardware configuration. And the hardware vendor is very helpful in
>
>giving them information that will bolster their arguments. Then again, your
>
>average DBA usually isn't a foremost expert in disk configurations.
>
> A consultant that is paid to come in and solve a performance problem
>
>then leave doesn't face the longer-term issues that arise from making ones
>
>colleagues look bad. My point is that there are some battles one wouldn't
>
>want to win.
>
>
>
>Dennis Williams
>
>DBA, 80%OCP, 100% DBA
>
>Lifetouch, Inc.
>
>dwilliams_at_lifetouch.com <mailto:dwilliams_at_lifetouch.com>
>
>
>
>
>
>-----Original Message-----
>
>Sent: Friday, May 23, 2003 8:27 PM
>
>To: Multiple recipients of list ORACLE-L
>
>
>
>
>
>
>
>Here's the thing. Disks are cheap. Maybe. But buying 5 disks is still
>
>cheaper than buying 10 disks. And it is NOT the people who actually
>
>work with the disks (DBAs, Sysadmins, even developers) who have the
>
>authority to order the disks.
>
>
>
>You need places and floor space to put the disks as well. It's not like
>
>you can just hang them from the ceiling.
>
>
>
>And even more so these days, and it seems most especially in the US
>
>companies, companies are trying to cut costs to the bone (we just went
>
>through an exercise where we practically had to jump through hoops just
>
>to get more memory for a production system which obviously needed the
>
>memory). And cutting costs means as few disks as possible, means reuse,
>
>consolidate and do without.
>
>
>
>It kind of reminds me of one of the sayings my parents (who grew up
>
>during the Great Depression here) have told me of that time: "use it
>
>up, make it do, or do without"
>
>
>
>we're in the "do without" phase
>
>
>
>Rachel
>
>--- Mogens_Nørgaard <mailto:mln_at_miracleas.dk> <mln_at_miracleas.dk> wrote:
>
>
>
>I'm beginning to think that we're having this wonderful discussion
>
>about
>
>RAID-5 time and again because the vendors keep coming up with
>
>arguments
>
>to prove that the general laws of nature don't apply to this
>
>particular
>
>technology.
>
>
>
>In the 50s we had a prime minister here in Denmark that said to the
>
>assembled parliament: "If that's the facts, then I deny the facts".
>
>Somehow you have to admire a guy like that.
>
>
>
>Since disks are now cheap, how on Earth is it that we allow various
>
>IO-vendors to bundle cheap disks, expensive cache which we won't
>
>need,
>
>and stupid technologies like RAID-5 into a box called SAN/NAS or
>
>whatever and charge 42 billion units of some real currency for it?
>
>
>
>Why don't we take our swords and shotguns and AK-74s (that's the
>
>small
>
>calibre), and point them at them until they actually just bundle a
>
>bunch
>
>of inexpensive disks into RAID 1+0 which we ALL know is superior to
>
>RAID-S, RAID-5 (notice how S looks like 5 - it's no coincidence).
>
>It's
>
>superior to RAID 0+1 even. It doesn't require (the expensive) cache
>
>either.
>
>
>
>One of my wonderful experiences was with a TelCo here, where the
>
>vendor
>
>kept saying that they should of course just add cache if performance
>
>wasn't sufficient. When they finally reached the 32GB (yep,
>
>thirty-two
>
>giga-bytes) of cache alone, they gave up, ripped the stuff apart and
>
>reconfigured it as RAID 0+1 and it finally performed on the small
>
>writes. We all loved it - except possibly the disk-vendor, who kept
>
>saying RAID-<some letter that resembles the number 5> was of course
>
>fantastic, and that they shouldn't listen to all these bitter, old,
>
>twisted Oracle people who just hated RAID-5 and the likes ... ahhh...
>
>
>
>just because!
>
>
>
>It's got nothing to do with Oracle or not. It's got to do with money
>
>and
>
>brains.
>
>
>
>RAID 1+0 is the best you can buy.
>
>RAID 0+1 is almost as good.
>
>RAID 5 sucks - in all its disguises and permutations - compared to
>
>0+1
>
>or 1+0.
>
>
>
>If RAID-5 is good enough for you, fine. But remember to test the
>
>good,
>
>old restore of the backup and time it so that you're prepared if it
>
>happens. It should of course take about four times as long as the
>
>backup.
>
>
>
>As Connor said (and Cary has said it in his excellent, but futile,
>
>RAID-5 paper) it's when you really, really need disk performance that
>
>
>
>RAID-5 will let you down.
>
>
>
>Yo.
>
>
>
>Mogens
>
>
>
>Arun Annamalai wrote:
>
>
>
>
>
>Hi George.
>
>
>
>I would recommend go with Raid 5 and its not that one transaction
>
>triggers that many writes as you have mentioned/calculated.
>
>It is all buffered with recovery mechanism.
>
>
>
>Have you thought about Raid4(Network appliance) hardware. (I am no
>
>
>
>way
>
>
>
>affliated with Network appliance, except that we use in our site.)
>
>Most of the big names use them, such as
>
>1, Yahoo. (you can see network appliance logo when you logout of
>
>
>
>your
>
>
>
>yahoo email account)
>
>2. Oracle
>
>3. Southwest airlines.
>
>
>
>I mean Yahoo, Southwest Airlnes are all heavy transaction oriented
>
>shop on a given per minute interval.
>
>The most common overhead of SAN is that the throughput of the
>
>
>
>switch
>
>
>
>that connects your server to the san storage.
>
>
>
>Also, heard that Net App (best in NAS) is colloborating with
>
>
>
>Hitachi
>
>
>
>(best in SAN storage) to get the best technology for storage and
>
>thoroughput performance. But this might take a while or might
>
>
>
>already
>
>
>
>be in the market. Check out with your local Hitachi or Netapp
>
>representatives.
>
>
>
>Hope this helps.
>
>-Arun.
>
>
>
>
>
> ----- Original Message -----
>
> From: George.Leonard_at_za.didata.com
><mailto:George.Leonard_at_za.didata.com>
>
> <mailto:George.Leonard_at_za.didata.com>
><mailto:George.Leonard_at_za.didata.com>
>
> To: Multiple recipients of list ORACLE-L
>
> <mailto:ORACLE-L_at_fatcity.com> <mailto:ORACLE-L_at_fatcity.com>
>
> Sent: Tuesday, May 20, 2003 2:06 AM
>
> Subject: Sizing - RAC, storage subsystem EMC
>
>
>
> Hi all, hope you can give some input ideas.
>
>
>
>
>
>
>
> I am in the process of designing a system for a client of ours
>
>
>
>for
>
>
>
> a proposal
>
>
>
>
>
>
>
> The sizing information I have been given is as follows.
>
>
>
>
>
>
>
> 58.1 million tickets/day at 351 bytes per record. The record
>
>
>
>was
>
>
>
> complete populated (all columns filled to max) in a table and
>
>
>
>then
>
>
>
> analyzed. Average row size 351 bytes.
>
>
>
> =~ 19 GB/day. Raw data. Plus overhead (indexes, temp space,
>
> rollback, some other data etc) here and there I have requested
>
>
>
>5 TB.
>
>
>
>
>
>
>
> We need to keep records for a month. Table design I am looking
>
>
>
>at
>
>
>
> is a date partition with a second level hash partition. This is
>
>
>
>so
>
>
>
> that I can move data in the oldest week/table space off line
>
>
>
>and
>
>
>
> write them to optical storage for possible retrieval at a later
>
> date (requirement).
>
>
>
>
>
>
>
> Of course this will be on locally managed table spaces with
>
>
>
>auto
>
>
>
> storage management for segments.
>
>
>
>
>
>
>
> Hardware:
>
>
>
> The database will be a Oracle RAC 9.2.0.4 on Sun cluster 3
>
>
>
>build
>
>
>
> on 2 x Sun StarFire V880, 4 CPU's, 4 GB RAM each,
>
>
>
> Connected to an EMC SAN via Fiber Channel
>
>
>
>
>
>
>
> I do not have more information about the EMC array at the
>
>
>
>moment.
>
>
>
> Hitachi has been mentioned. (excuse the spelling)
>
>
>
>
>
>
>
> Question I have.
>
>
>
>
>
>
>
> I have been asked how many writes the Database will be doing to
>
> the SAN per second.
>
>
>
> I have determined that I should expect about 2000
>
>
>
>tickets/second.
>
>
>
> The table in question will have 2 indexes.
>
>
>
>
>
>
>
> Now following rough guessing I said I should expect at least 16
>
> 000 writes/second
>
>
>
>
>
>
>
> This was done by say/assuming
>
>
>
>
>
>
>
> 2 writes for the redo log files (2 members)
>
>
>
> 2 writes for the control files (2 control files)
>
>
>
> 2 writes to index blocks
>
>
>
> 1 write to undo table space block
>
>
>
>
>
>=== message truncated ===
>
>
>
>
>
>__________________________________
>
>Do you Yahoo!?
>
>The New Yahoo! Search - Faster. Easier. Bingo.
>
>http://search.yahoo.com <http://search.yahoo.com>
>
>
>
>
>
>

-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.net
-- 
Author: =?ISO-8859-1?Q?Mogens_N=F8rgaard?=
  INET: mln_at_miracleas.dk

Fat City Network Services    -- 858-538-5051 http://www.fatcity.com
San Diego, California        -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).