Re: Database management techniques and frameworks

That isn't a moth!

Any improvements to PDBA toolkit are welcome. As Mladen points out, it is somewhat out of date.

But then again, it works, and it is fairly simple.

If you buy one on Mladens recommendation, I'll send him a check for 10% of what I make on it.

Mladen, I'll round it up to $0.10 for you.

Re the question asked about choice of monitoring platforms, mine is Linux, though most of my databases run on Windoze. It's much easier to manage from Linux.

The alert log monitoring scripts work from the database server, so most are running on Windoze. Should I get ambitious enough, I will rewrite it so that the Windoze portion is merely a server ( all X11 architecture ) and the client will be on Linux, and centralize the reporting portion ( emailing and paging ).

Automating everything you can so that emails merely have to be perused will simplify your life immensely.

Here's what I currently have automated.

Statspack report graphs via YAPPPACK. Important production databases have statspack running. Response times are daily graphed in 15 minute increments via YAPPACK, Statspack, DBI::Chart, and cron.

Objects that may run out of space are reported daily. These emails typically have a size of ~1920 bytes, so I don't actually read these unless the size of the email is larger than usual.

DBA_JOBS are monitored every 2 hours. I just scan down the email looking for a 'Y' in the 'Broken' column.

Alert log errors are emailed via a Perl script. This is found in the PDBA toolkit.

A script running on a Win2k server monitor important Oracle services on other Win2K/NT servers, and attempts to restart them if down. An email is sent when one is found to be down, and indicates the failure/success of the restart attempt.

All databases are checked every 5 minutes by logging into to them. Failures are recorded and sent the the DBA on call. (me). This script is configurable as to how many failures will be tolerated on a per database level before paging the dba, based on the critical times of day specified. Also in the PDBA toolkit.

A password server is used to supply passwords to all of the Linux based scripts, and some of the Win32 scripts. These are sent across the network encrypted in RC4. I considered using LDAP for this, but it was at the time too difficult to get OpenLDAP to work reliably. Also in the PDBA toolkit.

There might be more, I can't recall at the moment.

One word of advice regarding automation: Document the things you automate, even if it's only a checklist. It's surprising how easy it is not to notice that you are no longer receiving an email from an automated process because it's broken.

Scan the list occasionally to make sure everything is working.

In at least one case, I have a monitor that monitors another monitor. :)

Jared

On Fri, 2003-12-05 at 13:14, Mladen Gogala wrote:
> Ryan, have you tried PDBA toolkit? The address is:
> http://www.oreilly.com/catalog/oracleperl/pdbatoolkit/
> This toolkit has a plethora of very useful scripts. I seem to
> recollect an ugly looking O'Reilly book with moth on an orange
> overtone cover, which does a very good job on documenting it.
> The book is called "Perl for Oracle DBA". The PDBA toolkit is
> slightly out of date but still very useful, but the book is
> invaluable because it documents a whole lot of other tools like
> Oracle::OCI (a perversion) Apache::OWA, Apache::DBI and Mason,
> which are hard to come by and even harder to find examples that
> make sense. The author is Mr. Jared Still, otherwise known as the
> owner of this list. Please let me know if you purchase the book,
> because I'll have to charge Jared for commission.
>
>
>
>
> On 12/05/2003 03:44:32 PM, Ryan wrote:
> > one more point. Sorry for all the emails. I found that when writing scripts
> > for monitoring you really should follow an abstraction philosohpy similiar
> > to what you see in Object Oriented programming. Write utility scripts, use
> > data files, then have utility scripts that 'echo' out data from them like a
> > function.
> >
> > maintenance is much easier.
> > ----- Original Message -----
> > To: "Multiple recipients of list ORACLE-L" <ORACLE-L_at_fatcity.com>
> > Sent: Friday, December 05, 2003 1:49 PM
> >
> >
> > > Folks,
> > >
> > > I thought it'd be interesting to take a survey on what techniques and
> > > frameworks DBA's on this list use to manage their Oracle databases. I
> > > imagine that some of us manage only a single database and instance, but in
> > > those configurations where there are many instances, multiple databases,
> > > different platforms/versions, etc., what are some of the strategies for
> > > management in place? What daily tasks do you perform, and how do you
> > > organize them? How do you manage user requests (individually or as part
> > > of a larger environment)? How do you handle jobs? Organization
> > > techniques? Naming standards? User/application deployment framework,
> > > etc., etc.?
> > >
> > > (Obviously we could write a book about this -- there's an idea! -- but
> > > summaries and pointers would be interesting. Perhaps we can come up with
> > > a best practices document and associated framework for Oracle database
> > > management.)
> > >
> > > Thanks,
> > >
> > > Adam
> > > --
> > > Please see the official ORACLE-L FAQ: http://www.orafaq.net
> > > --
> > > Author:
> > > INET: AdamDonahue_at_maximus.com
> > >
> > > Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> > > San Diego, California -- Mailing list and web hosting services
> > > ---------------------------------------------------------------------
> > > To REMOVE yourself from this mailing list, send an E-Mail message
> > > to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> > > the message BODY, include a line containing: UNSUB ORACLE-L
> > > (or the name of mailing list you want to be removed from). You may
> > > also send the HELP command for other information (like subscribing).
> >
> > --
> > Please see the official ORACLE-L FAQ: http://www.orafaq.net
> > --
> > Author: Ryan
> > INET: ryan_oracle_at_cox.net
> >
> > Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> > San Diego, California -- Mailing list and web hosting services
> > ---------------------------------------------------------------------
> > To REMOVE yourself from this mailing list, send an E-Mail message
> > to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> > the message BODY, include a line containing: UNSUB ORACLE-L
> > (or the name of mailing list you want to be removed from). You may
> > also send the HELP command for other information (like subscribing).
> >
>
> Mladen Gogala
> Oracle DBA
>
>
>
> Note:
> This message is for the named person's use only. It may contain confidential, proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Wang Trading LLC and any of its subsidiaries each reserve the right to monitor all e-mail communications through its networks.
> Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of any such entity.
>
> --
> Please see the official ORACLE-L FAQ: http://www.orafaq.net
> --
> Author: Mladen Gogala
> INET: mladen_at_wangtrading.com
>
> Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> San Diego, California -- Mailing list and web hosting services
> ---------------------------------------------------------------------
> To REMOVE yourself from this mailing list, send an E-Mail message
> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> the message BODY, include a line containing: UNSUB ORACLE-L
> (or the name of mailing list you want to be removed from). You may
> also send the HELP command for other information (like subscribing).
>

-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.net
-- 
Author: Jared Still
  INET: jkstill_at_cybcon.com

Fat City Network Services    -- 858-538-5051 http://www.fatcity.com
San Diego, California        -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).