Home » Other » Marketplace » Oracle Direct Session Memory Access - Free and open source
Oracle Direct Session Memory Access - Free and open source [message #652214] |
Sat, 04 June 2016 07:40 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
Hi All,
I've been working on a application which unitilises direct memory access and reads data directly from the databases SGA. I'm hoping that some of you might be interested in looking at the current beta version and willing to provide me with some feedback, good or bad!
LINKS:
* GitHub for source code: https://github.com/dba-bots/DirectSessionMemoryAccess
* SourceForge for dsma-*.tar.gz: https://sourceforge.net/projects/dsma
Small summary:
DSMA attaches to an Oracle session and reads data directly from Oracle's SGA. At present DSMA will return the following equivalent SQL statement:
select SEQ#, EVENT, P1TEXT, P1,P2TEXT, P2, P3TEXT, P3, PROGRAM, PROCESS, SQL_HASH_VALUE, PREV_HASH_VALUE from v$session;
Contact details:
email: neilc@dba-bots.com
Follow us on Twitter: @dbabots
|
|
|
|
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652248 is a reply to message #652247] |
Sun, 05 June 2016 09:45 |
Frank Naude
Messages: 4587 Registered: April 1998
|
Senior Member |
|
|
Just tested it and it works as advertised (got it running is less than 10 minutes).
To compile, I had to change 'initdsma.bsh' to supply a password for SYS. This won't normally be required, but I had to do this:
sqlplus -S sys/oracle as sysdba
It core dumped with AMM. I then enabled ASMM (as per README) and bounced:
SQL> alter system set memory_target = 0;
SQL> startup force;
Recompiled:
./initdsma.bsh -d orcl -v 12102
And tested:
$ ./dsma.orcl 2
Time between wait 0.0000094472 secs
WAIT# 1 waiting for event: 'null event' Class:Other
P1: ''=65538 P2: ''=139661606903408 P3 ':'=81
command= ospid=0 sqlhv=0 psqlhv=1936681068
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652250 is a reply to message #652247] |
Sun, 05 June 2016 12:10 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
John Watson wrote on Sun, 05 June 2016 15:28Could your program be used for hang analysis, if the instance is on a state where you can't even log on? It would need to show columns such as EVENT and BLOCKING_SESSION.
Hi John,
I believe it would be helpful in such a situation.
I've managed to get EVENT, P1, P2 and P3 information from memory, but unfortunately BLOCKING_SESSION doesn't have a fixed offset so I'm unable to set this value at compile time to enable DSMA to get such information from memory. However in the next release I plan to add ROW_WAIT_OBJ# , ROW_WAIT_FILE# , ROW_WAIT_BLOCK# and ROW_WAIT_ROW# which would give a few more vital clues if DSMA was used in such a situation, would you agree?
I find DSMA really good as a session profiler. Given the very high sample rate, along with the events data and timing information it all really helps identify bottlenecks.
I'm also currently looking at add a number of flags to DSMA, which will enable a user to run in a certain mode, hang analyses or profiler so the relevant information can be included/excluded where applicable.
Regards
Neil
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652251 is a reply to message #652247] |
Sun, 05 June 2016 12:17 |
|
Michel Cadot
Messages: 68731 Registered: March 2007 Location: Saint-Maur, France, https...
|
Senior Member Account Moderator |
|
|
John Watson wrote on Sun, 05 June 2016 16:28Could your program be used for hang analysis, if the instance is on a state where you can't even log on? ...
No it can't as it requires to first connect as sysdba to retrieve some information like shmid and address of each session entry.
The first part can be workaround using sysresv instead of oradebug (and so without any connection).
The second is harder to workaround without connecting as x$ksuse (v$session underlying "table") can be split into several arrays. You can see that using the following query which gives the distinct gaps between entry; if your "sessions" parameter is big enough you will see several values (double the X if you are in 64-bit).
select distinct to_number(addr,'XXXXXXXX')-to_number(lag(addr) over(order by addr),'XXXXXXXX') from x$ksuse;
I think it is possible to not need this if you change the program and get the values from the SGA itself but this need a serious study on x$ksuse structure.
I wrote something like that in the previous century and upgrape it as I said a decade ago (up to 10gR2). The main problem is that in the current versions the SGA can be spread over several shared segments and all the structures are automatically split (v$latch/x$ksllt) or as soon as they are large enough (v$session/x$ksuse, v$session_event/x$ksles)...
I stopped working on this program (which original aim was to query any x$ table or SGA structures) because most of the time when I come at client's there was, for security safe, no C compiler on the servers and even when there is one they refuse we compile our own programs which is understandable.
You will find a definitive presentation of how to access directly the SGA by Kyle Hailey there and complete demonstration code by Miladin Modrakovic there.
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652252 is a reply to message #652251] |
Sun, 05 June 2016 12:35 |
John Watson
Messages: 8963 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
I see, thank you for explaining. I have never looked into the possibility, though I intend to do so. This is the motivation:
When Oracle introduced ASH, they cut the ground out from beneath all third party providers of administration tools because ASH can use a background process to go directly to the C data structures that make up the instance. This makes it possible to do per second sampling. Any third party product (such as Tanel Poder's Snapper, or the Quest product set) has to go through the SQL interface which can never be as efficient. Then in release 12 Oracle introduced Emergency Monitoring and Real-Time ADDM, which should let you query the instance even when you can't log on - if you have OEM running. That is also something no-one else can do.
If someone writes code that can do the same and doesn't need Enterprise Edition plus packs, it would be highly marketable. Let me know, people! My boss would be very interested in promoting it.
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652253 is a reply to message #652251] |
Sun, 05 June 2016 12:37 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
Hi Michel,
The tool hasn't been designed to be intrusive so I felt no reason not to use SYSDBA privs or tools DBAs know and use.
Michel Cadot wrote on Sun, 05 June 2016 18:17
John Watson wrote on Sun, 05 June 2016 16:28Could your program be used for hang analysis, if the instance is on a state where you can't even log on? ...
No it can't as it requires to first connect as sysdba to retrieve some information like shmid and address of each session entry.
Slightly unfair statement, so long as DSMA was setup prior to a database hang the tool could be used.
Michel Cadot wrote on Sun, 05 June 2016 18:17
The first part can be workaround using sysresv instead of oradebug (and so without any connection).
You are correct "sysresv" could be used, but then you might aswell go to /proc/[PMON_PID]/maps which would save calling "sysresv" in the first instance.
[Updated on: Sun, 05 June 2016 12:41] Report message to a moderator
|
|
|
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652257 is a reply to message #652255] |
Sun, 05 June 2016 13:06 |
John Watson
Messages: 8963 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
Quote:just how I see my clients working, they never allow a permanent program hacking the memory structures running I think this is what Uncle Oracle relies upon. The OEM agent does just this to enable the emergency monitoring facility, but because it OEM, no-one objects. Any third party product would have to be validated by a reputable source. Larry, of course, would never validate it.
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652258 is a reply to message #652255] |
Sun, 05 June 2016 13:34 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
Michel Cadot wrote on Sun, 05 June 2016 18:50
Quote:Slightly unfair statement, so long as DSMA was setup prior to a database hang the tool could be used.
Not unfair, just how I see my clients working, they never allow a permanent program hacking the memory structures running (maybe the term hacking is not the one you want to hear but it actually is and this is not a term of abuse for me just a method to work as long as it is done transparently and with the agreement of the client). They allow them ONLY when a hang situation occurs (if they allow it at all). Carefully note that for the client if you can read the memory you can read their data with no possible audit.
The program will run so long as it has been compiled prior to a database hang occurring. Whether or not such a tool would be aloud by a client is another matter..
Michel Cadot wrote on Sun, 05 June 2016 18:50
Quote:but then you might as well go to /proc/[PMON_PID]/maps
As far as I know, only if you are root. I mention sysresv because it is a standard Oracle program and nobody can blame you to use it (in the limits of your mission, of course).
/proc permissions are 0555, anyone has read
Michel Cadot wrote on Sun, 05 June 2016 18:50
Take no offense of what I say, it is just the thoughts of someone who has worked for a long time on this kind of subject (on both sides: the DBA and the security officer).
None taken.
|
|
|
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652309 is a reply to message #652252] |
Mon, 06 June 2016 15:40 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
John Watson wrote on Sun, 05 June 2016 18:35I see, thank you for explaining. I have never looked into the possibility, though I intend to do so. This is the motivation:
When Oracle introduced ASH, they cut the ground out from beneath all third party providers of administration tools because ASH can use a background process to go directly to the C data structures that make up the instance. This makes it possible to do per second sampling. Any third party product (such as Tanel Poder's Snapper, or the Quest product set) has to go through the SQL interface which can never be as efficient. Then in release 12 Oracle introduced Emergency Monitoring and Real-Time ADDM, which should let you query the instance even when you can't log on - if you have OEM running. That is also something no-one else can do.
If someone writes code that can do the same and doesn't need Enterprise Edition plus packs, it would be highly marketable. Let me know, people! My boss would be very interested in promoting it.
I've just released beta v2. Now you do not need to no the SID prior to running DSMA. Running DSMA without passing in a given SID will print an overview of all active sessions. You can then pass in a given SID to get a more detailed output regarding wait event history.
I've uploaded some screen shots on SourceForge
[Updated on: Mon, 06 June 2016 15:41] Report message to a moderator
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652310 is a reply to message #652309] |
Mon, 06 June 2016 16:10 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
Once you have ran initdsma.bsh for a given Oracle database you can call the program at anytime by running the executable DSMA.DBNAME
If you couldn't connect to the database, say via sqlplus DSMA will still work, so long as the SGA remains online and the DSMA.DBNAME executable was complied prior to this event occurring.
You just need to remember to re-run the initdsma.bsh after the database has bee bounced so it's there when you need it
[Updated on: Mon, 06 June 2016 16:23] Report message to a moderator
|
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652399 is a reply to message #652372] |
Wed, 08 June 2016 02:54 |
Frank Naude
Messages: 4587 Registered: April 1998
|
Senior Member |
|
|
I've installed v.3 as well. Let's use it to resolve a blocking lock:
$ ./dsma.orcl | grep sqlplus
SID EVENT PROGRAM OSPID
44 SQL*Net message from client sqlplus@localhost.localdomain (TNS V1-V3) 4155
47 enq: TM - contention sqlplus@localhost.localdomain (TNS V1-V3) 4187
So, let's look at both sessions:
$ ./dsma.orcl 44
Time between wait 0.0000048355 secs
WAIT# 407 waiting for event: 'SQL*Net message from client' Class:Idle
P1: 'driver id'=1413697536 P2: '#bytes'=1 P3 ':'=0
command=LOCK TABLE ospid=4155 sqlhv=937634959 psqlhv=937634959
$ ./dsma.orcl 47
Time between wait 0.0000032143 secs
WAIT# 64 waiting for event: 'enq: TM - contention' Class:Application
P1: 'name|mode'=1414332419 P2: 'object #'=92264 P3 ':table/partition'=0
command=DELETE ospid=4187 sqlhv=147427357 psqlhv=3933222116
From the above it's clear that session 47 is hanging, waiting for session 44 to release a lock.
Easy to diagnose on a not-so-busy database.
Some suggestions, if I may:
* Add BLOCKING_SESSION in the detail display (as indicated by John above).
* Introduce an option to only list active sessions.
SQL> SELECT event, status, blocking_session FROM v$session WHERE sid IN (44, 47);
EVENT STATUS BLOCKING_SESSION
------------------------------ -------- ----------------
SQL*Net message from client INACTIVE
enq: TM - contention ACTIVE 44
Best regards.
Frank
|
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652472 is a reply to message #652428] |
Thu, 09 June 2016 11:57 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
It's been a long week! But I'm pleased to announce our first stable release:
DirectSessionMemoryAccess-v1.0
You'll also be pleased to hear the release cycle will now slow right down! I hope now we've moved away from beta releases more people will be willing to give it a try and see what it has to offer.
A huge thank you to everyone who gave up there time to; download and try, write a review, leave a comment and provide feedback during the beta phase.
The GitHub wiki now includes; The storey behind DSMA, FAQs and usage cases
Cheers
Neil
[Updated on: Thu, 09 June 2016 13:57] Report message to a moderator
|
|
|
|
|
Re: Oracle Direct Session Memory Access - Free and open source [message #652522 is a reply to message #652254] |
Sat, 11 June 2016 08:20 |
|
dba-bots
Messages: 17 Registered: June 2016
|
Junior Member |
|
|
John Watson wrote on Sun, 05 June 2016 18:45Neil, you'll have seen my answer to Michel. Can you compare your tool and where it might go against Tanel Poder's Snapper, http://blog.tanelpoder.com/files/scripts/snapper.sql ? Many people swear by Snapper (I don't use it myself) but I've always thought that it suffers from going through the SQL interface. Your product could be really good (and monetizable).
Hi John,
Having had little chance to have a look at Snapper, which I myself have never used either; I now know both tools sample the GV$SESSION view. Maybe someone with experience of Snapper and who has tried DSMA would be willing to make a comparison?
Hopefully you might have had a look at DSMA yourself and I would be keen to hear your comparison.
Regards
Neil
[Updated on: Sat, 11 June 2016 11:33] Report message to a moderator
|
|
|
|
Goto Forum:
Current Time: Sat Jan 18 00:05:54 CST 2025
|