Re: TNS-12500: TNS:listener failed to start a dedicated server process
From: joel garry <joel-garry_at_home.com>
Date: Thu, 8 Jan 2009 14:17:59 -0800 (PST)
Message-ID: <4670563e-1a7e-4887-805c-0cdc8b21beba_at_35g2000pry.googlegroups.com>
On Jan 8, 12:25 pm, shweta.kapar..._at_googlemail.com wrote:
> On Jan 8, 8:22 pm, Steve Howard <stevedhow..._at_gmail.com> wrote:
>
>
>
>
>
> > On Jan 8, 2:26 pm, shweta.kapar..._at_googlemail.com wrote:
>
> > > On Jan 8, 2:34 pm, Steve Howard <stevedhow..._at_gmail.com> wrote:
>
> > > > On Jan 8, 9:29 am, shweta.kapar..._at_googlemail.com wrote:
>
> > > > > On Jan 8, 1:59 pm, Steve Howard <stevedhow..._at_gmail.com> wrote:
>
> > > > > > On Jan 8, 8:57 am, shweta.kapar..._at_googlemail.com wrote:
>
> > > > > > > Hi All
>
> > > > > > > We are badly being hit by the following error:
>
> > > > > > > 08-JAN-2009 12:43:05 * (connect_data=(server=dedicated)
> > > > > > > (service_name=XX)(INSTANCE_ROLE=primary)(failover_mode=(type=session)
> > > > > > > (method=basic)(backup=YY)(retries=180)(delay=5))(CID=(PROGRAM=oracle)
> > > > > > > (HOST=ZZ)(USER=oracle))(INSTANCE_NAME=AAAA)) * (ADDRESS=(PROTOCOL=tcp)
> > > > > > > (HOST=<IP>)(PORT=<PORT>)) * establish * BB * 12500
> > > > > > > TNS-12500: TNS:listener failed to start a dedicated server process
> > > > > > > TNS-12540: TNS:internal limit restriction exceeded
> > > > > > > TNS-12560: TNS:protocol adapter error
> > > > > > > TNS-00510: Internal limit restriction exceeded
> > > > > > > HPUX Error: 11: Resource temporarily unavailable
>
> > > > > > > This is repeating at the following interval. ( as a sample.).. It is
> > > > > > > intermittent.
>
> > > > > > > Start End
> > > > > > > 08/01/2009 12:40:03 08/01/2009 12:40:13
> > > > > > > 08/01/2009 12:41:03 08/01/2009 12:41:04
> > > > > > > 08/01/2009 12:42:04 08/01/2009 12:42:04
> > > > > > > 08/01/2009 12:43:04 08/01/2009 12:43:04
> > > > > > > 08/01/2009 12:43:05 08/01/2009 12:43:05
>
> > > > > > > In between above intervals connections are successful
>
> > > > > > > We are using "dedicated Server" connection.
>
> > > > > > > CPU /Mem utlisation is normal. No change seen in the pattern.
>
> > > > > > > Regards
>
> > > > > > > Shweta.
>
> > > > > > Hi Shweta,
>
> > > > > > How do your OS kernel parameter limits look, i.e. open files, max
> > > > > > processes, etc.?
>
> > > > > > Regards,
>
> > > > > > Steve- Hide quoted text -
>
> > > > > > - Show quoted text -
>
> > > > > Hi Steve
>
> > > > > We have 05 databases running in Host -A
>
> > > > > I have taken below o/p from 1st database. We ra facing this issue in
> > > > > all the databases almost at the same time.
>
> > > > > SQL>select * from v$resource_limit;
>
> > > > > RESOURCE_NAME CURRENT_UTILIZATION MAX_UTILIZATION
> > > > > INITIAL_AL LIMIT_VALU
> > > > > ------------------------------ ------------------- ---------------
> > > > > ---------- ----------
> > > > > processes 135
> > > > > 256 1000 1000
> > > > > sessions 134
> > > > > 252 1105 1105
> > > > > enqueue_locks 72
> > > > > 361 14154 14154
> > > > > enqueue_resources 72
> > > > > 72 50000 UNLIMITED
> > > > > ges_procs 133
> > > > > 253 1001 1001
> > > > > ges_ress 37791
> > > > > 50079 90367 UNLIMITED
> > > > > ges_locks 28140
> > > > > 42011 96306 UNLIMITED
> > > > > ges_cache_ress 5361
> > > > > 8522 0 UNLIMITED
> > > > > ges_reg_msgs 222
> > > > > 250 2230 UNLIMITED
> > > > > ges_big_msgs 17
> > > > > 28 2230 UNLIMITED
> > > > > ges_rsv_msgs 0
> > > > > 0 1000 1000
> > > > > gcs_resources 72634 81356
> > > > > 122432 122432
> > > > > gcs_shadows 26643
> > > > > 26663 67337 67337
> > > > > dml_locks 0
> > > > > 36 1024 UNLIMITED
> > > > > temporary_table_locks 0 2
> > > > > UNLIMITED UNLIMITED
> > > > > transactions 14
> > > > > 66 1215 UNLIMITED
> > > > > branches 2
> > > > > 28 1215 UNLIMITED
> > > > > cmtcallbk 4
> > > > > 29 1215 UNLIMITED
> > > > > sort_segment_locks 134 137
> > > > > UNLIMITED UNLIMITED
> > > > > max_rollback_segments 11
> > > > > 11 244 244
> > > > > max_shared_servers 1
> > > > > 1 20 20
> > > > > parallel_max_servers 4
> > > > > 121 241 241
>
> > > > > 22 rows selected.
>
> > > > > Thanks.
>
> > > > Hi Shweta,
>
> > > > I meant on the OS itself, i.e. HP-UX kernel limits. I have seen
> > > > exceptions such as the one you posted when processes, open files, etc.
> > > > were approaching/at their maximum limit as allowed/configured by the
> > > > system administrator.
>
> > > > HTH,
>
> > > > Steve- Hide quoted text -
>
> > > > - Show quoted text -
>
> > > Thanks Mark and Steve,
>
> > > Metalink Document 146859.1 describes the exact problem
>
> > > and suggested fix is -
>
> > > "The problem can be avoided by raising the value of UNIX kernel
> > > parameter:
>
> > > maxuprc - max number of processes per user
>
> > > Refer to the HP Operating system documentation for detailed steps."
>
> > > I need some clarification on this :
>
> > > a) Here "max number of processes per user" : does it referring to
> > > unix "oracle" user.
> > > because each processes are owned /fork by oracle user at operating
> > > system level?
>
> > > b) Any issues/side effects if we change the maxuprc.
>
> > > Regards
>
> > As long as the server can handle that many processes, it is just a
> > matter of increasing them. AFAIK, this can be done live, so ask your
> > HP-UX system administrator to bump them up.
>
> > Alos, maybe you have idle connections that could be released (a good
> > admin is probably going to ask you that before he/she bumps up the
> > number)?- Hide quoted text -
>
> > - Show quoted text -
>
> Thanks a lot Steve.
Date: Thu, 8 Jan 2009 14:17:59 -0800 (PST)
Message-ID: <4670563e-1a7e-4887-805c-0cdc8b21beba_at_35g2000pry.googlegroups.com>
On Jan 8, 12:25 pm, shweta.kapar..._at_googlemail.com wrote:
> On Jan 8, 8:22 pm, Steve Howard <stevedhow..._at_gmail.com> wrote:
>
>
>
>
>
> > On Jan 8, 2:26 pm, shweta.kapar..._at_googlemail.com wrote:
>
> > > On Jan 8, 2:34 pm, Steve Howard <stevedhow..._at_gmail.com> wrote:
>
> > > > On Jan 8, 9:29 am, shweta.kapar..._at_googlemail.com wrote:
>
> > > > > On Jan 8, 1:59 pm, Steve Howard <stevedhow..._at_gmail.com> wrote:
>
> > > > > > On Jan 8, 8:57 am, shweta.kapar..._at_googlemail.com wrote:
>
> > > > > > > Hi All
>
> > > > > > > We are badly being hit by the following error:
>
> > > > > > > 08-JAN-2009 12:43:05 * (connect_data=(server=dedicated)
> > > > > > > (service_name=XX)(INSTANCE_ROLE=primary)(failover_mode=(type=session)
> > > > > > > (method=basic)(backup=YY)(retries=180)(delay=5))(CID=(PROGRAM=oracle)
> > > > > > > (HOST=ZZ)(USER=oracle))(INSTANCE_NAME=AAAA)) * (ADDRESS=(PROTOCOL=tcp)
> > > > > > > (HOST=<IP>)(PORT=<PORT>)) * establish * BB * 12500
> > > > > > > TNS-12500: TNS:listener failed to start a dedicated server process
> > > > > > > TNS-12540: TNS:internal limit restriction exceeded
> > > > > > > TNS-12560: TNS:protocol adapter error
> > > > > > > TNS-00510: Internal limit restriction exceeded
> > > > > > > HPUX Error: 11: Resource temporarily unavailable
>
> > > > > > > This is repeating at the following interval. ( as a sample.).. It is
> > > > > > > intermittent.
>
> > > > > > > Start End
> > > > > > > 08/01/2009 12:40:03 08/01/2009 12:40:13
> > > > > > > 08/01/2009 12:41:03 08/01/2009 12:41:04
> > > > > > > 08/01/2009 12:42:04 08/01/2009 12:42:04
> > > > > > > 08/01/2009 12:43:04 08/01/2009 12:43:04
> > > > > > > 08/01/2009 12:43:05 08/01/2009 12:43:05
>
> > > > > > > In between above intervals connections are successful
>
> > > > > > > We are using "dedicated Server" connection.
>
> > > > > > > CPU /Mem utlisation is normal. No change seen in the pattern.
>
> > > > > > > Regards
>
> > > > > > > Shweta.
>
> > > > > > Hi Shweta,
>
> > > > > > How do your OS kernel parameter limits look, i.e. open files, max
> > > > > > processes, etc.?
>
> > > > > > Regards,
>
> > > > > > Steve- Hide quoted text -
>
> > > > > > - Show quoted text -
>
> > > > > Hi Steve
>
> > > > > We have 05 databases running in Host -A
>
> > > > > I have taken below o/p from 1st database. We ra facing this issue in
> > > > > all the databases almost at the same time.
>
> > > > > SQL>select * from v$resource_limit;
>
> > > > > RESOURCE_NAME CURRENT_UTILIZATION MAX_UTILIZATION
> > > > > INITIAL_AL LIMIT_VALU
> > > > > ------------------------------ ------------------- ---------------
> > > > > ---------- ----------
> > > > > processes 135
> > > > > 256 1000 1000
> > > > > sessions 134
> > > > > 252 1105 1105
> > > > > enqueue_locks 72
> > > > > 361 14154 14154
> > > > > enqueue_resources 72
> > > > > 72 50000 UNLIMITED
> > > > > ges_procs 133
> > > > > 253 1001 1001
> > > > > ges_ress 37791
> > > > > 50079 90367 UNLIMITED
> > > > > ges_locks 28140
> > > > > 42011 96306 UNLIMITED
> > > > > ges_cache_ress 5361
> > > > > 8522 0 UNLIMITED
> > > > > ges_reg_msgs 222
> > > > > 250 2230 UNLIMITED
> > > > > ges_big_msgs 17
> > > > > 28 2230 UNLIMITED
> > > > > ges_rsv_msgs 0
> > > > > 0 1000 1000
> > > > > gcs_resources 72634 81356
> > > > > 122432 122432
> > > > > gcs_shadows 26643
> > > > > 26663 67337 67337
> > > > > dml_locks 0
> > > > > 36 1024 UNLIMITED
> > > > > temporary_table_locks 0 2
> > > > > UNLIMITED UNLIMITED
> > > > > transactions 14
> > > > > 66 1215 UNLIMITED
> > > > > branches 2
> > > > > 28 1215 UNLIMITED
> > > > > cmtcallbk 4
> > > > > 29 1215 UNLIMITED
> > > > > sort_segment_locks 134 137
> > > > > UNLIMITED UNLIMITED
> > > > > max_rollback_segments 11
> > > > > 11 244 244
> > > > > max_shared_servers 1
> > > > > 1 20 20
> > > > > parallel_max_servers 4
> > > > > 121 241 241
>
> > > > > 22 rows selected.
>
> > > > > Thanks.
>
> > > > Hi Shweta,
>
> > > > I meant on the OS itself, i.e. HP-UX kernel limits. I have seen
> > > > exceptions such as the one you posted when processes, open files, etc.
> > > > were approaching/at their maximum limit as allowed/configured by the
> > > > system administrator.
>
> > > > HTH,
>
> > > > Steve- Hide quoted text -
>
> > > > - Show quoted text -
>
> > > Thanks Mark and Steve,
>
> > > Metalink Document 146859.1 describes the exact problem
>
> > > and suggested fix is -
>
> > > "The problem can be avoided by raising the value of UNIX kernel
> > > parameter:
>
> > > maxuprc - max number of processes per user
>
> > > Refer to the HP Operating system documentation for detailed steps."
>
> > > I need some clarification on this :
>
> > > a) Here "max number of processes per user" : does it referring to
> > > unix "oracle" user.
> > > because each processes are owned /fork by oracle user at operating
> > > system level?
>
> > > b) Any issues/side effects if we change the maxuprc.
>
> > > Regards
>
> > As long as the server can handle that many processes, it is just a
> > matter of increasing them. AFAIK, this can be done live, so ask your
> > HP-UX system administrator to bump them up.
>
> > Alos, maybe you have idle connections that could be released (a good
> > admin is probably going to ask you that before he/she bumps up the
> > number)?- Hide quoted text -
>
> > - Show quoted text -
>
> Thanks a lot Steve.
While it is usually either simply number of processes or things keep making too many processes, the fact that all your instances do this at once could be something a little more obscure, like running out of swap space or tcp issues. Watch it carefully! Easy to goof with many instances, especially when things go to heck because these erroring processes aren't cleaning up correctly.
How are you accessing the db - jdbc, odbc, sqlplus, what?
Also post your exact os, platform, oracle versions.
jg
-- _at_home.com is bogus. http://blog.wired.com/27bstroke6/2009/01/professed-twitt.htmlReceived on Thu Jan 08 2009 - 16:17:59 CST