Re: OEL - fork: Resource temporarily unavailable

From: Subodh Deshpande <deshpande.subodh_at_gmail.com>
Date: Thu, 22 Sep 2011 09:29:01 +0530
Message-ID: <CAJsOtB6QiznTx+bA-kWFzzfN+K08DSFcZvM4gc70owQx5xXA-g_at_mail.gmail.com>

check space where archives are written.. On 22 September 2011 09:02, Upendra N <nupendra_at_hotmail.com> wrote:

> We have a 2 node Oracle RAC clusters, both are running OEL 5.6 and Oracle
> 11g R2.
>
> ::::::::::::::
> /etc/enterprise-release
> ::::::::::::::
> Enterprise Linux Enterprise Linux Server release 5.6 (Carthage)
> ::::::::::::::
> /etc/oracle-release
> ::::::::::::::
> Oracle Linux Server release 5.6
> ::::::::::::::
> /etc/redhat-release
> ::::::::::::::
> Red Hat Enterprise Linux Server release 5.6 (Tikanga)
>
> From one of the node I am seeing the following error consistently..
>
> -bash: fork: Resource temporarily unavailable
>
> (22:55:17) root_at_proddb1: /var/tmp # ulimit -a
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 1056768
> max locked memory (kbytes, -l) 32
> max memory size (kbytes, -m) unlimited
> open files (-n) 100000
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 10240
> cpu time (seconds, -t) unlimited
> max user processes (-u) 1056768
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
>
>
>
> proddb1 is where the problem exists..
>
>
>
> Number of database connections between the boxes look fine:
>
> proddb1# ps -ef|grep -ic local=no
> 3800
>
>
> proddb2# ps -ef|grep -ic local=no
>
> 4500
>
>
>
> proddb1# ps -eLf|wc -l
>
> 32500
>
>
>
> proddb2# ps -eLf|wc -l
>
>
> 6500
>
>
>
>
> proddb1# strace lsof -o /tmp/lsof.out <--- Produces the following
> output
> .....
>
> .....
>
> close(99982) = -1 EBADF (Bad file descriptor)
> close(99983) = -1 EBADF (Bad file descriptor)
> close(99984) = -1 EBADF (Bad file descriptor)
> close(99985) = -1 EBADF (Bad file descriptor)
> close(99986) = -1 EBADF (Bad file descriptor)
> close(99987) = -1 EBADF (Bad file descriptor)
> close(99988) = -1 EBADF (Bad file descriptor)
> close(99989) = -1 EBADF (Bad file descriptor)
> close(99990) = -1 EBADF (Bad file descriptor)
> close(99991) = -1 EBADF (Bad file descriptor)
> close(99992) = -1 EBADF (Bad file descriptor)
> close(99993) = -1 EBADF (Bad file descriptor)
> close(99994) = -1 EBADF (Bad file descriptor)
> close(99995) = -1 EBADF (Bad file descriptor)
> close(99996) = -1 EBADF (Bad file descriptor)
> close(99997) = -1 EBADF (Bad file descriptor)
> close(99998) = -1 EBADF (Bad file descriptor)
> close(99999) = -1 EBADF (Bad file descriptor)
> open("/dev/null", O_RDWR) = 3
> close(3) = 0
> umask(0) = 022
> open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_sizeV442544, ...}) = 0
> mmap(NULL, 56442544, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b9c118e9000
> close(3) = 0
> getpid() = 760
> getgid() = 0
> getegid() = 0
> geteuid() = 0
> getuid() = 0
> stat("/dev", {st_mode=S_IFDIR|0755, st_sizea80, ...}) = 0
> open("/", O_RDONLY) = 3
> lseek(3, 1, SEEK_SET) = 1
> lstat("/proc/760/fd/3", {st_mode=S_IFLNK|0500, st_sized, ...}) = 0
> close(3) = 0
> open("/proc/mounts", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x2b9c14ebd000
> read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 4096) = 1228
> pipe([4, 5]) = 0
> pipe([6, 7]) = 0
> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
> child_tidptr=0x2b9c118e8670) = -1 EAGAIN (Resource temporarily unavailable)
> write(2, "lsof: can't fork: Resource tempo"..., 51) = 51
> exit_group(1) = ?
>
>
> Any help is appreciated.
>
> -Upendra
>
>
> --
> http://www.freelists.org/webpage/oracle-l
>
>
>

-- 
==============================
DO NOT FORGET TO SMILE TODAY
==============================


--
http://www.freelists.org/webpage/oracle-l

Received on Wed Sep 21 2011 - 22:59:01 CDT