Can't solve the "Too many open files" error

UNIX Solaris
1

I keep getting "Too many open files" in /var/adm/messages that I do not know how to fix.

I've searched for solution on this forum and have increased the file descriptors limit but the problem persists.

The problem is happening in a solaris zone....

/var/adm/messages log example

May 10 12:31:08 app.server.com procname[3962]: [ID 702911 local0.error] [1] Cannot open destination 'xyz.tmp' [Too many open files]

plimit 3962
3962:   procname
   resource              current         maximum
  time(seconds)         unlimited       unlimited
  file(blocks)          unlimited       unlimited
  data(kbytes)          unlimited       unlimited
  stack(kbytes)         8192            unlimited
  coredump(blocks)      unlimited       unlimited
  nofiles(descriptors)  3072            3072
  vmemory(kbytes)       unlimited       unlimited

pfiles 3962
.....
265 entries

ulimit -n -H
3072

ulimit -n -S
3072

Running a shell script that goes through /proc and runs pfiles on all the processes on the zone... 
Shows that it opened 2746 files... well below the limit.

Running the same script on the global zones....
shows 7522 files open (I'm guessing all zones + global)

global....

ulimit -n -H
65536

ulimit -n -S
256

---------- Post updated at 03:16 PM ---------- Previous update was at 03:14 PM ----------

Another thread mentioned that maybe it's a "too many concurrent open files"...

global...

# getconf OPEN_MAX
256


the problematic zone...

# getconf OPEN_MAX
3072

global

# echo "maxusers/D" | adb -k
physmem 1fe90d
maxusers:
maxusers:       2048


and I cant seems to run this on the zone...
2

Perhaps the script in question could be reworked to not open so many concurrent files?

3

The point is that the script in question (with proc id: 3962)... opened only 265 files... well
below the limit that I set.. :frowning:

4

And if you can't get it working one way perhaps you can get it working another... Why does your script open so many simultaneous files?

5

It's a binary executable that I do not have insight on how it works... :frowning:

6

This is a non-global zone, correct? Are you using resource management in that zone? Maybe process.max-file-descriptor is set too low.

7

I'm not using the resource management but.. here's the output for prctl....

ran from global zone....

# prctl -n process.max-file-descriptor 27160
process: 27160: procname
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
process.max-file-descriptor
        privileged      3.07K       -   deny                                 -
        system          2.15G     max   deny                                 -

process 27160 is similar to the problematic process posted earlier.. running in a zone.

---------- Post updated at 07:29 PM ---------- Previous update was at 07:28 PM ----------

it is privileged to 3.07k... matches the ulimit reading right?

8

Is your application 32 bit or 64 bit file /proc/3962/object/a.out ?

What Solaris release are you using ( cat /etc/release ) and is your system up to date with patches ?

If your application is 32 bit, there is a limitation of 256 file descriptors usable through the stdio library. This could explain why you have the too many open files error with only 265 open files (9 of which that could have been open directly with open, not through stdio fopen)

Have a look here for details and a workaround:
https://blogs.oracle.com/stw/entry/breaking\_the\_256\_file_descriptor
Solaris OS Solutions to 32-Bit stdio's 256 File-Descriptors Limitation

2 Likes
9

That fixed the problem. Thank you jlliagre!!!!