Presently I'm using a 3pp that uses fopen to open files and I encounter this problem of too many files open when it is trying to open a file. My application is done in java which interfaces with this 3pp. When I instantiate this 3pp it loads up some files but it is pops up the error saying that there are too many files open. However, right after that i try opening a file in my own application and it has no problems. Some of the questions that I have is:
1) Is there a way to know what files are open on a certain pid?
2) What exactly does pfiles pid show? Does it show the number of files that are open for tht particular pid?
3) file descriptors are used when files are opened but are there any other cases when file descriptors are used? (i.e. corba calls, commands such as ping, etc)
4) is there any way to increase a soft limit? I've already increased the hard limit to over 32k.
5) could the result of a truss with many EBADF represent anything related to this problem?
I answer this with some trepidation since I have no idea what a 3pp is.
2 Like lsof, it shows the file descriptors in use. That is not the same thing as the number of files open.
ping is a program and it uses file descriptors. It would not be reasonable to write a program that uses ping in such a way as consume it's own file descriptors, but unreasonable programmers exist. And there are many other ways to allocate a file descriptor. dup()/fcntl() may rival or even surpass open()/creat(). Network connections use file descriptors. lsof will tell you what each file descriptor is doing. It really is the tool of choice.
setrlimit() When a program needs a lot of fd's but didn't call setrlimit, it must be considered "broken". Most likely it is leaking fd's.
Well I guess that even if the truss was performed years ago on another system and on an unrelated process, it is possible that the output of the truss does represent something that is related to this problem.
What is the default limit for the user in question? Use ulimit -n to see.
You can change the soft limit for the user by adding ulimit -n <some number> to the user's .profile or the script that is starting your process. You will not be able to set the limit above the system hard limit.
You are getting two different things mixed up. The max of a soft limit is the hard limit. On SunOS 5.8 I just observed that, for file descriptors, my hard limit was 1024 and my soft limit was 256. I raised my soft limit to 512. I am using ksh which has a builtin ulimit. Switch to ksh and try it:
ulimit -n
ulimit -Hn
ulimit -Sn 512
ulimit -n
At this point I can open 512 files in a single process. But I'll admit that I've never tried to write such a program.
The next thing is how many files can stdio manage. I must admit that I was skeptical about the information in the link you provided. But "man stdio" not only backs it up, but points out that things are a tad worse...
That means that cranking the fd's up to 512 may not help much. And you may not reach 255 fopened files because a few fd's were burned for other things.
HP's docs claim that you can fopen() as many files as you have fd's. But I have never verified that.
It's hard to answer your question about "easily" reaching 255. I have never come close to that. I continue to suspect that your "3pp" is leaking fd's. Since you seem to have support, I would follow up with the vendor.
lsof works fine for me on SunOS 5.8. "I've tried lsof but it does not seem to work." is not much to go on. The best I can do with that is say "you must be doing something wrong." If you were to post a few more details, maybe someone can help.
You can increase your soft limit higher than 255 but you have to tweak it a little. This applies for solaris-intel, dunno for sparc or other unix specifically.
You open the file. The fd will be 0-255. Then, you have to re-open it and asking for a fd > 255. I dont know the exact functions though, search a bit but it can be done.