One of our mountpoint shows 100% but we have less data on that mountpoint. Pls help me to find which data/process holds the space.
bash-3.00$ cd /oracle/server_software/oracle10
bash-3.00$ du -sh *
0K admin
260M app
0K flash_recovery_area
0K lost+found
0K oradata
0K oraInventory
bash-3.00$ df -h .
Filesystem size used avail capacity Mounted on
/dev/dsk/c0d1s6 4.9G 4.9G 1.2M 100% /oracle/server_software/oracle10
A process can open a file, then delete it right away. The file continues to exist on disk, but no other process can see it
Try the pfiles command used with fuser:
# any process with a file open ( that does not show normally) will have to be visible here
fuser /oracle/server_software/oracle10
# for all of the pids that fuser reveals try:
pfiles [pid]
Then you can do something about stopping the process that filled the disk.
My server OS is Solaris 10, i do not have options -Dv. Hence i ran the fuser -c
# fuser -c /oracle/server_software/oracle10
/oracle/server_software/oracle10: 12949ctom 12666ctom 12470ctom 6596ctom 26743ctom 6338ctom 10372ctom 23683ctom 26807ctom 8949ctom 8947ctom 8945ctom 8943ctom 8941ctom 8939ctom 8937ctom 8935ctom 8933ctom 8931ctom 8929ctom 8923ctom 8921ctom 8919ctom 8917ctom 8915ctom 8913ctom 8911ctom 8909ctom 8907ctom 8905ctom 8903ctom 8901ctom 8899ctom 8897ctom 8895ctom 8893ctom 8891ctom 8889ctom 8887ctom 8885ctom 8883ctom 8881ctom 8879ctom 8877ctom 8875ctom 8873ctom 8871ctom 8869ctom 8867ctom 8865ctom 8863ctom 8861ctom 8859ctom 8857ctom 8855ctom 8853ctom 8851ctom 8849ctom 8847ctom 8845ctom 8843ctom 8841ctom 8839ctom 8837ctom 8835ctom 8833ctom 8831ctom 8829ctom 8827ctom 8825ctom 8823ctom 8821ctom 8819ctom 8817ctom 8815ctom 8813ctom 8811ctom 8809ctom 8807ctom 8805ctom 8803ctom 8801ctom 8799ctom 8797ctom 8795ctom 8793ctom 8791ctom 8789ctom 8787ctom 8785ctom 8783ctom 8781ctom 8779ctom 8775ctom 8710ctom 720m 26450ctom 28155ctom 445ctom 443ctom 441ctom 439ctom 437ctom 435ctom 433ctom 431ctom 429ctom 427ctom 425ctom 423ctom 421ctom 417ctom 415ctom 413ctom 411ctom 409ctom 406ctom 404ctom 402ctom 400ctom 396ctom 391ctom 389ctom 385ctom 383ctom 381ctom 379ctom 377ctom 375ctom 373ctom 371ctom 369ctom 367ctom 365ctom 363ctom 361ctom 359ctom 356ctom 354ctom 352ctom 350ctom 348ctom 346ctom 344ctom 342ctom 340ctom 338ctom 336ctom 334ctom 332ctom 330ctom 328ctom 326ctom 324ctom 322ctom 320ctom 318ctom 316ctom 314ctom 312ctom 310ctom 308ctom 303ctom 301ctom 299ctom 297ctom 294ctom 292ctom 290ctom 288ctom 286ctom 282ctom 158ctom 15287ctom 15277ctom 3714ctom 3712ctom 3709ctom 3707ctom 3705ctom 3703ctom 3701ctom 3699ctom 3697ctom 3695ctom 3693ctom 3691ctom 3689ctom 3687ctom 3685ctom 3683ctom 3681ctom 3679ctom 1599ctom 953ctom 949ctom 947ctom 943ctom 941ctom 939ctom 937ctom 935ctom 933ctom 931ctom 929ctom 927ctom 925ctom 921ctom 919ctom 917ctom 915ctom 913ctom 911ctom 680tom
Have you been deleting files in this partition (e.g. Oracle logs, Oracle database) ? These symptoms usually mean that files have been deleted while they are open by programs. In the end you will probably end up closing all applications which have files open on that partition.
Idea for a script to first list the detail of the processes which have the files open. Untested on Solaris, but tested on another unix. Substitute pfiles for ps when tested.
fuser -c /oracle/server_software/oracle10 2>&1 | tr -cd '[0-9] \n'|tr -s ' '| \
tr ' ' '\n'|sed -e "/^$/d" |sort -n | uniq | while read PID
do
ps -fp${PID} | grep -v "UID"
done
@methyl Your script would probably work but is quite convoluted.
@rock123 Here are simpler ways:
ps -fp $(echo $(fuser -c /oracle/server_software/oracle10 2>/dev/null) | tr ' ' ',')
and even:
pfiles $(fuser -c /oracle/server_software/oracle10 2>/dev/null)
Thanks jlliagre for the feedback. Its a modified extract of my script which has evolved over the years for variants of fuser and still works on O/S with limited command line length.
Your first script needs adjusting to be portable because there are multiple space characters between the PIDs in most editions of fuser and ps -fp coughs on the empty parameter.
I indeed only tested mine on Solaris and Gnu/Linux where it works fine. Too bad you still use OSes with broken fuser implementations (I think that was some Dynix if I recall our last conversation about it ...)
It actually doesn't need any adjustments. I used the echo command to remove these extra space characters.