Unable to list files in a directory

UNIX Solaris
1

Hi have a system running solaris with a mount point running 58% capacity used, but unfortunately when I type

ls -lrt

in a specific directory, it returns:

ls: Not enough space

. I suspect there are millions of files in this directory.
So what I did is to create a script like this:

touch -mt 201503040000 /var/tmp/ref1
touch -mt 201503092359 /var/tmp/ref2
find /directory/subdirectory -type f \ ( -newer /var/tmp/ref1 -a ! -newer /var/tmp/ref2 \) -exec rm {} \;

The script apparently runs, because does not return any error, but the free space on that subdir does not increase, or nothing appears to change.

Any help will be appreciated

2

Check your free inodes and also your swap space...

3

Not enough space to sort the results, probably. find, as you've discovered, can cope because it doesn't sort at all.

Try -exec echo instead of -exec rm to see if any files are matching at all.

4

List the files with

find . -ls
5

Try

df -g mountpoint

df -g /local
/local             (/local            ):         8192 block size          1024 frag size
528718676 total blocks  242831070 free blocks 237543884 available       31810688 total files
31649815 free files     30933022 filesys id
     ufs fstype       0x00000004 flag             255 filename length

You want to see how many files are on the filesystem and how many free inodes (file slots). This bypasses all directories. Your directory is either "broken" which means it is fsck time, or your directory has waaay too many entries.

6

Hi

Thank you for the replies, the

df -g /mount point

, produce me the follwoing results

 df -g /opt/optima
/opt/optima        (/dev/dsk/c0t0d0s4 ):         8192 block size          1024 frag size
174504966 total blocks   74857050 free blocks 73112002 available       10500672 total files
  503155 free files      8388620 filesys id
     ufs fstype       0x00000004 flag             255 filename length

the

find. ls

does list the files slowly.... and they are

csv

extension.
They want me to delete all march files...

swap space:

 swap -s
total: 256344k bytes allocated + 34752k reserved = 291096k used, 12758600k available

and

 swap -l
swapfile             dev  swaplo blocks   free
/dev/dsk/c0t0d0s1   32,9      16 20975120 20975120
7

A ufs filesystem becomes exponentially slow with the number of files per directory.
As a rule of thumb, do not have more than 10000 files per directory!
If this is Solaris 10, zfs would be the better alternative for such a work load.
Your method works, and can be augmented with a -ls to show what's being deleted.
Terminate the -exec rm with a + , that reduces the numbers of forks, i.e. is more efficient. In effect it does
rm file1 file2 file3 ... rather than rm file1; rm file2; rm file3; ... .
rm might prompt in certain conditions when it sees a terminal. rm -f prevents it.

find /directory/subdirectory -type f \( -newer /var/tmp/ref1 -a ! -newer /var/tmp/ref2 \) -ls -exec rm -f {} +
8

my script does not seem to be working, because the number of files continues the same, but, I found that when running:

 df -F ufs -o i /opt/optima
Filesystem             iused   ifree  %iused  Mounted on
/dev/dsk/c0t0d0s4   10000493  500179    95%   /opt/optima

and

 df -h /opt/optima
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0t0d0s4       83G    47G    35G    58%    /opt/optima

so clearly there is a problem here, please how can I go about delete this files, please help

---------- Post updated at 04:53 PM ---------- Previous update was at 02:38 PM ----------

Please, help... my script does not seem to be working...:

touch -mt 201501010000 /var/tmp/ref3
touch -mt 201503312359 /var/tmp/ref4
find /opt/optima/Interfaces/ERICSSON/ALL/ASN/12/parser/out/UNKNOWN -type f \( -newer /var/tmp/ref3 -a ! -newer /var/tmp/ref4 \) -ls -exec rm -f {} +

and:

df -F ufs -o i /opt/optima
Filesystem             iused   ifree  %iused  Mounted on
/dev/dsk/c0t0d0s4   10004339  496333    95%   /opt/optima

Any alternatives to this?

9

It works here. If it doesn't work for you, something's either wrong with your reference files, or your files themselves not matching the date range you think they ought.

Check your reference files, and also check that the files you want to delete are in the date range you want.

You can do that with find -ls as suggested by others already.

10

I have used the

find . ls

code to list the files, so it was showing files, so I did a

ls -lrt

of one files shown, and I could see the timestamp of Feb, hence my script that with includes all files from the month of february.
I beleive the problem lies because, as my script is deleting other systems are ftp files to that particular directory

11

Do you know what ls -lrt means?

-r: sort reverse
-t: sort by time

So you are sorting a list of one file backwards by time...

find . -ls should show dates, too, it does here, what does it show for you?

12

I get the impression that the repeated

df -F ufs -o i /opt/optima

shows that your ifree still shrinks (and iused still raises).
If that's the case then there could be a run-away process that is non-stop creating new files.
Here comes a method to find such a run-away process.
Search for all files created within the last 24 hours:

find /opt/optima -type f -mtime -1 -ls

Are there very many similar files with a common owner (column 5)?
List the most busy processes (top 20):

prstat 0 1 | head -20

Is there a process with the same owner (column 2)?

13

I beleive all the files have the same owner:

 find . -type f -mtime -1 -ls
9996072    1 -rw-r--r--   1 optima   sys           492 Apr  1 15:41 ./C20150401_1300_20150401_1400_MSC4_OPTIMA3_201504011300_2830_8_UNKNOWN_NCELHNDTOT__20150401162227332.csv
9990184    1 -rw-r--r--   1 optima   sys           446 Apr  1 11:32 ./C20150401_0900_20150401_1000_MSC4_OPTIMA2_201504010900_2826_tmp_10_UNKNOWN_SYS7IND__20150401113207285.csv
9993583    1 -rw-r--r--   1 optima   sys           453 Apr  1 14:15 ./C20150401_1200_20150401_1300_MSC4_OPTIMA2_201504011200_2829_3_UNKNOWN_SYS7IND__20150401151923600.csv
9993598    1 -rw-r--r--   1 optima   sys           453 Apr  1 14:15 ./C20150401_1200_20150401_1300_MSC4_OPTIMA2_201504011200_2829_8_UNKNOWN_SYS7IND__20150401152011337.csv

and the most busiest processes are not owned by sys:

 prstat 0 1 | head -20
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
 23415 root     2448K  672K cpu16   60    0   5:44:06 2.9% find/1
 29187 root     1616K 1512K cpu28   60    0   0:01:32 2.9% find/1
 29188 root     3360K 3256K cpu1    49    0   0:00:00 0.0% prstat/1
  1380 noaccess  226M  119M sleep   59    0   0:03:59 0.0% java/55
   384 root       11M 3976K sleep   59    0   0:00:00 0.0% smbd/1
   453 root     2096K   32K sleep   59    0   0:00:00 0.0% smcboot/1
   408 daemon   2464K   32K sleep   60  -20   0:00:00 0.0% lockd/2
   456 root     2576K  584K sleep   59    0   0:00:00 0.0% ttymon/1
   455 root     2096K   32K sleep   59    0   0:00:00 0.0% smcboot/1
   390 daemon   2872K   56K sleep   59    0   0:00:00 0.0% rpcbind/1
   154 root     2256K   16K sleep   59    0   0:00:00 0.0% drd/2
 29143 optima   1400K 1352K sleep   53    2   0:00:00 0.0% sh/1
   202 root     1448K    8K sleep   59    0   0:00:00 0.0% efdaemon/1
   114 root     2648K   32K sleep   59    0   0:00:00 0.0% syseventd/14
   380 root       11M 5296K sleep   59    0   0:00:00 0.0% smbd/1
Total: 299 processes, 525 lwps, load averages: 1.87, 1.34, 1.17

I beleive some other machine or source is sending files to that directory all the time. But I do not have the means to stop this

---------- Post updated at 09:36 PM ---------- Previous update was at 09:33 PM ----------

 find . -ls
 6862 992872 drwxr-xr-x   2 optima   sys      1016185856 Apr  1 21:06 .
9996072    1 -rw-r--r--   1 optima   sys           492 Apr  1 15:41 ./C20150401_1300_20150401_1400_MSC4_OPTIMA3_201504011300_2830_8_UNKNOWN_NCELHNDTOT__20150401162227332.csv
9990184    1 -rw-r--r--   1 optima   sys           446 Apr  1 11:32 ./C20150401_0900_20150401_1000_MSC4_OPTIMA2_201504010900_2826_tmp_10_UNKNOWN_SYS7IND__20150401113207285.csv
675490    1 -rw-r--r--   1 optima   sys           630 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_36_UNKNOWN_ID1__20150211214840896.csv
861214    1 -rw-r--r--   1 optima   sys           824 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_37_UNKNOWN_NBSFRV1PTOT__20150211214849663.csv
611404    1 -rw-r--r--   1 optima   sys           824 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_38_UNKNOWN_NBSFRV1PTOT__20150211214858517.csv
480538    1 -rw-r--r--   1 optima   sys           824 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_39_UNKNOWN_NBSFRV1PTOT__20150211214907389.csv
715105    1 -rw-r--r--   1 optima   sys           443 Feb 11 21:29 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_3_UNKNOWN_SYS7IND__20150211214916296.csv
89334    1 -rw-r--r--   1 optima   sys           824 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_40_UNKNOWN_NBSFRV1PTOT__20150211214925196.csv
79527    1 -rw-r--r--   1 optima   sys           521 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_41_UNKNOWN_NISDNCALL__20150211214934043.csv
89481    1 -rw-r--r--   1 optima   sys           521 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_42_UNKNOWN_NISDNCALL__20150211215323905.csv
611339    1 -rw-r--r--   1 optima   sys           521 Feb 11 21:33 ./C20150211_1100_20150211_1200_HLR1_OPTIMA2_201502111100_945_43_UNKNOWN_NISDNCALL__20150211215332823.csv
14

The owner is optima (not sys).
So the only candidate on your system is pid 29143

ptree 29143

shows the relevant process tree.
If this is NFS-exported then it could be sent from another system.

showmount

might show the NFS clients.
Or - I see smbd - CIFS-shared.

smbstatus

might show the CIFS clients.