Delete files older than certain days - Wrong !!!

kamathg · July 21, 2008, 9:17am

Guys,

I had raised a question about deleting files older than today in a specific directory and i got this as an answer

find ${ARCH_DEST}/*.gz -mtime +0 -exec rm -f {} \;

What happens when there aren't files that meet this criteria ? Can it delete any other directories ? I had a shocking output of the script yesterday as it looks to me that it has deleted quite a number of directories.

Could it be this command that generated this output ?
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/backup': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/files': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/files/lib': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/files/lib/libn10.a': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/files/lib32': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/files/lib32/libn10.a': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/etc': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/etc/config': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/original_patch/etc/xml': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/rac': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/scratch': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/files': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/files/lib': Is a directory
rm: cannot remove `product/10.2.0/.patch_storage/6397938_Sep_21_2007_07_19_39/files/lib/libn10.a': Is a directory

It removed quite a lot of files from a number of directories. It has worked before. Could it be this script ? I've made sure that the script cd's to the right directory

Thanks

Franklin52 · July 21, 2008, 10:14am

Add the "-type f" option to exclude directories:

find ${ARCH_DEST}/*.gz -mtime +0 -type f -exec rm -f {} \;

You should try it first with an ls command before using the rm command:

find ${ARCH_DEST}/*.gz -mtime +0 -type f -exec ls -l {} \;

Regards

kamathg · July 21, 2008, 11:55am

Thank you Franklin.
The ARCH_DEST is obtained from within a Oracle database, but accidentally the database down when this happened. Could it be that it ran the command for all the directories ?

Also, if no .gz files are available, the the return code is non-zero. basically, i am trying to see if the deletion .gz is successful and if it fails, report it as an error.

Something like this

Delete_Archive() {

LogDest="select value from v\$parameter where name='standby_archive_dest';"
runSql "${LogDest}"
ARCH_DEST="${COUNT}"
cd ${ARCH_DEST}

echo `pwd`

#find ${ARCH_DEST}/*.gz -mtime +0 -exec rm -f {} \;
rm *.gz

RC2=$?

if [[ $RC2 != 0 ]]
then
Mail_Subject="Dataguard on host `hostname`. Delete archive logs older than 24 hrs failed. Please investigate (SEV-4)"
ls -ltr | ${ARCH_DEST} > "${ORACLE_BASE}/oralogs/DELETE_ARCHIVE.log"
Mail_User "$Mail_Subject" "${ORACLE_BASE}/oralogs/DELETE_ARCHIVE.log"
exit 1
fi
}

Can this be coded better ? Can i put a condition saying if the number of files deleted is 0, then it's not an error ?

Thanks

Franklin52 · July 21, 2008, 1:06pm

echo `pwd` => pwd is sufficient

#find ${ARCH_DEST}/*.gz -mtime +0 -exec rm -f {} \;
rm *.gz => this deletes all .gz files!!

RC2=$? => now it gives the returncode of the rm command not the returncode of the find command

Replace the above code with:

pwd

find ${ARCH_DEST}/*.gz -mtime +0 -exec ls -l {} \; # Use ls -l first to be sure you get the expected files
# find ${ARCH_DEST}/*.gz -mtime +0 -exec rm -f {} \;
RC2=$?

Regards

sudhamacs · July 21, 2008, 1:36pm

put an if condition to check that ur sql returned the results successfully, otherwise the find command reads all the *.gz files from all the subdirectories and deletes.

methyl · July 22, 2008, 10:37am

sudhamacs and franklin52 make good points.

Test with "ls" first.
Check that ARCH_DEST is not blank.
Use "-type f" to only find files.

Here's another version with some more points:
This version:

has no limit on the number of filenames.
does nothing if there are no files to delete (and does not error).
works if there are spaces in the filename or directory name.
"escapes" the "*" and the "." in the "-name" parameter to avoid false matches.

#!/bin/ksh

if [ -d "${ARCH_DEST}" ]
then
find "${ARCH_DEST}/" -name \*\.gz -type f -mtime +0 -print | while read FILENAME
do
ls -ald "${FILENAME}"
# rm "${FILENAME}"
done
else
echo "ARCH_DEST incorrect or missing: ${ARCH_DEST}"
fi

BMDan · July 22, 2008, 11:59am

When testing with ls, remember to pass "-d", or you'll get strange results if you do encounter a directory somehow.

xtreemnet · August 19, 2008, 10:59pm

Hi

I am trying to move older files from adirectory to a archived folder. and I get the error "sh: /usr/bin/find: 0403-027 The parameter list is too long.". The following commnads is being used:

find /dir1/dir2/dir3/dir4/dir5/* -prune -type f -mtime +2 | xargs -l50 -i mv -f {} /dir1/dir2/dir3/dir4/dir6/

The source folder has around 6000 files. Approximately 2000 files per day. We would like to keep 2 days old files only.

Is there any other way this can be done?

Thanks,

Annihilannic · August 19, 2008, 11:32pm

Try:

find /dir1/dir2/dir3/dir4/dir5 -type f -prune -mtime +2 | xargs -l50 -i mv -f {} /dir1/dir2/dir3/dir4/dir6/

There's no need to specify '*' when find is quite capable of locating the files itself.

xtreemnet · August 20, 2008, 5:37pm

The error has disappeared but this command is picking up files from the subdirectories under dir5 as well. I want to archive only the files from currnet directory i.e. dir5.
I have made a slight change to the command. The target directory happens to be a subdirectory under the current directory.
How do I mdofiy this to pick up files from current directory only.
find /dir1/dir2/dir3/dir4/dir5 -type f -prune -mtime +2 | xargs -l50 -i mv -f {} /dir1/dir2/dir3/dir4/dir5/dir6

Annihilannic · August 20, 2008, 9:33pm

Sorry, should have realised that.

Try this:

find /dir1/dir2/dir3/dir4/dir5 -type d -a ! -name dir5 -prune -o -type f -mtime +2 | ...

xtreemnet · August 20, 2008, 10:31pm

Thanks for the quick response. Unfortunately this also did not work. It listed the subdirectories only.

Annihilannic · August 20, 2008, 10:46pm

I'll get there in the end.

find /dir1/dir2/dir3/dir4/dir5 -type d -a ! -name dir5 -prune -o -type f -mtime +2 -print | ...

xtreemnet · August 20, 2008, 10:51pm

This one did not list anything.

Annihilannic · August 20, 2008, 10:56pm

That one definitely works for me, both on HP-UX and Linux. What operating system are you using?

xtreemnet · August 20, 2008, 11:54pm

the command 'uname -a -v' returned:

AIX myserveraix4 3 5 00C7D5FF4C00

Annihilannic · August 21, 2008, 12:52am

Works for me on AIX too (although the only system I have access to is AIX 6.1 (yours is 5.3). Are you sure you copied the command correctly and replaced both instances of dir5 with the real directory name, in the second case without the path?

Annihilannic · August 21, 2008, 12:57am

e.g.

$ mkdir -p /tmp/testdir/subdir
$ touch 01010000 /tmp/testdir/oldfile /tmp/testdir/subdir/oldfile
$ touch /tmp/testdir/newfile /tmp/testdir/subdir/newfile
$ find /tmp/testdir | xargs ls -ld
drwxr-xr-x    3 username staff           256 Aug 21 04:54 /tmp/testdir
-rw-r--r--    1 username staff             0 Aug 21 04:54 /tmp/testdir/newfile
-rw-r--r--    1 username staff             0 Jan 01 2008  /tmp/testdir/oldfile
drwxr-xr-x    2 username staff           256 Aug 21 04:54 /tmp/testdir/subdir
-rw-r--r--    1 username staff             0 Aug 21 04:54 /tmp/testdir/subdir/newfile
-rw-r--r--    1 username staff             0 Jan 01 2008  /tmp/testdir/subdir/oldfile
$ find /tmp/testdir -type d -a ! -name testdir -prune -o -type f -mtime +2 -print | xargs ls -ld
-rw-r--r--    1 username staff             0 Jan 01 2008  /tmp/testdir/oldfile
$

xtreemnet · August 21, 2008, 6:53pm

The directory I am trying is named as _PROD_. Do you think the underscore would be the culprit?

I have tested this on another directory without underscore characters and it works fine. Is there a way get around this?

Annihilannic · August 25, 2008, 8:14pm

I repeated the above test using _PROD_ as the directory name and it still seems to behave as expected:

$ ksh -x /tmp/t
+ mkdir -p /tmp/_PROD_/subdir
+ touch 01010000 /tmp/_PROD_/oldfile /tmp/_PROD_/subdir/oldfile
+ touch /tmp/_PROD_/newfile /tmp/_PROD_/subdir/newfile
+ xargs ls -ld
+ find /tmp/_PROD_
drwxr-xr-x    3 username staff           256 Aug 26 00:11 /tmp/_PROD_
-rw-r--r--    1 username staff             0 Aug 26 00:11 /tmp/_PROD_/newfile
-rw-r--r--    1 username staff             0 Jan 01 2008  /tmp/_PROD_/oldfile
drwxr-xr-x    2 username staff           256 Aug 26 00:11 /tmp/_PROD_/subdir
-rw-r--r--    1 username staff             0 Aug 26 00:11 /tmp/_PROD_/subdir/newfile
-rw-r--r--    1 username staff             0 Jan 01 2008  /tmp/_PROD_/subdir/oldfile
+ xargs ls -ld
+ find /tmp/_PROD_ -type d -a ! -name _PROD_ -prune -o -type f -mtime +2 -print
-rw-r--r--    1 username staff             0 Jan 01 2008  /tmp/_PROD_/oldfile
$