Please help list/find files greater 1G move to different directory

dotran · April 1, 2014, 12:25pm

I have have 6 empty directory below. I would like write bash scipt if any files less "1000000000" bytes then move to "/export/home/mytmp/final" folder first and any files greater than "1000000000" bytes then move to final1, final2, final3, final4, final4, final5 and that depend see how many files, but I only want ONE file go into 1 directory. I run the command below with great than "1000000000" bytes and see 3 files then that 3 files should go 3 folders (final1, final2 and final3) only. If see more that 6 or 8 files greater than "1000000000" bytes then should go last folder "final5" . Is some kinda script loop through or command to do this? Please help with this task. Thanks

/export/home/mytmp/final  <= less "1000000000"
/export/home/mytmp/final1 <= greater "1000000000" but require only file move a
/export/home/mytmp/final2
/export/home/mytmp/final3
/export/home/mytmp/final4
/export/home/mytmp/final5
 
/export/home/mytmp/test1> ls -ltr test*.txt
-rwxrwxr-x   1 ca7prod  ftpusers 1073741824 Mar  4 11:14 test3.txt
-rwxrwxr-x   1 ca7prod  ftpusers 524288000 Mar  4 11:26 test4.txt
-rwxrwxr-x   1 ca7prod  ftpusers 629145600 Mar  4 11:28 test5.txt
-rwxrwxr-x   1 ca7prod  ftpusers 734003200 Mar  4 23:47 test7.txt
-rw-rw-r--   1 ca7prod  ftpusers      14 Mar 12 14:43 test.txt
-rw-------   1 ca7prod  ftpusers 52428800 Mar 31 15:24 test2.txt
-rw-rw-r--   1 ca7prod  ftpusers 104857600 Mar 31 15:27 test8.txt
-rw-rw-r--   1 ca7prod  ftpusers 1178599424 Mar 31 15:31 test9.txt
-rw-------   1 ca7prod  ftpusers 104857600 Mar 31 15:32 test10.txt
-rwxrwxr-x   1 ca7prod  ftpusers 104857600 Mar 31 15:37 test6.txt
-rw-rw-r--   1 ca7prod  ftpusers 157286400 Mar 31 15:38 test1.txt
-rw-rw-r--   1 ca7prod  ftpusers 1335885824 Mar 31 15:40 test11.txt
/export/home/mytmp/test1> ls -ltr test*.txt | awk '{if ($5 < 1000000000) print $9}' -exec mv {} /export/home/mytmp/final \;
test4.txt
test5.txt
test7.txt
test.txt
test2.txt
test8.txt
test10.txt
test6.txt
test1.txt
/export/home/mytmp/test1> ls -ltr test*.txt | awk '{if ($5 > 1000000000) print $9}' -exec mv {} /export/home/mytmp/xxx \;
test3.txt
test9.txt
test11.txt

Please with this script
/export/home/mytmp/test1> cat test.ksh

 
#!/bin/ksh
cd /export/home/mytmp/test1
###############################################################################
# Any files less than 1G move to final
###############################################################################
ls -ltr test*.txt | awk '{if ($5 < 1000000000) print $9}' -exec mv {} /export/home/mytmp/final \;
###############################################################################
# Any files greater than 1G move to final1, final2, final3, final4, final4, final5
###############################################################################
ls -ltr test*.txt | awk '{if ($5 > 1000000000) print $9}' |sed q | xxxxx bla bla
find /export/home/mytmp/test1 bla bla

rbatte1 · April 1, 2014, 1:22pm

have a further look at the options of the find command. There is one about size that should be very useful.

Have a go and let us know how you get on with the code you use (so other people can refer to it in future)

I hope that this helps,
Robin

dotran · April 1, 2014, 1:26pm

Can you give some example what's the find command to do this? Thanks

Don_Cragun · April 1, 2014, 1:56pm

I'm confused.

What do you want to do with files that contain exactly 1000000000 bytes?
If you have 6, 7, or 8 files that contain more than 1000000000 bytes, where should they go? (You have said that the 1st goes in final1; the 2nd goes in final2; the 3rd goes in final3; the 4th goes in final4; the 5th, the 9th, and any more files after that go in final5. But, you never specified a destination for the 6th, 7th, and 8th files that meet your criteria.)
You also said you want a bash script, but the first line of your sample code is #!/bin/ksh Do you really want a bash script, or do you want a Korn shell script?
What operating system and release are you using?

dotran · April 1, 2014, 2:15pm

What do you want to do with files that contain exactly 1000000000 bytes? Then move to final1, final2...etc. But only want 1 file go ino 1 directory.
If you have 6, 7, or 8 files that contain more than 1000000000 bytes, where should they go? (You have said that the 1st goes in final1; the 2nd goes in final2; the 3rd goes in final3; the 4th goes in final4; the 5th, the 9th, and any more files after that go in final5. But, you never specified a destination for the 6th, 7th, and 8th files that meet your criteria.) Let say I see 8 or 9 big files then last 3 or 4 files move to last folder "final5". Hope explain correct here. Thanks
You also said you want a bash script, but the first line of your sample code is #!/bin/ksh Do you really want a bash script, or do you want a Korn shell script? I mean use #!/usr/bin/bash or #!/bin/ksh and I don't know any different here.
What operating system and release are you using? SunOS Test1 5.10 Generic_147440-25 sun4v sparc SUNW,T5240

Don_Cragun · April 1, 2014, 6:26pm

You could try something like:

#!/bin/ksh
# cd /export/home/mytmp/test1
cld=0				# Current large directory
cutoff=${1:-1000000000}		# Min file size to be considered "big"
lt1G_d=/export/home/mytmp/final	# Directory to receive "small" files
ge1G_d_base="$lt1G_d"		# Base directory name to receive "big" files
ge1G_d_cnt=5			# Number of directories for "big" files

printf "Prepare to move text files smaller than %s bytes to directory:\n" \
	"$cutoff"
printf "\t%s\n" "$lt1G_d"
printf "and larger text files to directories:\n\t%s1 - %s%d\n" \
	"$ge1G_d_base" "$ge1G_d_base" "$ge1G_d_cnt"
ls -ltr *.txt | while read x x x x size x x x file
do	if [ "$size" -ge "$cutoff" ]
	then	echo mv -- "$file" \
			"$ge1G_d_base"$((cld = cld + 1 - (cld == ge1G_d_cnt)))
	else	echo mv -- "$file" "$lt1G_d"
	fi
done

If the output you get from this looks like the commands you want the script to execute, remove the echo in both places where it appears in the script and run it again.

dotran · April 1, 2014, 7:05pm

Thanks Mr Don C. It's really nice code and I still can't figure out why not move.

#!/bin/ksh
cd /export/home/mytmp/test1
cld=0 # Current large directory
cutoff=${1:-1000000000} # Min file size to be considered "big"
lt1G_d=/export/home/mytmp/final # Directory to receive "small" files
ge1G_d_base="$lt1G_d" # Base directory name to receive "big" files
ge1G_d_cnt=5 # Number of directories for "big" files
printf "Prepare to move text files smaller than %s bytes to directory:\n" \
"$cutoff"
printf "\t%s\n" "$lt1G_d"
printf "and larger text files to directories:\n\t%s1 - %s%d\n" \
"$ge1G_d_base" "$ge1G_d_base" "$ge1G_d_cnt"
ls -ltr test*.txt | while read x x x x size x x x file
do if [ "$size" -ge "$cutoff" ]
then echo mv "$file" \
"$ge1G_d_base"$((cld = cld + 1 - (cld == ge1G_d_cnt)))
else echo mv "$file" "$lt1G_d"
fi
done

I run the code but somehow go to folder final, final1, final2....etc and didn't see the files move at all. I am missing something here. Could please help again? Thanks

---------- Post updated at 06:05 PM ---------- Previous update was at 06:00 PM ----------

Wow....sorry Mr Don C. It's really worked. I didn't remove echo and run the code work like a charm. Thanks very much....for your script expert.