Selecting files between a user inputed date range

Shell Programming and Scripting
1

Hi all!

I'm working on a KSH script to select files between a user inputed date range (stored in a variable) and then move them and unzip them. I'm stuck at how to get the files between the user inputed date range selected. Any help would be greatly appreciated!

The files are as such:

1112858279 May 2 08:00 20080502_0800.zip
2226093319 Jul 1 15:16 20080502_1230.zip
2362831536 May 2 17:28 20080502_1700.zip
418436718 May 3 08:00 20080503_0800.zip

# User Prompting for email, start date and end date
print " Welcome to the MRR User Recovery Interface"
print ""
print ""
print " Please enter the email address to search for:"
read user
print ""
print " Enter start date to search (Format: YYYY/MM/DD):"
read sdate
print ""
print "Enter end date of search (Format: YYYY/MM/DD):"
read edate
print ""
print "I will now search for the email from $user"
print "starting at $sdate and ending at $edate."
print ""
print ""
print "${DATE} Search context email $user" >> logs/recovery.log
print "${DATE} Starting date $sdate" >> logs/recovery.log
print "${DATE} Ending date $edate" >> logs/recovery.log
# Eventually there will be some stuff here to sort the zip files by the sdate and edate and then copy them off to a work directory.
#sorting the files by sdate and edate
# find . -xdev -type f -newer /tmp/start -a ! -newer /tmp/finish -exec mv {} /newdir \;
 
# grab a list of the zip files. 
print "Now Getting a file list..."
print ""
# The following needs to be changed when putting into production. These file paths
# will *NOT* work. Make sure they are correct! You've been warned.
cd /nmltest/cbpnear/data/mrr/mrr
ls *.zip > /nmltest/cbpnear/data/mrr/ziplist
print "${DATE} ziplist created" >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
 
# Unzipping the files
# To do this, we need to feed the file into an array otherwise ksh will choke.
print "Now unzipping files into the recovery directory..."
print ""
    for zip in *.zip
    do
        dir=`echo $zip | sed 's/.\{4\}$//'`
        unzip -o -qq -d recovery/$dir $zip >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
    done
# moving old usermail.log to something else...
print "Creating a log of the user's emails..."
print ""
mv /nmltest/cbpnear/data/mrr/scripts/logs/usermail.log /nmltest/cbpnear/data/mrr/scripts/logs/$user-$DATE2.log
# Let's grep through the dir's to find our user...
print "Now searching for ${user}."
print ""
print "${DATE} Started search for ${user} in unzipped directories" >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
cd /nmltest/cbpnear/data/mrr/mrr/recovery/ >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
find . -type f -print |xargs grep -l $user >> /nmltest/cbpnear/data/mrr/scripts/logs/user.log
# Just going to remove the first two characters of the grep results and move them to a new file.
cat /nmltest/cbpnear/data/mrr/scripts/logs/user.log | sed 's/^..//' >> /nmltest/cbpnear/data/mrr/scripts/logs/usermail.log
rm /nmltest/cbpnear/data/mrr/scripts/logs/user.log >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
# let's move the files to the reprocessing directory.
print "${DATE} Moving files to the recovery folder" >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
print "Now moving the email messages to the recovery directory"
print ""
rm -rf /nmltest/cbpnear/data/mrr/mrr/reprocess/* >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
cd /nmltest/cbpnear/data/mrr/mrr/recovery/ >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
cat /nmltest/cbpnear/data/mrr/scripts/logs/usermail.log | while read line; do cp /nmltest/cbpnear/data/mrr/mrr/recovery/$line ../reprocess/. ; done >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
print "${DATE} Recovery of ${user} is complete." >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
2

You can filter the files out between the given dates with something like:

output_of_files | awk -F" |_" '$5 >= '$sdate' && $5 <= '$edate

If you get errors use nawk, gawk or /usr/xpg4/bin/awk on Solaris.

Regards

3

Franklin

Thanks for the help. Will this select on the system date/time stamp or the file name as it is in a YYMMDD format?

Thanks!

4

Sorry, you're asking for a date of this format YYYY/MM/DD in your script so first we have to remove the slashes.
The script select the timestamp from the lines. We use 2 field separators, a space or a underscore so the date is the bold portion (the 5th field) and we compare it with the 2 dates on the last line:

1112858279 May 2 08:00 20080502_0800.zip

output_of_files |
awk -F" |_" -v s=$sdate -v e=$edate '
BEGIN{gsub("/","",s);gsub("/","",e)}
$5 >= s && $5 <= e'

Regards

5

Ah ok, I'm following your now. Would it be easier to have the user input the dates with out the slashes?

If so, that would remove the needs for the

{gsub("/","",s);gsub("/","",e)}

Portion, right?

Thanks again!

6

Right, in that case you can remove this portion:

BEGIN{gsub("/","",s);gsub("/","",e)}

or you can use the first solution.

Regards

7

Awesome, many thanks Franklin!