Fetch files based on month

Shell Programming and Scripting
1

Hi All
I have a requirement like below.I have always 12 files got generated in my directory called /xx/out/

abc_2014_10_121.xml
abc_2014_09_345.xml
abc_2014_08_432.xml
abc_2014_07_123.xml
abc_2014_06_098.xml
abc_2014_05_569.xml
abc_2014_04_430.xml
abc_2014_03_235.xml
abc_2014_02_056.xml
abc_2014_01_450.xml
abc_2013_12_111.xml
abc_2013_11_222.xml

Out of these files I have to select the files based on the below condition and delete the rest of the files.

if it is a JAN month then fetch ----- Jan to Dec = 12 files
if it is a FEB month then fetch ----- Jan + Oct + Nov + Dec = 4 files
if it is a MAR month then fetch ----- Jan + Feb + Oct + Nov + Dec = 5 files
if it is a APR month then fetch ----- Jan + Feb + Mar + Oct + Nov + Dec = 6 files
if it is a MAY month then fetch ----- Apr + Jan + Feb + Mar = 4 files
if it is a JUN month then fetch ----- Apr + May + Jan + Feb + Mar = 5 files
if it is a JUL month then fetch ----- Apr + May + Jun + Jan + Feb + Mar = 6 files
if it is a AUG month then fetch ----- Jul + Apr + May + Jun = 4 files
if it is a SEP month then fetch ----- Jul + Aug + Apr + May + Jun = 5 files
if it is a OCT month then fetch ----- Jul + Aug + Sep + Apr + May + Jun = 6 files
if it is a NOV month then fetch ----- Oct + Jul + Aug + Sep = 4 files
if it is a DEC month then fetch ----- Oct + Nov + Jul + Aug + Sep = 5 files
can any one tell how to achieve this
Regards
Sri

2

If you know perl solving this would be a piece of cake imo...

3

What have you tried?

4

How about putting some more care and/or effort into formulating the specification?

5

Hi
Is this homework or some other kind of work? :slight_smile:

6

I have tried it by extracting the date and doing it by -1,-2.. inside a CASE .
Any better ideas other than this.

Regards
Sri

---------- Post updated at 11:51 PM ---------- Previous update was at 11:47 PM ----------

Hi Ongoto

This is not the homework. For some of the attributes in the files ,updation happens in the later months.That is the reason extracting the previous months files along with the current month.
Regards
Sri

7

@ chigurupati.dwh
Good. We're clear on that. :slight_smile:

IF statements might be easier to write than CASE.

Your conditions are kinda complicated because they change each month and cycle back every 3 months, so you would need to set different file age limits for each month.

Whadda think about testing each filename in a loop ?
Maybe something like this.

echo "for month $this_mnth"
echo "files excluded"
LF=$(( $last_file - 1 ))
for x in $(seq 0 "$LF");
do
    y=${files[x]}
    age=${y:9:2} && age=${age#0}
    if [[ $age -lt $this_mnth && $age -lt $lower_limit ]] || \
       [[ $age -gt $this_mnth && $age -lt $upper_Limit ]]; then
        echo $y
    else
        cp $srcdir/$y $fetchdir/
    fi
done

echo "
files fetched"
ls -1 $fetchdir | tr ' ' '\n' | sort -t '_' -k 3

--output--
for month 11
files excluded
abc_2014_01_450.xml
abc_2014_02_056.xml
abc_2014_03_235.xml
abc_2014_04_430.xml
abc_2014_05_569.xml
abc_2014_06_098.xml
abc_2013_12_111.xml

files fetched
abc_2014_07_123.xml
abc_2014_08_432.xml
abc_2014_09_345.xml
abc_2014_10_121.xml
abc_2013_11_222.xml
8

Following code utilizes the legendary eval command and should be run in the directory containing the xml files.

#!/bin/bash

#JAN skipped because all files are being kept
FEB=(01 10 11 12)
MAR=(02 "${FEB[@]}")
APR=(03 "${MAR[@]}")
MAY=(01 02 03 04)
JUN=(05 "${MAY[@]}")
JUL=(06 "${JUN[@]}")
AUG=(04 05 06 07)
SEP=(08 "${AUG[@]}")
OCT=(09 "${SEP[@]}")
NOV=(07 08 09 10)
DEC=(11 "${NOV[@]}")

process_date () {
    a=$1[@]

    #assembling appropriate find command (prefix + center + suffix)
    prefix="find . -maxdepth 1 -type f -name '*_*_*.xml' "

     for month in "${!a}"; do
      center="$center! -name '*_*_"$month"_*.xml' "
     done
    
    suffix="-delete"

    findcmd="$prefix$center$suffix"

    echo; echo "$findcmd"
    echo; echo -n "Execute above command?  Y/N "; read yesno

    case "$yesno" in
        [yY]) eval "$findcmd";;
        [nN]) exit;;
           *) echo "Invalid reply."; exit;;
    esac
}

#get abbreviated month name, e.g. NOV
DATE=$(date '+%b' | tr '[:lower:]' '[:upper:]')

case "$DATE" in
        JAN)    echo "It's January!"; exit;;
          *)    process_date $DATE;;
esac

Demo:

$ ls
abc_2013_11_222.xml  abc_2014_02_056.xml  abc_2014_05_569.xml  abc_2014_08_432.xml  delxml.sh
abc_2013_12_111.xml  abc_2014_03_235.xml  abc_2014_06_098.xml  abc_2014_09_345.xml
abc_2014_01_450.xml  abc_2014_04_430.xml  abc_2014_07_123.xml  abc_2014_10_121.xml
$ ./delxml.sh 

find . -maxdepth 1 -type f -name '*_*_*.xml' ! -name '*_*_07_*.xml' ! -name '*_*_08_*.xml' ! -name '*_*_09_*.xml' ! -name '*_*_10_*.xml' -delete

Execute above command?  Y/N y
$ ls
abc_2014_07_123.xml  abc_2014_08_432.xml  abc_2014_09_345.xml  abc_2014_10_121.xml  delxml.sh
$
1 Like
9

I think you meant "infamous".

You can avoid the eval. Never store quotes to parse them later, just store the parameters exactly as you want them -- minus quotes -- in an array or positional parameter. They will expand perfectly without any need to process them again.

If you return to this kind of code in a year, you won't know what it does:

PREFIX="something"
(10,000 lines)
SUFFIX="somethingelse"
$PREFIX $MIDDLE $SUFFIX

Don't bother stuffing it into $PREFIX and $SUFFIX... They don't change. Just use variables for things that vary, which will leave it as a readable if slightly cryptic find . -type d -maxdepth 1 "$@" -delete which you can figure out without having to find definitions for everything in it first.

-maxdepth is a GNU-only feature by the way.

     set --
     for month in "${!a}"; do
        set -- "$@" -name "*_*_${month}_*.xml"
     done

     # Change the "-print" to "-delete" once you've tested that it's deleting
     # the right files.
     find . -maxdepth 1 -type f -name '*_*_*.xml' "$@" -print
1 Like
10

Wow!!

This really gets interesting. :slight_smile: