Finding/Grep on files with date and hour in the file name

mihirvora16 · June 12, 2012, 11:51pm

Hi,
I have a folder structure as follows,

DATA -> 2012-01-01 -> 00 -> ABC_2012-01-03_00.txt
-> 01 -> ABC_2012-01-03_01.txt
-> 02 -> ABC_2012-01-03_02.txt
...
-> 23 -> ABC_2012-01-03_02.txt
-> 2012-01-02
-> 2012-01-03

So the dir DATA contains the above hierarchy,
User input Start and End Data
I want to grep on all those files between the input dates:confused:, I want a list of files which come under the input dates.
note that the files creation or modified date is not the date and hour contained in the file name so cannot use find newer/modified command.

So what could be the easiest way of doing it, After finding the files i want to grep a string "PQRS" on the selected files for the input dates.

Any help on this would be highly appreciated...!!! TIA

Chubler_XL · June 13, 2012, 1:06am

If your grep supports the -r (recursive search) option try this:

find_data.sh

#!/bin/bash
if [ $# -eq 0 -o $# -gt 3 ]
then
    echo "usage: $0 pattern [from-date [to-date]]">&2
    exit 2
fi
 
PATTERN=$1
F=${2:-0}
F=${F//-/}
T=${3:-99999999}
T=${T//-/}
 
for subdir in DATA/*
do
   DT=$( basename $subdir )
   DT=${DT//-/}
   [ $DT -lt $F -o $DT -gt $T ] && continue
   echo $subdir
done | xargs grep $PATTERN -r

Call it like this:

$ find_data.sh my_grep_pattern 2012-01-01 2012-06-13

mihirvora16 · June 13, 2012, 1:16am

Thanks for the quick reply Chubler_XL let me try this out.

---------- Post updated at 11:46 AM ---------- Previous update was at 11:40 AM ----------

chubler_xl:

If your grep supports the -r (recursive search) option try this:

find_data.sh

#!/bin/bash
# find_data.sh
# called with:   find_data.sh  pattern [from-data] [to-date]
PATTERN=$1
F=${2:-0}
F=${F//-/}
T=${3:-99999999}
T=${T//-/}
 
for subdir in DATA/*
do
   DT=$( basename $subdir )
   DT=${DT//-/}
   [ $DT -lt $F -o $DT -gt $T ] && continue
   echo $subdir
done | xargs grep $PATTERN -r

Call it like this:

$ find_data.sh my_grep_pattern 2012-01-01 2012-06-13

Now That worked pretty well, I was not aware of the for loop u mentioned above i used to doe ls -lR and then loop over... but i have multiple files inside the hour directory (00->) so i just want to look up into ABC_ file and not any other files since my search patter is also present under multiple files in the directory.

Chubler_XL · June 13, 2012, 1:19am

So are you saying you need to include hour in the from/to criteria?

mihirvora16 · June 13, 2012, 1:21am

No i dont want hour in the search criteria basically i want to loop over all the hours of the day but just the pattern search should happen over a selected file i.e. ABC_<Date><Hour>.txt as the file is present under all the hour dirs under a specific date...
Its like 00-23 hours for a day dir and ABC* file under all the hours dir...
got me?

Chubler_XL · June 13, 2012, 1:24am

Yes add --include "ABC_*" to grep line:

was

done | xargs grep $PATTERN -r

now

done | xargs grep $PATTERN -r --include "ABC_*"

mihirvora16 · June 13, 2012, 1:35am

Yeah that works just had to do --include great going mate... really appreciate...
Thanks,...

Could you just point me to any advanced learning guided online? PM me the same. Thanks again

---------- Post updated at 12:05 PM ---------- Previous update was at 11:59 AM ----------

Hey chubler_XL
could you explain in brief how exactly the above thing works.