[Solved] Fetching logs of last few days from logfile

Hi All,

I have a requirement to fetch logs of last 'N' days.
I am trying the following command which is working fine if it finds the date of that day in logfile.

START=`TZ="GMT+$((24*N))" date +"%Y %b %d"`

this is being used to fetch 'N'th day's date
and

awk '/'"$START"'/{p=1} /'"$NOW"'/p' <logfile.log

is being used to fetch the records from log file.

like if last 'N' day's date is 2012 Jun 17
and this date is present in file then it gives out the record from 2012 Jun 17 to NOW.

But if last 'N' day's date is 2012 Jun 17 and it is not present in log file then its giving the blank response , though there are records available for date 2012 Jun 16.

Please help me!

Thanks in advance!
Regards,
KD

Have a look at this thread:

Thanks but that is worked in cse if we kknow the exact date or timestamp.
Problem is that if that timestamo or date is not present in log file iit is giving whole file where aas it should give response as per my first post.

This should work, you don't have to give the exact timestamp:

cat file | awk -F, '{ if ($1>"2012-08-20 11:30" && $1<"2012-08-22 16:00") print }'

You are getting blank output for the logs for which last n days log is not present because you are setting p=1 only when log is present for last Nday

Use your variables directly...

awk -F, -v ST="$START" -v NW="$NOW" '{ if ($1 > ST && $1 < NW ) print }' file

But this will work only if date is in the format of yyyy mm dd

Hi Subbeh, it realy worked,
But its not giving logs corresponding to 2012-08-22 16:00,

where as i need all the logs corresponding any date from 2012-08-20 11:30 to 2012-08-22 16:00

can you give the sample of how date is appears in your log file

Hi Raj,

Its up to us how to define date variables...

We can define variables same as present in log files to match pattern....

make < to <= in the awk

---------- Post updated at 03:31 PM ---------- Previous update was at 03:30 PM ----------

but if month in date is in mon format in log than comparison result will be different

Hi Raj,
In log file date format is 2012 Jun 06

Also in your previous reply u suggested that the p=1... so what can i set there.
because if i set p=0 and searching from 2012 Aug 01 to now. Though in log file the data available for 2012 Aug 16 ..but its showing only data of 2012 Aug 30..

Yes..

If cannot compare date format then fetching logic needs to be changed....

setting p=0 wont work.
what you can do is use loop if log for last N day is not available then start can be modified for N-1 day and so on till NOW

Yes this can be done..
Good option, but it giving each time some output... and not sure at what valuue of 'p' there will be an existing date in log file.

So after each execution of command its giving some output.

CAN YOU GIVE SOME SAMPLE DATA and you required output so that i can come to some conclusion

Yes Raj, this can be done but problem is that we are not sure that at what value of 'p' there will be an existing date in logfile.
So for each value of 'p' command will get executed and hence resulting in some unexpected response.

---------- Post updated at 02:27 AM ---------- Previous update was at 02:21 AM ----------

sample data
:

2012 Aug 01 : sdfjsdfkljsdlfgjlsdfjgldjflgjsdfgasdfg
2012 Aug 05 10:12:dfkjsdlfksdfksdfks;dkf;sdkf;ksdfjsdf
2012 Aug 05 10:13: kfgsdklfjlsedjkflsdjkfljsdfgsda
2012 Aug 30 10:11: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf

So if i am asking data of last 30 days it should give whole data from 2012 Aug 01 to 2012 Aug 30.
if last 25 days then:
2012 Aug 05 10:12:dfkjsdlfksdfksdfks;dkf;sdkf;ksdfjsdf
2012 Aug 05 10:13: kfgsdklfjlsedjkflsdjkfljsdfgsda
2012 Aug 30 10:11: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf

But if i running it for last 29 days it should give
2012 Aug 05 10:12:dfkjsdlfksdfksdfks;dkf;sdkf;ksdfjsdf
2012 Aug 05 10:13: kfgsdklfjlsedjkflsdjkfljsdfgsda
2012 Aug 30 10:11: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf

but not of 2012 Aug 01

$cat file
2012 Aug 01 : sdfjsdfkljsdlfgjlsdfjgldjflgjsdfgasdfg
2012 Aug 05 10:12:dfkjsdlfksdfksdfks;dkf;sdkf;ksdfjsdf
2012 Aug 05 10:13: kfgsdklfjlsedjkflsdjkfljsdfgsda
2012 Aug 30 10:11: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf
2012 Aug 31 10:12: sdhfsdkfsdjflsjdfjsdf

#I don't know how <= is not working for the end date so add one more date to your end date...

$ awk -F ":" '{ if ($1 >= "2012 Aug 01" && $1 <= "2012 Aug 31" ) print }' file
2012 Aug 01 : sdfjsdfkljsdlfgjlsdfjgldjflgjsdfgasdfg
2012 Aug 05 10:12:dfkjsdlfksdfksdfks;dkf;sdkf;ksdfjsdf
2012 Aug 05 10:13: kfgsdklfjlsedjkflsdjkfljsdfgsda
2012 Aug 30 10:11: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf

$ awk -F ":" '{ if ($1 >= "2012 Aug 01" && $1 <= "2012 Sep 1" ) print }' file
2012 Aug 01 : sdfjsdfkljsdlfgjlsdfjgldjflgjsdfgasdfg
2012 Aug 05 10:12:dfkjsdlfksdfksdfks;dkf;sdkf;ksdfjsdf
2012 Aug 05 10:13: kfgsdklfjlsedjkflsdjkfljsdfgsda
2012 Aug 30 10:11: sdhfsdkfsdjflsjdfjsdf
2012 Aug 30 10:12: sdhfsdkfsdjflsjdfjsdf
2012 Aug 31 10:12: sdhfsdkfsdjflsjdfjsdf

and the code to extract last nth day's date is

START=`TZ="GMT+$((24*N))" date +"%Y %b %d"`

---------- Post updated at 02:50 AM ---------- Previous update was at 02:44 AM ----------

Yes Pamu, i can do this too, good option but in case if date format changed.
will this work?
Specially in awk -F ":" the ":" symbol. beacuse in another file this can be "-" or ";"

-F ":" this used as field separator. If date format changed by that means field separator changed then we have to implement that to the code. But this is not major change you can do it easily.

for START variable..

You can use below code to assign date variable to one day ahead of today's date...

START=$(date --date="tomorrow" +"%Y %b %d")