awk search pattern

Shell Programming and Scripting
1

Hi,

I have a log file which contains lines like below:

2010-07-19 07:13:19,021 ERROR system ...(text)
2010-07-19 07:22:03,427 ERROR system ...(text)
class com... (text)
2010-07-19 07:23:19,026 ERROR system ...(text)
class com... (text)

each line is a separate line... I am given the a
start time such as : "2010 07 17 07 13 19 "
end time such as: "2010 07 19 20 00 00"
and I have 2 variables which puts them to the format of "2010-07-19 07:13:19" and "2010-07-19 20:00:00

I'm not sure what to put such a search pattern to return the log lines in between (inclusive) the start and end times given, and on the example above, it should return lines afters it as well.

if I put the start time and end time of only the first time log line from above it should be:

2010-07-19 07:13:19,021 ERROR system ...(text)

if I put the start time and end time of only the second time log line from above it should be:

2010-07-19 07:22:03,427 ERROR system ...(text)
class com... (text)

if I put the start time and end time of only the third time log line from above it should be:

2010-07-19 07:23:19,026 ERROR system ...(text)
class com... (text)

or if i just put a start and end time including all it should return everything.. etc

This problem is kind of complicated... I would greatly appreciated if anyone can even lead me in the way of getting it to the correct search outputs.

Thanks

I have the following code which, only looks for the start time for exact matches, and returns.

awk -vs="$sstart" '$0~"^"s{p=1}!($0~"^"s) && /^2010/{if(p==1){print "\n"}p=o}p' "$f"

$sstart is the format of "2010-07-19 07:13:19", and $F is the file name.

2

Hey you can do it this way:

Inside your shell script, let's say the start daytime and end daytime are variables vstart and vend

then add the following in ur shell script:

vend=${vend:-vstart}

sed -n -e '/^'"${vstart}"'/,$p
/^'"${vend}"'/ { 
:up
n
/....-..-.. ..:..:../q
p
b up }' <your log file name>
3

Thanks for helping, but it's still not working the way i want it to be,

the line

vend=${vend:-vstart}

doesn't seem to do anything, vend is still vend...

---------- Post updated at 12:51 PM ---------- Previous update was at 12:22 PM ----------

Is there any built-in way of getting even a list of the times in between start and end? or do i need to make a function myself to do it specifically?

Thanks

4

Sorry there is a typo in my code. Please replace it as following:

vend=${vend:-$vstart} ## I missed a $ there

It works like this: if vend is not set then value of vstart is assigned to it.

5

Thanks, but... vstart and vend will always be set... and how does that help in finding out the lines in between?

6 
vend=${vend:-$vstart}

## the following sed script will stop printing when it finds vend in the log file. 
## So I am setting vend to vstart if you are not providing vend with any input value

sed -n -e '/^'"${vstart}"'/,$p
/^'"${vend}"'/ { 
:up
n
/....-..-.. ..:..:../q
p
b up }' <your log file name>
7

I am always always always providing a vend value,

this is not what I was looking for at all, but thanks anyways.

8

Assuming starttime and endtime are shell variables:

starttime="2010 07 19 07 13 19"
endtime="2010 07 19 07 30 00"

awk -F, -v stime="$starttime" -v etime="$endtime" '
BEGIN{gsub(" ","",stime);gsub(" ","",etime)}
/^[0-9]/{
  t=$1
  gsub("[ -:]","",t)
}
t >= stime{p=1}
t > etime{p=0}
p' logfile
9

Thanks for the input Franklin52, I input the start and end time as follows:

2010 07 19 10 29 00

2010 07 19 10 30 00

and got the following lines from the log file:

2010-07-19 07:37:16
2010-07-19 07:39:00
2010-07-19 07:55:13
2010-07-19 08:00:50
2010-07-19 08:40:04
2010-07-19 08:40:04
2010-07-19 09:18:36
2010-07-19 09:18:36
2010-07-19 09:37:10
2010-07-19 09:37:56
2010-07-19 09:37:56
2010-07-19 09:43:18
2010-07-19 09:43:18
2010-07-19 10:10:31
2010-07-19 10:24:39
2010-07-19 10:28:11
2010-07-19 10:28:11
2010-07-19 10:28:47
2010-07-19 10:28:47
2010-07-19 10:29:00
2010-07-19 10:29:00

I only wanted the last 2 to show up...

10

Can you post some real lines of your logfile and the desired output?

11

every file is very large... but I will try to minimize it, I did your search in a folder of log files, which has different dates. So it must have worked for the 2010-07-19 part, but the time part did not work out.

here is a shorter version of a log file:

I tried to output as much as I can without leaking confidential information, thanks.

The desired output would just be the date stamped lines and any lines following it which are not stamped, with the date in between the start and end input time, inclusively.

so if i use the start time as "2010 07 19 07 38 00"
end time as "2010 07 19 07 59 59", it should return:

12

This is what i get with the given file:

$ cat file
2010-07-19 07:37:16,372 ERROR [WebContainer : 10] class.......
2010-07-19 07:39:00,725 ERROR [WebContainer : 6] class.......
2010-07-19 07:55:13,357 ERROR [WebContainer : 10] class.......
Error code:************************
at *************getPrinters(FormBrokerProxy.java:202)
at ************init>(Unknown Source)
2010-07-19 08:00:50,210 ERROR [WebContainer : 3] class....
$ awk -F, -v stime="2010 07 19 07 38 00" -v etime="2010 07 19 07 59 59" '
BEGIN{gsub(" ","",stime);gsub(" ","",etime)}
/^[0-9]/{
  t=$1
  gsub("[ -:]","",t)
}
t >= stime{p=1}
t > etime{p=0}
p' file
2010-07-19 07:39:00,725 ERROR [WebContainer : 6] class.......
2010-07-19 07:55:13,357 ERROR [WebContainer : 10] class.......
Error code:************************
at *************getPrinters(FormBrokerProxy.java:202)
at ************init>(Unknown Source)
$

Am I missing something?

13

Hmm this is weird, originally I made a mistake and changed the variable names to something else I did not declare, but now I just copy your code, or fix the variable names, nothing is getting returned?! I am using cygwin if that helps... I have no idea what's going on.

14

Have you used a windows editor or is this a file in windows format?

Try to convert them with:

tr -d '\r' < windowsfile > unixfile
15

I am confident these are in unix format, at the beginning I was having trouble reading in the '\r' then I fixed everything up...

Before I used

awk -F, -v stime="$sstart" -v etime="$eend" '

But those variables weren't declared, I used it to search the whole folder of log files, which contained several log files with different dates ( each log file contains all lines within the day ), but the result returned to me was everything in the day, as shown in my previous posts when I was confused.