Any time you're trying to compare dates as strings you're doomed to failure if your strings contain a year that is not in the high order position, a month that is an abbreviated English month name instead of a month number, and/or days of month that are sometimes one digit and sometimes two digits. You need to be comparing date strings that in the same format and contain the same number of characters (unless you're going to convert everything to Seconds since the Epoch and perform a numeric comparison). The optimum string comparison format until the year 10000 is: YYYYmmddHHMMSS
. You could try adding milliseconds to the end of that if you want to, but I don't think GNU date will give you anything other than 0 for milliseconds if you ask it to give you a date and time that is 1800 seconds ago. (And, if you tell it to give you a date and time that is 30 minutes ago, it will probably also give you 0 for the seconds part of your timestamp.
Note that I'm guessing on that, I don't have access to a GNU date
utility. I do have access to a ksh
version 93u+ which has a printf
statement of the form:
printf "%(GNU_date_format_string)T\n" '1800 seconds ago'
that will give me date and time strings from 30 minutes ago (where GNU_date_format_string
is a GNU date format string without the leading <plus-sign> character.
The following script seems to do what you want using the Korn shell on macOS Mojave version 10.14.3 to create a test log file with timestamps from 1900 seconds ago up to 1700 seconds ago in 15 second intervals to verify that it is converting dates so it starts printing records from the log file that are no more than 30 minutes old. If you comment out the printf
statements that are printing dates and uncomment the date
commands that are currently commented out, this code should work with either bash
or ksh
on a Linux system with a GNU date
utility installed.
If you invoke this script with an argument (any argument), the awk
script will print out debugging information showing how the split()
function split up the lines in the date format you want to process until it finds a timestamp that meets your criteria.
#!/bin/ksh
# Create sample logfile for this test creating entries with two different date
# and time formats.
for ((i=1900; i>1700; i-=15))
do TZ=UCT0 LC_ALL=C printf \
"%(<%b %e, %Y, %I:%M:%S,%3N %p %Z> <$i seconds ago>)T\n" \
"$i seconds ago"
# LC_ALL=C date -u --date "$i seconds ago" \
# "+<%b %e, %Y, %I:%M:%S,%3N %p %Z> <$i seconds ago>"
TZ=UCT0 LC_ALL=C printf \
"%(%Y-%m-%dT%H:%M:%S.%3N+0000: $i seconds ago)T\n" \
"$i seconds ago"
# LC_ALL=C date -u --date "$i seconds ago" \
# "+%Y-%m-%dT%H:%M:%S.%3N+0000: $i seconds ago"
done > logfile
printf 'Using logfile containing:\n'
cat logfile
printf '\nstarting awk at about '
TZ=UCT0 LC_ALL=C printf '%(%Y-%m-%d %H:%M:%S,%3N)T\n'
#LC_ALL=C date -u '%Y-%m-%d %H:%M:%S,%3N'
start_date=$(TZ=UCT0 LC_ALL=C printf '%(%Y%m%d%H%M%S)T' '1800 seconds ago')
#start_date=$(LC_ALL=C date -u '+%Y%m%d%H%M%S' --date '1800 seconds ago')
printf 'start_date=%s\n' "$start_date";date -u
awk -v start="$start_date" -v Log=$# '
BEGIN { split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec", m2b)
for(i = 1; i <= 12; i++)
b2m[m2b] = sprintf("%02d", i)
}
{ if($1 ~ /</) {
if(print_it) {
print
next
}
} else next
split($0, fields, /[<> ,:]+/)
if(Log) for(i=1; i<=12; i++) printf("fields[%d]=%s\n",i,fields)
if(fields[5] == 12)
fields[5] = "00"
if(fields[9] == "PM")
fields[5] += 12
linedate = fields[4] b2m[fields[2]] sprintf("%02d", fields[3]) \
fields[5] fields[6] fields[7]
if(Log)printf("linedate:%s from %s\n", linedate, substr($0,1,45))
if(linedate >= start) {
print_it = 1
print
}
}' logfile
Running this script a few minutes ago produced the following output:
Using logfile containing:
<Feb 12, 2019, 11:48:31,000 PM GMT> <1900 seconds ago>
2019-02-12T23:48:31.000+0000: 1900 seconds ago
<Feb 12, 2019, 11:48:46,000 PM GMT> <1885 seconds ago>
2019-02-12T23:48:46.000+0000: 1885 seconds ago
<Feb 12, 2019, 11:49:01,000 PM GMT> <1870 seconds ago>
2019-02-12T23:49:01.000+0000: 1870 seconds ago
<Feb 12, 2019, 11:49:16,000 PM GMT> <1855 seconds ago>
2019-02-12T23:49:16.000+0000: 1855 seconds ago
<Feb 12, 2019, 11:49:31,000 PM GMT> <1840 seconds ago>
2019-02-12T23:49:31.000+0000: 1840 seconds ago
<Feb 12, 2019, 11:49:46,000 PM GMT> <1825 seconds ago>
2019-02-12T23:49:46.000+0000: 1825 seconds ago
<Feb 12, 2019, 11:50:01,000 PM GMT> <1810 seconds ago>
2019-02-12T23:50:01.000+0000: 1810 seconds ago
<Feb 12, 2019, 11:50:16,000 PM GMT> <1795 seconds ago>
2019-02-12T23:50:16.000+0000: 1795 seconds ago
<Feb 12, 2019, 11:50:31,000 PM GMT> <1780 seconds ago>
2019-02-12T23:50:31.000+0000: 1780 seconds ago
<Feb 12, 2019, 11:50:46,000 PM GMT> <1765 seconds ago>
2019-02-12T23:50:46.000+0000: 1765 seconds ago
<Feb 12, 2019, 11:51:01,000 PM GMT> <1750 seconds ago>
2019-02-12T23:51:01.000+0000: 1750 seconds ago
<Feb 12, 2019, 11:51:16,000 PM GMT> <1735 seconds ago>
2019-02-12T23:51:16.000+0000: 1735 seconds ago
<Feb 12, 2019, 11:51:31,000 PM GMT> <1720 seconds ago>
2019-02-12T23:51:31.000+0000: 1720 seconds ago
<Feb 12, 2019, 11:51:46,000 PM GMT> <1705 seconds ago>
2019-02-12T23:51:46.000+0000: 1705 seconds ago
starting awk at about 2019-02-13 00:20:11,614
start_date=20190212235011
Wed Feb 13 00:20:11 UTC 2019
<Feb 12, 2019, 11:50:16,000 PM GMT> <1795 seconds ago>
<Feb 12, 2019, 11:50:31,000 PM GMT> <1780 seconds ago>
<Feb 12, 2019, 11:50:46,000 PM GMT> <1765 seconds ago>
<Feb 12, 2019, 11:51:01,000 PM GMT> <1750 seconds ago>
<Feb 12, 2019, 11:51:16,000 PM GMT> <1735 seconds ago>
<Feb 12, 2019, 11:51:31,000 PM GMT> <1720 seconds ago>
<Feb 12, 2019, 11:51:46,000 PM GMT> <1705 seconds ago>
Maybe this will give you something you can build on.