SH script to split squid log by date

Hi,

I really need your help to make a script to split a large squid's log file into a multiple files, each of them containing the log entries for every logged date.

To achieve the result I planned to use the function "date" with the first log field as a parameter using this syntax:

date -d @(ten number squid timestamp field) +%F

to obtain the desired filename for splitted logs.

Example:
# date -d @1232163496 +%F
will return: 2009-01-16

The idea is to analize the first field of each line in the log file and append whole line to the corresponding file.

Any help would be apreciated,

I'm a newbie �Someone can help me?

How does your file looke like and what is the desired output? Do you get the epoch time from a file?

Regards

Hi Franklin52

I want to analize squid's log file "access.log", it look like this:

1232935790.356 1795 192.168.0.10 TCP_MISS/200 551 GET http://www.cubachat ...
1232935790.383 1645 192.168.0.55 TCP_MISS/302 619 GET http://images.imagef ...
1232935791.271 888 192.168.0.55 TCP_MISS/302 618 GET http://images.imagef ...
1232935796.064 1444 192.168.0.55 TCP_MISS/302 619 GET http://images.imagef ...
1232935797.477 1412 192.168.0.55 TCP_MISS/302 618 GET http://images.imagef ...
1232955307.426 527 192.168.0.10 TCP_CLIENT_REFRESH_MISS/407 2116 GET Free Download Manager - absolutely free download accelerator and manager ...
1232955307.703 804 192.168.0.10 TCP_MISS/407 2072 GET http://webmail.rimed ...
1232955315.375 1266 192.168.0.10 TCP_MISS/407 1614 GET http://webmail.rimed ...
1232955322.071 1633 192.168.0.10 TCP_MISS/407 1614 GET http://webmail.rimed ...

The first field of the log is the date and time in UNIX format (seconds since 1/1/1970). Analizing this field (as proposed in my first post), two dates as registered: jan 25 2009 and jan 26 2009, thus two files must be generated:

2009-01-25 containing:
1232935790.356 1795 192.168.0.10 TCP_MISS/200 551 GET http://www.cubachat ...
1232935790.383 1645 192.168.0.55 TCP_MISS/302 619 GET http://images.imagef ...
1232935791.271 888 192.168.0.55 TCP_MISS/302 618 GET http://images.imagef ...
1232935796.064 1444 192.168.0.55 TCP_MISS/302 619 GET http://images.imagef ...
1232935797.477 1412 192.168.0.55 TCP_MISS/302 618 GET http://images.imagef ...

and
2009-01-26 containing:
1232955307.426 527 192.168.0.10 TCP_CLIENT_REFRESH_MISS/407 2116 GET Free Download Manager - absolutely free download accelerator and manager ...
1232955307.703 804 192.168.0.10 TCP_MISS/407 2072 GET http://webmail.rimed ...
1232955315.375 1266 192.168.0.10 TCP_MISS/407 1614 GET http://webmail.rimed ...
1232955322.071 1633 192.168.0.10 TCP_MISS/407 1614 GET http://webmail.rimed ...

Any help would be apreciated,

Thanks

Try one of these solutions (not tested). If you have gawk:

gawk -F"." '{d=strftime("%Y-%m-%d", $1);print > d}' file

Otherwise:

awk -F"." '{"date -d @" $1 " +%F" | getline d;print > d;close("date -d @" $1 " +%F")}' file

Regards

Hi, :slight_smile:

Thank you so much!!

It works very good.