perform a check based on number of @ in a log line

Shell Programming and Scripting
1

Hello,

I am intending to perform a check based on number of "@" , present in a line in a log file . The idea is basically to perform a check on cc or bcc sender, based on an email log, which shows all the for email address. Say if the number of @ is more than 30, I will consider it as a mass emailer, then an email alert will be sent to a specified email address.

Highly appreciate any advice/little script on this.

Thank you

2

YMMV:

echo 'foo@a.com, bar@b.net, fred@c.info' | nawk '{ print gsub("@", "")}'
3 
awk -F"@" '{print NF}' logfile|while read line
do
 if [ $line -gt 30 ]
 then
   echo "possible mass mailer"
   break
 fi
done
4 
$ echo 'foo@a.com, bar@b.net, fred@c.info' | awk -F"@" '{print NF}'
4
5

Hi,

Thank you, each one is great.

Now onto next riddle,

Email logs record time in the following example format.

day hour
### #####
2009-02-23 18:50:45
2009-02-23 14:12:19
2009-02-24 08:50:45

Even if I run a cron to check the log file every 5 min or 3 min, I might miss the last 3 or 5 mins, ( I if use time parameter in the script to check logs only for the current hour, say with date command ) , because the last part of time will contain ( say 9:00 , after 8:55 )

Any ideas how to use the system time, as well not missing any line in the log file for last 1 ( or look like check, every last 2 hours log is better ) due to time factor.

6

oh and the check for presence and number of "@" should be per line, and not as a whole file, and then if any particular line has "@" > 30 times , then an alert will be sent out

7

Hi krishmaths , vgersh99

Both of your script/oline show the counts or the number.

Can you propose something that will print the line number, the count of "@" 's present and the line itself in 3 colums

say,

line number count of @ present display the line
########## ################# ##############
5 80 <content>

Here is a real world test log
#########################

2009-02-20 11:17:38 [15620] SMTP connection from ([z.z.z1.46]) [189.x.x.x]:49177 I=[y.y.y.12]:26 closed by QUIT
2009-02-20 11:18:35 [16261] H=([z.z.z1.46]) [189.x.x.x]:49178 I=[y.y.y.12]:26 Warning: Sender rate 4.7 / 1h
2009-02-20 11:18:47 [16261] 1LaZ1O-0004EH-Mk <= mariana@sumergibawew.tld H=([z.z.z1.46]) [189.x.x.x]:49178 I=[y.y.y.12]:26 P=esmtpa A=fixed_login:mariana@sumergibawew.tld S=56834 id=C5C4420B.BF2%mariana@sumergibawew.tld T="FICHA TECNICA DE LA CAJA PELICAN 1090" from <mariana@sumergibawew.tld> for ez@elmerzambrano.tldx evs64@prodigy.net.mx subaquatec@ecotoursvallarta.tldx ventas@grupoarcosoreste.tldx ugalde64@prodigy.net.mx iusan25@asdasdasdastld bethorecuperado@testmail.tldx robdam@testmail.tldx romanocesar53@testmail.tldx romanoc04@asdasdasdastldx dainzu@msm.tldx foro5inko@testmail.tldx ramonsange@asdasdasdastldx industriasjac@prodigy.net.mx patricia@gilsama.tldx araceligenis@gilsama.tldx sbelmont@prodigy.net.mx compras@promovago.tldx audiorenta@prodigy.net.mx valentin.andrade@dileds.tldx francisca.fuente@sgs.tldx moises.cruz@sgs.tldx ottovogel@prodigy.net.mx gabriel.torres@ge.tldx andrew_1355@testmail.tldx tomas.padillamoreno@xailasasa.tldx roradiqui@xailasasa.tldx francisco-l@dm-corporativo.tld thernandez@c-m-a.tld normahernandez@medihelp.tld edith.dorantes@lgseguridad.tldx ventas@protechseguridad.tldx eolvera@impypub.tld jecera@asdasdasdastldx technosub@ver.megared.net.mx tht@geofisica.unam.mx eescamilla@tdesa.tldx mzlopezm@deacero.tldx victormr@ilsamexico.tldx miguel_hernandez@praxair.tldx mateein@prodigy.net.mx hopper.arreguin@testmail.tldx maricarmen@ponchito.tldx ifarfan@tvazteca.tld a.b.hempe@xailasasa.tldx gvc68@terra.tld ventas_ips1@prodigy.net.mx grupo_hermasa@testmail.tldx alf2421@testmail.tldx thaliareyes@xailasasa.tldx orivera@amtec.tld blopez@suinma.tldx
2009-02-20 11:18:50 [16261] SMTP connection from ([z.z.z1.46]) [189.x.x.x]:49178 I=[y.y.y.12]:26 closed by QUIT
2009-02-20 11:19:20 [16587] H=([z.z.z1.46]) [189.x.x.x]:49180 I=[y.y.y.12]:26 Warning: Sender rate 5.6 / 1h
2009-02-20 11:19:27 [16587] 1LaZ23-0004JX-8G <= mariana@sumergibawew.tld H=([z.z.z1.46]) [189.x.x.x]:49180 I=[y.y.y.12]:26 P=esmtpa A=fixed_login:mariana@sumergibawew.tld S=57440 id=C5C44238.BF3%mariana@sumergibawew.tld T="ESTUCHES Y LINTERNAS PELICAN PROMOCION DEL MES" from <mariana@sumergibawew.tld> for mariana@sumergibawew.tld technosub@ver.megared.net.mx gerencia@avasesores.tldx juana_nafri@asdasdasdastld merchandisemex@jsav.tldx snovoa@dolphindiscovery.tldx mdaisyrg@xailasasa.tldx letym@grupoaei.tldx mhccastaneda@testmail.tldx ezavalaa@prodigy.net.mx ssefmesina@testmail.tldx carlos@avicolarioja.tld capri_790117@asdasdasdastld emonreal2000@testmail.tldx jdvaldezleal@asdasdasdastld raul_montalvo@sgs.tldx clientes@hamelectrica.tldx cesar.campillo@sytd.tld everth.palmer@optimalpressure.tldx
2009-02-20 11:19:31 [16587] SMTP connection from ([z.z.z1.46]) [189.x.x.x]:49180 I=[y.y.y.12]:26 closed by QUIT
2009-02-20 11:19:53 [16806] H=([z.z.z1.46]) [189.x.x.x]:49181 I=[y.y.y.12]:26 Warning: Sender rate 6.6 / 1h
2009-02-20 11:20:10 [16806] 1LaZ2i-0004N4-0H <= mariana@sumergibawew.tld H=([z.z.z1.46]) [189.x.x.x]:49181 I=[y.y.y.12]:26 P=esmtpa A=fixed_login:mariana@sumergibawew.tld S=57438 id=C5C44259.BF4%mariana@sumergibawew.tld T="ESTUCHES Y LINTERNAS PELICAN PROMOCION DEL MES" from <mariana@sumergibawew.tld> for conpras@lumen.tld ednoriega@intelcompras.tldx eduardo@smartsonicsupply.tldx gabrielm@grupoatr.tldx juliowitrago@testmail.tldx gustavoescarcega@asdasdasdastld dmecanico@fabatsa.tld apca50@testmail.tldx coloresyaditivos@axtel.net jlmerodio@testmail.tldx recepcion_mexico@globaldrilling.tld jsandoval@pinnocle-home.tld felix.gonzalez@adsmexicana.tldx aygservicios_especializados@asdasdasdastld desisti@desistimexico.tldx marecidive@testmail.tldx beto.garcia3@xailasasa.tldx egarcia@crockettint.tldx mangel@mexichen.tldx browningmexico@asdasdasdastld jmenab@mebaimportaciones.tld lisandoval@xailasasa.tldx disi_ventas@asdasdasdastld cturati@turavision.tld erikpalme@testmail.tldx daniela_salinas@testmail.tldx bom_alfonso@testmail.tldx charliees@asdasdasdastldx greg_vo@asdasdasdasit alberto.anguino.aa@lanxess.tldx emoreno@codeciphers.co.uk info@mundosubmarino.tld jrodriguez@sipsamex.tldx ernesto.matias@cfe.gob.mx ruly_benhur@testmail.tldx israelguerrero@asdasdasdastld ftorres@geolect.tld mariana@sumergibawew.tld freddypv35@xailasasa.tldx ramces.villanueva@vro.tld tropa212003@asdasdasdastldx eduardo.lopez@grainger.tldx contreras@defensa.tld jesus.gonzalez@admexicana.tldx compras@vro.tld fullservice@ndt.tld laurahinojosa2003@asdasdasdastld oscar.mar@psi-ind.tld colin_arturo@testmail.tldx rolmale@gamail.tldx juan.iskander@xailasasa.tldx mzlopezm@deacero.tldx raul@serecom.tld alejandrordz73@testmail.tldx victormr@ilsamexico.tldx gbriones@wittman.tld bksystem2@asdasdasdastld guisami@testmail.tldx aicsageo@testmail.tldx ozzflores@maill.risoul.tld fision_energias@infosel.net.mx medina.torres@asdasdasdastldx cds@ver.megared.net.mx loose@prodigy.net.mx cbprovento@asdasdasdastld cbravosdc@prodigy.net.mx stovar@meyersound.tld
2009-02-20 11:20:13 [16806] SMTP connection from ([z.z.z1.46]) [189.x.x.x]:49181 I=[y.y.y.12]:26 closed by QUIT
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h
2009-02-20 11:20:29 [17082] H=([z.z.z1.46]) [189.x.x.x]:49182 I=[y.y.y.12]:26 Warning: Sender rate 7.5 / 1h

Thanks

8 
awk -F"@" '{print FNR, NF, $0}' myFile
9

awk -F"@" '{print FNR, NF, $0}' /root/testlog | grep fixed_login

gives me desired result, like:

72 ( is the count of @ )

72 2009-02-20 11:20:10 [16806] 1LaZ2i-0004N4-0H <= mariana@sumergibawew.tld H=([z.z.z1.46]) [189.x.x.x]:49181 I=[y.y.y.12]:26 P=esmtpa A=fixed_login:mariana@sumergibawew.tld S=57438 id=C5C44259.BF4%mariana@sumergibawew.tld T="ESTUCHES Y LINTERNAS PELICAN PROMOCION DEL MES" from <mariana@sumergibawew.tld> for conpras@lumen.tld ednoriega@intelcompras.tldx eduardo@smartsonicsupply.tldx gabrielm@grupoatr.tldx juliowitrago@testmail.tldx gustavoescarcega@asdasdasdastld dmecanico@fabatsa.tld apca50@testmail.tldx coloresyaditivos@axtel.net jlmerodio@testmail.tldx recepcion_mexico@globaldrilling.tld jsandoval@pinnocle-home.tld felix.gonzalez@adsmexicana.tldx aygservicios_especializados@asdasdasdastld desisti@desistimexico.tldx marecidive@testmail.tldx beto.garcia3@xailasasa.tldx egarcia@crockettint.tldx mangel@mexichen.tldx browningmexico@asdasdasdastld jmenab@mebaimportaciones.tld lisandoval@xailasasa.tldx disi_ventas@asdasdasdastld cturati@turavision.tld erikpalme@testmail.tldx daniela_salinas@testmail.tldx bom_alfonso@testmail.tldx charliees@asdasdasdastldx greg_vo@asdasdasdasit alberto.anguino.aa@lanxess.tldx emoreno@codeciphers.co.uk info@mundosubmarino.tld jrodriguez@sipsamex.tldx ernesto.matias@cfe.gob.mx ruly_benhur@testmail.tldx israelguerrero@asdasdasdastld ftorres@geolect.tld mariana@sumergibawew.tld freddypv35@xailasasa.tldx ramces.villanueva@vro.tld tropa212003@asdasdasdastldx eduardo.lopez@grainger.tldx contreras@defensa.tld jesus.gonzalez@admexicana.tldx compras@vro.tld fullservice@ndt.tld laurahinojosa2003@asdasdasdastld oscar.mar@psi-ind.tld colin_arturo@testmail.tldx rolmale@gamail.tldx juan.iskander@xailasasa.tldx mzlopezm@deacero.tldx raul@serecom.tld alejandrordz73@testmail.tldx victormr@ilsamexico.tldx gbriones@wittman.tld bksystem2@asdasdasdastld guisami@testmail.tldx aicsageo@testmail.tldx ozzflores@maill.risoul.tld fision_energias@infosel.net.mx medina.torres@asdasdasdastldx cds@ver.megared.net.mx loose@prodigy.net.mx cbprovento@asdasdasdastld cbravosdc@prodigy.net.mx stovar@meyersound.tld

Now, I want something like,

awk -F"@" '{print FNR, NF, $0}' /root/testlog | grep fixed_login

the first colum gives the number of @.

Now, we want a code that will check each of the line for ( awk -F"@" '{print FNR, NF, $0}' /root/testlog | grep fixed_login | awk {'print $1}' ) and if the value for any one line is > 30 , it should declare that line a mass emailer and alert the line number or append all the lines, that has more than 30 "@" to and file and email it to a particular address.

Any ideas appreciated

10 
awk -F"@" '/fixed_login/ && NF > 30 {print FNR, NF, $0}' /root/testlog
11

ok that wondeful, vgersh99

Now, what i want run a cron , that will check the log file for last 60 mings of of logs generated in the log file , every 5 min, for the above command and if it finds one it will email.

Any idea on how can we can most recent 60 mins of generated logs, every 5 mins.

Thanks

12

man find

man crontab

13

thanks