Mr_47
1
Hi,
I am trying to create a shell script in unix platform, hence will need info on how to start and any ideas from you guys. million thx.
Objective: Shell script to search all entries from 1 file(a.out) to all other files and extract the search output to 1 file (c.out).
Situation,
1) a.out file contains item that i need to search.
more a.out
mailadm@tm.net.my
tm_network@tm.net.my
tmm_cic@tm.net.my
webmaster@tm.net.my
tmcops@tm.net.my
dnsteam@tm.net.my
tm_quake@tm.net.my
tm_billing@tm.net.my
peering@tm.net.my
tm_dco@tm.net.my
2) list of other files ( need to search all content from a.out file)
ls -al|grep mail.log
-rw-r--r-- 1 mailsrv mail 10431447 May 3 2011 mail.log.01052011.gz
-rw-r--r-- 1 mailsrv mail 24449972 Jun 3 2011 mail.log.01062011.gz
-rw-r--r-- 1 mailsrv mail 23294154 Jul 3 2011 mail.log.01072011.gz
-rw-r--r-- 1 mailsrv mail 9886937 Jul 3 2012 mail.log.01072012.gz
-rw-r--r-- 1 mailsrv mail 19758797 Aug 3 2011 mail.log.01082011.gz
-rw-r--r-- 1 mailsrv mail 21122727 Aug 3 00:31 mail.log.01082013.gz
-rw-r--r-- 1 mailsrv mail 12600862 Sep 3 2011 mail.log.01092011.gz
-rw-r--r-- 1 mailsrv mail 21982491 Nov 3 2011 mail.log.01112011.gz
-rw-r--r-- 1 mailsrv mail 13511183 May 4 2011 mail.log.02052011.gz
-rw-r--r-- 1 mailsrv mail 12473063 Jul 4 2011 mail.log.02072011.gz
-rw-r--r-- 1 mailsrv mail 19321867 Jul 4 2012 mail.log.02072012.gz
-rw-r--r-- 1 mailsrv mail 22605484 Aug 4 2011 mail.log.02082011.gz
-rw-r--r-- 1 mailsrv mail 18831478 Aug 4 00:32 mail.log.02082013.gz
-rw-r--r-- 1 mailsrv mail 15772747 Sep 4 2011 mail.log.02092011.gz
-rw-r--r-- 1 mailsrv mail 22903663 May 5 2011 mail.log.03052011.gz
.
.
.
-rw-r--r-- 1 mailsrv mail 17611225 Aug 31 2011 mail.log.29082011.gz
-rw-r--r-- 1 mailsrv mail 17393355 Feb 1 2012 mail.log.30012012.gz
-rw-r--r-- 1 mailsrv mail 25865466 Jul 2 2011 mail.log.30062011.gz
-rw-r--r-- 1 mailsrv mail 13508155 Aug 1 2011 mail.log.30072011.gz
-rw-r--r-- 1 mailsrv mail 19499004 Aug 1 00:31 mail.log.30072013.gz
-rw-r--r-- 1 mailsrv mail 12118182 Sep 1 2011 mail.log.30082011.gz
-rw-r--r-- 1 mailsrv mail 9555015 Aug 2 2011 mail.log.31072011.gz
-rw-r--r-- 1 mailsrv mail 20240410 Aug 2 00:32 mail.log.31072013.gz
-rw-r--r-- 1 mailsrv mail 12667014 Sep 2 2011 mail.log.31082011.gz
-rw-r--r-- 1 mailsrv mail 189438124 Nov 2 2012 mail.log.31102012
-rw------- 1 mailsrv mail 76248835 Aug 26 12:43 mail.log_current
output should be something like this,
file should be name as c.out
more c.out
26-Aug-2013 09:03:11.10 19c0.2c507b.9406946 tcp_fromvirus reprocess E 90 prvs=3950219457=mohamadzarifi@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <01af01cea1f8$069b1070$13d13150$@tm.net.my> mailsrv pp-inbound.tm.net.my ([192.168.10.91])
26-Aug-2013 09:03:11.99 68ff.1.28 reprocess D 90 prvs=3950219457=mohamadzarifi@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <01af01cea1f8$069b1070$13d13150$@tm.net.my> mailsrv
26-Aug-2013 09:32:34.81 19c0.2c61ff.9421218 tcp_fromvirus reprocess E 36 noradila@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <58AF33FACEC1B7409319333B74FA8E2B05F413C5@SMSGVS32.tm.my> mailsrv pp-inbound.tm.net.my ([192.168.10.85])
26-Aug-2013 09:32:35.48 68ff.1.112 reprocess D 37 noradila@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <58AF33FACEC1B7409319333B74FA8E2B05F413C5@SMSGVS32.tm.my> mailsrv
26-Aug-2013 11:04:51.87 19c0.2c9fab.9473076 tcp_fromvirus reprocess E 40 nurhanani@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <02F70B247ED17F48B8889E578A3F066008300942@SMSGVS33.tm.my> mailsrv pp-inbound.tm.net.my ([192.168.10.91])
26-Aug-2013 11:04:52.42 68ff.1.508 reprocess D 40 nurhanani@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <02F70B247ED17F48B8889E578A3F066008300942@SMSGVS33.tm.my> mailsrv
26-Aug-2013 11:30:50.55 19c0.2cb274.9488313 tcp_fromvirus reprocess E 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <8D26E305AED74660975E208051D72F2F@B12562PCTM01> mailsrv pp-inbound.tm.net.my ([192.168.10.84])
26-Aug-2013 11:30:51.11 6db5.1.54 reprocess D 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <8D26E305AED74660975E208051D72F2F@B12562PCTM01> mailsrv
26-Aug-2013 11:35:00.89 19c0.2cb636.9491404 tcp_fromvirus reprocess E 19 mdkamarulfadzli@vads.com rfc822;mailadm@tm.net.my mailadm@tm.net.my <D0E896F4E4B10E40B4375570AE94878108B57C6C@SMSGVS21.tm.my> mailsrv pp-inbound.tm.net.my ([192.168.10.89])
26-Aug-2013 11:35:01.20 6db5.1.60 reprocess D 20 mdkamarulfadzli@vads.com rfc822;mailadm@tm.net.my mailadm@tm.net.my <D0E896F4E4B10E40B4375570AE94878108B57C6C@SMSGVS21.tm.my> mailsrv
26-Aug-2013 11:48:12.15 19c0.2cbf0e.9498526 tcp_fromvirus reprocess E 6 nurhanani@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <02F70B247ED17F48B8889E578A3F066008300996@SMSGVS33.tm.my> mailsrv pp-inbound.tm.net.my ([192.168.10.85])
26-Aug-2013 11:48:12.53 6db5.1.97 reprocess D 6 nurhanani@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <02F70B247ED17F48B8889E578A3F066008300996@SMSGVS33.tm.my> mailsrv
26-Aug-2013 12:36:03.61 19c0.2ce165.9528027 tcp_fromvirus reprocess E 992 prvs=3950303a3a=adlinothman@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <521ABDDD.9070701@tm.net.my> mailsrv pp-inbound.tm.net.my ([192.168.10.89])
26-Aug-2013 12:36:05.60 6db5.1.246 reprocess D 992 prvs=3950303a3a=adlinothman@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <521ABDDD.9070701@tm.net.my> mailsrv
note: original structure of other files that need to be search is similar to the output at c.out
Have you tried something like:
gzcat mail.log.*.gz | cat - mail.log*[!g][!z] | grep -f a.out > c.out
1 Like
What about zgrep?
(untested):
zgrep -f a.out maillog.*.gz > c.out
Mr_47
4
yup similar idea but cant use grep -f
grep: illegal option -- f
Usage: grep -hblcnsviw pattern file . . .
Try /usr/xpg4/bin/grep instead
1 Like
Mr_47
6
tested to remove -f, its running but result seems not right
gzcat mail.log.*.gz | cat - mail.log*[!g][!z] | grep a.out > c.out
cat: cannot open mail.log*[!g][!z]
[root] ls
a.out c.out mail.log.02072012.gz mail.log.10092008.gz mail.log.16082013.gz mail.log_current.gz
[root|mx1.tm.net.my:/jes/SUNWmsgsr/log/zz] more c.out
02-Jul-2012 00:34:43.13 3e31.905d6e.32070899 tcp_fromvirus tcp_lmtpcs E 25 madabout@vps.madaboutdeal.sg rfc822;Innoipoh@tm.net.my @mstore2.secure.tmnet.lmtp:innoi
poh@lmtpcs-daemon <E1SlN6n-00039U-QG@vps.madaboutdeal.sg> mailsrv pp-inbound.tm.net.my ([192.168.10.91])
02-Jul-2012 00:34:43.54 4c25.75.387 tcp_lmtpcs D 25 madabout@vps.madaboutdeal.sg rfc822;Innoipoh@tm.net.my @mstore2.secure.tmnet.lmtp:innoipoh@lmtpcs
-daemon <E1SlN6n-00039U-QG@vps.madaboutdeal.sg> mailsrv mstore2.secure.tmnet dns;mstore2.secure.tmnet (TCP|192.168.10.100|42012|192.168.10.150|225) (mail1.secure.tm
net -- Server LMTP [Sun ONE Messaging Server 6.2-8.04 [built Feb 28 2007]]) smtp;250 2.1.5 innoipoh@lmtpcs-daemon and options OK.
02-Jul-2012 02:13:15.61 3e31.907710.32093863 tcp_fromvirus tcp_lmtpcs E 10 elviracovaultrp93@yahoo.com rfc822;acoutech@tm.net.my @mstore2.secure.tmnet.lmtp:acoute
ch@lmtpcs-daemon <1341166392.79347.androidMobile@web140501.mail.bf1.yahoo.com> mailsrv pp-inbound.tm.net.my ([192.168.10.89])
02-Jul-2012 02:13:15.67 4f50.93.483 tcp_lmtpcs R 10 elviracovaultrp93@yahoo.com rfc822;acoutech@tm.net.my @mstore2.secure.tmnet.lmtp:acoutech@lmtpcs-
daemon <1341166392.79347.androidMobile@web140501.mail.bf1.yahoo.com> mailsrv mstore2.secure.tmnet dns;mstore2.secure.tmnet (TCP|192.168.10.100|49067|192.168.10.150|
225) (mail1.secure.tmnet -- Server LMTP [Sun ONE Messaging Server 6.2-8.04 [built Feb 28 2007]]) lmtp;550 5.2.2 Delivery failed: Over quota
02-Jul-2012 05:37:04.63 3e31.90971b.32122534 tcp_fromvirus tcp_lmtpcs E 57 pabounce@gsol.globalsources.com rfc822;acoutech@tm.net.my @mstore2.secure.tmnet.lmtp:ac
outech@lmtpcs-daemon <1146025520.1341178620109.JavaMail.web@turquoise1.globalsources.com> mailsrv pp-inbound.tm.net.my ([192.168.10.90])
02-Jul-2012 05:37:04.72 55e7.0d2.677 tcp_lmtpcs R 57 pabounce@gsol.globalsources.com rfc822;acoutech@tm.net.my @mstore2.secure.tmnet.lmtp:acoutech@lm
tpcs-daemon <1146025520.1341178620109.JavaMail.web@turquoise1.globalsources.com> mailsrv mstore2.secure.tmnet dns;mstore2.secure.tmnet (TCP|192.168.10.100|58432|192
.168.10.150|225) (mail1.secure.tmnet -- Server LMTP [Sun ONE Messaging Server 6.2-8.04 [built Feb 28 2007]]) lmtp;550 5.2.2 Delivery failed: Over quota
02-Jul-2012 06:00:30.54 3e31.909a28.32125129 tcp_fromvirus tcp_lmtpcs E 15 email@mail-greatoutdoors.com rfc822;rkslim@streamyx.com @mstore1.secure.tmnet.lmtp:rksl
im%streamyx.com@lmtpcs-daemon <GreatOutdoors_com.6436prxv9z2.fp5f@mail-greatoutdoors.com> mailsrv pp-inbound.tm.net.my ([192.168.10.89])
02-Jul-2012 06:00:30.59 56aa.0b0.562 tcp_lmtpcs D 15 email@mail-greatoutdoors.com rfc822;rkslim@streamyx.com @mstore1.secure.tmnet.lmtp:rkslim%stream
yx.com@lmtpcs-daemon <GreatOutdoors_com.6436prxv9z2.fp5f@mail-greatoutdoors.com> mailsrv mstore1.secure.tmnet dns;mstore1.secure.tmnet (TCP|192.168.10.100|59372|192
.168.10.146|225) (mail2.secure.tmnet -- Server LMTP [Sun ONE Messaging Server 6.2-8.04 [built Feb 28 2007]]) smtp;250 2.1.5 rkslim%streamyx.com@lmtpcs-daemon and op
tions OK.
---------- Post updated at 03:59 PM ---------- Previous update was at 03:58 PM ----------
zgrep -f a.out maillog.*.gz > c.out
ksh: zgrep: not found
You need -f to do this. /usr/xpg4/bin/grep has a -f option.
In your example there were also:
-rw-r--r-- 1 mailsrv mail 189438124 Nov 2 2012 mail.log.31102012
-rw------- 1 mailsrv mail 76248835 Aug 26 12:43 mail.log_current
unzipped files that also needed to be searched, hence the cat construct...
If they are not there, you could simply try:
gzcat mail.log.*.gz | /usr/xpg4/bin/grep -f a.out > c.out
1 Like
Try gzgrep instead of zgrep
Mr_47
9
fully tested, sorry wasn't aware of the non gz file. ( thanks for reminding)
now it perfectly work and most importantly I learnt something today tq again Gurus here!
[root|] gzcat mail.log.*.gz | cat - mail.log*[!g][!z] | /usr/xpg4/bin/grep -f a.out > c.out
[root] du -sh *
1K a.out
15K c.out
18M mail.log.02072012.gz
94M mail.log.10092008.gz
19M mail.log.16082013.gz
97M mail.log_current
[root] more c.out
02-Jul-2012 09:47:57.80 3e31.90fdd0.32211442 tcp_fromvirus reprocess E 4 prvs=0530c4a896=adlinothman@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <4FF0FD
EC.2090906@tm.net.my> mailsrv pp-inbound.tm.net.my ([192.168.10.90])
02-Jul-2012 09:47:57.89 5d62.1.99 reprocess D 5 prvs=0530c4a896=adlinothman@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <4FF0FDEC.2090906@t
m.net.my> mailsrv
.
.
.
26-Aug-2013 12:36:05.60 6db5.1.246 reprocess D 992 prvs=3950303a3a=adlinothman@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <521ABDDD.907070
1@tm.net.my> mailsrv
26-Aug-2013 12:47:51.78 19c0.2ce850.9533558 tcp_fromvirus reprocess E 5 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <3218DC26A184
48FA9A3986F282E30370@B12562PCTM01> mailsrv pp-inbound.tm.net.my ([192.168.10.90])
26-Aug-2013 12:47:52.17 6db5.1.270 reprocess D 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <3218DC26A18448FA9A3986
F282E30370@B12562PCTM01> mailsrv
26-Aug-2013 12:56:28.05 19c0.2cecf3.9538437 tcp_fromvirus reprocess E 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <D45E34DD7994
452DA2BD89F2F33348FC@B12562PCTM01> mailsrv pp-inbound.tm.net.my ([192.168.10.90])
26-Aug-2013 12:56:28.43 6db5.1.290 reprocess D 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <D45E34DD7994452DA2BD89
F2F33348FC@B12562PCTM01> mailsrv
26-Aug-2013 14:10:38.06 19c0.2d14a9.9571984 tcp_fromvirus reprocess E 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <FFF520403FF9
4D99BFB9131E27F38A9F@B12562PCTM01> mailsrv pp-inbound.tm.net.my ([192.168.10.90])
26-Aug-2013 14:10:38.55 73c8.1.48 reprocess D 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <FFF520403FF94D99BFB9131
E27F38A9F@B12562PCTM01> mailsrv
26-Aug-2013 14:19:28.03 19c0.2d19a3.9575905 tcp_fromvirus reprocess E 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <1C43C9DD2B40
4E7BA46D9D3DD4D09074@B12562PCTM01> mailsrv pp-inbound.tm.net.my ([192.168.10.89])
26-Aug-2013 14:19:28.44 73c8.1.84 reprocess D 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <1C43C9DD2B404E7BA46D9D3
DD4D09074@B12562PCTM01> mailsrv
26-Aug-2013 14:27:23.15 19c0.2d1d59.9578921 tcp_fromvirus reprocess E 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <F7D67C7674D6
433EB0D124AC7BB6A32F@B12562PCTM01> mailsrv pp-inbound.tm.net.my ([192.168.10.84])
26-Aug-2013 14:27:23.66 21.1.18 reprocess D 6 prvs=3950b1d694=tmcops@tm.net.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <F7D67C7674D6433EB0D124AC7
BB6A32F@B12562PCTM01> mailsrv
26-Aug-2013 15:26:49.35 19c0.2d3fb7.9607210 tcp_fromvirus reprocess E 150 bakarh@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <02035C11A44A5E44B5C11A00BD
4DB8C9076F90FD@SMSGVS32.tm.my> mailsrv pp-inbound.tm.net.my ([192.168.10.85])
26-Aug-2013 15:26:50.36 21d.1.4 reprocess D 151 bakarh@tm.com.my rfc822;mailadm@tm.net.my mailadm@tm.net.my <02035C11A44A5E44B5C11A00BD4DB8C9076F90F
D@SMSGVS32.tm.my> mailsrv
[root|mx1.tm.net.my:/jes/SUNWmsgsr/log/zz]