Dealing with log files

charudpss · January 19, 2009, 10:06pm

Hi ,
My requirement is that i need to search for a number of strings in a log file and print them with line numbers.The search should be date wise.
The sample log file is :

Jan 17 02:45:34 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2648E: Message backed out to a queue; node 'com.morleyfm.eai.b
enchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowInput'. : UKBRKR1P_B
.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngine/ImbMqInp
utNode.cpp: 3477: ImbMqInputNode::eligibleForBackout: ComIbmMQInputNode: com/mor
leyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/eai/
common/GlobalInputSub#FCMComposite_1_1

Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2232E: Error detected whilst handling a previous error in node
 'com.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.Thro
w'. : UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlo
wEngine/ImbDataFlowNode.cpp: 387: ImbDataFlowNode::logExceptionList: ComIbmThrow
Node: com/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/
morleyfm/eai/common/GlobalInputSub#FCMComposite_1_5
Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2230E: Error detected whilst processing a message in node 'com
.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowEnvir
onment'. : UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/Da
taFlowEngine/PluginInterface/ImbJniNode.cpp: 892: ImbJniNode::evaluate: ComIbmJn
iNode: com/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com
/morleyfm/eai/common/GlobalInputSub#FCMComposite_1_2

Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2230E: Error detected whilst processing a message in node 'com
.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.Begin'. :
 UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngi
ne/PluginInterface/ImbJniNode.cpp: 902: ImbJniNode::evaluate: ComIbmJniNode: com
/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/
eai/common/GlobalInputSub#FCMComposite_1_8

Jan 17 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2230E: Error detected whilst processing a message in node 'com
.morleyfm.eai.benchmark.RetrieveBenchmarksFromODR.Global_Exception_Format_sub.EA
I.HUB.EXCEPTION.IN.ANY.WBI'. : UKBRKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a:
/build/S500_P/src/DataFlowEngine/ImbMqOutputNode.cpp: 849: ImbMqOutputNode::eval
uate: ComIbmMQOutputNode: com/morleyfm/eai/benchmark/RetrieveBenchmarksFromODR#F
CMComposite_1_5.Global_Exception_Format_sub#FCMComposite_1_1

Jan 18 02:47:43 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2628E: Exception condition detected on input node 'com.morleyf
m.eai.benchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowInput'. : UKB
RKR1P_B.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngine/I
mbMqInputNode.cpp: 2127: ImbMqInputNode::readQueue: ComIbmMQInputNode: com/morle
yfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/eai/co
mmon/GlobalInputSub#FCMComposite_1_1

Jan 18 02:47:44 srim6165 MQSIv500[22581]: [ID 702911 user.error] (UKBRKR1P_B.LZ_
BENCHMARKS)[30]BIP2648E: Message backed out to a queue; node 'com.morleyfm.eai.b
enchmark.RetrieveBenchmarksFromODR.EAI.BENCHMKS.WBI.WBI.FlowInput'. : UKBRKR1P_B
.40b370fc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowEngine/ImbMqInp
utNode.cpp: 3477: ImbMqInputNode::eligibleForBackout: ComIbmMQInputNode: com/mor
leyfm/eai/benchmark/RetrieveBenchmarksFromODR#FCMComposite_1_1.com/morleyfm/eai/
common/GlobalInputSub#FCMComposite_1_1

Jan 18 08:42:17 srim6165 MQSIv500[22576]: [ID 702911 user.error] (UKBRKR1P_B.ROU
TING2)[38]BIP2232E: Error detected whilst handling a previous error in node 'com
.morleyfm.eai.routers.JPMCentralGatewayHandler.EAI.COMMON_NON_SEQ.JPM.WBI.Throw'
. : UKBRKR1P_B.13e26cfc-1901-0000-0080-d51319f22e3a: /build/S500_P/src/DataFlowE
ngine/ImbDataFlowNode.cpp: 387: ImbDataFlowNode::logExceptionList: ComIbmThrowNo
de: com/morleyfm/eai/routers/JPMCentralGatewayHandler#FCMComposite_1_1.com/morle
yfm/eai/common/subflows/CommonInputSub#FCMComposite_1_5

Now i want to search that string for date Jan 17..How do I extract the contents of Jan 17 using shell script so that i can go ahead with the search ?

Thanks in advance !!

ynilesh · January 19, 2009, 11:01pm

You can grep for "Jan 17".

charudpss · January 19, 2009, 11:06pm

Hi ,
Thanks for the reply.But this will only give the first line containing Jan 17 from the log file.I need all the lines.

danmero · January 19, 2009, 11:39pm

Try awk.

awk '$1==month && $2!=day{f=0}$1==month && $2==day{f=1}f' month=Jan day=17 file

charudpss · January 20, 2009, 12:46am

Thanks danmero !!...
Can I pass the month and date as command line argument in this code as i will be using this awk in a shell script.

charudpss · January 20, 2009, 3:48am

Hi,
Thanks alot.The Problem is solved now.

ynilesh · January 20, 2009, 8:23am

cat log.file | grep -B 5 "Jan 17"

This will print below 5 lines of the search.

-nilesh