I am reading a value from a file and want to store the value in a dynamic array as i don't know the number of occurrences of the value in that file.
How can i do that and then later fetch that value from array
You do it straightforward. An array is just another variable after all and variables in shell scripts can be created just by using them.
Alas you haven't said which shell you are using, so as your penance for omitting this vital information i just suppose you are using Korn shell (ksh) and if you don't you will have to translate my solution to your shell yourself. ;-))
Suppose the following input file. We want all the lines "data=<value>" and store the "<value>" in an indexed array (1-based):
# this is a test file
lalala
data=first array element
some more meaningless text
data=second array element
data=third array element
# we do not want this line to do anything
data=last array element
foo bar
Now the script, which will read it and put the content into an array, like stated above:
#! /bin/ksh
# first some declarations
typeset -i Index=0
typeset Line=""
typeset Input="/path/to/file.above"
# At first we fill the array:
(( Index = 1 )) # we want to start our indices with 1
grep "^data=" "$Input" | while read Line ; do
typeset Array[$Index]="${Line##data=}" # split off "data=", leaving value
(( Index += 1 )) # increment index
done
# now let us try to display the array contents:
(( Index = 1 )) # reset our array index
# ${#Array[*]} evaluates to the number of elements in the array
while [ $Index -le ${#Array[*]} ] ; do
print - "display array index ${Index}:\t ${Array[$Index]}"
(( Index += 1 ))
done
exit 0
I hope this helps.
bakunin
Thanku for the reply..It was helpful
I am facing another issues
I have two arrays:
Taskid
comp
I want to fetch values from file based on values in these arrays. My file name is Sri1.log
I have written this code:
for var in "${Taskid[@]}"
do
for var1 in "${comp[@]}"
do
while read line
do
if[ grep -w 'Task started' && $var1]
then
cut -d"|" -f1 Sri1.log >> time.txt
fi
done< Sri1.log
done
done
But gives the error
syntax error near unexpected token 'then'
Please guide me to resolve this
There must be a space on both side of "[" and "]"
Inner commands should be first executed, $(..) or `...`
Also, when using && constructs, you should probably use ..
if [ "$(grep -w 'Task started')" ] && [ $var1 ]
then
cut -d"|" -f1 Sri1.log >> time.txt
fi
Thanku it worked
but
it is giving error
[: too many arguments
[: binary operator expected
[: unary operator expected
Can you post the updated version of script/code ?
Thanx for the help..I worked but currently I am stuck in
I want only those lines from the file which have value in array and keyword "Task started"
i have written the following code
for var1 in "${comp[@]}"
do
while read line
do
if [ " $(echo $line | grep -w 'Task started') " ] && [ " $(echo $line | grep "$var1") " ]
then
echo $line >>time1.txt
fi
done< Sri1.log
echo 'calculating time' >> time1.txt
done
but its not working as it is giving me all the lines from the file
Please help in this
Ok. You have series of loops over loops which might be inefficient.
From the post #3, I came to know that there is one more array.
We might help you in a better way if you could tell us the actual requirement with examples.
I guessed something like.. You want to search in a file for records from couple of files including the word "Task started".
Please give us the sample data from all the files and final output you require.
Also, The O/S and Shell version you are using.
I have a file which is seperated by pipe "|" operator
and I want to fetch those lines from the file which has the
the keyword "task started" and the value which i have stored in the array
A line from the file
2012-05-09 10:29:14.285 | SYNC | 00050 | DEBUG | Customer_Rece | Exchange | Task started | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000003 | xengine:CERR to ECR CF | {previous-component=Validation,operties={CSystemID=MS, BSystemID=A1F8D828-15, RId=IUxBNzKlZfghcMxf101209090000024_1, ParserSchema=/apphema_CERR.ecs, IsCorrection=false},}
The value in my array comp is
xengine:CERR to ECR CF
since this line has 'Task started' and value in array
the output required is
10:29:14.285 xengine:CERR to ECR CF Customer_Rece
Won't this work for you?
# Create a file containing the patterns. (I think you already having that)
for var1 in "${comp[@]}"
do
echo $var1
done > pattern.list
grep -w "Task started" Sri1.log | grep -f pattern.list
data
$ cat pattern.list
xengine:CERR to ECR CF
xengine:CERR to ECR FF
xengine:CERR to ECR RF
$ cat Sri1.log
2012-05-09 10:29:14.285 | SYNC | 00050 | DEBUG | Customer_Rece | Exchange | Task started | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000003 | xengine:CERR to ECR CF | {previous-component=Validation,operties={CSystemID=MS, BSystemID=A1F8D828-15, RId=IUxBNzKlZfghcMxf101209090000024_1, ParserSchema=/apphema_CERR.ecs, IsCorrection=false},}
2012-05-09 10:29:14.285 | SYNC | 00050 | DEBUG | Customer_Rece | Exchange | Task going on | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000003 | xengine:CERR to ECR CF | {previous-component=Validation,operties={CSystemID=MS, BSystemID=A1F8D828-15, RId=IUxBNzKlZfghcMxf101209090000024_1, ParserSchema=/apphema_CERR.ecs, IsCorrection=false},}
2012-05-09 10:29:14.285 | SYNC | 00050 | DEBUG | Customer_Rece | Exchange | Task started | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000003 | XXXXXXX:CERR to ECR CF | {previous-component=Validation,operties={CSystemID=MS, BSystemID=A1F8D828-15, RId=IUxBNzKlZfghcMxf101209090000024_1, ParserSchema=/apphema_CERR.ecs, IsCorrection=false},}
2012-05-09 10:29:14.285 | SYNC | 00050 | DEBUG | Customer_Rece | Exchange | Task ended | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000003 | XXXXXXX:CERR to ECR CF | {previous-component=Validation,operties={CSystemID=MS, BSystemID=A1F8D828-15, RId=IUxBNzKlZfghcMxf101209090000024_1, ParserSchema=/apphema_CERR.ecs, IsCorrection=false},}
Given clx's files sri1.log and pattern.list, would this be an option for you:
while read var; do grep -Ei "task started.*$var" sri1.log | cut -d"|" -f1,5,9; done <pattern.list
resulting in
2012-05-09 10:29:14.285 | Customer_Rece | xengine:CERR to ECR CF
I tried the option given but its not taking the keyword "Task Started".
Can you give another alternate for this?
Are you sure to grep the exact case (upper/lower)?
I can see it varies with "Tast Started", "task started" or "Tast started". Just check and use the correct string to match.
If you are not sure about the case and want to do a case-independent search,
use "grep -i".
grep -iw "Task started" Sri1.log | grep -f pattern.list
also try
grep -i "Task started" Sri1.log | grep -f pattern.list
I have tried all the options but its not working.
The file pattern.list has:
B678C56D-96DA-4FFC-B40E-9A032A2EB12E| xengine:CERR to ECR CF
B678C56D-96DA-4FFC-B40E-9A032A2EB12E| service.adapter:UCF Version Creator
4ECF997A-BA1A-4497-B48F-6CA27414567A| service.adapter:UCF Writer_2
The file Sri1.log has:
2012-05-09 10:29:14.284 | SYNC | 00041 | DEBUG | Customer_Receive Payment | Task | Task started on a component | B678C56D-96DA-4FFC-B40E-9A032A2EB12E | xengine:CERR to ECR CF | {}
2012-05-09 10:29:14.285 | SYNC | 00050 | DEBUG | Customer_Receive Payment | Exchange | Exchange sent | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000003 | xengine:CERR to ECR CF | {previous-component=Validation,{ClaimS RemittanceId=IUxBNzKlZfghcMxf101209090000024_1}
2012-05-09 10:29:14.285 | SYNC | 00051 | DEBUG | Customer_Receive Payment | Exchange | Exchange processing started | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000003 | xengine:CERR to ECR CF | {previous-component=Validation, Split, exchange-properties={ClaimSystemID= RemittanceId=IUxBNzKlZfghcMxf101209090000024_1}
2012-05-09 10:29:28.677 | SYNC | 00055 | DEBUG | Customer_Receive Payment | Exchange | Exchange created | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000004 | xengine:CERR to ECR CF | {previous-component=Validation, Split, exchange-properties={BAMSystem, RemittanceId=IUxBNzKlZfghcMxf101209090000024_1}, output-name=Split Good}
2012-05-09 10:29:28.679 | SYNC | 00041 | DEBUG | Customer_Receive Payment | Task | Task started on a component | B678C56D-96DA-4FFC-B40E-9A032A2EB12E | service.adapter:UCF Version Creator | {}
2012-05-09 10:29:28.680 | SYNC | 00050 | DEBUG | Customer_Receive Payment | Exchange | Exchange sent | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000004 | service.adapter:UCF Version Creator | {previous RemittanceId=IUxBNzKlZfghcMxf101209090000024_1}
2012-05-09 10:29:28.680 | SYNC | 00051 | DEBUG | Customer_Receive Payment | Exchange | Task started | B678C56D-96DA-4FFC-B40E-9A032A2EB12E-0000004 | service.adapter:UCF Version Creator | {previous-component=CERR to ECR CF, exchange-properties={BAMSystemID=A1F8D828-15D0-4DE2-8B6E-672BD8, RemittanceId=IUxBNzKlZfghcMxf101209090000024_1}
2012-05-09 10:32:05.439 | SYNC | 00040 | INFO | EUO_Payment Validation | Task | Task started | C3EDAE4F-2F73-402F-AABE-1C3B9FEC4314 | subroute.endpoint:Payment Split Completion | {}
2012-05-09 10:32:05.451 | SYNC | 00050 | DEBUG | EUO_Payment Validation | Exchange | Exchange sent | 4ECF997A-BA1A-4497-B48F-6CA27414567A-0000006 | service.adapter:UCF Writer_2 | {}
I want lines having the word "Task started" and the value stored in the file pattern.list
I tried the following code :
while read var
do
grep -iw "Task started" Sri1.log | grep -f pattern.list | cut -d"|" -f1,5,9
done <pattern.list
the output its giving is
2012-05-09 10:29:14.284 | Customer_Receive Payment | xengine:CERR to ECR CF
2012-05-09 10:29:14.285 | Customer_Receive Payment | xengine:CERR to ECR CF
2012-05-09 10:29:28.677 | Customer_Receive Payment | xengine:CERR to ECR CF
2012-05-09 10:29:28.679 | Customer_Receive Payment | service.adapter:UCF Version Creator
2012-05-09 10:29:28.680 | Customer_Receive Payment | service.adapter:UCF Version Creator
but the output should be:
2012-05-09 10:29:28.680 | Customer_Receive Payment | service.adapter:UCF Version Creator
2012-05-09 10:32:05.439 | EUO_Payment Validation |subroute.endpoint:Payment Split Completion
Could you provide the results/error messages? As clx points out, the -i option to grep makes case irrelevant, so it should find the keywords regardless of case. Even different white space around "task started" does not matter...
You can even leave out the "E"xtended regexp option in my proposal, it's redundant.
---------- Post updated at 12:44 PM ---------- Previous update was at 12:10 PM ----------
To me, the problem seems to be: your entries in pattern.list do NOT match the lines in sri1.log:
pattern.list:
B678C56D-96DA-4FFC-B40E-9A032A2EB12E| xengine:CERR to ECR CF
sri1.log:
B678C56D-96DA-4FFC-B40E-9A032A2EB12E | xengine:CERR to ECR CF
So you would have to adapt either... maybe allow for some regexp in pattern.list?
Do the spaces matter during grep??
According to me the problem is in matching the word "Task started"
Since i tried matching with some other pattern in pattern.list
Do we have any other method of doing that?
Yes. In your case, grep would try to search for the exact string you would pass with-in double quotes.
I you are not sure about the spaces and cases of the string, use the regular expression search. something like below..
grep -E '[Tt]ask +[Ss]tarted' ...
If the logs/text are generated by an automated process the text MUST be same always !!
I have multiple conditions to check.
So i am using different loops to check each condition
I have 3 arrays
Remitid
Taskid
comp
I want to check file Sri1.log if it has the values stored in these arrays one by one. Means firstly for 1 value of Remitid it has 1st value of Taskid and then 1st value of comp array and then fetch value from array which has word task started
The code I have written is
while var1 in "${Remitid[@]}"
do
for var2 in "${Taskid[@]}"
do
for var3 in "${comp[@]}"
do
while read var4
grep
done < Sri1.log
done
done
done
But i don't know what to write in "grep". How to use all variables along with "Task started"
Please help
Try this:
dropping the
while read var4 grep done < Sri1.log
in your last example use
grep -Ei "task started.*($var1|$var2|$var3)" sri1.log
plus cut'ting the respective fields you need. This proposal relies on "task started" being positioned left to all variable's data. If that does not work, please provide all details in your files, arrays, error msgs, what have you.
btw, your first "while" should be a "for"
I tried running the script but it is giving error
Unable to initialize device PRN
What does this mean??