Grep number between Square [] brackets

I wanted to store the number inside the square bracket between colon( : ) and closing suqre bracket(]) in some variable.

Suppose I have lines like :

Input file :

 
20140320 00:08:23.846 INFO  [WebContainer : 84] - anything in line
20140320 00:08:23.846 Test  [translate : 589] - Virtual and lab lab anything
20140320 00:08:23.846 Data  [anything : 60] - anything in line
20140320 00:08:23.847 anyting  [anything : 5] - anything in line 
 

I need data in one variable to store all the values : 84, 589,60 and 5

Can anyone please help me

What have you tried so far?

Do you mean you want one long string or an array where you can address each element?

A few more questions:-

• How many records are we considering?
• What operating system version and level?
• What shell are you using?
• What are your preferred tools?

.... and to re-iterate anbu23's question, What have you tried so far?

Robin

1. I am trying to store this values in one variable like (temp = 84|589|60|5)
   Because I have hundreds or thousands of lines and don't know how many enries are there. So I am planning to give seperator (|) to store values.

2. In both SunOS and Linux systeam

3. I am using
   #!/bin/ksh

4. no tools directly in Linux and Sunos systeam

Thanks you

What are you going to do with this variable with hundreds of thousands of values separated by vertical bars in it once you get it? If you're going to process each value, you'd probably be much better off processing each value as you find it piped into a loop to process the values. Storing a set of values in a variable as a string that is about a quarter of a megabyte long and then processing that string doesn't seem like a good approach.

And, once more: What have you tried so far? If you refuse to answer this question, it will likely take a lot longer to get a resolution to your problem. We're here to help you learn how to use Linux and UNIX tools; not to write your code for you. And even more importantly, if you show us what you've done, we might get a much better idea of what you're trying to do!

temp=$( awk -F [\]\[] '{split($2, a, " : "); print a[2]}' ORS='|' input_file)

Thanks a lot SriniShoo

I tried to delete the duplicate entry and sorting using following code, but not getting. need help

temp=$(awk -F [\]\[] '{split($2, a, " : "); print a[2]}' ORS='|' $INPUTFILE  | sort -u)

---------- Post updated at 05:53 PM ---------- Previous update was at 05:40 PM ----------

Hi Don Cragun,

I have log file which is mixed of all users, so perticular user log not getting from common log file. I have already captured lines based on userid. But thread I wanted to get in one variable. So here thread value is in between squre brackets after colon ( : ).

Can you please explain deleting duplicate entries and sorting
How do you need the output

I am getting duplicate entry in variable temp for example temp=2|2|3|4|4.
I was trying to get the temp=2|3|4. So I added sort -u at last, but not getting. it is right way to get using sort -u or any other options are available.

temp=$(grep "nk1577" $INPUTFILE | awk -F [\]\[] '{split($2, a, " : "); print a[2]}' ORS='|' | sort -u)

temp=$(grep "nk1577" $INPUTFILE | awk -F [\]\[] '{split($2, a, " : "); b[a[2]]++} END {for(x in b) {a[n++] = x+0}; asort(a); for(i = 1; i <= n; i++) {print a}}' ORS='|')

No!
Sort is used to sort lines into order.
You requested that there be only one line of output, and the code samples that you have been given provide a single line of output as you requested; so sort will have no affect.

If you would have answered the questions I asked in message #5 in this thread, the volunteers here trying to help you might be able to help you find a solution to your problem instead of trying to patch together something that probably still won't meet your needs.

Please answer the questions I asked earlier so we can help you find a solution will meet your needs.

Longhand using __builtins__ on OSX 10.7.5, ksh, default terminal.

#/bin/ksh
# get_vals.sh
ifs_str="$IFS"
IFS="$IFS"'[]'
echo "20140320 00:08:23.846 INFO  [WebContainer : 84] - anything in line
20140320 00:08:23.846 Test  [translate : 589] - Virtual and lab lab anything
20140320 00:08:23.846 Data  [anything : 60] - anything in line
20140320 00:08:23.847 anyting  [anything : 5] - anything in line" > /tmp/file1
temp=""
text=""
while read line
do
	temp=($line)
	text=$text${temp[5]}'|'
done < /tmp/file1
echo "$text"
IFS="$ifs_str"
exit 0

Results:-

Last login: Tue Apr  1 09:03:46 on ttys000
AMIGA:barrywalker~> chmod 755 get_vals.sh
AMIGA:barrywalker~> ./get_vals.sh
84|589|60|5|
AMIGA:barrywalker~> _

EDIT:

Added text="" for fullness.

You can adjust it a little towards the end to have:-

:
:
:
done < /tmp/file1
text="${text%|}"
echo "$text"
:
:

This will trim off the trailing | if that is what is required.

Robin

Hi Don Cragun,

I have log file which is mixed of all users, so perticular user log not getting from common log file. I have already captured lines based on userid. But thread I wanted to get in one variable. So here thread value is in between squre brackets after colon ( : ).
[/quote]

You cannot usefully execute sort on a single record (i.e. your variable)

$ var="1|2|4|3|5"
$ echo $var | sort
1|2|4|3|5
$

You would need to put the sort in before building the variable up.

Have you tried the suggestion from wisecracker?

An alternate might be:-

cat get_vals.sh
#/bin/ksh
# get_vals.sh
ifs_str="$IFS"
IFS="$IFS"'[]'
echo "20140320 00:08:23.846 INFO  [WebContainer : 84] - anything in line
20140320 00:08:23.846 Test  [translate : 589] - Virtual and lab lab anything
20140320 00:08:23.846 Data  [anything : 60] - anything in line
20140320 00:08:23.847 anyting  [anything : 5] - anything in line" > /tmp/file1
temp=""
text=""
while read line
do
        temp=($line)
        echo ${temp[5]}            # Just put to standard output for sort and next loop
done < /tmp/file1 | sort -un | while read item
do
        text="${text}${item}|"
done

text="${text%\|}"
echo "$text"
IFS="$ifs_str"
exit 0

$ ./get_vals.sh
5|60|84|589

I repeat: What are you going to do with this variable with hundreds of thousands of values separated by vertical bars in it once you get it? Now that you've said you want to get rid of duplicates, maybe it is only thousands of elements in the list instead of hundreds of thousands, but it still seems like you are focusing on an output format that is going to make it hard for you to use the results!

After you answer that question, also answer the following:

1. Do you care if the values are sorted, or do you just want to be sure your list has no duplicates?
2. Is it OK if the list separator is a <newline> character instead of a vertical bar? (If not, explain why!)

If you refuse to answer these three simple questions, I feel that we are all wasting our time in this thread.

I just wanted no duplicate entry,So if we will do sort and then remove duplicate it will be easy.
Any seperator is fine.

Thanks for helping

The following seems pretty easy to me and doesn't need sort or uniq to get a list with no duplicates:

#!/bin/ksh
var=$(awk -F '( : )|[]]' '!($2 in a) {a[$2]; print $2}' Input)
printf 'var=%s\n' "$var"

If the file Input contains:

20140320 00:08:23.846 INFO  [WebContainer : 84] - anything in line
20140320 00:08:23.846 Test  [translate : 589] - Virtual and lab lab anything
20140320 00:08:23.846 Data  [anything : 60] - anything in line
20140320 00:08:23.847 anyting  [anything : 5] - anything in line
20140320 00:08:23.846 INFO  [WebContainer : 84] - anything in line
20140320 00:08:23.846 Test  [translate : 589] - Virtual and lab lab anything
20140320 00:08:23.846 Data  [anything : 60] - anything in line
20140320 00:08:23.847 anyting  [anything : 5] - anything in line

the output produced is:

var=84
589
60
5

I still question the need for storing a list of hundreds of thousands of values in a variable. In most cases it would be better to process the values one at a time as in:

awk -F '( : )|[]]' '!($2 in a) {a[$2]; print $2}' Input | while read num
do      ... do whatever you want with $num ...
done

Hi Don Cragun,

It is working.... Thank you very much.