Help require for vLookup Utility same as in EXCEL

Hi guys,

can someone please help me with a vlookup utility in shell script that does the work similar to Excel.

Ex:
If File1.txt is

1
2
11
4
12

and File2.txt is

1 tab1
2 tab2
3 tab3
4 tab4
5 tab5

then the utility should create File3.txt with the below mentioned output:

1 tab1
2 tab2
11
4 tab4
12

the separator between columns will be tab(\t).

Also, is it possible, if File1.txt and File2.txt has multiple columns then, the lookup should happen from Column 3 of File1.txt to Column 6 of File2.txt and return Column 9 in File3.txt

Please help

From the above examples you could try:-

#!/bin/ksh
while read f1
do
   f2line=`grep "^${f1}tab" File2.txt`
   f2="${f2line#*tab}"
   echo "${f1}tab${f2}"
done < File1.txt > File3.txt

That might get you started, but the follow-up question is a bit harder.

Robin
Liverpool/Blackburn
UK


For the follow up question, consider:-

```text
#!/bin/ksh
cut -f3 -d "tab" File1.txt | while read f1
do
   f2line=`cut -f6,9 -d "tab" File2.txt|grep "^${f1}tab"
   f2="${f2line#*tab}"
   echo "${f1}tab${f2}"
done > File3.txt
```



Does that do the job?

These may be a little slow as they are running a while read loop.  Perhaps someone can suggest a good awk that will run faster.  You will notice the difference with larger files as that may read the files once where my suggestion will re-read File2.txt for every line in File1.txt

Robin
1 Like

For your first request, try

$ awk 'FNR==NR{Ar[$1]=$0;next} $1 in Ar {$0=Ar[$1]}1' FS="\t" OFS="\t" file2 file1
1    tab1
2    tab2
11
4    tab4
12

For your second, pls post samples, as I don't quite understand it.

1 Like

Thans a ton Rudi.....this works...

For 2nd request, please find below the scenario:

File1.txt:

nu1    eu11    10
nu2    eu12    2
nu3    eu13    14
nu4    eu14    4
nu5    eu15    9
nu6    eu16    0
nu7    eu17    7
nu8    eu18    12
nu9    eu19    9
nu10    eu20    1

File2.txt:

ea1    1    dc1    ek1    vivek1
ea2    2    dc2    ek2    vivek2
ea3    11    dc3    ek3    vivek3
ea4    4    dc4    ek4    vivek4
ea5    5    dc5    ek5    vivek5
ea6    6    dc6    ek6    vivek6
ea7    7    dc7    ek7    vivek7
ea8    12    dc8    ek8    vivek8
ea9    9    dc9    ek9    vivek9
ea10    10    dc10    ek10    vivek10

Expected Output, File3.txt:

nu1    eu11    10    vivek10
nu2    eu12    2    vivek2
nu3    eu13    14    
nu4    eu14    4    vivek4
nu5    eu15    9    vivek9
nu6    eu16    0    
nu7    eu17    7    vivek7
nu8    eu18    12    vivek8
nu9    eu19    9    vivek9
nu10    eu20    1    vivek1

Lookup 3rd column in File1.txt against 2nd column in File2.txt and vlookup the value of 5th column in the output against the same

Please refer attachment fpr sample files

Thanks

I think your formatting has gone a bit wild when you've pasted the file contents in. Could you have another go. Can you wrap them with Code Tags too.

Does my follow up code above work perchance? If not, how badly is it wrong?

Robin

---------- Post updated at 03:17 PM ---------- Previous update was at 03:15 PM ----------

Oops, crossed postings. I will have a look-see.

Robin

---------- Post updated at 03:23 PM ---------- Previous update was at 03:17 PM ----------

Syntax error and then no matching output. I will think again.

Sorry.

---------- Post updated at 03:30 PM ---------- Previous update was at 03:23 PM ----------

Syntax error corrected, but no hits. Your input file File2.txt does not have anything in columns 6 & 9 to work with.

My code is now:-

cut -f3 -d "tab" File1.txt | while read f1
do
   f2line=`cut -f6,9 -d "tab" File2.txt|grep "^${f1}tab"`
   f2="${f2line#*tab}"
   echo "${f1}tab${f2}"
done > File3.txt

... and seems to work when I add more text on each line in File2.txt

Have I mis-interpreted your needs?

Robin

Hi Robin...

i am getting the following error while executing the second code...

line 4: unexpected EOF while looking for matching ``'
line 8: syntax error: unexpected end of file

pls help

---------- Post updated at 09:02 PM ---------- Previous update was at 09:01 PM ----------

hi robin,

there is something wrong going in the formatting...can you please download the attached files for samples...

thanks

---------- Post updated at 09:05 PM ---------- Previous update was at 09:02 PM ----------

im getting this error:

line 6: syntax error near unexpected token `done'
line 6: `done > File3.txt'

I will attach my script just in case it's the copy/paste through the thread.

Robin

Still not:

line 7: syntax error near unexpected token `done'
line 7: `done > File3b.txt

---------- Post updated at 09:28 PM ---------- Previous update was at 09:25 PM ----------

Still not getting through:

line 7: syntax error near unexpected token `done'
line 7: `done > File3b.txt

Very odd. It works fine for me on AIX 4.3.3, 5.1 (yes I know they're pretty old!) & 6.1 and also on RH Linux 6. Did you save the attachment and FTP it to the server, or copy & paste the contents?

Robin

im running this through Cygwin

Well, I think that is just an X-windows server. What is the underlying Unix server you are using?

Did you just paste the contents of my file into the screen or did you save them, FTP them to the server and then run them as a script?

Robin

i saved the content into another text file, renamed it as .sh and then executed it on Windows-machine through Cygwin...i am using cygwin as i do not have any linux machine to execute...:frowning:

Could I direct you to look at the Cygwin home page here

The section that states what it isn't makes me worry that we're trying to do something that Cygwin is not designed for. You might well be better with a DOS batch file, an Excel Macro or some Visual Basic code etc.

If anyone wants to leap in and correct me, feel free as I am happy to learn.

Robin

Me2....however i am using Cygwin for all my shell scripts and it is working fine....let me see if i can get a linux server from my friend...or will ask him to test their...whether it works...

alternately if someone has some suggestion to this problem....pls WELCOME..

also if someone can help me with a different code for my second request....that would be great... :slight_smile:

thanks

@vivekgupta: use vi and you will see the problem: ^M characters
correct it with dos2unix command

@rbatte1: the forums think the world is a PC even though we are UNIX. When I copied and pasted in UNIX, voila! DOS carriage returns. Your code is just fine - assuming the OP actually cut and pasted it and removed ^M characters.

1 Like

Thanks JIM.....it did really helped.....thanks a ton !!! :slight_smile:

---------- Post updated at 11:49 PM ---------- Previous update was at 11:43 PM ----------

THANKS A TON FOR ALL OF YOU HELPING ME RESOLVE THIS PROBLEM...

RUDI

JIM

ROBIN

THANKS A TON.... :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile:

---------- Post updated 01-05-13 at 09:54 AM ---------- Previous update was 01-04-13 at 11:49 PM ----------

Hi Robin,

the output that i am getting frm ur script my second request is:

10    vivek10
2    vivek2
14    
4    vivek4
9    vivek9
0    
7    vivek7
12    vivek8
9    vivek9
1    vivek1

I need the output in the below mentioned format, the contents of File1.txt should be as it is, only the last column should be appended to the output file File3.txt with the looked up value...

nu1    eu11    10    vivek10
nu2    eu12    2    vivek2
nu3    eu13    14    
nu4    eu14    4    vivek4
nu5    eu15    9    vivek9
nu6    eu16    0    
nu7    eu17    7    vivek7
nu8    eu18    12    vivek8
nu9    eu19    9    vivek9
nu10    eu20    1    vivek1

can u pls look into this and help...

thanks...

Sorry for my late reply - you can adapt my earlier proposal easily. Try

$ awk 'FNR==NR{Ar[$2]=$5;next} $3 in Ar {$0=$0 FS Ar[$3]}1' FS="\t" OFS="\t" file2 file1
nu1    eu11    10   vivek10
nu2    eu12    2    vivek2
nu3    eu13    14
nu4    eu14    4    vivek4
nu5    eu15    9    vivek9
nu6    eu16    0
nu7    eu17    7    vivek7
nu8    eu18    12   vivek8
nu9    eu19    9    vivek9
nu10   eu20    1    vivek1
1 Like

This is working as it should be...THANKS A TON.... :slight_smile:

---------- Post updated at 05:57 PM ---------- Previous update was at 05:14 PM ----------

Hi Rudi....with the current code we would be able to fetch single column value....if we need to fetch multiple column for both the requirement, how do we go about it....can you please help....?

With a small tweak, I have the following for putting the whole of the line from File1.txt in the output:-

#!/bin/ksh

while read f1line
do
   f1="${f1line##*tab}"
   f2line=`cut -f6,9 -d "tab" File2.txt|grep "^${f1}tab"`
   f2="${f2line#*tab}"
   echo "${f1line}tab${f2}"
done < File1.txt > File3b.txt

Is that better?

Robin

---------- Post updated at 12:51 PM ---------- Previous update was at 12:48 PM ----------

Sorry, I'd been away from my desk with the reply all keyed, so I pressed submit to find a follow up!

Okay, well, it depends what multiple column values you are after. If it's easy to say that columns 2 & 3 in File1.txt must match columns 6 & 7 in File2.txt then we could work on that. Can you describe the rules you need applied?

Thanks,
Robin

so File1.txt is:

nu1    eu11    10
nu2    eu12    2
nu3    eu13    14
nu4    eu14    4
nu5    eu15    9
nu6    eu16    0
nu7    eu17    7
nu8    eu18    12
nu9    eu19    9
nu10    eu20    1

File2.txt is:

ea1    1    dc1    ek1    vivek1    vv1    ww1
ea2    2    dc2    ek2    vivek2    vv2    ww2
ea3    11    dc3    ek3    vivek3    vv3    ww3
ea4    4    dc4    ek4    vivek4    vv4    ww4
ea5    5    dc5    ek5    vivek5    vv5    ww5
ea6    6    dc6    ek6    vivek6    vv6    ww6
ea7    7    dc7    ek7    vivek7    vv7    ww7
ea8    12    dc8    ek8    vivek8    vv8    ww8
ea9    9    dc9    ek9    vivek9    vv9    ww9
ea10    10    dc10    ek10    vivek10    vv10    ww10

the output should be File3.txt:

nu1    eu11    10    vivek10    ww10    kk10
nu2    eu12    2    vivek2    ww2    kk2
nu3    eu13    14            
nu4    eu14    4    vivek4    ww4    kk4
nu5    eu15    9    vivek9    ww9    kk9
nu6    eu16    0            
nu7    eu17    7    vivek7    ww7    kk7
nu8    eu18    12    vivek8    ww8    kk8
nu9    eu19    9    vivek9    ww9    kk9
nu10    eu20    1    vivek1    ww1    kk1

Please find attached the files for sample and reference...

So what we did earlier is lookup column 3 in File1.txt in Column 2 in File2.txt and extracted Column 5 from File2.txt...now on same condition extract Column5 and Column 7 in the output file...so multiple column to be extracted instead of one from File2.txt and the output should be as available in the attached File3.txt