Want to read data from a file name.txt and search it in another file and then matching...

ektubbe · January 31, 2012, 12:29pm

Hi Frnds...
I have an input file name.txt and another file named as source.. name.txt is having only one column and source is having around 25 columns...i need to read from name.txt line by line and search it in source file and then save the result in results file..
I have a rough idea about the same...that a loop should be taken for the same and it searching can be done by awk filter as source file is quite big...and then the output can be redirected to results..
can someone plzz help me out with this..

~Thanks
Priya...:wall:

frappa · January 31, 2012, 12:34pm

Hi,

if I understood what u're looking to accomplish:

for currline in `cat name.txt`
do
grep "${currline}" source >> result
done

you may test the procedure on a subset of the whole data to check if it's consistent with your needs

see ya
fra

ektubbe · January 31, 2012, 12:37pm

frappa:

Hi,

if I understood what u're looking to accomplish:
for currline in `cat name.txt`
do
grep "${currline}" source >> result
done
you may test the procedure on a subset of the whole data to check if it's consistent with your needs

see ya
fra

Thanks a lot buddy...but i need to read the file as well..also the file from which i need to search the data is quite big so grep may get fail...
i m stuck with the use of awk here... :wall:

gary_w · January 31, 2012, 12:57pm

ksh93 on Solaris:

#!/usr/dt/bin/dtksh

#  Declare constants
typeset -r RESULTSFILE=results
typeset -r INFILE=name.txt
typeset -r SOURCEFILE=source

#  Clear or create output file.
> $RESULTSFILE

while read line_in
do
  print "Search results for [${line_in}]:" >> $RESULTSFILE
  grep "$line_in" $SOURCEFILE >> $RESULTSFILE
done < $INFILE

exit 0

ektubbe · February 1, 2012, 1:36am

gary_w:

ksh93 on Solaris:

#!/usr/dt/bin/dtksh

#  Declare constants
typeset -r RESULTSFILE=results
typeset -r INFILE=name.txt
typeset -r SOURCEFILE=source

#  Clear or create output file.
> $RESULTSFILE

while read line_in
do
  print "Search results for [${line_in}]:" >> $RESULTSFILE
  grep "$line_in" $SOURCEFILE >> $RESULTSFILE
done < $INFILE

exit 0

hi buddy..thanks for the same...but it is not searching the msisdn i.e. numbers present in the name.txt file...i feel we need to use grep or awk for the same...plzz some1 help me out wid d same... :wall:

knight_eon · February 1, 2012, 2:14am

It will be great for us as well to see the Sample input file, data file and ur expected output. Like this, all can only give wild guesses and you will all the time end up banging your head on the wall :wall: like the way you are doing for past few posts :rolleyes:

ektubbe · February 1, 2012, 2:24am

Here is the data file name.txt :

msisdn
--------------
09311424354
09128734473
09487736472

source file :

msisdn             | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS
09376536263 | 63473627632 | shubhra | yes | VAS | 2011-07-07 | WAP
09655423434 | 13312353343 | shubhi | no | non VAS | 2011-05-04 | SMS
09473874383 | 36267267372 | amita | yes | VAS | 2011-04-05 | IVR

now i want to sort this source file on the basis of msisdn present in name.txt and output should be stored in output file as given below :

msisdn             | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS

As these are the matching values.
See if you can help out with it.....it would be A great help .....
~Thanks
Priya...

knight_eon · February 1, 2012, 3:02am

You can check this code

 
#!/bin/bash

while read MSISDN; do
        awk -F'|' '/^'"$MSISDN"'/ {print $0}' source.txt >> output

done < msidsn.txt

ektubbe · February 1, 2012, 5:31am

Thanks a tonn :)

parthmittal2007 · February 1, 2012, 5:39am

cat name.txt | while read line
do
grep -e $line source.txt >> resultfile.txt
done

Output-:
msisdn | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS

Hope this might help you

ektubbe · February 1, 2012, 7:59am

Thanks a tonn buddy..

---------- Post updated at 07:59 AM ---------- Previous update was at 07:58 AM ----------

it works with other unix systems but i am using below one :
Linux magnolia 2.6.16.60-0.21-smp #1 SMP Tue May 6 12:41:02 UTC 2008 x86_64 x86_64 x86_64 GNU/Linux

It is showing incorrect output with the awk filter..is it that awk doesnt work fine with the above system???

knight_eon · February 2, 2012, 12:45am

If you can please post the output of the above script here?

---------- Post updated at 11:15 AM ---------- Previous update was at 11:01 AM ----------

parthmittal2007:

cat name.txt | while read line
do
grep -e $line source.txt >> resultfile.txt
done
```[/b]

 
Output-:
msisdn | imsi | name | subscription | product | date | recharge type
09311424354 | 09488439820 | ekta | yes | VAS | 2012-12-07 | IVR
09128734473 | 83974738734 | priya | no | Non VAS | 2011-08-11 | SMS
 
Hope this might help you

What if the source.txt contains for eg. 09311424354 in imsi field as well? or somewhere else?

Rather this will be more robust and will eliminate possibility of above scene:

cat name.txt | while read line
do
grep -e "^${line}" source.txt >> resultfile.txt
done
```[/b]

 
Including ^ \(carat\) above will ensure that the pattern is searched at the beginning of the line only, not in between :\)

ektubbe · February 2, 2012, 12:53am

knight_eon:

If you can please post the output of the above script here?

---------- Post updated at 11:15 AM ---------- Previous update was at 11:01 AM ----------

What if the source.txt contains for eg. 09311424354 in imsi field as well? or somewhere else?

Rather this will be more robust and will eliminate possibility of above scene:
cat name.txt | while read line
do
grep -e "^${line}" source.txt >> resultfile.txt
done
Including ^ (carat) above will ensure that the pattern is searched at the beginning of the line only, not in between

helloo..
the source file is quite big.....say 800000 records and name.txt is having 8000 records so i m not sure whether grep will not take that much of time...

knight_eon · February 2, 2012, 12:58am

And what output you are getting when you run this command:

#!/bin/bash

while read MSISDN; do
        awk -F'|' '/^'"$MSISDN"'/ {print $0}' source.txt >> output

done < name.txt

parthmittal2007 · February 2, 2012, 2:24am

hi grep is quiet fast..

ektubbe · February 15, 2012, 12:20pm

hi frnz...
thnx fr dis much help....
nw i want a bit more extended version of this filteration script...
i have 5 files A,B,C,D and E located at /home/anubha
I have a file F located at /home/anubha/ed
File F has some records which can be matched in A,B,C,D and E
and another set of files on the basis of these filteration should be created i.e. G,H,I,J and K to be stored at /home/anubha/priya
Plsss help me out....pls pls pls..pls pls pls...:wall::wall:

~Thanks : Priya........