Hey guys,
I have a file with an ID which I'm using to grep out the original record from another file. Problem is I have special characters in the original file, and grep is returning only a partial record. How can I get around this?
Appreciate your help!
Pete
Could you post an example of the data? cat -v filename should show it with ^M and so forth instead of nonprinting chars.
I suspect it's not grep that's truncating the results, but your terminal. A carriage return in the middle of the value could send the cursor back to the beginning of the line and overwrite previous content.
Thanks for the prompt reply! I changed some of the data to protect client obligations, but the format is the same. This is the results of cat -v FILE | grep "0000000000000000001" :
0000000000000000001XX 0000000000000000 000002011-09-12 00000000000000 00000 00000 000000 00 0000000.00 0000 0000 0000010.00 00000.00000000000000N 00000000000000 000 ^@0 0000000000000000 0000000000000000 ^@ 0000000000000000 0000000000000000 0000000000 Y^
Problem is I have 50,000 rejected records out of a 5MM record file, and it would be impractical to loop through a file 50,000 times to get the data out with the cat -v | grep method.
Original method I was attempting to use is:
cat $reject_file | grep ID_NO | awk '{print $2}' | xargs -I {} grep {} SLS_TBL_20110914_001.dat
cat -v doesn't seem so impractical when you've got one useless use of cat in there already. 
Just seeing the record doesn't tell me what's going wrong. I wasn't suggesting using cat -v to fix the file -- I need to see what the strange characters are.
How about:
cat -v datafile | grep "\^" | head
That should capture a few lines with strange chars in them.