Help with file editing while keeping file format intact

Shell Programming and Scripting
1

Hi,

I am having a file which is fix length and comma seperated. And I want to replace values for one column.
I am reading file line by line in variable $LINE and then replacing the string.
Problem is after changing value and writing new file temp5.txt, formating of original file is getting lost. It is removing all the blank spaces.

Sample value: Original File

LINE="000001077,03190, ,                                          1 Main Street,                                                       ,                                                       ,                                                       ,     Foxboro                            ,           02035,1,MA,12-13-2010 22:43:42,12-13-2010 22:43:42"

I want to replace MA with 00021

I am reading file line by line in variable $LINE and then replacing the string

echo $LINE | sed "s/",MA,"/",00021,"/g" >>temp5.txt

temp5.txt:

000001077,03190, , 1 Main Street, , , , Foxboro , 02035,1,00021,12-13-2010 22:43:42,12-13-2010 22:43:42

Please help.

Thanks

2

U may try:

echo $LINE | sed 's/,MA,/,00021,/g' >>temp5.txt

But I don't see where is the problem. How original file is being affected.

3

The problem is in echo $LINE . You need to use:

echo "$LINE"

to keep spacing intact.

But: you do not need to read line by line as sed can do all that in one go ( sed '...' file )

1 Like
4

Thanks for the reply.

Sorry my mistake.

Original file looks like:

000001077,03190, ,                                          1 Main Street,                                                       ,                                                       ,                                                       ,     Foxboro                            ,           02035,1,MA,12-13-2010 22:43:42,12-13-2010 22:43:42

Its a fix length file. So having number of blank spaces around the column. But after I use 'sed', it is removing all blank spaces.

---------- Post updated at 04:45 PM ---------- Previous update was at 04:39 PM ----------

Hi

I tried with quotes. But it is still not working.

echo "$LINE" | sed "s/",$SERVICE_CODE,"/",$SERVICE_CODE_NUM,"/g" >>temp5.txt

simple echo "$LINE" keeps the spaces. But with sed it's not working.

5

Check if this helps.

echo "$LINE" | sed "s/,$SERVICE_CODE,/,$SERVICE_CODE_NUM,/g" >>temp5.txt

OR just (if suitable)

echo "$LINE" | sed "s/$SERVICE_CODE/$SERVICE_CODE_NUM/g" >>temp5.txt

If not, how is the output, any error? Pls post the result.
Also as Scrutinizer told, U can process whole file in one go, no need to read file line by line

sed "s/$SERVICE_CODE/$SERVICE_CODE_NUM/g" inputFile >temp5.txt
6 
echo "$LINE" | sed "s/,$SERVICE_CODE,/,$SERVICE_CODE_NUM,/g" >>temp5.txt
 
echo "$LINE" | sed "s/$SERVICE_CODE/$SERVICE_CODE_NUM/g" >>temp5.txt

Both is giving same output as before, removing all space. It does not give any error.

-----------------------------

sed "s/$SERVICE_CODE/$SERVICE_CODE_NUM/g" inputFile >temp5.txt

-----------------------------
works fine. but i am not sure if i can use it.
Because I need to do some validations on each line before updating it.

7

How are you reading lines in LINE variable, Pls post that peice of code.
Also some sort of validations can be done in sed too (but all kind of validations).
U may post the validations to be done in lines to see if sed can do that or not.

8

I have one Extracted CSV file and one param file. Param file has a list of values and their mapping values. I want to replace one of the field in CSV with the matching mapping value in param file.
Example:

CSV: 123,   ABCD,  487274
        324,   ASKJ,  472844
Param file:   export ABCD=20
                 export XYZ=44

ABCD should get replaced with 20.

I am reading CSV as

cat ${DATA_FILE} | while read LINE

Note: I can change format of param file if require.

9

If CSV file is has one space after comma (That's how above CSV file looks like), then following one line awk should do the needful.

awk -F'( |=)' 'NR==FNR{a[$2]=$3;next;}{FS=", ";for(i=1;i<=NF;i++) if(a[$i]) $i=a[$i];print}' OFS=", " Param_File CSV_File

If field no/column no in CSV to be replaced is fixed.. say 2nd (Only values in 2nd columns are supposed to be replaced then)

awk -F'( |=)' 'NR==FNR{a[$2]=$3;next;}{FS=", ";if(a[$2]) $2=a[$2];print}' OFS=", " Param_File CSV_File

If there is no space in CSV file after comman, these commands need to be modified a little.