Normalization Using Shell Scripting.

Shell Programming and Scripting
1

Hi All, I am having a file having below three lines or maybe more than 3 lines. The first line will be always constant.

### Line 1 ####
Transformation||Transformation Mapplet Name||Transformation Group||Partition Index||Transformation Row ID||Error Sequence||Error Timestamp||Error UTC Time||Error Code||Error Message||Error Type||Transformation Data||Source Mapplet Name||Source Name||Source Row ID||Source Row Type||Source Data 
### Line 2 ####
exp_Concat||N/A||Input||1||4||1||03/01/2010 08:42:33||1267450953||11132||Transformation [exp_Concat] had an error evaluating output column [ID_OUT]. Error message is [<<Expression Fatal Error>> [ABORT]: ID CANT BE NULL VALUE\n... f:ABORT(u:'ID CANT BE NULL VALUE')].\n||3||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000||N/A||SQ_EMP||4||0||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000
### Line 3 ####
lkp_Concat||N/A||Input||1||3||1||03/01/2010 08:42:33||1267450953||11132||NULL||3||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000||N/A||SQ_EMP||4||0||N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000

Out put expected :

Transformation = exp_Concat
Transformation Mapplet Name = N/A
.
.
.
Source Data = N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000

Transformation = lkp_Concat
Transformation Mapplet Name = N/A
.
.
.
Source Data = N:,D:VENKAR,D:CON,D:HYD,D:367288.0000000000

is it possible using shell script. Please if so let me know.

2

Post your script what you have tried so far.Then,we will help you.

3 
sed 's/||/|/g' file | awk 'BEGIN{FS="|"}NR==1{for(i=1;i<=NF;i++)head=$i}NR>1{for(i=1;i<=NF;i++)print head,"=",$i}'

cheers,
Devaraj Takhellambam

4

You try the following script which will do your requirement.

j=0
for line in `sed 's/ //g;s/||/|/g' Transinput`
do
if [[ `echo $line|grep -v "^#.*"` ]];then
if [[ $j -eq 0 ]]
then
for((k=1;k<=17;k++))
do
arr[$k]=`echo $line | cut -d '|' -f $k `
done
let j+=1
else
for((k=1;k<=17;k++))
do
echo "${arr[$k]} = `echo $line | cut -d '|' -f $k `"
done
echo -e "\n"
let j+=1
fi
fi
done
5

Hi vivekraj and devtakh, thanks a lot for your reply.