Hi Scripting Gurus,
I am trying to parse a csv file and generate a new output file.
The input file will be a variable length in turns of rows and columns.
output file will have 8 columns. we have three columns from the header for each set.
just to give little bit more clarification each row will have n number of sets as seen in the input file we have two sets .
we can have two solutions one is split each row into two rows or bring set2 under set 1 and so on if we have more sets.
|-----------set 1----------|----------set 2----------------|
INPUT FILE
Header,4567,USD,BMI,,4568,USD,BMI,
Date,LEVEL,NET,TOTAL,MCAP,LEVEL,NET,TOTAL,MCAP
22-Dec-97,27.06492573,,20.9289574,,26.17032566,,20.60702136,
23-Dec-97,27.11627283,,20.96866346,,26.50090757,,20.86732796,
24-Dec-97,27.17448579,,21.01367878,,26.53347955,,20.89299111,
25-Dec-97,27.17854775,,21.02091981,,26.66484879,,20.99775394,
26-Dec-97,27.13228338,,20.98513719,,26.55463469,,20.91096371,
Desired output
Date, index_id,currency,index_pro, level, net, total, mcap
22-Dec-97, 4567, USD, BMI, 27.06492573, , 20.9289574 ,
22-Dec-97, 4568, USD, BMI, 26.17032566, , 20.60702136,
23-Dec-97, 4567, USD, BMI, 27.11627283, , 20.96866346,
23-Dec-97, 4568, USD, BMI, 26.50090757, , 20.86732796,
24-Dec-97, 4567, USD, BMI, 27.17448579, , 21.01367878,
24-Dec-97, 4568, USD, BMI, 26.53347955, ,20.89299111,
25-Dec-97, 4567, USD, BMI, 27.17854775, ,21.02091981,
25-Dec-97, 4568, USD, BMI, 26.66484879, ,20.99775394,
26-Dec-97, 4567, USD, BMI, 27.13228338, ,20.98513719,
26-Dec-97, 4568, USD, BMI, 26.55463469, ,20.91096371,
here is the script i wrote and iam having issues geting the proper out put.
#!/bin/sh
hdr=`cat inputfile.csv|grep 'Header'|awk -F"," '{ t += NF - 1 } END { print t }'`
hdr_str=`cat inputfile.csv|grep 'Header'`
hdr=`expr $hdr + 1`
while read line
do
i=2;
j=1;
dte=`echo $line | awk -F"," '{ print $1 }'`
k=2;
str_tmp="";
l=1;
while [ $k -le ${hdr} ]
do
if [ $l -le 3 ]; then
str_id_cur_iprv=`echo $hdr_str | awk -F"," '{ print $'$k' }'`
if [ $l -eq 1 ]; then
str_tmp=$str_id_cur_iprv
l=`expr $l + 1`
k=`expr $k + 1`
else
l=`expr $l + 1`
k=`expr $k + 1`
str_tmp=$str_tmp","$str_id_cur_iprv
fi
else
break
fi
done
str=${dte}","$str_tmp
while [ $i -le ${hdr} ]
do
if [ $j -le 4 ]; then
var=`echo $line | awk -F"," '{ print $'$i' }'`
str=${str}","${var}
i=`expr $i + 1`
j=`expr $j + 1`
else
echo $str >> output_file.dat
str=${dte}","$str_tmp
j=1;
fi
done
if [ $j -ne 1 ]; then
echo $str >> output_file.dat
str=${dte}","$str_tmp
fi
done < inputfile.csv
here is the out put i get with the above script.
Header,4567,USD,BMI,4567,USD,BMI,
Header,4567,USD,BMI,4568,USD,BMI,
Date,4567,USD,BMI,LEVEL,NET,TOTAL,MCAP
Date,4567,USD,BMI,LEVEL,NET,TOTAL,MCAP
22-Dec-97,4567,USD,BMI,27.06492573,,20.9289574,
22-Dec-97,4567,USD,BMI,26.17032566,,20.60702136,
23-Dec-97,4567,USD,BMI,27.11627283,,20.96866346,
23-Dec-97,4567,USD,BMI,26.50090757,,20.86732796,
24-Dec-97,4567,USD,BMI,27.17448579,,21.01367878,
24-Dec-97,4567,USD,BMI,26.53347955,,20.89299111,
25-Dec-97,4567,USD,BMI,27.17854775,,21.02091981,
25-Dec-97,4567,USD,BMI,26.66484879,,20.99775394,
26-Dec-97,4567,USD,BMI,27.13228338,,20.98513719,
26-Dec-97,4567,USD,BMI,26.55463469,,20.91096371,
Gurus can any one take a look at my script and modify it to get the correct form.
Thanks
vkr