Need a script to parse data and output to csv

Shell Programming and Scripting
1

I am not too savvy with arrays and am assuming that what I am looking for needs arrays. This is my requirement.

So I have the raw data that gets updated to a log as shown below


[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = Newark
    Age = 20
    Name= John
}

[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = NYC
    Age = 22
    Name = Mary
}

There are blocks of data in the sense, a block starts from '[' and closes at '}'. So from the above there are 2 blocks.

The fields are what appears between '{' and '}'. These fields are not fixed, in the sense, i will run the script and supply the fields, sometimes I will supply say 3 fields - Address, Age and Name and other times more or less.

Can a script be made to parse this data and output the data into csv?

So the output for the above would look as

DateTime                  Address   Age     Name
20171102 10:00:00  Newark    20        John
20171102 10:00:00  NYC         22        Mary

If i had say another field supplied say Subject, then the output would be

[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = Newark
    Age = 20
    Name = John
    Subject = Math
}

[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = NYC
    Age = 22
    Name = Mary
    Subject = Literature
}

I would the output in csv as

DateTime                  Address   Age     Name    Subject
20171102 10:00:00  Newark    20        John     Math
20171102 10:00:00  NYC         22        Mary     Literature

Please can some one advise. The farthest I have got is doing awk and parsing the data, but I dont know how to call it by block and the align it in columns

2

Hello sidnow,

Could you please try following and let me know if this helps you.

awk 'BEGIN{
  print "DateTime                  Address   Age     Name"
}
/^}/{
  print date1,add,age,name;
  next
}
/StudentInfo/{
  gsub(/\[|\]/,"");
  date1=$1 $2;
  next
}
/Address/{
  add=$3;
  next
}
/Age/{
  age=$3;
next
}
/Name/{
  name=NF==2?$2:$3;
  next
}'   Input_file

Thanks,
R. Singh

3

Try also

awk  -F"[= ]*"  '
/^[[]/  {V = $1 " " $2 " " ++CNT
         gsub (/[][]/, "", V)
         next
        }

NF < 3  {next
        }

        {LN[V]
         HD[$2]
         MX[V,$2] = $3
        }

END     {H = "DateTime"
                        printf "%20s", H; for (i in HD) printf "%10.10s", i; print "";
         for (j in LN) {printf "%20s", j; for (i in HD) printf "%10.10s", MX[j,i]; print ""}
        }
' file
            DateTime       Age      Name   Subject   Address
 20171102 10:00:00 4        22      MaryLiterature       NYC
 20171102 10:00:00 3        20      John      Math    Newark
 20171102 10:00:00 2        22      Mary                 NYC
 20171102 10:00:00 1        20      John              Newark

We need the running count CNT to differentiate between single records, small additional code can suppress it if need be.