Strings between 2 characters

Shell Programming and Scripting
1

Hi
I have a wired string pattern ( mongo output) which I need to convert to only values.

"_id" : ObjectId("59280d9b95385c78b73252e4"), "categorySetId" : NumberLong(1100000041), "categorySetName" : "PROD GROUP", "serviceableProductFlag" : "N", "categoryId" : NumberLong(1053), "pid" : "800-319-03", "productFamily" : "PP", "productType" : "SEATS", "subGroup" : "PP SER", "description" : "^AY,EECH, NG-C", "inventoryItemId" : NumberLong(200699), "itemStatusMfg" : "S-INTV", "organizationIdMfg" : NumberLong(90000), "src" : "orcl", "syncedOn" : NumberLong("1495797136138"), "CreationDate" : ISODate("2017-05-26T11:12:16.138Z"), "CreatedBy" : "tool", "LastUpdatedDate" : ISODate("2017-05-26T11:12:16.138Z"), "LastUpdatedBy" : "tool", "itemFamilyDesc" : "PP FAMILY", "itemFamilyGroupId" : 750, "itemFamilyGroupName" : "PP SERIES PRODUCTS"

I want output like

59280d9b95385c78b73252e4,1100000041,PROD GROUP,N,1053,800-319-03,PP, SEATS,PP SER ,'^AY,EECH, NG-C', 200699,S-INTV,90000,orcl,1495797136138,2017-05-26T11:12:16.138Z,tool,2017-05-26T11:12:16.138Z,tool,PP FAMILY,750,PP SERIES PRODUCTS

I have tried to achieve with some help by using below ask

awk -F':|, *"' '{ r=""; for(i=2;i<=NF;i+=2) {gsub(/^ *([^(]+\()?|"|\)$/,"",$i); if(index($i,",")!=0){ $i="\047"$i"\047" } r=(r!="")? r","$i : $i } print r }' text.txt

But the date is getting split which I want like

2017-05-26T11:12:16.138Z but coming as 2017-05-26T11,16.138Z . Can someone help me with same ?

Regards

2 
awk -F: '
{
ll=split($0, b, ",");
for (i=1; i<=ll; i++) {
   if ((gsub("\"", "\"", b) % 2) || ! match(b, "\"")) {
      if (match(b, "\" *$")) { t=t b; b=t; t=""; }
      else {t=t b ","; continue; }
   }
   a[c++]=b;
}
NF=0;
for (i=0; i<c; i++) {
   f=split(a,e, "[()\"]");
   if (f==3) {sub("[^:]*: *", "", e[3]); o[d++]=e[3];}
   if (f==5) {o[d++]=e[f-1];}
   if (f==7) {o[d++]=e[f-2];}
   if (o[d-1] ~ /,/) o[d-1]="\x027" o[d-1] "\x027";
   $(++lo)=o[(d-1)];
}
print $0;
}
' OFS=, infile
3

Thank you so much

4

The demanding part is that the field separator , is part of some fields as well, so we need to spend some effort dealing with that. Try

awk -F"\"" -vOFS="" '
        {for (i=2; i<=NF; i+=2) {if (gsub (/,/, "\001", $i)) $i = "\047" $i "\047"
                                 gsub (/:/, "\002", $i)
                                }
        }
1
' file |  awk -F, '
        {for (i=1; i<=NF; i++)  {sub (/^.*: /, _, $i)
                                 gsub (/^[^(]*\( ?| ?\)[^)]*$/, _, $i)
                                }
         gsub ("\001", ",")
         gsub ("\002", ":")
        }
1' OFS=","
59280d9b95385c78b73252e4,1100000041,PROD GROUP,N,1053,800-319-03,PP,SEATS,PP SER,'^AY,EECH, NG-C',200699,S-INTV,90000,orcl,1495797136138,2017-05-26T11:12:16.138Z,tool,2017-05-26T11:12:16.138Z,tool,PP FAMILY,750,PP SERIES PRODUCTS
5

Or, to condense it into one single awk script:

awk -vSQ="'" -vCA=$'\001' -vCB=$'\002' '
        {FS  = "\""
         OFS = ""
         $0  = $0
         for (i=2; i<=NF; i+=2) {if (gsub (/,/, CA, $i)) $i = SQ $i SQ
                                 gsub (/:/, CB, $i)
                                }
         FS  = ","
         OFS = ","
         $0  = $0
         for (i=1; i<=NF; i++)  {sub (/^.*: /, _, $i)
                                 gsub (/^[^(]*\( ?| ?\)[^)]*$/, _, $i)
                                }
         gsub (CA, ",")
         gsub (CB, ":")
        }
1
' file
6

And here's another approach in case you're comfortable with Perl and regular expressions:

$
$ perl -plne 's/"\w+"\s+:(\s+\w+\("|\s+\w+\(|\s+"|\s+)//g;
              s/"\s*$//;
              s/"\),\s+|\),\s+|",\s+/~/g;
              s/(\d+),\s+/$1~/g;
              s/([^~]*,[^~]*)/chr(39).$1.chr(39)/eg;
              s/~/,/g
             ' text.txt
59280d9b95385c78b73252e4,1100000041,PROD GROUP,N,1053,800-319-03,PP,SEATS,PP SER,'^AY,EECH, NG-C',200699,S-INTV,90000,orcl,1495797136138,2017-05-26T11:12:16.138Z,tool,2017-05-26T11:12:16.138Z,tool,PP FAMILY,750,PP SERIES PRODUCTS
$
$

The idea is to transform the input string via a series of substitutions using regexes.

1 Like