Help me pls : splitting single file in unix into different files based on data

Shell Programming and Scripting
21

I observed above that you have two patterns at the end of the line..

{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}

And assuming Program Files\\Ab Initio is same pattern for the all file names..

try this..

awk -F "\.mdc|\.mpc" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if($0~/Program Files\\\\Ab Initio/){n=split($1,a,"\\");x++;fn=a[n]x;{print s > fn ;s=""}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' file
1 Like
22

Thanks a lot for answering my ques.
Using these files i need to do some more analysis. will come back to you if necessary

---------- Post updated at 01:31 PM ---------- Previous update was at 12:59 PM ----------

awk -F "\\" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if($0~/E:\\\\Program Files\\\\Ab Initio/){split($11,a,".");x++;fn=a[1]x;{print s > fn ;s=""}}
else if($0~/C:\\\\Program Files\\\\Ab Initio/){split($13,a,".");x++;fn=a[1]x;{print s > fn ;s=""}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' file

This is not giving me desired output.
Files saperation is not being done and i am getting only one file which is having C:\\ line and that even extracting lines is wrong.

Can u please look into this again

---------- Post updated at 01:47 PM ---------- Previous update was at 01:31 PM ----------

awk -F "\.mdc|\.mpc" '{if($0~/Layout\|\$\[\[recor/){s=$0;} else if($0~/Program Files\\\\Ab Initio/){n=split($1,a,"\\");x++;fn=a[n]x;{print s > fn ;s=""}} else if(s){s=s"\n"$0} else{if(fn){print > fn}}}' fileEven this is not giving correct O/P.awk: warning: escape sequence `\.' treated as plain `.'

This is giving a file with name C1 and all the data of the original file is being copied.Please look into this , bcz in my view if this gives me correct o/p , my problem is solved.

---------- Post updated at 01:50 PM ---------- Previous update was at 01:47 PM ----------

.mpc , .mdc , .mp : these are the only three extensions to the file names

23

just one correction needed.. escape sequence for "."

Now try..

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if($0~/program files\\\\Ab Initio/){n=split($1,a,"\\");x++;fn=a[n]x;{print s > fn ;s=""}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' file

Just check above highlighted one in your file to match the pattern. is this same or not..

run this - Below command should give all the partition names which are used to create files.

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '/program files\\\\Ab Initio 1438/{n=split($1,a,"\\");print a[n]}' file
1 Like
24 
 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '/program files\\\\Ab Initio 1438/{n=split($1,a,"\\");print a[n]}' temp.txt
awk: warning: escape sequence `\|' treated as plain `|'
Filter_by_Expression
Input_File
Join
Input_File
Input_File
Join
Input_File
Output_File
Partition_by_Key_and_Sort
Partition_by_Key_and_Sort
Partition_by_Key_and_Sort
Partition_by_Key_and_Sort

One join is missing i need 13 partitions.

That line it missed is below:

 
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}

---------- Post updated at 03:49 PM ---------- Previous update was at 03:32 PM ----------

I executed the following code and got around 101 lines (i.e. in this file i need to get 101 partitions)

cat temp1.txt | grep "{30001002|XXparameter|!prototype_path|" | grep "Ab Initio"

 
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 14\\Ab Initio GDE\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Rollup.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Filter by Expression - (Transform).mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|930|0|2710|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|936|0|2736|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Rollup.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1276|0|3943|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1282|0|3962|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1333|0|4101|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Departition\\Gather.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 14\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Lookup_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Lookup_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Lookup_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1461|0|4466|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Rollup.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}

but using this code:

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '/program files\\\\Ab Initio 1438/{n=split($1,a,"\\");print a[n]}' temp1.txt
i got only 18 partitions
 
 
awk: warning: escape sequence `\|' treated as plain `|'
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression
Join
Input_File
Replicate
Reformat
Output_File
Lookup_File
Lookup_File
Filter_by_Expression
Filter_by_Expression
Partition_by_Key_and_Sort
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression

Please look into this.

---------- Post updated at 03:49 PM ---------- Previous update was at 03:49 PM ----------

I executed the following code and got around 101 lines (i.e. in this file i need to get 101 partitions)

cat temp1.txt | grep "{30001002|XXparameter|!prototype_path|" | grep "Ab Initio"

 
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 14\\Ab Initio GDE\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Rollup.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\Filter by Expression - (Transform).mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|930|0|2710|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|936|0|2736|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Transform\\Join.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Rollup.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1276|0|3943|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1282|0|3962|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Transform\\reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1333|0|4101|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Departition\\Gather.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 14\\Ab Initio GDE\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Lookup_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Lookup_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Lookup_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{2010501005|XXGpvertex|1461|0|4466|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Miscellaneous\\Replicate.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Reformat.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Rollup.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Sort\\Partition_by_Key_and_Sort.mp|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE\\Components\\Transform\\Dedup_Sorted.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Transform\\Filter_by_Expression.mpc|3|2|Pw$|@{0|}}

but using this code:

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '/program files\\\\Ab Initio 1438/{n=split($1,a,"\\");print a[n]}' temp1.txt

i got only 18 partitions

 
awk: warning: escape sequence `\|' treated as plain `|'
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression
Join
Input_File
Replicate
Reformat
Output_File
Lookup_File
Lookup_File
Filter_by_Expression
Filter_by_Expression
Partition_by_Key_and_Sort
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression
Filter_by_Expression

Please look into this.
I am working on the smallest files and then if it works fine i am checking with others

25

try with this..

awk -F "\\\.mdc\|3|\\\.mpc\|3|\\\.mp\|3" 'NF > 1{n=split($1,a,"\\");print a[n]}' file
26

No not even names of the partitions are comming.
Un desired o/p

27

Ohh sorry for this..

now try .. it should give...

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" 'NF > 1{n=split($1,a,"\\");print a[n]}' file
28 
{30001002|XXparameter|!prototype_path|C:\\gui\\src\\mpc\\Sort-Merge\\Sort.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\gui\\src\\mpc\\Partition\\Hash.mpc|3|2|Pw$|@{0|}}

These two are also getting generated with other files which is not required
Bcz these are sub components of a main component, so can we make 'Abi initio' also as another searching criteria and name of the file as specified above is good.

Thanks in advance.

29

yeah that's good...:slight_smile:

just add this..

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" 'NF > 1 && /Abi initio/{n=split($1,a,"\\");print a[n]}' file
30

I tried this

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" 'NF > 1 && /Abi initio/{n=split($1,a,"\\");print a[n]}' temp1.txt

It is giving warning as below and no output.in the sense it's not printing the file names.

awk: warning: escape sequence `\|' treated as plain `|'
31

these are just warnings...

verify below highligted one with your file is this same or not..? May be some uppercase or lower case change.. try:)

In your previous given inputs it is "Ab Initio". Please check...

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" 'NF > 1 && /Abi initio/{n=split($1,a,"\\");print a[n]}' temp1.txt
32

Thanks yaar ,
I am getting names but the numbering to the files......
I mean
Join1
Input2....

Pls include this part also to this code:

 
 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" 'NF > 1 && /Ab Initio/{n=split($1,a,"\\");print a[n]}' temp1.txt

I am testing with some other files, If every thing is correct :slight_smile:
Thanks a lot

33

use this..

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if(NF > 1 && $0 ~ /Ab Initio/){n=split($1,a,"\\");x++;fn=a[n]x;{print s > fn ;s="";print > fn}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' file

Now it should work..

sorry for one mistake:o

34 
 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" 'NF > 1 && /Ab Initio/{n=split($1,a,"\\");print a[n]}' temp1.txt
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if(NF > 1 && $0 ~ /program files\\\\Ab Initio/){n=split($1,a,"x;%7bprint/"]\\");x++;fn=a[n]x;{print s > fn ;s=""}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' temp1.txt

Should i use as mentioned above??? I dont think so .
even

 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if(NF > 1 && $0 ~ /program files\\\\Ab Initio/){n=split($1,a,"x;%7bprint/"]\\");x++;fn=a[n]x;{print s > fn ;s=""}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' temp1.txt

This is not giving required.
Please solve this .
Thanks alot in advance

35

edited in previous post..

Just use that script as whole don't add any extra code to it...:slight_smile:

If still not working attach one input file to it. :slight_smile:

36

Just another doubt ....
can we create these files in a directory???
if yes
can we make the directory name as the file name we are using???

For ex:

 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if(NF > 1 && $0 ~ /Ab Initio/){n=split($1,a,"\\");x++;fn=a[n]x;{print s > fn ;s="";print > fn}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' temp1.txt

I want these files to be in a directory with the directory name as temp1.

If possible can u please modify the code which solves this problem...

37

give your temp folder path as a variable..

use this..
It will create files in your given path..

awk -v dir_path="/home/temp1/" -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if(NF > 1 && $0 ~ /Ab Initio/){n=split($1,a,"\\");x++;fn=dir_path""a[n]x;{print s > fn ;s="";print > fn}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' file

Hope this helps you:)

38 
 
Output_File73:
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|write_metadata||3|8|s=|@{0|}}
{30001002|XXparameter|eme_dataset_location|$\{PROJECT_DIR\}/data/serial/lookup/m_cdp_cdm_gl_prod_lvl_2_lookup.dat|3|9||@{0|}}


 
Output_File77:
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Output_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|write_metadata||3|8|s=|@{0|}}
{30001002|XXparameter|eme_dataset_location|$\{PROJECT_DIR\}/data/serial/lookup/cdp2_uedw_v_thrd_prty_orig.lkp|3|9||@{0|}}

Hi ,
Please check the code above.
Even though It is a output , but it is a lookup o/p . I dont want the name of the file to be output for these kind.
So, the below code is working fine.

 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if(NF > 1 && $0 ~ /Ab Initio/){n=split($1,a,"x;%7bprint/"]\\");x++;fn=a[n]x;{print s > fn ;s="";print > fn}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' temp2.txt

after this code execution can we make anything so that the file name changes if

 
{30001002|XXparameter|eme_dataset_location|$\{PROJECT_DIR\}/data/serial/lookup/cdp2_uedw_v_thrd_prty_orig.lkp|3|9||@{0|}}

So if we have lookup in the above line . i.e cat the file with names output and input and grep this particular pattern and if it is present change the file name to lookup accordingly.
I also mentioned I/p bcz i think this trouble may occur in I/P file also.

Thanks a lot in advance :slight_smile:

39

Now its getting more complicated...

try this..

awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){if(fn && s){print s > fn;s=$0;}else{s=$0}}
else if(NF > 1 && $0 ~ /Ab Initio/){n=split($1,a,"\\");x++;fn=a[n]x;{s=s"\n"$0}}
else if($0 ~ /PROJECT_DIR/ && $0 ~ /serial\/lookup/){fn="lookup"x;s=s"\n"$o}
else if(s){s=s"\n"$0}
}END{print s > fn}' file
40

There is difference in O/P for these 2 codes:

 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){s=$0;}
else if(NF > 1 && $0 ~ /Ab Initio/){n=split($1,a,"x;%7bprint/"]\\");x++;fn=a[n]x;{print s > fn ;s="";print > fn}}
else if(s){s=s"\n"$0}
else{if(fn){print > fn}}}' temp1.txt

 
awk -F "\\\.mdc\||\\\.mpc\||\\\.mp\|" '{if($0~/Layout\|\$\[\[recor/){if(fn && s){print s > fn;s=$0;}else{s=$0}}
else if(NF > 1 && $0 ~ /Ab Initio/){n=split($1,a,"x;%7bs=s%22/n%22$0"]\\");x++;fn=a[n]x;{s=s"\n"$0}}
else if($0 ~ /PROJECT_DIR|serial\/lookup/){fn="lookup"x;s=s"\n"$o}
else if(s){s=s"\n"$0}
}END{print s > fn}' temp1.txt

For the first i am getting 102 files generated.This is correct .
But for the second where 'lookup' change is done , i am getting only 42 files.
I am not able to analyse these two and find where it is going wrong .
can u please look into this issue.

My point of view is as the first code is working good. Can we make a change of the file name like this:

cat all files like 'Output_File' and then | grep "{30001002|XXparameter|eme_dataset_location|$\{PROJECT_DIR\}/data/serial/lookup"
if present change corresponding file name to lookup. Here numbering convention is not necessary for lookup.So, if u feel numbering convention is difficult to obtain, u can leave it.
Thanks a lot in advance.