In this case there are 11 headers, but if there are N headers for which I want to get the value in $3, how would be a way to make a shortest script and avoid write this up to N?
awk -F"[ =]+" '
BEGIN {HD = "STAGE|ID|NAME|TYPE|DFRUL|ADDR|RRUL|SPRR|ISGALW|ISUTWD"
for (MX=n=split (HD, HDArr, "|"); n>0; n--) SRCH[HDArr[n]]
print HD
}
!NF {for (i=1; i<=MX; i++) printf "%s%s", RES[HDArr], (i == MX)?ORS:OFS
delete RES
}
$1 in SRCH {RES[$1] = $2
}
END {for (i=1; i<=MX; i++) printf "%s%s", RES[HDArr], (i == MX)?ORS:OFS
}
' OFS="|" file
STAGE|ID|NAME|TYPE|DFRUL|ADDR|RRUL|SPRR|ISGALW|ISUTWD
1|0|JFMSC|MLRR|PERMISSION|1001|PERMISSION|TRUE|FALSE|FALSE
1|2|PLLSJS|MLRR|PERMISSION|||||
1|4|AAAARR|MLRR|RESTRICT|890087|PERMISSION|FALSE|FALSE|FALSE
1|0|PPROOA|RRHN|PERMISSION|7034|PERMISSION|FALSE|FALSE|FALSE
will extract data based on the header fields and is thus very flexible. Unfortunately you didn't define the record terminator which would define the point at which to print a line. And, the records are non-uniformly structured: you have "incomplete" records (rec 2) and records with duplicate field entries (rec 3). You need to define very clearly and accurately what to extract and what to print, and adapt the script accordingly.
I'm surprised when an awk script is written in that way, I almost don't understan the logic to customize it.
The record initiator will be always the line STAGE = 1
The Record terminator will be the empty line before the next Record Initiator. The issue is that in the original file the records with duplicate fields (rec 3) has empty lines before the occurence of next Record Initiator. And between Record Terminator and next Record Initiator (in green) could be more than one empty line and some garbaje lines (in red).
At the end of file appears and END string.
This would be a better representation of input file:
some garbaje
some garbaje
some garbaje
STAGE = 1
ID = 0
NAME = JFMSC
TYPE = MLRR
DFRUL = PERMISSION
ADDR = 1001
RRUL = PERMISSION
SPRR = TRUE
ISGALW = FALSE
ISUTWD = FALSE
some garbaje
some garbaje
STAGE = 1
ID = 2
NAME = PLLSJS
TYPE = MLRR
DFRUL = PERMISSION
some garbaje
some garbaje
STAGE = 1
ID = 4
NAME = AAAARR
TYPE = MLRR
DFRUL = RESTRICT
ADDR = 3553
RRUL = PERMISSION
SPRR = FALSE
ISGALW = FALSE
ISUTWD = FALSE
ADDR = 66444
RRUL = PERMISSION
SPRR = FALSE
ISGALW = FALSE
ISUTWD = FALSE
ADDR = 890087
RRUL = PERMISSION
SPRR = FALSE
ISGALW = FALSE
ISUTWD = FALSE
some garbaje
some garbaje
STAGE = 1
ID = 0
NAME = PPROOA
TYPE = RRHN
DFRUL = PERMISSION
ADDR = 7034
RRUL = PERMISSION
SPRR = FALSE
ISGALW = FALSE
ISUTWD = FALSE
--- END
The duplicated fields within a record should be printed in different lines in the output. In the output, the records that don't have fields repeated only have one line.
See below output (lines in green are the values for record 3) and input of my post #6.
Excellent rdrtx1 for yout solution and Chubler_XL for the modification. With both solution it seems to work pretty nice even when in real file the strings have leading spaces.
Chubler_XL,
I see this solution has several arrays. Is there a tool to test and see what has stored each part of an awk program, the content of the arrays, etc?
Something similar to IRB(interactive Ruby) which we can test small parts of the ruby program. This in order to understand better the logic and how it works.
I'm not aware of any tool like that. You could simply print out the contents of the arrays in the END block like this:
awk '
BEGIN {lines=0; column_count=0}
$2 !~ /=/ || NF != 3 {next}
! column[$1]++ {columns[column_count++]=$1}
$1 ~ /^STAGE*$/ || $1 SUBSEP lines in column_data{lines++}
{column_data[$1, lines]=$3}
END {
for (i in column) print "column[" i "]="column
for (i in columns) print "columns[" i "]="columns
for (i in column_data) {
k=i
gsub(SUBSEP,",",k)
print "column_data[" k "]="column_data
}
}
' infile
You can see that:
column[] is index by the column name and contains a count of the number of rows the column appears in (less 1)
columns[] is indexed by the column order and contains the column name.
column_data[] is a 2D array with index of <column name>,<row> and contains value of the data for that cell.
column[] is only used to detect the first occurrence of a column and add it to columns[]