Merge files with multi-line stanzas

Hello.
I try to merge few /etc/qconfig files in AIX (from few servers into one file on new server).
Could you please help me what would be the best way to merge files that contain multi-line stanzas, like this:

stanza1:
  attr1 = value1
  attr2 = value2
  attr3 = value3
stanza2:
  attr1 = value1
stanza3:
  attr1 = value1
  attr2 = value2

Thanks in advance.

Without knowing what you want the output to be after you merge (what appears to be a single file) it is hard to explain what you need to do. Or have you shown us the output you want but not shown us the input files that you want to merge to produce that output?

For us to help you do what you want to do we need sample input files (shown in CODE tags) and the corresponding output that should be produced by merging those input files (again shown in CODE tags).

1 Like

Ok, sorry, I wasn't to clear. I will give example on real files I'm trying to merge. I'll paste only parts of files, as an example.

Source file 1:

62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101

Source file 2:

pnr5:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101
62_133_3_zebra:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
oss-prnsrv2_3:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9102
oss-prnsrv2_1:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9100
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100

Target file should be like this:

62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101
62_133_3_zebra:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
oss-prnsrv2_3:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9102
oss-prnsrv2_1:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9100
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100

Thanks for any help.

I don't understand what the rules are for your merge. You say that the target file should contain two stanzas wth the heading hp@Jet133: , two stanzas with the heading hp@JetNL95: , and three stanzas with the heading hp@JetNL97: .

I would have thought that the purpose of merging stanzas would be to get rid of stanzas with the same name. Since that clearly is not your requirement, please explain the rules you want to use to determine which stanzas are to be copied into the result file and which stanzas are to be dropped.

1 Like

Ok, more explanation from me.
File structure is - two-line stanza defining a print queue (which refers to a device) and after that multi-line stanza defining this device. In examples above I have a situation, that multi print queues use the same device - that's why device definition is duplicated.
After I've made some checks - we can get rid of additional device definitions, so correct target file should be like:

62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101
62_133_3_zebra:
        device = hp@Jet133
oss-prnsrv2_3:
        device = hp@JetNL97
oss-prnsrv2_1:
        device = hp@JetNL97
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100

This still is far from clear and complete. What should happen if there are several different "stanzas" for the same device, as for hp@JetNL97 , that points to ports 9100 , 9101 , and 9102 ? And, are there any requirements on the order of output lines?
This is based an some assumptions on my side; it keeps the last device definition encountered and scraps the earlier ones. Plus, it has been developed and tested on my linux system; it may need some adaptions to run on your AIX. Still, I think it comes close to what you posted as the desired output:

awk '
!/^ /           {IX = $0
                 if (!(IX in TMP)) NIX[++CNT] = IX
                 delete TMP[IX]
                 next
                }
                {TMP[IX] = TMP[IX] RS $0
                }
END             {for (i=1; i<=CNT; i++) print NIX  TMP[NIX]
                }
' file[12]
62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9100
62_133_3_zebra:
        device = hp@Jet133
oss-prnsrv2_3:
        device = hp@JetNL97
oss-prnsrv2_1:
        device = hp@JetNL97
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100

Keeping the 1st occurrence of stanzas with the same first line produces the requested output and makes the code a little simpler:

awk '
!/^[ \t]/ {
	copy = !($1 in stanza)
	stanza[$1]
}
copy' file[12]

If someone else wants to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk .

Guys, thank you very much!
That did the trick.