How to extract xml attribute values using awk inline.?

I am trying to extract specific XML attribute values for search pattern <factories.*baseQueueName' from resources.xml .
my scripts works ok,, but to extract 3 values this code does echo $line three times, it could be 'n' times. How can I use awk to extract matching pattern values in-line or efficiently than I am doing.

resources.xml
    <factories xmi:type="resources.jms.mqseries:MQQueue" xmi:id="MQQueue_11111" name="Queue1" jndiName="jms/Queue1" description="Queue1" category="TEST" persistence="APPLICATION_DEFINED" priority="APPLICATION_DEFINED" specifiedPriority="0" expiry="APPLICATION_DEFINED" specifie dExpiry="0" baseQueueName="TEST.QUEUE1" baseQueueManagerName="" useNativeEncoding="false" integerEncoding="Normal" decimalEncoding="Normal" floatingPointEncoding="IEEENormal" targetClient="JMS" queueManagerHost="" queueManagerPort="0" serverConnectionChannelName="" userName="" password="{xor}" readAhead="NO"/>
	
    <factories xmi:type="resources.jms.mqseries:MQQueue" xmi:id="MQQueue_22222" name="Queue2" jndiName="jms/Queue2" description ="Queue2" category="TEST" persistence="APPLICATION_DEFINED" priority="APPLICATION_DEFINED" specifiedPriority="0" expiry="APPLICATION_DEFINED" specifiedExpiry="0" baseQueueName="TEST.QUEUE2" baseQueueManagerName="" useNativeEncoding="false" integerEncoding="Normal" decimalEncoding="Normal" floa tingPointEncoding="IEEENormal" targetClient="JMS" queueManagerHost="" queueManagerPort="0" serverConnectionChannelName="" userName="" password="{xor}" readAhead="NO"/>

# .. around 20+ similar lines like above. 

	
grep '<factories.*baseQueueName' resources.xml | while read line; do
	QUEUE_JNDI_NAME=$( echo $line | grep -Po 'jndiName=\D\S+\D'      | cut -d'"' -f2 )
	BASE_QUEUE_NAME=$( echo $line | grep -Po 'baseQueueName=\D\S+\D' | cut -d'"' -f2 )
	QUEUE_NAME=$(      echo $line | grep -Po 'name=\D\S+\D'          | cut -d'"' -f2 )
	echo "$QUEUE_JNDI_NAME,$BASE_QUEUE_NAME,$QUEUE_NAME"
	done

output:

jms/Queue1,TEST.QUEUE1,Queue1
jms/Queue2,TEST.QUEUE2,Queue2
# .. around 20+ similar output lines like above.

Maybe you want something like:

awk '
BEGIN {	dqsERE = "\"[^\"]*\""
	EREs[++nEREs] = " jndiName=" dqsERE
	EREs[++nEREs] = " baseQueueName=" dqsERE
	EREs[++nEREs] = " name=" dqsERE
	for(i = 1; i <= nEREs; i++)
		offset = index(EREs, "=") + 1
}
/<factories.*baseQueueName/ {
	out = ""
	for(i = 1; i <= nEREs; i++) {
		if(match($0, EREs))
			out = out substr($0, RSTART + offset,
				RLENGTH - offset - 1)
		out = out ((i < nEREs) ? "," : "")
	}
	print out
}' resources.xml

As always, if you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk .

Would this do it?

perl -nle '@r=/(?:jndiN|baseQueueN|n)ame="([^"]+)/g and print join ",",@r[1,2,0]' resources.xml

Output:

jms/Queue1,TEST.QUEUE1,Queue1
jms/Queue2,TEST.QUEUE2,Queue2

Aia, thanks for trying,, I knew a wicked one liner like you gave could do this..
Don, please check performance stats,, I am bash/awk lover,, we have work to do..

 
grep '<factories.*baseQueueName' resources.xml | perl -nle '@r=/(?:jndiN|baseQueueN|n)ame="([^"]+)/g and print join ",",@r[1,2,0]'

# 153 XML tag lines scanned from one resources.xml file. time <command> gives these stats
real    0m0.007s, user    0m0.004s, sys     0m0.002s  -- Perl solution OMG
real    0m0.028s, user    0m0.025s, sys     0m0.002s  -- Don's awk solution
real    0m0.928s, user    0m0.458s, sys     0m0.409s  -- My general public :-) solution. Why sys taking so long here! not fair.

is there a way to combine first regex '<factories.*baseQueueName' into second one as well?

I wonder if any basic shell commands like sed/awk/grep can match what you are able to do with perl .. I would love to see simplified awk solution to beat perl .

I asked you if it would work, because <factories.*baseQueueName appeared to me, quite a long regex to verify a line. However, if you really need it, grep is not necessary.

perl -nle '/<factories.*baseQueueName/ and @r=/(?:jndiN|baseQueueN|n)ame="([^"]+)/g and print join ",",@r[1,2,0]' resources.xml

or, if the order of the strings are always the same:

perl -nle '@r=/<factories.*name="([^"]+)"\sjndiName="([^"]+)".*baseQueueName="([^"]+)/ and print join ",",@r[1,2,0]' resources.xml

performance is same with in-line first part of regex in perl vs grep + perl .

Can you explain what is the meaning of below two expressions in your command?

'?:' 
"([^"]+)

(?:) does not create a captured group. Anything inside () would be saved into a group; we do not want that sometime, (like in that occasion).
"([^"]+) match a " and keeps matching, as a captured group, anything until it meets another " . That last " is not included in the group.

Using the following bash script on OS X El Capitan (version 10.11.5) with a 2.8 GHz Intel Core i7 (4 core) processor and a 1TB SSD holding my data and code, the following script:

#!/bin/bash
printf 'perl results:\n'
time perl -nle '/<factories.*baseQueueName/ and @r=/(?:jndiN|baseQueueN|n)ame="([^"]+)/g and print join ",",@r[1,2,0]' resources.xml

printf '\nawk results:\n'
time awk '
BEGIN {	# Define ERE to match double-quoted string.
	dqsERE = "\"[^\"]*\""

	# Construct array of extended regular expression to match attributes...
	# First, the attribute name...
	EREs[++nEREs] = " jndiName="
	EREs[++nEREs] = " baseQueueName="
	EREs[++nEREs] = " name="

	# Save the lengths of the attribute names and add an ERE to match the
	# double-qouted string following the attribute name.
	for(i = 1; i <= nEREs; i++) {
		offset = length(EREs) + 1
		EREs = EREs dqsERE
	}
}
/<factories.*baseQueueName/ {
	# We have an XML line to process.
	# Clear the output string.
	out = ""
	for(i = 1; i <= nEREs; i++t) {
		# For each desired attribute, look for a match...
		if(match($0, EREs))
			# A match was found for this attribute, add the data
			# from the double-quoted string to the output string.
			out = out substr($0, RSTART + offset,
				RLENGTH - offset - 1)
		# Whether or not a match was found, add a field separator to
		# the output string.
		out = out ((i < nEREs) ? "," : "")
	}
	# Print the accumulated output string.
	print out
}' resources.xml

printf '\nOriginal script results:\n'
time {	grep '<factories.*baseQueueName' resources.xml | while read line; do
	QUEUE_JNDI_NAME=$( echo $line | grep -o 'jndiName="[^"]*"'      | cut -d'"' -f2 )
	BASE_QUEUE_NAME=$( echo $line | grep -o 'baseQueueName="[^"]*"' | cut -d'"' -f2 )
	QUEUE_NAME=$(      echo $line | grep -o 'name="[^"]*"'          | cut -d'"' -f2 )
	echo "$QUEUE_JNDI_NAME,$BASE_QUEUE_NAME,$QUEUE_NAME"
	done
}

produces output with the average times (from 10 runs):

perl results:
jms/Queue1,TEST.QUEUE1,Queue1
jms/Queue2,TEST.QUEUE2,Queue2

real	0m0.007s
user	0m0.002s
sys	0m0.003s

awk results:
jms/Queue1,TEST.QUEUE1,Queue1
jms/Queue2,TEST.QUEUE2,Queue2

real	0m0.002s
user	0m0.001s
sys	0m0.001s

Original script results:
jms/Queue1,TEST.QUEUE1,Queue1
jms/Queue2,TEST.QUEUE2,Queue2

real	0m0.017s
user	0m0.011s
sys	0m0.016s

Note that grep on OS X does not have a -P option, so I had to modify your script to use basic REs instead of perl REs.

Note that even with commented awk code, my awk script runs in 1/3 the time needed for Aia's perl script (with the grep folded into the perl script).

Could we assume that you didn't time the grep | perl pipeline, but instead just timed the perl script that did not select only lines matching the pattern <factories.*baseQueueName ; or is awk really that much slower on your system compared to perl ?

I have SuSe Linux 11 VM with 8GM RAM(300 MB free) 2 CPU cores.

# XML has 151 matching xml nodes(lines)
 perl -nle '/<factories.*baseQueueName/ and @r=/(?:jndiN|baseQueueN|n)ame="([^"]+)/g and print join ",",@r[1,2,0]' resources.xml | wc -l
151

# I did two tests with the same script you tried on your end.
perl results:
real    0m0.008s	real    0m0.008s  
user    0m0.007s  	user    0m0.007s
sys     0m0.002s	sys     0m0.002s

awk results:
real    0m0.066s  	real    0m0.026s
user    0m0.026s    user    0m0.007s
sys     0m0.001s	sys     0m0.002s

Original script results:
real    0m0.997s    real    0m1.102s
user    0m0.621s    user    0m0.644s
sys     0m0.680s	sys     0m0.690s

It is interesting to note that on your two runs, the timings for the two perl runs were similar and the timing for the two bash , grep , cut runs were similar, but the awk timings were radically different. It is also interesting to note that on the 2nd awk run, the user and sys times were identical to the perl user and sys times, but the elapsed time was grossly longer for awk . Were you running your timing tests on an otherwise idle system?

What timing results do you get running this stripped down awk code a few times:

#!/bin/bash
printf 'awk results:\n'
time awk 'BEGIN{d="\"[^\"]*";E[1]=" jndiName=";E[2]=" baseQueueName=";E[n=3]=" name=";for(i=1;i<=n;i++){O=length(E)+1;E=Ed}}/<factories.*baseQueueName/{o="";for(i=1;i<=n;i++){if(match($0,E))o=o substr($0,RSTART+O,RLENGTH-O);o=o ((i<n)?",":"")}print o}' resources.xml

Don,
With awk I am noticing wild swings within few minutes gap. At the same time perl solution performance is pretty consistent with little variation.

I am dropping my original solution from contest. So I ran below three solutions,, multiple times also after few minutes gap.. Here are the ruff averages I am seeing..
It can't be IO since awk && perl are printing to console..

time awk 'BEGIN{d="\"[^\"]*";E[1]=" jndiName=";E[2]=" baseQueueName=";E[n=3]=" name=";for(i=1;i<=n;i++){O=length(E)+1;E=Ed}}/<factories.*baseQueueName/{o="";for(i=1;i<=n;i++){if(match($0,E))o=o substr($0,RSTART+O,RLENGTH-O);o=o ((i<n)?",":"")}print o}' resources.xml
real    0m0.007s
user    0m0.005s
sys     0m0.002s
_________
real    0m0.028s
user    0m0.025s
sys     0m0.002s


perl
time perl -nle '/<factories.*baseQueueName/ and @r=/(?:jndiN|baseQueueN|n)ame="([^"]+)/g and print join ",",@r[1,2,0]' resources.xml
real    0m0.008s
user    0m0.006s
sys     0m0.002s

time perl -nle '@r=/<factories.*name="([^"]+)"\sjndiName="([^"]+)".*baseQueueName="([^"]+)/ and print join ",",@r[1,2,0]' resources.xml
real    0m0.007s
user    0m0.005s
sys     0m0.002s

One might guess that perl is used frequently on your system and awk is used infrequently. If that is the case, perl will always be in your cache (and will get consistent timings) while after a few minutes of inactivity awk will drop out of your cache and the first run after after it has dropped out of the cache will have to be reloaded from disk (needing more time to be loaded) and subsequent runs (while it is in the cache) run on par with perl .