parsing xml file

annar · November 10, 2011, 6:58am

Hello!

We need to parse weblogic config.xml file and display rows in format:

machine:listen-port:name:application_name

In our enviroment the output should be (one line for every instance):

 
Crm-Test-Web:8001:PIA:peoplesoft
Crm-Test-Web:8011:PIA:peoplesoft
Crm-Test-Web:8021:PIA:peoplesoft
:9999:WebLogicAdmin:

The config.xml file looks like:

<?xml version='1.0' encoding='UTF-8'?>
<domain xmlns="http://xmlns.oracle.com/weblogic/domain" xmlns:sec="http://xmlns.oracle.com/weblogic/security" xmlns:wls="http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http:... ...">
<name>peoplesoft</name>
<domain-version>10.3.3.0</domain-version>
<security-configuration>
<name>peoplesoft</name>
<realm>
<sec:authentication-provider xsi:type="wls:default-authenticatorType"></sec:authentication-provider>
<sec:authentication-provider xsi:type="wls:default-identity-asserterType">
<sec:active-type>AuthenticatedUser</sec:active-type>
</sec:authentication-provider>
<sec:role-mapper xmlns:xac="http://xmlns.oracle.com/weblogic/security/xacml" xsi:type="xac:xacml-role-mapperType"></sec:role-mapper>
<sec:authorizer xmlns:xac="http://xmlns.oracle.com/weblogic/security/xacml" xsi:type="xac:xacml-authorizerType"></sec:authorizer>
<sec:adjudicator xsi:type="wls:default-adjudicatorType"></sec:adjudicator>
<sec:credential-mapper xsi:type="wls:default-credential-mapperType"></sec:credential-mapper>
<sec:cert-path-provider xsi:type="wls:web-logic-cert-path-providerType"></sec:cert-path-provider>
<sec:cert-path-builder>WebLogicCertPathProvider</sec:cert-path-builder>
<sec:name>myrealm</sec:name>
<sec:password-validator xmlns:pas="http://xmlns.oracle.com/weblogic/security/providers/passwordvalidator" xsi:type="pas:system-password-validatorType">
<sec:name>SystemPasswordValidator</sec:name>
<pas:min-password-length>8</pas:min-password-length>
<pas:min-numeric-or-special-characters>1</pas:min-numeric-or-special-characters>
</sec:password-validator>
</realm>
<default-realm>myrealm</default-realm>
<credential-encrypted>{AES}H+LpfH72QMbs6SzJPEAgQSDLzuKuDWf8gfnMJCjVjxHPBUgXKSGs0VP0xNngwj2wDZtEmuX99q/3dSxLSW2khhki1acWq1XtG/MytpJTM1grhNQv4rxtPeUjMs2xWxCx</credential-encrypted>
<node-manager-username>system</node-manager-username>
<node-manager-password-encrypted>{AES}neDvoLnRRW2UiGENxwO2EiCDsQa0Q1KMU109Nby0Vfs=</node-manager-password-encrypted>
<enforce-valid-basic-auth-credentials>false</enforce-valid-basic-auth-credentials>
</security-configuration>
<server>
<name>WebLogicAdmin</name>
<ssl>
<name>WebLogicAdmin</name>
<listen-port>443</listen-port>
</ssl>
<log>
<name>WebLogicAdmin</name>
<file-name>./logs/WebLogicAdmin_weblogic.log</file-name>
<log-file-severity>Info</log-file-severity>
<redirect-stdout-to-server-log-enabled>true</redirect-stdout-to-server-log-enabled>
</log>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>50</thread-count>
</execute-queue>
<listen-port>9999</listen-port>
<web-server>
<name>WebLogicAdmin</name>
<web-server-log>
<name>WebLogicAdmin</name>
<file-name>./logs/WebLogicAdmin_access.log</file-name>
<logging-enabled>false</logging-enabled>
</web-server-log>
<https-keep-alive-secs>120</https-keep-alive-secs>
</web-server>
<iiop-enabled>false</iiop-enabled>
<listen-address></listen-address>
<staging-directory-name>./stage</staging-directory-name>
<upload-directory-name>./upload</upload-directory-name>
<staging-mode>nostage</staging-mode>
<graceful-shutdown-timeout>30</graceful-shutdown-timeout>
<msi-file-replication-enabled>true</msi-file-replication-enabled>
<custom-identity-key-store-file-name>piaconfig/keystore/pskey</custom-identity-key-store-file-name>
<custom-identity-key-store-type>JKS</custom-identity-key-store-type>
<custom-trust-key-store-file-name>piaconfig/keystore/pskey</custom-trust-key-store-file-name>
<custom-trust-key-store-type>JKS</custom-trust-key-store-type>
</server>
<server>
<name>PIA</name>
<ssl>
<name>PIA</name>
<enabled>true</enabled>
<listen-port>8443</listen-port>
<server-private-key-alias>Crm-Test-Web</server-private-key-alias>
<identity-and-trust-locations>KeyStores</identity-and-trust-locations>
</ssl>
<log>
<name>PIA</name>
<file-name>./logs/PIA_weblogic.log</file-name>
<log-file-severity>Info</log-file-severity>
<redirect-stdout-to-server-log-enabled>true</redirect-stdout-to-server-log-enabled>
</log>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>50</thread-count>
</execute-queue>
<machine>Crm-Test-Web</machine>
<listen-port>8001</listen-port>
<listen-port-enabled>true</listen-port-enabled>
<cluster xsi:nil="true"></cluster>
<web-server>
<name>PIA</name>
<web-server-log>
<name>PIA</name>
<file-name>./logs/PIA_access.log</file-name>
<logging-enabled>false</logging-enabled>
</web-server-log>
<https-keep-alive-secs>120</https-keep-alive-secs>
</web-server>
<iiop-enabled>false</iiop-enabled>
<listen-address>172.22.12.96</listen-address>
<java-compiler>javac</java-compiler>
<staging-directory-name>./stage</staging-directory-name>
<upload-directory-name>./upload</upload-directory-name>
<staging-mode>nostage</staging-mode>
<graceful-shutdown-timeout>30</graceful-shutdown-timeout>
<msi-file-replication-enabled>true</msi-file-replication-enabled>
<client-cert-proxy-enabled>false</client-cert-proxy-enabled>
<custom-identity-key-store-file-name>piaconfig/keystore/pskey</custom-identity-key-store-file-name>
<custom-identity-key-store-type>JKS</custom-identity-key-store-type>
<custom-trust-key-store-file-name>piaconfig/keystore/pskey</custom-trust-key-store-file-name>
<custom-trust-key-store-type>JKS</custom-trust-key-store-type>
<server-diagnostic-config>
<wldf-diagnostic-volume>Off</wldf-diagnostic-volume>
</server-diagnostic-config>
</server>
<server>
<name>PIA1</name>
<ssl>
<name>PIA1</name>
<enabled>true</enabled>
<listen-port>8443</listen-port>
<server-private-key-alias>Crm-Test-Web</server-private-key-alias>
<identity-and-trust-locations>KeyStores</identity-and-trust-locations>
</ssl>
<log>
<name>PIA1</name>
<file-name>./logs/PIA1_weblogic.log</file-name>
<log-file-severity>Info</log-file-severity>
<redirect-stdout-to-server-log-enabled>true</redirect-stdout-to-server-log-enabled>
</log>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>50</thread-count>
</execute-queue>
<machine>Crm-Test-Web</machine>
<listen-port>8011</listen-port>
<listen-port-enabled>true</listen-port-enabled>
<cluster xsi:nil="true"></cluster>
<web-server>
<name>PIA1</name>
<web-server-log>
<name>PIA1</name>
<file-name>./logs/PIA1_access.log</file-name>
<logging-enabled>false</logging-enabled>
</web-server-log>
<https-keep-alive-secs>120</https-keep-alive-secs>
</web-server>
<iiop-enabled>false</iiop-enabled>
<listen-address>172.22.12.96</listen-address>
<java-compiler>javac</java-compiler>
<staging-directory-name>./stage</staging-directory-name>
<upload-directory-name>./upload</upload-directory-name>
<staging-mode>nostage</staging-mode>
<graceful-shutdown-timeout>30</graceful-shutdown-timeout>
<msi-file-replication-enabled>true</msi-file-replication-enabled>
<client-cert-proxy-enabled>false</client-cert-proxy-enabled>
<custom-identity-key-store-file-name>piaconfig/keystore/pskey</custom-identity-key-store-file-name>
<custom-identity-key-store-type>JKS</custom-identity-key-store-type>
<custom-trust-key-store-file-name>piaconfig/keystore/pskey</custom-trust-key-store-file-name>
<custom-trust-key-store-type>JKS</custom-trust-key-store-type>
<server-diagnostic-config>
<wldf-diagnostic-volume>Off</wldf-diagnostic-volume>
</server-diagnostic-config>
</server>
<server>
<name>PIA2</name>
<ssl>
<name>PIA2</name>
<enabled>true</enabled>
<listen-port>8443</listen-port>
<server-private-key-alias>Crm-Test-Web</server-private-key-alias>
<identity-and-trust-locations>KeyStores</identity-and-trust-locations>
</ssl>
<log>
<name>PIA2</name>
<file-name>./logs/PIA2_weblogic.log</file-name>
<log-file-severity>Info</log-file-severity>
<redirect-stdout-to-server-log-enabled>true</redirect-stdout-to-server-log-enabled>
</log>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>50</thread-count>
</execute-queue>
<machine>Crm-Test-Web</machine>
<listen-port>8021</listen-port>
<listen-port-enabled>true</listen-port-enabled>
<cluster xsi:nil="true"></cluster>
<web-server>
<name>PIA2</name>
<web-server-log>
<name>PIA2</name>
<file-name>./logs/PIA2_access.log</file-name>
<logging-enabled>false</logging-enabled>
</web-server-log>
<https-keep-alive-secs>120</https-keep-alive-secs>
</web-server>
<iiop-enabled>false</iiop-enabled>
<listen-address>172.22.12.96</listen-address>
<java-compiler>javac</java-compiler>
<staging-directory-name>./stage</staging-directory-name>
<upload-directory-name>./upload</upload-directory-name>
<staging-mode>nostage</staging-mode>
<graceful-shutdown-timeout>30</graceful-shutdown-timeout>
<msi-file-replication-enabled>true</msi-file-replication-enabled>
<client-cert-proxy-enabled>false</client-cert-proxy-enabled>
<custom-identity-key-store-file-name>piaconfig/keystore/pskey</custom-identity-key-store-file-name>
<custom-identity-key-store-type>JKS</custom-identity-key-store-type>
<custom-trust-key-store-file-name>piaconfig/keystore/pskey</custom-trust-key-store-file-name>
<custom-trust-key-store-type>JKS</custom-trust-key-store-type>
<server-diagnostic-config>
<wldf-diagnostic-volume>Off</wldf-diagnostic-volume>
</server-diagnostic-config>
</server>
<server>
<name>RPS</name>
<ssl>
<name>RPS</name>
<enabled>true</enabled>
<listen-port>8443</listen-port>
<identity-and-trust-locations>KeyStores</identity-and-trust-locations>
</ssl>
<log>
<name>RPS</name>
<file-name>./logs/RPS_weblogic.log</file-name>
<log-file-severity>Info</log-file-severity>
<redirect-stdout-to-server-log-enabled>true</redirect-stdout-to-server-log-enabled>
</log>
<instrument-stack-trace-enabled>false</instrument-stack-trace-enabled>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>50</thread-count>
</execute-queue>
<listen-port>8080</listen-port>
<web-server>
<name>RPS</name>
<web-server-log>
<name>RPS</name>
<file-name>./logs/RPS_access.log</file-name>
<logging-enabled>false</logging-enabled>
</web-server-log>
<https-keep-alive-secs>120</https-keep-alive-secs>
</web-server>
<iiop-enabled>false</iiop-enabled>
<listen-address></listen-address>
<staging-directory-name>./stage</staging-directory-name>
<upload-directory-name>./upload</upload-directory-name>
<staging-mode>nostage</staging-mode>
<msi-file-replication-enabled>true</msi-file-replication-enabled>
<custom-identity-key-store-file-name>piaconfig/keystore/pskey</custom-identity-key-store-file-name>
<custom-identity-key-store-type>JKS</custom-identity-key-store-type>
<custom-trust-key-store-file-name>piaconfig/keystore/pskey</custom-trust-key-store-file-name>
<custom-trust-key-store-type>JKS</custom-trust-key-store-type>
</server>
<server>
<name>PSEMHUB</name>
<ssl>
<name>PSEMHUB</name>
<enabled>false</enabled>
<listen-port>8002</listen-port>
<identity-and-trust-locations>KeyStores</identity-and-trust-locations>
</ssl>
<log>
<name>PSEMHUB</name>
<file-name>./logs/PSEMHUB_weblogic.log</file-name>
<log-file-severity>Info</log-file-severity>
<redirect-stdout-to-server-log-enabled>true</redirect-stdout-to-server-log-enabled>
</log>
<instrument-stack-trace-enabled>false</instrument-stack-trace-enabled>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>50</thread-count>
</execute-queue>
<listen-port>8081</listen-port>
<web-server>
<name>PSEMHUB</name>
<web-server-log>
<name>PSEMHUB</name>
<file-name>./logs/PSEMHUB_access.log</file-name>
<logging-enabled>false</logging-enabled>
</web-server-log>
</web-server>
<iiop-enabled>false</iiop-enabled>
<listen-address></listen-address>
<staging-directory-name>./stage</staging-directory-name>
<upload-directory-name>./upload</upload-directory-name>
<staging-mode>nostage</staging-mode>
<msi-file-replication-enabled>true</msi-file-replication-enabled>
<custom-identity-key-store-file-name>piaconfig/keystore/pskey</custom-identity-key-store-file-name>
<custom-identity-key-store-type>JKS</custom-identity-key-store-type>
<custom-trust-key-store-file-name>piaconfig/keystore/pskey</custom-trust-key-store-file-name>
<custom-trust-key-store-type>JKS</custom-trust-key-store-type>
</server>
<server>
<name>PSOL</name>
<ssl>
<name>PSOL</name>
<enabled>false</enabled>
<listen-port>6002</listen-port>
<identity-and-trust-locations>KeyStores</identity-and-trust-locations>
</ssl>
<log>
<name>PSOL</name>
<file-name>./logs/PSOL_weblogic.log</file-name>
<log-file-severity>Info</log-file-severity>
<redirect-stdout-to-server-log-enabled>true</redirect-stdout-to-server-log-enabled>
</log>
<instrument-stack-trace-enabled>false</instrument-stack-trace-enabled>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>50</thread-count>
</execute-queue>
<listen-port>6001</listen-port>
<web-server>
<name>PSOL</name>
<web-server-log>
<name>PSOL</name>
<file-name>./logs/PSOL_access.log</file-name>
<logging-enabled>false</logging-enabled>
</web-server-log>
</web-server>
<iiop-enabled>false</iiop-enabled>
<listen-address></listen-address>
<staging-directory-name>./stage</staging-directory-name>
<upload-directory-name>./upload</upload-directory-name>
<staging-mode>nostage</staging-mode>
<graceful-shutdown-timeout>30</graceful-shutdown-timeout>
<msi-file-replication-enabled>true</msi-file-replication-enabled>
<custom-identity-key-store-file-name>piaconfig/keystore/pskey</custom-identity-key-store-file-name>
<custom-identity-key-store-type>JKS</custom-identity-key-store-type>
<custom-trust-key-store-file-name>piaconfig/keystore/pskey</custom-trust-key-store-file-name>
<custom-trust-key-store-type>JKS</custom-trust-key-store-type>
</server>
<cluster>
<name>peoplesoftCluster</name>
<multicast-address>239.192.0.0</multicast-address>
<multicast-port>7001</multicast-port>
<cluster-messaging-mode>unicast</cluster-messaging-mode>
</cluster>
<embedded-ldap>
<name>peoplesoft</name>
<credential-encrypted>{AES}C41eNE8xTc8w1i/+kZblVFUqK+vW9GheuNpZX96kJb0jd2UuAnYZqwewV32L76H3</credential-encrypted>
</embedded-ldap>
<configuration-version>10.3.3.0</configuration-version>
<app-deployment>
<name>peoplesoft</name>
<target>PIA,PIA1,PIA2</target>
<module-type>ear</module-type>
<source-path>applications/peoplesoft</source-path>
<sub-deployment>
<name>PORTAL.war</name>
<target>peoplesoftCluster</target>
</sub-deployment>
<sub-deployment>
<name>/PSIGW</name>
<target>peoplesoftCluster,PIA</target>
</sub-deployment>
<sub-deployment>
<name>/PSINTERLINKS</name>
<target>peoplesoftCluster,PIA</target>
</sub-deployment>
<sub-deployment>
<name>/PSOL</name>
<target>PIA,PSOL</target>
</sub-deployment>
<sub-deployment>
<name>/PSEMHUB</name>
<target>PIA,PSEMHUB</target>
</sub-deployment>
<sub-deployment>
<name>/pspc</name>
<target>PIA</target>
</sub-deployment>
<sub-deployment>
<name>/testsuite</name>
<target>PIA</target>
</sub-deployment>
<sub-deployment>
<name>/wsrptest</name>
<target>PIA</target>
</sub-deployment>
<sub-deployment>
<name>/helloportletapp</name>
<target>PIA</target>
</sub-deployment>
<deployment-order>1</deployment-order>
<security-dd-model>DDOnly</security-dd-model>
<staging-mode>nostage</staging-mode>
</app-deployment>
<app-deployment>
<name>HttpClusterServlet</name>
<target>RPS</target>
<module-type>war</module-type>
<source-path>applications/HttpClusterServlet</source-path>
<deployment-order>1</deployment-order>
<security-dd-model>DDOnly</security-dd-model>
<staging-mode>nostage</staging-mode>
</app-deployment>
<app-deployment>
<name>HttpProxyServlet</name>
<target>RPS</target>
<module-type>war</module-type>
<source-path>applications/HttpProxyServlet</source-path>
<deployment-order>1</deployment-order>
<security-dd-model>DDOnly</security-dd-model>
<staging-mode>nostage</staging-mode>
</app-deployment>
<app-deployment>
<name>PORTAL</name>
<target>PIA,PIA1,PIA2</target>
<module-type>war</module-type>
<source-path>applications/peoplesoft/PORTAL.war</source-path>
<security-dd-model>DDOnly</security-dd-model>
</app-deployment>
<machine xsi:type="unix-machineType">
<name>Crm-Test-Web</name>
<node-manager>
<nm-type>SSL</nm-type>
<listen-port>5556</listen-port>
<debug-enabled>false</debug-enabled>
</node-manager>
</machine>
<admin-server-name>WebLogicAdmin</admin-server-name>
</domain>

Thank you very much!

zaxxon · November 10, 2011, 7:03am

What about if you'd mark at least the places in this small xml snippet (maybe marking bold) where the values are that you need?
While we are at it - what have you tried so far?

annar · November 10, 2011, 7:10am

We have a script that works for a previous version of weblogic (config.xml file looks completly different). We tried to change the old script, or use awk command. But it was unsuccessfull.

Thank you for your fast feedback.

radoulov · November 10, 2011, 7:36am

If you can install XMLgawk:

xgawk -lxml 'XMLCHARDATA { 
  data[XMLPATH] = $0 
  }
  XMLENDELEM ~ /machine|listen-port/ &&
  XMLPATH ~ /domain\/server\/(listen-port|machine)/ { 
    print data["/domain/server/machine"], data["/domain/server/listen-port"],
    data["/domain/server/name"], data["/domain/name"]
    }' OFS=: sample.xml

I get this output with your xml:

:9999:WebLogicAdmin:peoplesoft
Crm-Test-Web:9999:PIA:peoplesoft
Crm-Test-Web:8001:PIA:peoplesoft
Crm-Test-Web:8001:PIA:peoplesoft
Crm-Test-Web:8001:PIA1:peoplesoft
Crm-Test-Web:8011:PIA1:peoplesoft
Crm-Test-Web:8011:PIA1:peoplesoft
Crm-Test-Web:8011:PIA2:peoplesoft
Crm-Test-Web:8021:PIA2:peoplesoft
Crm-Test-Web:8021:PIA2:peoplesoft
Crm-Test-Web:8080:RPS:peoplesoft
Crm-Test-Web:8081:PSEMHUB:peoplesoft
Crm-Test-Web:6001:PSOL:peoplesoft

annar · November 16, 2011, 4:27am

Hello!
We installed XMLgawk and ran your script.
We get wrong results and too many rows:

:9999:WebLogicAdmineoplesoft
Crm-Test-Web:9999:PIA: peoplesoft
Crm-Test-Web:8001:PIA: peoplesoft
Crm-Test-Web:8001:PIA: peoplesoft
Crm-Test-Web:8001:PIA1: peoplesoft
Crm-Test-Web:8011:PIA1: peoplesoft
Crm-Test-Web:8011:PIA1: peoplesoft
Crm-Test-Web:8011:PIA2: peoplesoft
Crm-Test-Web:8021:PIA2: peoplesoft
Crm-Test-Web:8021:PIA2: peoplesoft
Crm-Test-Web:8080:RPS: peoplesoft
Crm-Test-Web:8081:PSEMHUB: peoplesoft
Crm-Test-Web:6001:PSOL: peoplesoft

We expecting to get only 4 next rows:

:9999:WebLogicAdmin: peoplesoft
Crm-Test-Web:8001:PIA: peoplesoft
Crm-Test-Web:8011:PIA1: peoplesoft
Crm-Test-Web:8021:PIA2: peoplesoft

What we need to change in the script?

Thank you

radoulov · November 16, 2011, 5:22am

Yes, the code need to be corrected.
Using the latest xml attached to your post the following code produces the following result:

xgawk -lxml 'XMLCHARDATA {
  d[XMLPATH] = $0
  }
XMLENDELEM == "name" {
  XMLPATH == "/domain/server/name" && name = d["/domain/server/name"]
  XMLPATH == "/domain/name"        && domain_name = d["/domain/name"]
  }
XMLENDELEM == "listen-port" && XMLPATH == "/domain/server/listen-port" {
  listen_port = d["/domain/server/listen-port"]
  }
XMLENDELEM == "machine" && XMLPATH == "/domain/server/machine" {
  machine = d["/domain/server/machine"]
  }
XMLENDELEM == "server" {  
  print machine, listen_port, name, domain_name
  }' OFS=: sample.xml

Output:

:9999:WebLogicAdmin:peoplesoft
Crm-Test-Web:8001:PIA:peoplesoft
Crm-Test-Web:8011:PIA1:peoplesoft
Crm-Test-Web:8021:PIA2:peoplesoft
Crm-Test-Web:8080:RPS:peoplesoft
Crm-Test-Web:8081:PSEMHUB:peoplesoft
Crm-Test-Web:6001:PSOL:peoplesoft

Is it wrong?

Edit: rewritten using the switch statement:

xgawk -lxml 'XMLCHARDATA { d[XMLPATH] = $0 }
{ 
  switch (XMLENDELEM) {
    case "name": 
      switch (XMLPATH) { 
        case "/domain/server/name":  
          name = d["/domain/server/name"]
          break
        case "/domain/name":
          domain_name = d["/domain/name"]
          break
        }
    case "listen-port":
      XMLPATH == "/domain/server/listen-port" && 
        listen_port = d["/domain/server/listen-port"]
      break
    case "machine": 
      XMLPATH == "/domain/server/machine" && 
        machine = d["/domain/server/machine"]
      break
    case "server":  
      print machine, listen_port, name, domain_name
      break  
    }
  }' OFS=: sample.xml

annar · November 16, 2011, 5:28am

Hi!
Yes, this is wrong, because:

only PIA listen to port 8001
only PIA1 listen to port 8011
only PIA2 listen to port 8021
and only WeblogicAdmin listen to port 9999.

So we expecting to get only 4 rows.

---------- Post updated at 10:28 AM ---------- Previous update was at 10:27 AM ----------

Also, we need only PIA, PIA1,PIA2, admin.

Thank you.

radoulov · November 16, 2011, 5:28am

But in my last output all this is true. The only difference seems to be related to the number of rows you're expecting.

radoulov · November 16, 2011, 5:30am

OK, but based on what criteria? Some pattern?

annar · November 16, 2011, 7:42am

This is OK now!
Thank you.