Curl and write custom result to file

Shell Programming and Scripting
1

I have the following script,

for i in $(cat MyWebApps); do curl -u manager:tH1s1s4f3k3p4ssw0Rd http://10.10.10.10:7529/manager/jmxproxy/"?get=Catalina:type=Manager,context=/$i,host=localhost&att=activeSessions"; done

Which gives me an output like this

OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp1,host=localhost' - activeSessions = 2
OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp2,host=localhost' - activeSessions = 5
OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp3,host=localhost' - activeSessions = 180
OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp4,host=localhost' - activeSessions = 300
OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp5,host=localhost' - activeSessions = 72
OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp6,host=localhost' - activeSessions = 89

Based on the output, I need to write in a csv file as following, can you help on this?

2019-04-19-10:05,MyWebApp1,2
2019-04-19-10:05,MyWebApp2,5
2019-04-19-10:05,MyWebApp3,180
2019-04-19-10:05,MyWebApp4,300
2019-04-19-10:05,MyWebApp5,72
2019-04-19-10:05,MyWebApp6,89

INFO:
SUSE Linux Enterprise Server 11
Thank you.

2

Any attempts / ideas / thoughts from your side? What have you learned from your other threads, e.g. this one?

You might want to add your OS, shell, and tools' versions so people know what can be done and what to avoid.

3

Thanks Rudic,
info for server added.

4

Thanks for the server info. What about the other questions?

5

I tryied multiple versions of the script adding awk grep etc, none worked, so i asked for help, let me know if you have any ideas,
thank yuo.

6

Show your attempts and where and how they failed.

7 
for i in $(cat MyWebApps); do curl -u manager:tH1s1s4f3k3p4ssw0Rd http://10.10.10.10:7529/manager/jmxproxy/"?get=Catalina:type=Manager,context=/$i,host=localhost&att=activeSessions" |awk -F, '{print $6}'; done

And prints

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    97  100    97    0     0  21661      0 --:--:-- --:--:-- --:--:-- 32333
9
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    96  100    96    0     0  11370      0 --:--:-- --:--:-- --:--:-- 13714
8

That is incompatible with the output you showed in post #1. Looks like something is profoundly messed up here. Run the curl statement for one single webapp, and post the entire session's log here.

9

So when i add any commands to the first script example below,i get a lot more stuff, that could be the reason that i cant ge't what i need,

for i in $(cat MyWebApps); do curl -u manager:tH1s1s4f3k3p4ssw0Rd http://10.10.10.10:7529/manager/jmxproxy/"?get=Catalina:type=Manager,context=/$i,host=localhost&att=activeSessions" | grep 'context'; done

OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp1,host=localhost' - activeSessions = 0
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    95  100    95    0     0  26942      0 --:--:-- --:--:-- --:--:-- 47500
OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp2,host=localhost' - activeSessions = 0
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    96  100    96    0     0  25552      0 --:--:-- --:--:-- --:--:-- 48000

When i run the curl fo one single webapp it prints one single line

curl -u manager:tH1s1s4f3k3p4ssw0Rd http://10.10.10.10:7529/manager/jmxproxy/"?get=Catalina:type=Manager,context=/MyWebApp1,host=localhost&att=activeSessions"

OK - Attribute get 'Catalina:type=Manager,context=/MyWebApp1,host=localhost' - activeSessions = 2
10

A bit difficult to believe. When you grep for "context", why should it print all those lines? What's the contents of the file MyWebApps ?

11

Please note RudiC that i'm curling a tomcat manager Sessions tab specific,
i'm not curling the full page, the contente of the MyWebApps are only all the names of the webapps.
Thanks for you help.

12

Hello RudiC,
finally i figured it out,
the worng output was a terminal issue,

for i in $(cat MyWebApps); do curl -u manager:tH1s1s4f3k3p4ssw0Rd http://10.10.10.10:7529/manager/jmxproxy/"?get=Catalina:type=Manager,context=/$i,host=localhost&att=activeSessions" |awk '{print $5," "$9}'|  awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0 }' | awk '{gsub("Catalina:type=Manager,context=/", "");print}' | awk '{gsub(",host=localhost", "");print}' >> sessions.log; done

Let me know if you have any better ideas on this.
Thank you for the support.

1 Like
13

Good you found a solution on your own. How about some improvements? read the MyWebApps file in lieu of cat ting it in a "command substitution" (saves a process creation), write to the log file outside the loop (avoids opening / closing it over and over again), and combine five awk executions into one, like

while read i; do curl ...$i... | awk -F"[,/ ]" -vDT="$(date +"%Y-%m-%d-%H:%M")" -vOFS="," '{print DT, $7, $NF}'; done < MyWebApps > sessions.log