Grep echo awk print all output on one line

Hello,

I've been trying to find the answer to this with Google and trying to browse the forums, but I haven't been able to come up with anything. If this has already been answered, please link me to the thread as I can't find it.

I've been asked to write a script that pulls a list of our CPE then have the script dig through the show.version file to pull a specific entry from there and display it all on one line.

This is the code I've come up with, but the display is all wrong. Note: I am no programmer/scripter, this is something new I'm learning for a recent promotion.

for CPE_DEVICE in `ls -d *cpe.domain.net`; do; echo $CPE_DEVICE; grep JUNOS $CPE_DEVICE/show.version | grep boot | awk '{print $5}'; done

My output comes out like this

device1-r0.cpe.domain.net/
[10.4R6.5]
device2-e0.cpe.domain.net/
device3-r0.cpe.domain.net/
[11.4R7.5]

I decided to dissect my code to see where the fault could be. I found that this half works fine in displaying the CPE devices on their own line each

for CPE_DEVICE in `ls -d *cpe.domain.net`; do; echo $CPE_DEVICE;

I'm not sure what I've done wrong in the second half of my code, but my output should look like this

device1-r0.cpe.domain.net/[10.4R6.5]
device2-e0.cpe.domain.net/[11.4R7.5]
device3-r0.cpe.domain.net/[11.4R7.5]

It's not that easy for us to tell what output comes from where. Did you try to stop echo from printing a <newline> char? On some systems (you fail to mention yours), this can be done with the -n option. Or, use printf with an adequate format string.

Please be aware that your entire pipe could probably be replaced by one single awk command.

Programming in ZSH, everything we use is based on FreeBSD.

This line gives me all my CPE in a list just fine.

for CPE_DEVICE in `ls -d *cpe.domain.net`; do; echo $CPE_DEVICE;

This half is supposed to pull the version running for each CPE and display that next to the CPE name.

grep JUNOS $CPE_DEVICE/show.version | grep boot | awk '{print $5}'; done

I tried using -n with echo and it didn't help. I don't really know anything about this scripting stuff, so all of this is new to me.

After some research I was thinking that maybe a single awk command would work, but I'm not quite there yet in figuring it out.

Read man echo to find out how to suppress the <newline> char. Or use printf if your zsh provides it.

What would be the result of

awk '/JUNOS/ && /boot/ {sub (/[^/]*$/, "", FILENAME); print FILENAME $5}' *cpe.domain.net/show.version

?

I tried echo -n but it still gives me the output wrong. It puts a lot of the devices and their software version on the same line, but I still get lines where multiple devices show together on one line with one software version.

The output comes out like this with echo -n added.

device1-e0.cpe.domain.net/[11.4R8.5]
device2-e1.cpe.domain.net/device3-e0.cpe.domain.net/[11.4R10.3]
device4-e1.cpe.domain.net/[11.4R10.3]

When I ran what you tried with the proper information filled (filename, domain) in I get this.

awk: extra ] at source line 1
 context is
        /JUNOS/ && /boot/ {sub >>>  (/[^/] <<< 
awk: syntax error at source line 1
awk: illegal statement at source line 1
        extra ]

The directory I'm running this in has a directory for each CPE device. So this script here pulls a list of all the devices and puts them in a nice neat list for me, no problems.

for CPE_DEVICE in `ls -d *cpe.domain.net`; do; echo $CPE_DEVICE;

The later half of my code is supposed to pull the show.version information, but GREPing out the word boot and printing the 5th column which is the actual software version. It does do that, however the output isn't where it should be.

grep JUNOS $CPE_DEVICE/show.version | grep boot | awk '{print $5}'; done

The end result should look like this, where the first half the script pulled each CPE device and the second half pulls the version

device1-r0.cpe.domain.net/[10.4R6.5]
device2-e0.cpe.domain.net/[11.4R7.5]
device3-r0.cpe.domain.net/[11.4R7.5]

I apologize if I'm not explaining properly or just talking in circles here.

Seems to be an awk version problem. Try to escape the / within the square brackets, like [^\/]* .

Alright, that allowed the script to run, however, it only gave me a list of the software versions without the device names.

Drop the sub (...); and post the result.

awk '/JUNOS/ && /boot/ {print devices $5}' *cpe.domain.net/show.version

That gave me the list of software versions.

I see in this '/JUNOS/ && /boot/ how it replaces my grep from my original example.

I'll use this and some time studying AWK to see if I can figure out the rest on my own when I have more time and I will post my results.

Thank you!

That is NOT what I posted. "devices" is not defined and thus the empty string. Pls post the result of

awk '/JUNOS/ && /boot/ {print FILENAME $5}' *cpe.domain.net/show.version

I apologize, I had assumed FILENAME was something I could change. I went back through what you suggested before, kept FILENAME, fixed the sub string and everything displayed exactly how I wanted it.

awk '/JUNOS/ && /boot/ {sub (/[^\/]* .$/, "", FILENAME); print FILENAME $5}' *cpe.domain.net/show.version

Thanks again for your assistance!

---------- Post updated at 02:38 AM ---------- Previous update was at 02:37 AM ----------

Just wanted to do a follow up since I'm at work and have had more time to play with this.

This script awk '/JUNOS/ && /boot/ {sub (/[^\/]* .$/, "", FILENAME); print FILENAME $5}' *cpe.domain.net/show.version got me started on what the boss wanted me to do.

Output:

device1-e0.cpe.domain.net/show.version[11.4R8.5]
device2-e0.cpe.domain.net/show.version[11.4R10.3]
device3-e1.cpe.domain.net/show.version[11.4R10.3]
device4-e0.cpe.domain.net/show.version[11.4R10.3]
device5-e1.cpe.domain.net/show.version[11.4R10.3]
device6-e0.cpe.domain.net/show.version[11.4R7.5]
device7-e1.cpe.domain.net/show.version[11.4R7.5]
device8-e0.cpe.domain.net/show.version[11.4R8.5]

From there, I had to change the way this prints so that it shows column 5 of show.version first, followed by a space, then the device name. After that, the boss wanted me to sort by the software version. Lastly, that script above left in some output I wasn't too fond of, particularly in showing /show.version at the end of each device name, so I had to cut that out. Below is the modified script meeting all the requirements, along with a sample of the output.

Script:
awk '/JUNOS/ && /boot/ {sub (/[^\/]* .$/, "", FILENAME); print $5 " " FILENAME}' *cpe.domain.net/show.version | sort -k5 | cut -d / -f 1

Output:

[10.0R4.7] device1-r0.cpe.domain.net
[10.0R4.7] device2-r0.cpe.domain.net
[10.0R4.7] device3.cpe.domain.net
[10.0R4.7] device4.cpe.domain.net
[10.0S6.1] device5-e0.cpe.domain.net
[10.0S6.1] device6-e0.cpe.domain.net
[10.1R4.4] device7-e0.cpe.domain.net
[10.1S1.3] device8.cpe.domain.net
[10.4R3.4] device9-r0.cpe.domain.net

Thanks again, RudiC! I definitely like the awk approach instead of the one I initially posted. Maybe I can get the boss on board with doing it that way too.

Glad to hear!

You seem a bit undecided if the trailing / should appear in your device string or not after getting rid of the file name. The sub (...) is there for exactly that - remove the show.version part, so your cut should be unnecessary. Obviously, the sub doesn't do what it is expected to. I guess that due to the space that you introduced which was not present in my original proposal. Playing around a bit with the regex in sub might bring you to where you want to be. You could also substitute the exact string : sub ("show.version", "", FILENAME) .

I'm somewhat surprised of the key that you use in sort ...

I definitely don't need the /show.version to appear. It doesn't hurt anything, I just prefer not to see it.

I don't fully understand what's going on in {sub (/[^\/]* .$/, "", FILENAME) , so I don't get how to modify it to do what I want.

I used sort -k5 because the boss needed the list sorted by the software version, so we can easily see which devices need upgraded. If there's a way to do that with sub as well, I'd love to learn how to make that happen.

Also, I've moved on to turning this script into a function called cpever . The problem I'm running into is that I need to be able to run it from anywhere, and I now need to cut out a bunch of stuff before the actual device name and the /show.version at the end.

Example:

cpever () {
	awk '/JUNOS/ && /boot/ {sub (/[^\/]* .$/, "", FILENAME); print $5 " " FILENAME}' 
~/svn/nw_config_data/*cpe.domain.net/show.version | sort -k5

Output:
[10.0R4.7] /home/clmbn-eng2/a/rwalker/svn/nw_config_data/device-r0.cpe.domain.net/show.version

I need to make /home/clmbn-eng2/a/rwalker/svn/nw_config_data/ and /show.version disappear so my output can go back to looking like this [10.0R4.7] device1-r0.cpe.domain.net

Using the default field separator, there's just two fields in your file, so why use -k5 for sorting?
As you don't show the directory structure, and seem to keep changing targets, I'd suggest to play around with the sub regexes until it fits what you need. Also, I still don't understand why you keep the space in the regex making it pointless.
For your function, why don't you cd into the correct directory at the beginning?

I used sort -k5 because I need it to sort by the 5th column from my print $5 , which it does. Is there a better way of doing it?

Initially, I was doing cd into the directory and running the function there and that's fine, everything displays properly (with the function edited of course to remove what I added). What I would like to do is be able to run my function from any directory as I have a few different job functions, and it's easier on me to stay in my main directory where I do a lot of my work. Eventually, I'll have everything I need automated into an alias or function so I don't have to jump around at all.

I added the space in print $5 " " FILENAME to make the output look the way I want it. I like this [10.0R4.7] device1-r0.cpe.domain.net over this [10.0R4.7]device1-r0.cpe.domain.net I just like how that looks.

Here is where I'm at now

cpever () {
	awk '/JUNOS/ && /boot/ {sub ("show.version", "", FILENAME); print $5 " " FILENAME}' ~/svn/nw_config_data/*cpe.domain.net/show.version | sort -k5
}

Which gives me the output of:
[10.0R4.7] /home/clmbn-eng2/a/rwalker/svn/nw_config_data/device-r0.cpe.domain.net/

A little more tinkering and I should get it.

Using a key outside the range makes sort ignore it and use the entire line for a key which by accident is what you want.
print $5 will print one single field which in the output is field 1 (becoming the default key, then).
You can use another sub (...) to remove the leading path from the file name.

---------- Post updated at 11:09 ---------- Previous update was at 11:08 ----------

You could cd to the working dir, and cd back to the original dir when finished.

So, I should have used sort -g instead? I tested with that and it worked too.

I know I can cd in and out as I need to. It's not that big of a deal, it's just something I wanted to accomplish with a function.

You can cd in and out in a function.

---------- Post updated at 11:30 ---------- Previous update was at 11:29 ----------

Why -g ?

-g because it sorts numerically, which I need based on the software version number. Older software versions need to be at the top of my list, which -g has also accomplished.

If this also works by accident, I'm fine with that.

---------- Post updated at 07:12 AM ---------- Previous update was at 06:37 AM ----------

I added cd to my function to jump into that directory, run the script, then cd back to where I want to be and everything is exactly as I need it.

You've taught me a lot in the last day or so, RudiC, whether it may appear that way or not haha!

Came back to playing with this. I found that just a plan old sort does the sort as I need. RudiC, I think you were trying to point me in that direction previously with your questions about why use -k5 and -g

After presenting this to the boss, he really liked it and wants to get it implemented in the system for everyone to use, but not without a few changes. This brings me back to a previous problem and introduced new issues (of course, new features = new problems to debug).

What I have currently is:

cpever () {
	awk '/JUNOS/ && /boot/ {sub ("/show.version", "", FILENAME); print $5 " " FILENAME}' ~/svn/nw_config_data/*cpe.domain.net/show.version | sort
}

Which then outputs:
[10.0R4.7] /home/clmbn-eng2/a/rwalker/svn/nw_config_data/device-r0.cpe.domain.net

I've been trying to strip this out out /home/clmbn-eng2/a/rwalker/svn/nw_config_data/ so that my output looks like this instead [10.0R4.7] device-r0.cpe.domain.net but I can't get it right. To test it I replaced "/show.version" with the full path, and that got rid of it, but left the /show.version .

RudiC, you had mentioned another sub(...) but I failed horribly at implementing it. I tried setting it up like this:

cpever () {
	awk '/JUNOS/ && /boot/ {sub ("/show.version", "", FILENAME); sub ("/home/clmbn-eng2/a/rwalker/svn/nw_config_data/"); print $5 " " FILENAME}' ~/svn/nw_config_data/*cpe.domain.net/show.version | sort
}

but I get this error, and I just am not sure how to do it right

awk: syntax error at source line 1
 context is
        /JUNOS/ && /boot/ {sub ("/show.version", "", FILENAME); sub >>>  ("/home/clmbn-eng2/a/rwalker/svn/nw_config_data/") <<< 
awk: illegal statement at source line 1

Notice that in the path after the /a/ is my username. If I want this to work for everyone, I can't have my username there. So, I tried setting it up like this "/home/clmbn-eng2/a/*/svn/nw_config_data/" to use the * as a wildcard, but that doesn't work and just shows me the full path again. I checked the /home/clmbn-eng2/a/ directory and I see all the different users there, so I'm wondering if maybe in this instance * doesn't work as a wildcard search? If that's the case, how do I wildcard it to make this command friendly for everyone?

Once I sort out how to wildcard that part of the directory, and remove the filepath up to the device name AND the "/show.version" I can move on to what the boss wants next. He wants one command that contains all 3 cpe subdomains we have that can be hit with a variable like -c , -e , or -o . I'm still researching how to do all the different variables and stuff with awk, so for now I have this setup as a starting point for after I learn some more and figure out the multiple subs to strip out stuff I don't want to show:

cver () {
	#Need a help defined by -h which would show the different variables available to the user. Also suggest grep | "xx" to find specific versions
	#cver without any variables should kick back -h message, if possible, to user that they need to define a variable such as -c, -e, -o
	#cver without those variables is no fun having everything shown to you at once	
	awk '/JUNOS/ && /boot/ {sub ("/show.version", "", FILENAME); print $5 " " FILENAME}' ~/svn/nw_config_data/*cpe.domain.net/show.version | sort
	#I want cpe to be accessed by -c
	#Example: cver -c
	#
	awk '/JUNOS/ && /boot/ {sub ("/show.version", "", FILENAME); print $5 " " FILENAME}' ~/svn/nw_config_data/*cpe2.domain.net/show.version | sort
	#I want cpe2 to be accessed by -e
	#Example: cver -e
	#
	awk '/JUNOS/ && /boot/ {sub ("/show.version", "", FILENAME); print $5 " " FILENAME}' ~/svn/nw_config_data/*cpe3.domain.net/show.version | sort
	#I want cpe3 to be accessed by -o
	#Example: cver -o
}