Remove string between two special characters

Shell Programming and Scripting
1

Hi All,

I have a variable like

AVAIL="\
BACK:bkpstg:testdb3.iad.expertcity.com:backtest|\
#AUTH:authstg:testdb3.iad.expertcity.com:authiapd|\
TEST:authstg:testdb3.iad.expertcity.com:authiapd|\
"

What I want to do here is that If a find # before any entry, remove the entire string between # and first | (including # and |). SO the resulting string would be like -

AVAIL="\
BACK:bkpstg:testdb3.iad.expertcity.com:backtest|\
TEST:authstg:testdb3.iad.expertcity.com:authiapd|\
"

With the help of this forum itself, I found the solution as below -

echo "${AVAIL}" | sed -e "s/#[^|]*|//g"

But I am not able to exactly understand how above is working. My initial understanding was that #[^|]*| deletes all entries encapsulated between # and first pipe except the ones encapsulated between two pipes but that doesn't seem to explain above transalation. I have played a good amount with sed but this one seems to be tough to understand.

Can anybody please help understand the functioning of above sed trick ?

Thanks

2 
$ echo $AVAIL | nawk -F"[#|]" '{if ($0~/#/) {print $1"|"$4}}'
3

Thanks for the nawk trick.

Sorry. I was not much clear earlier. Its not necessary that every entry between pipes will contain only 4 fields seperated by colon (:). We might have more 5 or 6 or any number of fields. So probably $4 won't be a good idea.

Moreover, nawk is not installed on our linux boxes and we don't want to install it extra when sed trick is solving the purpose. We would stick to sed.

Please help me understanding original sed trick I pasted as to how its working ?

:confused:

4

Maybe this can help (| is special in perl regex and # in regex with \x option, so it's needed to escape them here):

echo "$AVAIL" | perl -pe 's/
\#       # start your sequence with #
[^|]*    # as many any but "|" chars in the sequence as possible
\|       # finish with |
//xg'
5

Thanks for the great explaination Yazu.

But I am still confused. Say, $AVAIL is like below -

#AUTH:authstg:testdb3.iad.expertcity.com:authiapd|BACK:authstg:testdb3.iad.expertcity.com:authiapd|

According to your explaination, it will start searching from # and will search as many chars as possible till last pipe (or first pipe ?). If its searching till last pipe, the end result of

echo "${AVAIL}" | sed -e "s/#[^|]*|//g"

should be (since ^| instructing it to ignore pipe)

|

Please correct me if I wrong in understanding above.....
:wall:

6

Any but "|"
Look at your string, find sharp, then look at all chars but "|" and then find "|" and change this sequence to nothing.
BTW, "Mastering regular expressions" is really a good book.