ksh String Manipulation - removing variables from within a variable

Hi. I'd like to remove all values in a string variable that also exist in a second variable. What is the appropriate approach to take here? I can use a 'For' loop and check each element and then populate a new string. But is there a cleaner, simpler way?

E.g. I have the following 2 variables

NAMES="John Paul George Ringo"
EXCLUDE_NAME="Ringo"

What would be the simplest way to update $NAMES so that all $EXCLUDE_NAME members have been removed?
This works but there must be better way:

for X in ${NAMES} ; do
    if [[ ! "$EXCLUDE_NAME" =~ "$X" ]]; then
       NEW_NAMES="$NEW_NAMES $X"
    fi
done

Many thanks

Simple form using bash without utilities.
Longhand OSX 10.14.1, default bash terminal.

Last login: Thu Jan 31 18:23:42 on ttys000
AMIGA:amiga~> NAMES=( John Paul George Ringo )
AMIGA:amiga~> EXCLUDE_NAME='Ringo'
AMIGA:amiga~> STRING=""
AMIGA:amiga~> for (( N=0; N<${#NAMES[$@]}; N++ ));do if [ "${NAMES[$N]}" != "${EXCLUDE_NAME}" ]; then STRING="${STRING} ${NAMES[$N]}"; fi; done
AMIGA:amiga~> echo ${STRING}
John Paul George
AMIGA:amiga~> _

Have fun...

You need to post the ksh version that you use. ksh93 has "Parameter Expansion". man ksh :

In your case:

echo ${NAMES//$EXCLUDE_NAME}
 John Paul George

In case $EXCLUDE_NAME contains several space separated strings, you'd need to run a loop across all these strings.

Your solution is not only correct - it is so even to the extent that this will work in ksh88 too. In terms of parameter expansion ksh88 and ksh93 are very much identical.

Of course, in ksh88 you would have to write:

NAMES="Mick Keith Ron Bill Charlie"
EXCLUDE_NAME="Bill"
echo "${NAMES/${EXCLUDE_NAME}/}"

instead to arrive at Mick Keith Ron Charlie . And beware of the -satisfaction option....

bakunin

Hi RudiC & bakunin...

I learn something new every day.
That expansion, new to me, works in bash too.
(Sadly it is not POSIX compliant though.)

Super stuff.
Thanks a mil'.

Bazza.

@bakunin: I can't get no ... oh!

Parameter expansion gets to combine array expansion with the substitution expansions in both bash and ksh93:

$ namea=( John Paul George Ringo )
$ exclude=Ringo
$ echo ${namea[@]//$exclude/}
John Paul George 

but notice this difference between the two shells:
bash:

$ nameb=( ${namea[@]//$exclude/} )
$ echo ${#nameb[@]}
3

ksh93:

$ nameb=( ${namea[@]//$exclude/} )
$ echo ${#nameb[@]}
4

so in bash you could use this method to delete an item from an array but in ksh93 it will simply empty the item.

Andrew

Still, this is string substitution.
// is global substitution (retry if successful)
/ is one substitution
Only ksh93 has it correctly implemented (and bash-4, while bash-2 and bash-3 have a little bug in it).
For example, Solaris ksh88:

echo "${NAMES//${EXCLUDE_NAME}}"
ksh: "${NAMES//${EXCLUDE_NAME}}": bad substitution

Hmm, in this case i stand corrected. I only used it with the sh version in AIX (which is a ksh88) and there it works. So either it is a bug from porting it to Solaris or IBM fixed it when porting the ksh to AIX. Interesting to know that this feature seems not so global as i thought it to be.

/Addendum: only now i recognised a rather careless typo in my example above, which i have now corrected: instead of the correct ${NAMES/${EXCLUDE_NAME}/} (replace ${EXCLUDE_NAME} with nothing) i used ${NAMES//${EXCLUDE_NAME}} (replace nothing with ${EXCLUDE_NAME} ??), which is fo course wrong. You may want to try again with Solaris as, alas, my trusted old U05 seems not to work any more.

@apmcd74:
Please notice that when you define a variable with:

cname=( word1 word2 word3 [...] )

you create an array, not a string. The parameter expansion ${cname/searchword/replacement} deals with strings though. i.e.

variable="++old++"
echo ${variable/old/NEW}
++NEW++

bakunin

Thanks for the reply. Appreciate it.

When you want to give thanks to a member for their help, please click on the "thumbs up" icon in the post(s) you want to thank them.

Everyone here appreciates a bit of thanks for their work.

Gave a thumbs up as well. Definitely appreciate the feedback and excellent discussion.

My point was that you can do string manipulation per element, or for all elements in an array of strings, thus:

$ array=( john paul george ringo )
$ echo ${array[0]/h/}
jon
$ for btl in ${array[@]^}
> do
>   echo $btl
> done
John
Paul
George
Ringo
$

As I said, this works for bash and ksh93 on my Ubuntu systems; I haven't tried it out on ksh88.

Andrew

I'm happy (or should I be sorry?) to be able to correct your humble self-correction: the original expression was NOT wrong but corresponds to the second form quoted below. man ksh :

I pointed out earlier that // means "global". That makes sense because an empty search would not make sense.
Now RudiC opened my eyes for the /# and /% modifiers. Again, makes sense!

OMG bash-4 comes with even more modifiers:

 array=( john paul george ringo )
 for btl in ${array[@]}; do echo "$btl"; done 
john
paul
george
ringo
 for btl in ${array[@]^}; do echo "$btl"; done
John
Paul
George
Ringo
 for btl in ${array[@]^^}; do echo "$btl"; done
JOHN
PAUL
GEORGE
RINGO

I like these ones in particular:

echo ${array[@]//g}
john paul eore rino
echo ${array[@]//[og]}
jhn paul ere rin

Back to question#1.
If you really want simple strings, you need nested for loops.
Still things look easy if you hide the complexity in a function.

#!/bin/sh
NAMES_without(){
  for i in $NAMES
  do
    for j
    do
      [ "$i" = "$j" ] && continue 2
    done
    echo "$i"
  done
}

NAMES="John
Paul
George
Ringo"
EXCLUDE_NAMES="Ringo
Paul"
NAMES=$(NAMES_without $EXCLUDE_NAMES)
echo "$NAMES"

Output is

John
George

As you see, I have taken a newline as separator. Often this is handier than a space.

@MadeInGermany: Why should multiple / repeated "Pattern Substitution" not work on simple strings? No loop needed. With the variables as defined in post #16 (and falling back to bash and it's extglob / extended pattern matching):

echo "${NAMES//@(${EXCLUDE_NAMES//$'\n'/|})}"
John

George

You may apply another expansion to remove duplicate separators.

@RudiC, that seems to only work in an interactive shell.?

@MadeInGermany: no, works as well in a shell script. Don't forget to

shopt -s extglob

prior to executing the (complex) expansion.

echo $BASH_VERSION 
4.4.19(1)-release

Right, I forgot to set the extglob!