Using sed in a loop/to remove lines contained in variable from file

I've tried numerous commands, but I am not sure how to use sed in a loop. This is what I have:

VARZ contains CARD_FILE_LIST and it also contains CARD_FILE_LIST2
so
echo "$VARZ"
CARD_FILE_LIST
CARD_FILE_LIST2

I have a file with 60 lines in /tmp/testfile it and I want those lines deleted from the file. Note that VARZ may not always contain CARD_FILE_LIST or CARD_FILE_LIST2. It may contain another line. I want the command to delete whatever values are contained by VARZ preferably with sed or awk.

I have tried:

for i in `cat /tmp/testfile`;do echo $i |sed "s/$VARZ//g"`;done
sed "/$VARZ/d" /tmp/testfile
sed '/'"${VARX}"'/d /tmp/testfile

I have the old version of sed so I can't use sed -i to make changes but obviously I could output it to another file and rename that one if I need to. I just want to get the command/loop working.

Any help is appreciated.

That won't work, because the substitute command in sed cannot take multiple lines..

If you are using bash and your OS supports it, try something like:

grep -vFf <(echo "$VARZ") /tmp/testfile

Otherwise you could try:
Try

echo "$VARZ" | grep -vfF - /tmp/testfile

Note that partial matches will also remove the entry, so if you do not want that it will need to be further refined..

Running sed 60 times to remove 60 lines is like making 60 telephone calls to say 60 words.

1 Like

It might be good to turn the short file into a script of piped together sed's or whatever, so one pas on the long file is all that is needed. This can be done in a file or env var, using eval.

Really, all that is needed is a generated sed script for one sed, unless you need more speed. 'sed' is not a shell command, but a different script interpreter.