I am trying to delete lines from a file with the sed command.
I have two files. File a.txt 200,000 records. Here is a sample of the file.
File a.txt
2401ABC0212345678 plus 250 characters of other data
2401ABC0212345678 plus 250 characters of other data
2402ABC0212345678 plus 250 characters of other data
2403ABC0212345678 plus 250 characters of other data
2405ABC0298765432 plus 250 characters of other data
2407ABC0276543211 plus 250 characters of other data
2407ABC0265432198 plus 250 characters of other data
2401ABC0212345678 plus 250 characters of other data
2401ABC0245231448 plus 250 characters of other data
2402ABC0212345678 plus 250 characters of other data
2403ABC0222245678 plus 250 characters of other data
2405ABC0247895678 plus 250 characters of other data
2407ABC0212345678 plus 250 characters of other data
2407ABC0212345678 plus 250 characters of other data
File B has about 5,000 records.
2401ABC0212345678
2402ABC0212345678
2407ABC0265432198
If the first 18 characters of a line in file a.txt matches an entry from file b.txt I need to delete the line from file a.txt
Normally file B only contains one or two entires so I use
sed 's/2401ABC0212345678//' a.txt > a.txt
Since I don't want to run the command 5,000 time I am looking for a way to feed the entries from file b.txt into my
sed command
sed 's/next line from file b.txt //' a.txt > a.txt
If there is a better way to do this than with the sed command I am open to any other suggestions.
Thanks for your help.
nawk 'FNR==NR {a[$1];next} !($1 in a)}' b.txt a.txt
First let me say thank you this worked great.
I am new to awk and have been trying to dissect your answer so I can expand it for future uses. Here is what I have come up with. If you have time to provide any additional explanation it would be appreciated.
nawk
'
FNR==NR
I don't understand what this is doing.
FNR is the current record number in the current file. FNR is
incremented each time a new record is read.
NR is the number of input records awk has processed since the
beginning of the program's execution.
{a[$1];next}
I think the "a "refers to the first input file, an the $1 is the first
column of data with in that file.
!($1 in a)}'
The ! is NOT. and $1 again refers to the first column of data
b.txt
a.txt
nawk '
# FNR will equal NR for the FIRST file specified on the command line - read the
# definition of FNR and NR in 'man nawk'
# The associated action '{...}' executed for the FIRST file.
# 'a' is the name of an array indexed by the FIRST field ($1) of the current
# record/line from the FIRST file.
# 'next' - skip the rest of the actions, and go on to the 'next' record of the
# current file.
FNR==NR {a[$1];next}
# We get here when the FIRST file has already been completely processed
# when the ABOVE condition (FNR== NR) is no longer 'true' - processing
# SECOND file from the command line
# If the value in the FIRST column ($1) in the current record/line is NOT (!)
# in arra 'a' (populated from the FIRST file), print the current record/line.
# The absence of the ACTION associated with the condition is the same
# as printing the ENTIRE record/line.
!($1 in a)}
# End of code - specifying the input files: b.txt is the FIRST, a.txt is the SECOND.
' b.txt a.txt
HTH