Hello All-
I need help from the community while we migrate from ksh to bash. The below command is failing for my application:
awk ‘$1 == “1” {print $2} ‘ | xargs rm;
Could you share an alternative awk or sed command to replace the above. I will appreciate your help. Thanks in advance.
Welcome!
It should work in all shells and on every OperatingSystem.
Why do you think it does not?
Perhaps you feed it wrongly?
It needs two words per input line, and it will run rm on the 2. word if the 1. word is 1
Exactly what does “failing” mean? Does it give a diagnostic, do nothing, or remove the wrong file? Linux tries its best to explain its actions in the event of failure.
I can think of several possible causes:
.. This command gets the file list from stdin, and you do not show the source of that. Have you forgotten to redirect that input from a file, or run in the wrong directory, or moved that file as part of the migration?
.. Does that list contain absolute names or relative names, or a mixture? Have those files been moved to different parts of the directory tree during the migration?
.. Has that list of files been edited on a Windows machine? That is likely to put a CR character at the end of every line, which Linux will assume is part of the filename. awk can fix that, if you know how to ask it.
.. Have the failing files, or any directories in their paths, had their permissions changed during the migration? Refusing read permissions on any part of the directory tree, or write permissions on the file, will stop this from working.
We cannot suggest an alternative command until we understand why the current one is failing. As my eminent colleague states, this command is correct in any known Unix/Linux environment, and any awk version since about 1980.
Addendum:
Another possible reason for failure: throwing random filenames into xargs is a bad idea. Any special characters in a filename (space, tab, CR, newline, quotes, apostrophies, backslashes and others) will cause the filename to be split into several words, or mangled in other interesting ways. xargs has options to deal with this, the safest of which is to terminate each filename with a NUL character.
Thanks @MadeInGermany and @Paul_Pedant for your reply.
The specific command is like below:
cd /temp; wc -l Test*Error.csv | awk ‘$1 == “1” {print $2}’|xargs rm;
It should work as intended. However, throwing error did not finish OK, reply = ‘123’
We are trying to find an alternate to delete one liner error files. This is the application issue as the command works in shell but not in our application.
Thanks again.
Thank you for the corrected command. We will use the same in our application and update you with the results.
Thanks again.
This snippet has some "pretty-print" characters!
For shell and awk you need the ASCII characters:
cd /temp; wc -l Test*Error.csv | awk '$1 == "1" {print $2}' | xargs rm
Looks similar but consists of ASCII characters only.
xargs is quick and dirty, works well in many cases.
The following shell command is 100% robust, even in border cases (no file found, very many files found, space character in file name, ...):
cd /temp && for fn in Test*Error.csv; do [ -f "$fn" ] && [ $( wc -l < "$fn" ) -eq 1 ] && rm "$fn"; done
Your original snippet and this one is failing because of the quote characters.
Use straight single quotes '...', not curly quotes ‘...’.
I hope that one file does not have several million lines !
At the cost of an extra process, this would limit the cost of counting all the lines.
$( head -n 2 < "$fn" | wc -l)
Or using a single awk process:
$(awk ‘FNR > 1 { exit (1) }’ "$fn")
In fact, as awk also has the filename, you might as well have it delete the file anyway.
awk -v SQ="'" '
FNR > 1 { exit }
END { if (FNR == 1) system (sprintf ("rm -f %s\n", SQ FILENAME SQ)) }
' "${fn")
Good idea, only read two lines!
With the read builtin:
cd /temp && for fn in Test*Error.csv; do [ -f "$fn" ] && { read && ! read; } < "$fn" && rm "$fn"; done
One read must succeed, the second read must fail; then it is one line long and is deleted.
The following variant will delete empty files also:
cd /temp && for fn in Test*Error.csv; do [ -f "$fn" ] && { read; ! read; } < "$fn" && rm "$fn"; done
The second read must fail.