Print filename and last line using awk

Shell Programming and Scripting
1

Hi,
using awk command I want to print filenames and the last line of each file, in a single command line statement.
I want to use 'awk', because I want to add more functionality to this logic later.

I tried the following on *.sh files in the current directory

find . -type f -name "*.sh" -exec awk 'NR = FNR {print FILENAME NR}' {} \;

but the above command prints all line numbers of every file and not just the last line
I understand that 'NR' stands for line number and not the line content itself

How can I print the filename, along with ONLY the last line (content) of the file?

Thanks,
-sri

2

You could use something like this:

find . -type f -name "*.sh" -exec awk '{s=$0};END{print FILENAME,s}' {} \;
3

Thanks Subbeh, it works !
Now, how can I print the filename and the last line of only those files where the last line is not like '' (empty) ?

In other words, I want list all filenames together with their last line where the file doesn't end with a new line

Thanks
-sri

4

You can simply use this:

find . -type f -name "*.sh" -exec awk '{s=$0};END{if(s)print FILENAME,s}' {} \;
5

... or simply

 find . -type f -name "*.sh" -exec awk 'END{if ($0) print FILENAME,$0}' {} \;
6

AWK is not required to preserve the value of the field variables ($0, $1, etc...) for the END section and some implementations don't. Subbeh's approach is the safest, most portable solution.

Regards,
Alister

7

Works with the original awk, mawk, nawk, gawk and gawk in its traditionnal mode. If OP needs to process big files, it's about two times faster.

See also awk.info

8

Thanks all, here's what i am trying to do, I am close but not there yet
I want to print the last line of script where the file doesn't terminate with a newline.
But checking if the last line is empty or not, i want to get the list, but it doesn't work exactly that way

Consider the following files, '$' in bold black is the command prompt, '$' in bold blue is the formatting character from -e flag

$ cat c.sh
apple$ cat b.sh
apple
$ cat a.sh
apple.sh

$ cat -e c.sh
apple$ cat -e b.sh
apple$
$ cat -e a.sh
apple.sh$
$
$

Both files b.sh and c.sh seem to have the last line as 'apple', but in c.sh the end of file is on the same line as 'apple' and in b.sh it's in the next line

I want to print filename and last line for only files of type c.sh and not b.sh

$ find . -type f -name "*.sh" -exec awk '{s=$0};END{if(s)print FILENAME,s}' {} \;
./b.sh apple
./c.sh apple
$

The above awk command doesn't differentiate between b.sh and c.sh

Thanks,
-sri

9

Elaborating on Subbeh's, how about:

awk '$0{last=$0};END{print FILENAME,last}'
10

Hi Ripat,
the new command now shows one more incorrect result too. The result should only display for c.sh

I attached the 3 .sh files here

$ find . -type f -name "*.sh" -exec awk '{s=$0};END{if(s)print FILENAME,s}' {} \;
./b.sh apple
./c.sh apple
$ find . -type f -name "*.sh" -exec awk '$0{last=$0};END{print FILENAME,last}' {} \;
./a.sh apple.sh
./b.sh apple
./c.sh apple

Thanks,
-sri

11

Windows files with CRLF?

12

I ftpd files unix to windows. So files in unix were originally created in windows.
Either way, i want to detect files not ending with a newline

13 
$ tail -1 c.sh | wc -l
0

Don't know if this will behave the same way on your OS, but on my ubuntu, if i tail -1 c.sh | wc -l i will get a 0 instead of 1 when there is no end of line.

So this may help to detect such file with no newline character.(A check that the file is not empty should be carried out as well)

14

hi ctsgnb, this c.sh has only one line without having a new line that's why "wc -l" returns zero. c.sh can have multiple lines without having newline at the end of the file, so checking for zero doesn't work

I want to print list of filenames along with their last line where the file doesn't end with a new line
Thanks

15

If you check that c.sh exists an is not empty and tail -1 c.sh| wc -l returns 0

then this file meet the condition you are looking for.
So checking for zero is not sufficient but if brought with a test -s then it should work doesn't it ?
At least, that was what i wanted to point out in my previous post.

16

The files can be zero or non-zero size.

"ALL i want is a single line command that will print all filenames in the current directory along with the file last line where the file doesn't end with a newline"

A tested working solution as answer will be appreciated

17

Files with the last newline character missing are not proper UNIX format. Awk just processes such a file as if the file did have a closing newline. So inside an awk script it cannot be detected whether or not it was present in the file..

You would need to use another way to detect whether it is present, or just convert all files to proper format, for example:

awk 1 file > newfile

GNU sed or BSD sed could use in-place editing, something like:

sed -i'.bak' '' file

--

Alister is right though, that does not work with POSIX awk or nawk on Solaris and HPUX for example. That page does not include an example where $0 is used in the END section BTW..

18

I believe what you should expect from this forum is proposals, suggestions, and consultancy so you can come up with a solution yourself AND test it by yourself. Anything beyond that is nice but not a given.
However . . . -- this is a proposal that works on my linux box BUT it does not cope with empty files. Please feel free to use and test it or dump it...

for i in *.sh; do  [ '0a' != $(od -An -tx1 -j$(( $(stat -c%s $i) - 1 )) $i) ] && { echo -ne $i"\t"; tail -n1 $i; } done
a.sh    apple$

The $ sign is the next command prompt as no newline char is output - as requested.

19

Yes, it does. Look for Print the last line of a file (emulates "tail -1") 2/3 down the page.
awk.info Handy One Liners

20

As has already been pointed out, any attempt to process an incomplete line at the end of a file with awk may or may not work. The standards say that tail (when using options that count lines), grep, and sed as well as awk produce unspecified results if the input files they process are not text files. (And, by definition, a non-empty text file's last character is a <newline>.)

A side note on the discussion about $0 in END actions in awk: The standards are silent about whether $0, $1, ... $NF are valid in END actions. However they require that FILENAME, FNR, and NR have the values they had for the last line read when used in END actions. They also require that:

Since NF has to be preserved, I don't see why $0 through $NF woudn't also be preserved under the same conditions; but the standards don't require awk to do so.

Rudi has given you a script that will work on some systems, but it won't work on OS X, Solaris, or several other UNIX and UNIX-like systems.

The following script doesn't use anything that isn't specified by the standards, so it should work on almost any system. It was tested using ksh, but should work for any POSIX conforming shell.

This script will tell you if a file is missing the terminating <newlin>, but it does not attempt to print the incomplete line. Although I could do that in a shell script, if I had a REAL need to portably print incomplete lines, I'd do it in C; not shell.

for i in *.sh
do
        if [ ! -e "$i" ] || [ ! -r "$i" ]
        then    printf "%s: not a readable regular file\n" "$i"
        elif [ ! -s "$i" ]
        then    printf "%s: empty file\n" "$i"
        else    # Use 012 below for ASCII, 045 for EBCDIC
                if ! tail -c -1 "$i" | od -b | grep -Fq ' 012'
                then    printf "%s: missing <newline>\n" "$i"
                fi
        fi
done