Read from last point

Shell Programming and Scripting
1

Hi again,

first thanks for all your suggestions. This forum it is very useful.
I have a question. Is it possible to read from the last line a file was closed. For example, imagine that i've got a file with a LOT of timestamps :

1467387616.868717770
1467387616.874189609
1467387616.888759180
1467387616.894188020
1467387616.908742591
1467387616.914199431
(..)

and i need to read it in block of 70 lines each time, BUT since the file is updating quite fast i need to return to read it form the last (awk?) command without missing any line.

i don't know itf i render the idea or if it's possible. Maybe i need a program instead.

Thanks!
Kind Regards

2

You can save the last timestamp processed by your awk script and reload that value in your shell script before calling awk the next time and then have awk skip over all lines with a timestamp that is less than or equal to the saved timestamp. For example:

#!/bin/ksh
IAm=${0##*/}
TSF="$HOME/.LastTimeStamp"

if [ ! -f $TSF ]
then	printf '%s: Time stamp file (%s) not found.\n\tProcessing all timestamps.\n' \
	    "$IAm" "$TSF" >&2
	ts=0
else	read ts < "$TSF"
fi
awk -v TS="$ts" -v TSF="$TSF" '
$1 + 0 <= TS {
	# Skip previous processed timestamps.
	next
}
{	# Do what every you want to do with the new timestamp...
	# Save last completed time stamp.
	print $1 > TSF
	close(TSF)
}' input_file...

Obviously, you should also verify that the read was successful, but this might work as a starting point for what you need.

1 Like
3

i'll give a try. Thanks for your support.

Kind Regards

4

Using Perl

#!/usr/bin/perl
open (FH,"/path/to/file.txt") or die "cannot open file \n";
my $pos=0;
while (1) {
  seek(FH,$pos,0);
  while (<FH>) {
    print $_;
  }
  $pos=tell(FH);
}
5 
#!/bin/bash
f=0
while [ ${f} -eq 0 ] ; do
    for i in {1..70} ; do
        read a || { f=1 ; break ; }
        echo "${a}"
    done
done < <(tac yourfile.txt)