Removing lines from a file being used by another process using SHELL script

Hi All,

Need a small help in writing a shell script which can delete a few lines from a file which is currently being used by another process.

File gets appended using tee -a command due to which its size is getting increased.
Contents like :

[INFO] 25/09/2012 05:18 Run ID:56579677-1

My requirement is to remove lines which are more than 1 month old...
for this i will cut month from the line i.e. 09 and backtrack to fetch the previous value i.e. 8
and remove all lines from top till the row fetched...

Query is can i remove the lines in the same file ? (what if another process comes to write in that file using tee -a)..

Thanks,
Nik

This is how I remove unwanted lines in syslog, it will give you the idea...

cat syslog.log|grep -vFi -e sudo -e tty?? \
-e smbd -e nmbd \
-e "above message repeats " \
-e " registrar/tcp: Conn" \
-e "-prod" >tata
sleep 2
cat tata>syslog.log

I would guess that it depends on how you remove the top lines. This is at the bash prompt...

$ echo x > file1

$ ( ( echo a ; sleep 30 ; echo b ) | tee -a file1 ) &
[1] 7000

$ a


$ sed -i 1d file1
sed: cannot rename ./sedCDnEEq: Permission denied

$ cat file1
x
a

$ awk '{a[NR]=$0}END{close(FILENAME);for(i=2;i<=NR;i++)print a>FILENAME}' file1

$ cat file1
a

$ b

[1]+  Done                    ( ( echo a; sleep 30; echo b ) | tee -a file1 )

$ cat file1
a
b

$

In the example above, I created a file then submitted a background process to append to the file. Using sed -i to remove the first line didn't work, but awk worked okay.

Are you in linux? Try flock command. Beware: file locks are NOT mandatory, so every command accessing the file must do it through flock.

Here is an example:

$ cat file
25/08/2012
31/08/2012
4/09/2012
5/10/2012

$ flock file -c "sed '1,/\/08\/2012/ d' <file >tmpfile; cat tmpfile >file" & flock file -c "echo yadayadayada |tee -a file >/dev/null"
[1] 22160
[1]+  Completato              flock file -c "sed '1,/\/08\/2012/ d' <file >tmpfile; cat tmpfile >file"

$ cat file
4/09/2012
5/10/2012
yadayadayada

To have a real feel of what's happening, reset the original file content and use some sleeps, as in:

flock file -c "sed '1,/\/08\/2012/ d' <file >tmpfile; cat tmpfile >file; sleep 10" & \
sleep 3; echo "sed has worked; cat has worked, here is the file:"; cat file; \
echo; echo "waiting the last 7 seconds for tee..."; \
flock file -c "echo yadayadayada |tee -a file >/dev/null"; \
echo "tee has been able to write just now. Here is the file:"; cat file

To sum it up:

a) to delete your old lines, use flock file -c "sed ... <file >tmpfile; cat tmpfile >file" ;

b) prepend flock file -c "... to your tee -a file command accessing the file.
--
Bye