loop through files in each folder and perform sed

Dear all,

I have few log folders in directory (FILE) and i need to perform sed on each files inside each folders. Here is my script, and i wish to rename each file back to the same file name after modification. Can anyone guide me on my script below?

eg:

folder1 contain files ABC.log,ABCD.log
folder 2 contain files EFG.log,EFGH.log

for toclean in `ls $FILE/*`
do
basetoclean=`basename $toclean .txt`
sed '1,/============/d' $toclean > $toclean.txt
sed -e '1d' -e '2d' -e '/^$/,/Output/d' $toclean.txt > $toclean.log
done

Thanks alot and looking forward for your help!!

A few things jump out at me straight off:

for toclean in `ls $FILE/*`
do
basetoclean=`basename $toclean .txt`   # note 1 below
sed '1,/============/d' $toclean > $toclean.txt   # note 2 below
sed -e '1d' -e '2d' -e '/^$/,/Output/d' $toclean.txt > $toclean.log  #note 3
done

1) I don't understand your need to take the basename of the original file. Not to mention that you dont use the variable. Also, given your examples both indicate that the files have the suffix .log it seems removing .txt seems wrong.

2) If $toclean.txt is really just a temp file, then it is better to create a temporary file in /tmp than to use some form of the original filename. For example:

sed 's/foo/bar/'  $toclean >/tmp/cleanscript_$$.tmp1

The name makes some previsions to avoid collisions if more than one instance of the script is running concurrently, and all temp files can be easily cleaned up with one rm command at the end.

3) It appears you are trying to overlay the original file, however since the sufix wasn't stripped from the original name you'll end up with something line path/file.log.log

Further, it's not a good idea to overlay the file directly (in my opnion). Disk fill, and other things causes commands to incorrectly generate, or fail to generate, output, so until you are sure that the command was successful, I'd write the second output to another temp file and check the return code. If the return code indicates success, then move the tmp file to the original file.

Finally remove any temp files that you created. Assuming all were created with the example name above:

rm -f /tmp/cleanscript_$$.*

Hope this helps you move forward.

Thansk agama for your help!!!

I put a tmp as you said and remove the tmp after that. Things work well!!
Thanks again!!