Modify text file using sed

Shell Programming and Scripting
1

Hello all,

I have some text files I need to do the following on:

Delete banner page (lines 1-56) --I am doing this using sed
Remove ^M --I am doing this using vi
Remove trailer page --this can vary based on the contents of the file, it usually starts with ***************************

I am able to do the first two steps manually by just issuing the commands against the filename, but I am wondering if there is a systematic approach that I can use to make this happen all in one simple script.

2

^M seem to be of DOS origin in which case you should convert the file first using dos2ux or dos2unix depending of your ux flavor

3

Systematic approach is a good idea, the awk language can handle a lot of this in one invocation.

Lots of people don't have dos2unix but you don't even need it. awk or tr can do it.

awk '{ sub(/\r/, ""); } # Delete carriage returns
NR <= 56 { next } # Don't print lines from banner page
# Quit once you find ***********
/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/ { exit }
# Otherwise, print all lines
{ print }' < inputfile > outputfile
4

Thank you for steering me in the right direction, I think I am almost there. My goal is to take a file, "massage" it, and output the contents. I am expecting the below to massage JEFFTEST files and output JEFFTEST.TIMESTAMP (Whatever that may be)

find /opt/test -type f -name "JEFFTEST" | while read name

do
	timestamp=$(date +%s)
	awk '{ sub(/\r/, ""); } # Delete carriage returns
	NR <= 56 { next } # Don't print lines from banner page
	# Quit once you find ***********
	/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/ { exit }
	# Otherwise, print all lines
	{ print }' < $name > $name.$timestamp
	
done
5

Test a single file before you test thousands of files.

6

A bit more efficient and robust:

timestamp=$(date +%s)
find /opt/test -type f -name "JEFFTEST" |
while read name
do
   awk '
    NR <= 56 { next } # Don't print lines from banner page
    { sub(/\r/$, ""); } # Delete carriage return at the end of the lines
    # Quit once you find ***********
    /\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/ { exit }
    # Otherwise, print all lines
    { print }
   ' < "$name" > "$name.$timestamp"
done