Removing consecutive lines in a file

We have very large transaction logs that have transactions which start with a line that starts with 'Begin :' and ends with a line that starts with 'End :'. For most transactions there is valid data between those two lines.

I am trying to get rid of lines that look like this:

Begin : Transaction Id 1245 04-Oct-2011 04:01:48 Username smith
End : Transaction Id 1245 04-Oct-2011 04:01:49

So, get rid of the first line, then get rid of the very next line if that next line starts with 'End :'. If there is data between, I want to preserve those lines throughout the entire file.

Thank you !

Try:

perl -lp0e 's/^Begin.*\nEnd.*\n//mg' file

Didn't test it.

Using awk

awk '/^End :/ && A { A=""; next }
  A { print A ; A="" }
  /^Begin :/ { A=$0 ; next }
  1' infile

Both of these gave me syntax errors.

Perl: Substitution replacement not terminated at -e line 1.
awk: syntax error near line 1
awk: bailing out near line 1

use nawk (instead of awk) if on Solaris.

nawk ran but did not change the file ...

Is the format exactly as you posted in your exampe ie "Begin :" with a capital B and space between n and colon?

Try this?

awk '/^Begin/{getline}/^End/{next}1' input_file

--ahamed

I cut and pasted but it doesn't look like there are the right amount of spaces in my post - there should be three spaces after 'Begin' and before the ':'. With 'End', there are five spaces before the ':'.
Let me try pasting again.

Begin : Transaction Id 1234 04-Oct-2011 13:44:22.24 Username smith
End : Transaction Id 1234 04-Oct-2011 13:44:22.25

I modified and ran again: nawk '/^End && A { A=""; next } A { print A ; A="" } /^Begin { A=$0 ; next } 1' 20111005.trl > testnawk &

I think this may have solved it but need to do some testing on a much bigger file.

Is this finding 'End :' putting that in a place holder, then finding if 'Begin :' is the next line above and deleting both or printing everything but those lines ?

Thank you !

Yes it's storing the line in A and printing later, have a look a ahamed101's solution it just calls getline to throw away the being line and process the line following.

Also, to post data as is put it between

```text
 and 
```

tags.

Looks like it truncated my whitespace yet again ...
Will keep you posted on this - pretty sure it works !
Thank you !

---------- Post updated at 02:15 PM ---------- Previous update was at 02:13 PM ----------

The getline solution took out all of my 'Begin :' lines ...except for one in the middle of the file.

Thank you though !

The getline solution just egrep -ve '^Begin|^End'

If you only want to retain the Begin/End block who contains at least 1 or more lines (and keeping the Begin and End lines of those blocks) then you can go for :

nawk '{A[NR]=$0}/^Begin/{b=NR}/^End/&&((NR-b)>1){for(i=b-1;++i<=NR;) print A}' infile

you can then play with b or b-1 as well as <NR or <=NR of the "for" loop depending on your need of the Bein/End lines or not