Insert contents of file after keyword

Shell Programming and Scripting
1

Hello, I a have a file i.e:

This is line 1
...
This is line x
END
This is line x+2 *<WANT TO REPLACE WITH BLANK LINE AND CONTENTS OF FILE1*
...
This is line  y
END
This is line  y+2 *<WANT TO REPLACE WITH BLANK LINE AND CONTENTS OF FILE2*
...
This is line z
END
This is line z+1   *<DO NOT REPLACE THE 3RD OCCURENCE*

I want to replace the lines after the line with END with a blank line and after a new line with the contents of files but only for the first 2 occurrences.

Output:

This is line 1
...
This is line x
END
<BLANK LINE>
LINE1 FROM FILE1
LINE2 FROM FILE1
...
This is line  y
END
<BLANK LINE>
LINE1 FROM FILE2
LINE2 FROM FILE2
...
This is line z
END
This is line z+1   *<NO CHANGE ON 3RD  OCCURENCE*

Any help much appreciated.

2

What code have you tried so far?

3

Nothing for this part. I have 7 files that I need to modify daily and need to automate. This is tha last part I have to fix , and I have no Ideas. To reach at this point I used sed and awk, but now I am stuck :frowning:

image
image1058Ă—724 36.7 KB

4

@drbiloukos,
just an observation - every time I see awk/grep/sed/tr/kitchenSink in the code, something smells wrong.
Also, it sounds like a classic XY problem.
I'd suggest starting with the clean slate rather than addressing your "current" issue.
You need to go from "here" to "there".
Define what the here/there are with small representative data sample for BOTH.
Try to do it ALL in awk - I don't think you really need anything else look at your current "stuck state".
Give it whirl and see how far you get.

2 Likes
5

Well Iam not advanced user and do not script every day. Now I have to automate a daily job. Combining multiple commands works ok for my task. I know it is not optimal...

6

The following you can do in the shell:

# function
insert_at_END(){
  cnt=0
  while IFS= read -r line
  do
    echo "$line"
    if [[ $line == END* ]] && [[ $# -gt 0 ]]
    then
      if [[ -n $1 ]]
      then
        echo ""
        cat "$1"
      fi
      shift
    fi
  done
}

# Now use the function
insert_at_END TSAPI_HEADER2.txt TSAPS_HEADER2.txt <AFFILE0_AD30_TSAPI_2411240010.dat >AFFILE0_AD30_TSAPI_2411240010.new
7

Seems almost correct, but

Line 4:
  cnt=0
  ^-- SC2034 (warning): cnt appears unused. Verify use (or export if used externally).

besides, this puts a newline and file content in between END and its following line, instead of replacing the following line.

I've got something like:

#!/usr/bin/env bash
insert_at_END ()
{
    # this will find numbers of lines for first $# occurrences of 'END' string in a file redirected to stdin
    mapfile -t END_lines < <(grep -nm $# '^END$' "$(readlink -e /dev/stdin)" | tr -d ':[A-Z]');
    cnt=1;
    idx=0;
    while IFS= read -r line; do
        if [[ "$line" == END* ]] && [[ $# -gt 0 ]]; then
            if [[ -n "$1" ]]; then
                echo "$line";
                echo "";
                cat "$1";
            fi;
            shift;
        else
            if (( cnt != END_lines[idx]+1 )); then
                echo "$line";
            else
                ((idx++));
            fi;
        fi;
    ((cnt++));
    done
}

and then you run it exactly the same way, as @MadeInGermany presented above. Of course, you can also avoid using mapfile (and grep/readlink/tr "dependencies") by simply implementing other "skip next line mechanism", e.g. based on additional variable inside the first "if statement", like e.g.:

#!/usr/bin/env bash
insert_at_END ()
{
    while IFS= read -r line; do
        if [[ "$line" == END* ]] && [[ $# -gt 0 ]]; then
            if [[ -n "$1" ]]; then
                echo "$line";
                echo "";
                cat "$1";
            fi;
            skip_next_line='yes';
            shift;
        else
            if [[ -z "$skip_next_line" ]]; then
                echo "$line";
            else
                unset skip_next_line; 
            fi;
        fi;
    done
}
1 Like
8

Thank you. I will try to test it today.

9

That's an excellent observation! I've noticed that this kind of XY Problem frequently arises in these forums. People often attempt to address the surface-level issue (a "band-aid solution") rather than tackling the root cause of the problem.

This tendency isn't limited to technical discussions—it extends to health as well. Many people focus on treating symptoms rather than addressing the underlying lifestyle choices that lead to the core issues of disease.

10

Yes, anything else is better!
Most simple: another read line after the cat.
The skip_next_line approach is ok.
I would go for a skip_lines

insert_at_END(){
  skip_lines=0
  while IFS= read -r line
  do
    (( skip_lines && skip_lines-- )) && continue
    echo "$line"
    if [[ $line == END* ]] && [[ $# -gt 0 ]]
    then
      if [[ -n $1 ]]
      then
        echo ""
        cat "$1"
        skip_lines=1
      fi
      shift
    fi
  done
}
11

There's a very interesting and enlightening discussion/thread about the XY problem on SE.
Everything boils down to the following "scenario/observation":

    User wants to do X.
    User doesn't know how to do X, but thinks they can fumble their way to a solution if they can just manage to do Y.
    User doesn't know how to do Y either.
    User asks for help with Y.
    Others try to help user with Y, but are confused because Y seems like a strange problem to want to solve.
    After much interaction and wasted time, it finally becomes clear that the user really wants help with X, and that Y wasn't even a suitable solution for X.

The problem occurs when people get their train of thought stuck on one approach 
and become unable to take a step back. Remaining open to having a new look at the
bigger picture, these people might find their way back to X and continue 
searching for alternative solutions.

There're other linked references I found interesting to read that have helped me personally in the past in a number of ways.

1 Like