Change nth depending on matiching a pattern in Y line

Shell Programming and Scripting
1

Hi,

I have below file, each line that starts with /* marks the beginning of the a new job.

   /* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /home2/proddata/bin/moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/tmp/cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/home2/autosys/cmdsprod/profiles/daily
ENDJOB

  /* ----------------- cmdsMlyDdrgRcvF -----------------
FILE_TRIGGER CMMD003F
  AGENT CMDSHP
  USER autosys
  AFTER CMMM002J
  /* "watch prepareDDRG.data(www, xtmp.htm, Tpmplant.exe"
  FILENAME /home3/ref_files/download/prepareDDRG.data
  ENVAR ENV=/home2/autosys/cmdsprod/profiles/daily
ENDJOB

  /* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /home2/proddata/bin/moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/tmp/cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/home2/autosys/cmdsprod/profiles/daily
ENDJOB

i want to get rid of the line ENVAR ENV=/home2/autosys/cmdsprod/profiles/daily whereever the new job starts with FILE_TRIGGER. Please assist

2

Hi,
Try:

sed -n -e '/\/\*/,/ENDJOB/{/FILE_TRIGGER/H' -e '/ENVAR ENV/!b' -e 'x' -e '/FILE_TRIGGER/!b' -e 'x' -e 'p' -e 'x' -e '}' file

Regards.

3

Here is an awk solution:

awk '
  $2 ~ "^FILE_TRIGGER" {gsub("\nENVAR ENV=[^\n]*\n","")}
  NF{printf("%s", "/* -"$0)}' RS='/[*] -' FS='\n\n' infile
4 
>>cat test2.sh
awk '
$2 ~ "^FILE_TRIGGER" {gsub("\nENVAR ENV=[^\n]*\n","")}
NF{printf("%s", "/* -"$0)}' RS='/
[*] -' FS='\n\n' txt
 
>>./test2.sh
awk: syntax error near line 2
awk: bailing out near line 2

---------- Post updated at 07:09 PM ---------- Previous update was at 07:07 PM ----------

Sed didnt work either it just displayed one line

>>sed -n -e '/\/\*/,/ENDJOB/{/FILE_TRIGGER/H' -e '/ENVAR ENV/!b' -e 'x' -e '/FILE_TRIGGER/!b' -e 'x' -e 'p' -e 'x' -e '}' txt
ENVAR ENV=/home2/autosys/cmdsprod/profiles/daily

---------- Post updated at 07:15 PM ---------- Previous update was at 07:09 PM ----------

Just to be clear, I want my output to look like this

 
/* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
CMDNAME /home2/proddata/bin/moveTPMPLANT.sh
AGENT CMDSHP
USER proddata
AFTER CMMU001J
/* "DDRG monthly processing. This job can FAIL if the files are n
ENVAR STDOUT=/tmp/cmdsMlyMoveTPMPLANTJ
ENVAR ENV=/home2/autosys/cmdsprod/profiles/daily
ENDJOB
 
/* ----------------- cmdsMlyDdrgRcvF -----------------
FILE_TRIGGER CMMD003F
AGENT CMDSHP
USER autosys
AFTER CMMM002J
/* "watch prepareDDRG.data(www, xtmp.htm, Tpmplant.exe"
FILENAME /home3/ref_files/download/prepareDDRG.data
ENDJOB
 
/* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
CMDNAME /home2/proddata/bin/moveTPMPLANT.sh
AGENT CMDSHP
USER proddata
AFTER CMMU001J
/* "DDRG monthly processing. This job can FAIL if the files are n
ENVAR STDOUT=/tmp/cmdsMlyMoveTPMPLANTJ
ENVAR ENV=/home2/autosys/cmdsprod/profiles/daily
ENDJOB

this example has just one file trigger my real file has multiple of these and are all in between UNIX_JOB

5

You're using Solaris. Try /usr/xpg4/bin/awk or nawk inplace of awk

6 
>>cat test2.sh
/usr/xpg4/bin/awk '
$2 ~ "^FILE_TRIGGER" {gsub("\nENVAR ENV=[^\n]*\n","")}
NF{printf("%s", "/* -"$0)}' RS='/
[*] -' FS='\n\n' txt

OUTPUT-------

/* -  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyDdrgRcvF -----------------
FILE_TRIGGER CMMD003F
  AGENT CMDSHP
  USER autosys
  AFTER CMMM002J
  /* -* "watch prepareDDRG.data(www, xtmp.htm, Tpmplant.exe"
  FILENAME /* -home3/* -ref_files/* -download/* -prepareDDRG.data
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB
7

Sorry I thought your input file was double spaced (blank line between each data line)

You should use

```text
 and 
```

tags around your data not ``` and ``` .

Also are some of those lines indented with spaces, it looks like they might be...This should work a little better:

/usr/xpg4/bin/awk '
  $2 ~ "^ *FILE_TRIGGER"{gsub(" *ENVAR ENV=/home2/[^\n]*\n","")}
  NF{printf "%s","/* -"$0}' RS='/[*] -' FS='\n' infile
8

No luck

 
>>cat test2.sh
/usr/xpg4/bin/awk '
  $2 ~ "^ *FILE_TRIGGER"{gsub(" *ENVAR ENV=/home2/[^\n]*\n","")}
  NF{printf "%s","/* -"$0}' RS='/
[*] -' FS='\n' txt
[meandmy(autosys)]/usr/autosys/shells/TWS
>>./test2.sh
/* -  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyDdrgRcvF -----------------
FILE_TRIGGER CMMD003F
  AGENT CMDSHP
  USER autosys
  AFTER CMMM002J
  /* -* "watch prepareDDRG.data(www, xtmp.htm, Tpmplant.exe"
  FILENAME /* -home3/* -ref_files/* -download/* -prepareDDRG.data
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB
9

Yes, I just picked up that data lines are indented - the updated code in post #7 should work OK for your actual file format.

10

I dont think its working.. see below. i used the code from #7 and still messed up result

>>cat test2.sh
/usr/xpg4/bin/awk '
  $2 ~ "^ *FILE_TRIGGER"{gsub(" *ENVAR ENV=/home2/[^\n]*\n","")}
  NF{printf "%s","/* -"$0}' RS='/
[*] -' FS='\n' txt

>>./test2.sh
/* -  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyDdrgRcvF -----------------
FILE_TRIGGER CMMD003F
  AGENT CMDSHP
  USER autosys
  AFTER CMMM002J
  /* -* "watch prepareDDRG.data(www, xtmp.htm, Tpmplant.exe"
  FILENAME /* -home3/* -ref_files/* -download/* -prepareDDRG.data
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB
11

Still looks like some trouble slight tweaking is needed with the RS=value. Try this adjustment:

awk '
  $2 ~ "^ *FILE_TRIGGER"{gsub(" *ENVAR ENV=/home2/[^\n]*\n","")}
  NF{printf "%s","/* -"$0}' RS='/\\* -' FS='\n' infile
12

Nope - yu see its changeing all the

/

to

/* -

. I dont want that. onlything i need is if job is FILE_TRIGGER.. remove the line that has

ENAVR ENV=/home2******

>>cat test2.sh
/usr/xpg4/bin/awk '
  $2 ~ "^ *FILE_TRIGGER"{gsub(" *ENVAR ENV=/home2/[^\n]*\n","")}
  NF{printf "%s","/* -"$0}' RS='/\\* -' FS='\n' txt
[meandmy(autosys)]/usr/autosys/shells/TWS
>>./test2.sh
/* -  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyDdrgRcvF -----------------
FILE_TRIGGER CMMD003F
  AGENT CMDSHP
  USER autosys
  AFTER CMMM002J
  /* -* "watch prepareDDRG.data(www, xtmp.htm, Tpmplant.exe"
  FILENAME /* -home3/* -ref_files/* -download/* -prepareDDRG.data
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB

  /* -* ----------------- cmdsMlyMoveTPMPLANTJ -----------------
UNIX_JOB CMMM002J
  CMDNAME /* -home2/* -proddata/* -bin/* -moveTPMPLANT.sh
  AGENT CMDSHP
  USER proddata
  AFTER CMMU001J
  /* -* "DDRG monthly processing. This job can FAIL if the files are n
  ENVAR STDOUT=/* -tmp/* -cmdsMlyMoveTPMPLANTJ
  ENVAR ENV=/* -home2/* -autosys/* -cmdsprod/* -profiles/* -daily
ENDJOB
13

Apologies, support for multi-character FS is a GNU extension and will not be supported in Solaris awk.

Are those blank lines between each job? If we can rely on that the following should work:

/usr/xpg4/bin/awk '
  $2 ~ "^ *FILE_TRIGGER"{gsub(" *ENVAR ENV=/home2/[^\n]*\n","")}
    NF{print $0 FS}' RS= FS='\n' txt
14

Sorry, I read about a little fast and I thought you just wanted to get the line in blocks with FILE_TRIGGER.So, my sed solution is KO.

Regards.

---------- Post updated at 01:43 PM ---------- Previous update was at 11:38 AM ----------

I really do bad read, try this sed solution (very simple):

sed -e '/FILE_TRIGGER/,/ENDJOB/{/ENVAR ENV/d;}' file

Regards.