Command/script to match a field and print the next field of each line in a file.

pocodot · August 10, 2015, 9:42am

Hello,

I have a text file in the below format:

Source                                                                                              Destination                                                                              State          Lag        Status
CQA02W2K12pl:D:\CAQA                                                                            infrabkpsrvpl:/vol/CQA02W2K12srv/CAQA                                                Snapvaulted    13:46:44   Idle
DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    13:46:43   Idle

And the file has hundreds of such entries.

What I'm interested in is the lag column- which is always in the format hh:mm:ss. I want to read the file line by line, check if the "lag" column in greater than 48:00:00 (just matching the hour field would be enough) and if the lag is greater than 48:00:00, append that particular line to another file.

How can I achieve this?

I tried awk, but since some of the line descriptions have spaces, $4 is not always lag column.

Thanks,
Pocodot

vgersh99 · August 10, 2015, 10:40am

here's something to start with:

awk '{print $(NF-1)}' myFile

pocodot · August 10, 2015, 10:13pm

Thanks! Here's something else i'm trying based on your reply:

 awk '{if($(NF-1) < 48) print $0}'   OP.txt > output.txt

This command would be perfect for me, but $(NF-1) is in the format hh:mm:ss.

Is there any way I can just compare the hh part ?

Thanks,
Pocodot

Aia · August 10, 2015, 10:44pm

awk  '$(NF-1) ~ /..:..:../ {split($(NF-1), a, ":"); if(a[1] < 48)print} OP.txt > output.txt

Don_Cragun · August 10, 2015, 10:53pm

Assuming there are never any spaces in the Status field, try:

awk -F'[[:space:]]+|:' '$(NF-3) + 0 >= 48' OP.txt >> output.txt

Note the >> redirection operator instead of > . (You said you wanted to append to the destination file; not replace its current contents.)

pocodot · August 10, 2015, 10:53pm

Thats perfect! Thanks!

Aia · August 10, 2015, 11:04pm

I guess it doesn't work on my shell.

[aia@ludus tmp]$ nl pocodot.file
     1  Source                                                                                              Destination                                                                              State          Lag        Status
     2  CQA02W2K12pl:D:\CAQA                                                                            infrabkpsrvpl:/vol/CQA02W2K12srv/CAQA                                                Snapvaulted    13:46:44   Idle
     3  DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    13:46:43   Idle

[aia@ludus tmp]$ awk -F'[[:space:]]+|:' '$(NF-3) + 0 >= 48' pocodot.file
[aia@ludus tmp]$ echo $?
0

It appears that the logic is broken

[aia@ludus tmp]$ awk -F'[[:space:]]+|:' '{print $(NF-3) + 0, ">=", 48}' pocodot.file
0 >= 48
13 >= 48
13 >= 48

Don_Cragun · August 10, 2015, 11:23pm

aia:

I guess it doesn't work on my shell.

[aia@ludus tmp]$ nl pocodot.file
   1  Source                                                                                              Destination                                                                              State          Lag        Status
   2  CQA02W2K12pl:D:\CAQA                                                                            infrabkpsrvpl:/vol/CQA02W2K12srv/CAQA                                                Snapvaulted    13:46:44   Idle
   3  DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    13:46:43   Idle

[aia@ludus tmp]$ awk -F'[[:space:]]+|:' '$(NF-3) + 0 >= 48' pocodot.file
[aia@ludus tmp]$ echo $?
0

I guess you didn't read the requirements in post #1:

There are no lag fields in the sample input file that meet the criteria, so nothing is printed. With an input file like:

Source                                                                                              Destination                                                                              State          Lag        Status
CQA02W2K12pl:D:\CAQA                                                                            infrabkpsrvpl:/vol/CQA02W2K12srv/CAQA                                                Snapvaulted    13:46:44   Idle
DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    13:46:43   Idle
DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    48:46:43   Idle
DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    48:00:00   Idle
DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    8:00:00   Idle

the output appended to the output file is:

DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    48:46:43   Idle
DBIntranetpl:E:\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL                                  infrabkpsrvpl:/vol/DBIntranetsrv/MSSQL10                                                Snapvaulted    48:00:00   Idle

Further tests could be added to exclude the last line if the desire is to only copy lags truly greater than 48:00:00.

Aia · August 10, 2015, 11:44pm

Actually, I did, however, since the OP came back with another criteria that superseded the original first post and since there was not clear indication on your part, I tested based on the final request which I just had posted.

pocodot:

Thanks! Here's something else i'm trying based on your reply:
 awk '{if($(NF-1) < 48) print $0}'   OP.txt > output.txt
This command would be perfect for me, but $(NF-1) is in the format hh:mm:ss.

Is there any way I can just compare the hh part ?

Thanks,
Pocodot

Another consideration is that the test can not be reversed without unintended consequences.

[aia@ludus tmp]$ awk -F'[[:space:]]+|:' '$(NF-3) + 0 < 48 {print $(NF-3)}' pocodot1.file
Destination
13
13
8

Don_Cragun · August 11, 2015, 12:21am

Hi Aia,
You saw new requirements; I saw non-working code. We'll have to let the OP clarify the intent.

My code was written to meet the stated requirements. If the requirement is less than 48 hours instead of greater than (or greater than or equal to) 48 hours, that would be:

awk -F'[[:space:]]+|:' ' NR > 1 && $(NF-3) + 0 < 48' OP.txt >> output.txt

pocodot · August 17, 2015, 8:28am

@Don Cragun and @Aia,

Here is what I used based on Aia's input.

/usr/kerberos/bin/rsh infrabkpsrvplp snapvault status | awk  '$(NF-1) ~ /..:..:../ {split($(NF-1), a, ":");if(a[1] > 48){printf("\n");printf("Backup Source: ");print$1;printf("Destination Filer: infrabkpsrvplp\n");printf ("Time Since Last Backup: ");print$(NF-1)}}' >> /var/adm/snapvault/body

Instead of dumping the output on a file and then awk'ing the file, I sent the output to a "body" file then I would just mail myself the content of body. Which I recieve as below:

 Backup Source: m1storagepp:D:\EM1\EM1-Applications\MATRIX-Applications-LG\Design
  Destination Filer: infrabkpsrvplp
  Time Since Last Backup: 172:55:03
   
  Backup Source: m1storagepp:D:\EM1\EM1-Applications\MATRIX-Applications-PortfolioManagement\Dev\InhouseTools
  Destination Filer: infrabkpsrvplp
  Time Since Last Backup: 220:55:04

Thanks again for your support and apologies for not responding earlier- Onsite visits

-Pocodot.

---------- Post updated at 05:58 PM ---------- Previous update was at 05:57 PM ----------

Also, I'm happy to answer any questions you may have.