Print Line as per the dependent sequence in shell script.

krishnaswarnkar · June 7, 2019, 6:59am

Hi i have a file like this as shown below:

DA        PROCESS_ID IDENTIFIER          DA_FILE                        STATUS          WAITING_FOR                   SCOPED_DEPENDENT

1836       21000    01052019             BH90P.TEMP.DA1836.FTP             W            NULL                                    S

1836       22000    01052019             BH90P.TEMP.DA1836.FTP             W            1836/21000                              S

1836       23000    01052019             BH90P.TEMP.DA1836.FTP             W            1836/22000                              S

1836       24000    01052019             BH90P.TEMP.DA1836.FTP             W            1836/23000                              S

1836       25000    01052019             BH90P.TEMP.DA1836.FTP             W            1836/24000;1801/24000;2248/24000        S

1836       26000    01052019             BH90P.TEMP.DA1836.FTP             W            1836/25000                              S

1836       27000    01052019             BH90P.TEMP.DA1836.FTP             W            1836/26000                              S

1836       ETL31    01052019             BH90P.TEMP.DA1836.FTP             W            1836/27000                              S

1836       ETL32    01052019             BH90P.TEMP.DA1836.FTP             W            1836/ETL31                              S

1836       ETL33    01052019             BH90P.TEMP.DA1836.FTP             W            1836/ETL32                              S

2120       24000    01052019             BH90P.*                           W            1801/ETL33;1836/ETL33;2248/ETL33        D

2120       25000    01052019             BH90P.*                           W            2120/24000                              D

2207       24000    01052019             BH90P.NORDEASE.DA2207.FTP         W            2120/25000                              D

2207       25000    01052019             BH90P.NORDEASE.DA2207.FTP         W            2207/24000;2208/24000;2209/24000        D

2208       24000    01052019             BH90P.NORDEASE.DA2208.FTP         W            2120/25000                              D

2208       25000    01052019             BH90P.NORDEASE.DA2208.FTP         W            2208/24000                              D

2209       24000    01052019             BH90P.NORDEASE.DA2209.FTP         W            2120/25000                              D

2209       25000    01052019             BH90P.NORDEASE.DA2209.FTP         W            2209/24000                              D

2248       24000    01052019             BH90P.NORDEAFI.DA2248.FTP         W            1836/24000                              D

2248       25000    01052019             BH90P.NORDEAFI.DA2248.FTP         W            2248/24000                              D

2248       26000    01052019             BH90P.NORDEAFI.DA2248.FTP         W            2248/25000                              D

2248       27000    01052019             BH90P.NORDEAFI.DA2248.FTP         W            2248/26000                              D

2248       ETL31    01052019             BH90P.NORDEAFI.DA2248.FTP         W            2248/27000                              D

2248       ETL32    01052019             BH90P.NORDEAFI.DA2248.FTP         W            2248/ETL31                              D

2248       ETL33    01052019             BH90P.NORDEAFI.DA2248.FTP         W            2248/ETL32                              D

1577       ETL31    01052019             BH90P.NORDEAFI.DA1577.FTP         W            1592/27000;1596/27000;1836/27000        D

1577       ETL32    01052019             BH90P.NORDEAFI.DA1577.FTP         W            1577/ETL31                              D

1577       ETL33    01052019             BH90P.NORDEAFI.DA1577.FTP         W            1577/ETL32                              D

1588       ETL31    01052019             BH90P.NORDEAFI.DA1588.FTP         W            1592/27000;1596/27000;1836/27000        D

1588       ETL32    01052019             BH90P.NORDEAFI.DA1588.FTP         W            1588/ETL31                              D

1588       ETL33    01052019             BH90P.NORDEAFI.DA1588.FTP         W            1588/ETL32                              D

1590       24000    01052019             BH90P.NORDEAFI.DA1590.FTP         W            1836/23000                              D

1590       25000    01052019             BH90P.NORDEAFI.DA1590.FTP         W            1590/24000                              D

1590       26000    01052019             BH90P.NORDEAFI.DA1590.FTP         W            1590/25000                              D

1590       27000    01052019             BH90P.NORDEAFI.DA1590.FTP         W            1590/26000                              D

1590       ETL31    01052019             BH90P.NORDEAFI.DA1590.FTP         W            1590/27000                              D

1590       ETL32    01052019             BH90P.NORDEAFI.DA1590.FTP         W            1590/ETL31                              D

1590       ETL33    01052019             BH90P.NORDEAFI.DA1590.FTP         W            1590/ETL32                              D

1592       24000    01052019             BH90P.NORDEAFI.DA1592.FTP         W            1836/23000                              D

1592       25000    01052019             BH90P.NORDEAFI.DA1592.FTP         W            1592/24000                              D

1592       26000    01052019             BH90P.NORDEAFI.DA1592.FTP         W            1592/25000                              D

1592       27000    01052019             BH90P.NORDEAFI.DA1592.FTP         W            1592/26000                              D

1592       ETL31    01052019             BH90P.NORDEAFI.DA1592.FTP         W            1592/27000                              D

1592       ETL32    01052019             BH90P.NORDEAFI.DA1592.FTP         W            1592/ETL31                              D

1592       ETL33    01052019             BH90P.NORDEAFI.DA1592.FTP         W            1592/ETL32                              D

1596       24000    01052019             BH90P.NORDEAFI.DA1596.FTP         W            1836/23000                              D

1596       25000    01052019             BH90P.NORDEAFI.DA1596.FTP         W            1596/24000                              D

1596       26000    01052019             BH90P.NORDEAFI.DA1596.FTP         W            1596/25000                              D

1596       27000    01052019             BH90P.NORDEAFI.DA1596.FTP         W            1596/26000                              D

1596       ETL31    01052019             BH90P.NORDEAFI.DA1596.FTP         W            1596/27000                              D

1596       ETL32    01052019             BH90P.NORDEAFI.DA1596.FTP         W            1596/ETL31                              D

1596       ETL33    01052019             BH90P.NORDEAFI.DA1596.FTP         W            1596/ETL32                              D

1598       24000    01052019             BH90P.NORDEAFI.DA1598.FTP         W            1836/23000                              D

1598       25000    01052019             BH90P.NORDEAFI.DA1598.FTP         W            1598/24000                              D

1598       26000    01052019             BH90P.NORDEAFI.DA1598.FTP         W            1598/25000                              D

1598       27000    01052019             BH90P.NORDEAFI.DA1598.FTP         W            1598/26000                              D

1598       ETL31    01052019             BH90P.NORDEAFI.DA1598.FTP         W            1598/27000                              D

1598       ETL32    01052019             BH90P.NORDEAFI.DA1598.FTP         W            1598/ETL31                              D

1598       ETL33    01052019             BH90P.NORDEAFI.DA1598.FTP         W            1598/ETL32                              D

1801       24000    01052019             BH90P.NORDEASE.DA1801.FTP         W            1836/24000                              D

1801       25000    01052019             BH90P.NORDEASE.DA1801.FTP         W            1801/24000                              D

1801       26000    01052019             BH90P.NORDEASE.DA1801.FTP         W            1801/25000                              D

1801       27000    01052019             BH90P.NORDEASE.DA1801.FTP         W            1801/26000                              D

1801       ETL31    01052019             BH90P.NORDEASE.DA1801.FTP         W            1801/27000                              D

1801       ETL32    01052019             BH90P.NORDEASE.DA1801.FTP         W            1801/ETL31                              D

1801       ETL33    01052019             BH90P.NORDEASE.DA1801.FTP         W            1801/ETL32                              D

In this file Column 6 contains dependencies values , i want to read file line by line and Print each line if its dependencies is already printed.

for example if dependency i s then we can print line no 1 having PID - 21000 , similarly of PID-24000 if Dependency 1836/23000 is printed then only it can proceed to print.

How can i achieve the same using shell script because there are few cases where multiple dependecies are available like for PID-25000
1836/24000;1801/24000;2248/24000

Can any one help me with shell script achieve the same.

RudiC · June 7, 2019, 11:41am

More guessing than reading / understanding / knowing your spec, I came up with

awk '{D[$1,$2]++; for (n=split($6, T, ";"); n; n--) if (T[n] in D) print}' SUBSEP="/" file

. Works?

krishnaswarnkar · June 10, 2019, 2:37am

Hi Rudic ,

Checked with your reply but its giving the same output .
Let me clarify once more,
I want to arrange jobs in sequence , as we can see in first line if Waiting for Value in Null for "1836 21000" then it comes in number 1 in sequence for execution.
similarly for second line Waiting for Value is "1836/21000" so "1836 22000" comes in second number in sequence for execution.
:
:
:
For line "1836 25000 01052019 BH90P.TEMP.DA1836.FTP W 1836/24000;1801/24000;2248/24000"
Waiting for Value is "1836/24000;1801/24000;2248/24000" so we know the "1836 24000" is already in sequence before this job but "1801 24000" and "2248 24000" somewhere later in the so its necessary put those before this in sequence and also check their waiting for value as well ...

Its like a Once job having multiple dependency but i want to arrange them in sequence using shell script so that i can execute it using my job scheduler.

RudiC · June 10, 2019, 6:18am

No, it suppresses lines that don't have a dependency met.

Do you mean you need ALL dependencies met? Why not say so? What would the desired output look like (small but representative sample)?

What do you mean?

krishnaswarnkar · June 10, 2019, 8:43am

Thanks Rudic ,
Added a sequence column , Need to arrange in the sequence numbers as mentioned in the last field so that no Job goes on wait if arranged in this manner.

DA PROCESS_ID IDENTIFIER 1836 21000 01052019 1836 22000 01052019 1836 23000 01052019 1836 24000 01052019 1836 25000 01052019 1836 26000 01052019 1836 27000 01052019 1836 ETL31 01052019 1836 ETL32 01052019 1836 ETL33 01052019 2120 24000 01052019 2120 25000 01052019 2207 24000 01052019 2207 25000 01052019 2208 24000 01052019 2208 25000 01052019 2209 24000 01052019 2209 25000 01052019 2248 24000 01052019 2248 25000 01052019 2248 26000 01052019 2248 27000 01052019 2248 ETL31 01052019 2248 ETL32 01052019 2248 ETL33 01052019 1577 ETL31 01052019 1577 ETL32 01052019 1577 ETL33 01052019 1588 ETL31 01052019 1588 ETL32 01052019 1588 ETL33 01052019 1590 24000 01052019 1590 25000 01052019 1590 26000 01052019 1590 27000 01052019 1590 ETL31 01052019 1590 ETL32 01052019 1590 ETL33 01052019 1592 24000 01052019 1592 25000 01052019 1592 26000 01052019 1592 27000 01052019 1592 ETL31 01052019 1592 ETL32 01052019 1592 ETL33 01052019 1596 24000 01052019 1596 25000 01052019 1596 26000 01052019 1596 27000 01052019 1596 ETL31 01052019 1596 ETL32 01052019 1596 ETL33 01052019 1598 24000 01052019 1598 25000 01052019 1598 26000 01052019 1598 27000 01052019 1598 ETL31 01052019 1598 ETL32 01052019 1598 ETL33 01052019 1801 24000 01052019 1801 25000 01052019 1801 26000 01052019 1801 27000 01052019 1801 ETL31 01052019 1801 ETL32 01052019 1801 ETL33 01052019 DA_FILE STATUS WAITING_FOR SCOPED_DEPENDENT SEQUENCE
BH90P.TEMP.DA1836.FTP W NULL S 1
BH90P.TEMP.DA1836.FTP W 1836/21000 S 2
BH90P.TEMP.DA1836.FTP W 1836/22000 S 3
BH90P.TEMP.DA1836.FTP W 1836/23000 S 4
BH90P.TEMP.DA1836.FTP W 1836/24000;1801/24000;2248/24000 S 7
BH90P.TEMP.DA1836.FTP W 1836/25000 S 8
BH90P.TEMP.DA1836.FTP W 1836/26000 S 9
BH90P.TEMP.DA1836.FTP W 1836/27000 S 10
BH90P.TEMP.DA1836.FTP W 1836/ETL31 S 11
BH90P.TEMP.DA1836.FTP W 1836/ETL32 S 12
BH90P.* W 1801/ETL33;1836/ETL33;2248/ETL33 D 26
BH90P.* W 2120/24000 D 27
BH90P.NORDEASE.DA2207.FTP W 2120/25000 D 28
BH90P.NORDEASE.DA2207.FTP W 2207/24000;2208/24000;2209/24000 D 31
BH90P.NORDEASE.DA2208.FTP W 2120/25000 D 29
BH90P.NORDEASE.DA2208.FTP W 2208/24000 D 32
BH90P.NORDEASE.DA2209.FTP W 2120/25000 D 30
BH90P.NORDEASE.DA2209.FTP W 2209/24000 D 33
BH90P.NORDEAFI.DA2248.FTP W 1836/24000 D 6
BH90P.NORDEAFI.DA2248.FTP W 2248/24000 D 19
BH90P.NORDEAFI.DA2248.FTP W 2248/25000 D 20
BH90P.NORDEAFI.DA2248.FTP W 2248/26000 D 21
BH90P.NORDEAFI.DA2248.FTP W 2248/27000 D 23
BH90P.NORDEAFI.DA2248.FTP W 2248/ETL31 D 24
BH90P.NORDEAFI.DA2248.FTP W 2248/ETL32 D 25
BH90P.NORDEAFI.DA1577.FTP W 1592/27000;1596/27000;1836/27000 D 42
BH90P.NORDEAFI.DA1577.FTP W 1577/ETL31 D 43
BH90P.NORDEAFI.DA1577.FTP W 1577/ETL32 D 44
BH90P.NORDEAFI.DA1588.FTP W 1592/27000;1596/27000;1836/27000 D 45
BH90P.NORDEAFI.DA1588.FTP W 1588/ETL31 D 46
BH90P.NORDEAFI.DA1588.FTP W 1588/ETL32 D 47
BH90P.NORDEAFI.DA1590.FTP W 1836/23000 D 48
BH90P.NORDEAFI.DA1590.FTP W 1590/24000 D 49
BH90P.NORDEAFI.DA1590.FTP W 1590/25000 D 50
BH90P.NORDEAFI.DA1590.FTP W 1590/26000 D 51
BH90P.NORDEAFI.DA1590.FTP W 1590/27000 D 52
BH90P.NORDEAFI.DA1590.FTP W 1590/ETL31 D 53
BH90P.NORDEAFI.DA1590.FTP W 1590/ETL32 D 54
BH90P.NORDEAFI.DA1592.FTP W 1836/23000 D 34
BH90P.NORDEAFI.DA1592.FTP W 1592/24000 D 35
BH90P.NORDEAFI.DA1592.FTP W 1592/25000 D 36
BH90P.NORDEAFI.DA1592.FTP W 1592/26000 D 37
BH90P.NORDEAFI.DA1592.FTP W 1592/27000 D 55
BH90P.NORDEAFI.DA1592.FTP W 1592/ETL31 D 56
BH90P.NORDEAFI.DA1592.FTP W 1592/ETL32 D 57
BH90P.NORDEAFI.DA1596.FTP W 1836/23000 D 38
BH90P.NORDEAFI.DA1596.FTP W 1596/24000 D 39
BH90P.NORDEAFI.DA1596.FTP W 1596/25000 D 40
BH90P.NORDEAFI.DA1596.FTP W 1596/26000 D 41
BH90P.NORDEAFI.DA1596.FTP W 1596/27000 D 58
BH90P.NORDEAFI.DA1596.FTP W 1596/ETL31 D 59
BH90P.NORDEAFI.DA1596.FTP W 1596/ETL32 D 60
BH90P.NORDEAFI.DA1598.FTP W 1836/23000 D 61
BH90P.NORDEAFI.DA1598.FTP W 1598/24000 D 62
BH90P.NORDEAFI.DA1598.FTP W 1598/25000 D 63
BH90P.NORDEAFI.DA1598.FTP W 1598/26000 D 64
BH90P.NORDEAFI.DA1598.FTP W 1598/27000 D 65
BH90P.NORDEAFI.DA1598.FTP W 1598/ETL31 D 66
BH90P.NORDEAFI.DA1598.FTP W 1598/ETL32 D 67
BH90P.NORDEASE.DA1801.FTP W 1836/24000 D 5
BH90P.NORDEASE.DA1801.FTP W 1801/24000 D 13
BH90P.NORDEASE.DA1801.FTP W 1801/25000 D 14
BH90P.NORDEASE.DA1801.FTP W 1801/26000 D 15
BH90P.NORDEASE.DA1801.FTP W 1801/27000 D 16
BH90P.NORDEASE.DA1801.FTP W 1801/ETL31 D 17
BH90P.NORDEASE.DA1801.FTP W 1801/ETL32 D 18

vgersh99 · June 10, 2019, 10:13am

Just to quote RudiC:

A desired output would be helpful as well....
Also, you've been asked to start using code tags when posting code/data sample...

krishnaswarnkar · June 11, 2019, 2:01am

Hi Rudic,

Hope you got the desired output , basically i need a field which shows the number in which the Jobs can be arranged.