usage of Awk command for output

Shell Programming and Scripting
1

Hi Experts,

I have a Text file generated as below;

<NAME>
    NEW#<technicalName><TAB> <Version>
    OLD#<technicalName><TAB> <Version>

e.g.

CH_PPV_AUDIT_DISTRIBUTOR
        NEW#EL_CFG_FTP_DISTRIBUTOR      2.1.0.upc2
        OLD#EL_CFG_FTP_DISTRIBUTOR      2.1.0.upc1
CH_PPV_AUDIT_ENCODER
        NEW#UPC_CFG_AUDIT_REJECTED_ENCODER      1.1.0.
        OLD#UPC_CFG_CH_PPV_AUDIT_ENCODER        1.0.2.

I need to generate the output as follows

CH_PPV_AUDIT_DISTRIBUTOR
       EL_CFG_FTP_DISTRIBUTOR    2.1.0.upc1  -->   2.1.0.upc2
CH_PPV_AUDIT_ENCODER
        UPC_CFG_AUDIT_REJECTED_ENCODER      1.0.2. --> 1.1.0.
     
<NAME>
        <TechnicalNewName>    oldversion  -->   NewVersion

Kindly help me in generating such an output.

Thanks in Advance.

Regards
Rajan

2

try this...

awk -F "[# ]" '{if($0 ~/#/){s=$3;getline;print $2" "$3,s}else{print }}' OFS=" --> " file
3

Please use code tags for code and data samples.

sed '/^NEW#/{;s///;h;d;}
/^OLD#/{;s///;G;s/\n/ --> /;}' file
4

Hi,

I used this awk command as below;

/usr/local/bin/gawk -F "[#]" '{if($0 ~/#/){s=$3;getline;print $2" "$3,s}else{print }}' OFS=" --> "

And the output i got is as this:

CH_PPV_AUDIT_DISTRIBUTOR
EL_CFG_FTP_DISTRIBUTOR  2.1.0.upc1  -->
CH_PPV_AUDIT_ENCODER
UPC_CFG_CH_PPV_AUDIT_ENCODER    1.0.2.  -->

As seen the new version is not printing. I tried to modify and see but no luck. Please have a look.

5

there are two field separator and space is also one of the field separator... and you have used only # as field separator
try with awk -F "[# ]"

try this also..

awk -F "[# ]" 'NF == 1{print}
/NEW#/{s=$NF}
/OLD#/{print $2" "$3,s}' OFS=" --> " file

and as Elixir mentioned above please use code tags. see this

6

Have you tried the sed command? I hope it does not matter which tool you use to achieve your goal.
And again, please use code tags for code and data samples.

7

Hi,

Thanks for your answer. It worked as of now. But i have noticed one more improvement that need to be implemented. As stated below

Input:

Name
 NEW#TechnicalName1 Value1
 OLD#TechnicalName1  Value2
 NEW#TechinalName2 Value21
 OLD#TechnicalName2 Value 22

output

Name
 TechnicalName1 Value2 --> TechnicalName1 Value1
 TechinalName2 Value22 --> TechinalName2 Value21

how to achieve this senario?

Many Many thanks to All....!!!!

8

Again, have you tried the sed command?
Try it on both the input samples and check.
The awk command will need some more change when the number of words in the OLD and NEW lines exceeds 3. As the requirement (as far as I can understand) is to remove the OLD and NEW tags along with the pound signs, the awk script should have been written using sub function.

9

just need to do small change...

awk -F "[# ]" 'NF == 1{print}
/NEW#/{s=$2" "$3}
/OLD#/{print $2" "$3,s}' OFS=" --> " file
10

With small corrections (remove anchoring & multiple spaces), elixir_sinari's proposal should work:

$ sed '/NEW#/ {s///;h;d}; /OLD#/ {s///;G;s/\n/ -->/;s/  */ /g}' file
CH_PPV_AUDIT_DISTRIBUTOR
 EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1 --> EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
CH_PPV_AUDIT_ENCODER
 UPC_CFG_CH_PPV_AUDIT_ENCODER 1.0.2. --> UPC_CFG_AUDIT_REJECTED_ENCODER 1.1.0.
$

Pamu's suggestion needs a "+" after the -F"[# ]" 'cos there's many spaces out there, and needs to use $3 and $4 for output.

11

Hi,

Thanks all for your effort. Let me just put what exactly i am looking for;
I have attached an input file and here what i do from it;

cat -vt diffStreamVersions.txt |sed 's/SQL>//g' | sed 's/ //g' | tr ':' '\n' |tr '-' '\t' | tr '|' '\t' | sort -n |
/usr/local/bin/gawk -F";" '{OFS="\t";if(x!=$1){print "\n"$1;x=$1};print "\t"$3}'

Output is as follows;

CH_PPV_AUDIT_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!BehaviourOnError Abort
NEW!OutputFileNameExtractionMask
NEW!Subdirectory
NEW!TargetDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/${Affiliate}_PPV
NEW!TempDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/${Affiliate}_PPV/temp
OLD!BehaviourOnError Restart
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/CH_PPV
OLD!TempDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/CH_PPV/temp
CH_PPV_AUDIT_ENCODER
NEW#UPC_CFG_AUDIT_REJECTED_ENCODER 1.1.0.
OLD#UPC_CFG_CH_PPV_AUDIT_ENCODER 1.0.2.
NEW!OutputDescriptionFileName vod.xml
OLD!OutputDescriptionFileName audit_file_description.xml
NEW?AUDIT_REJECTED 0
NEW?RDC_AUDIT_OUT 0
OLD?AUDIT_REJECTED 0
OLD?RDC_AUDIT_OUT 0
OLD?VALIDATION_AUDIT_OUT 0
NEW@UPC_CFG_AUDIT_REJECTED_ENCODER REJ
OLD@UPC_CFG_CH_PPV_AUDIT_ENCODER REJ
CH_PPV_BACKUP
NEW#UPC_CFG_BACKUP 1.2.1.
NEWNODE
NEW#EL_APP_FTP_DISTRIBUTOR 2.1.4.upc1
NEW#EL_LIB_SFTP 1.1.0.
NEW!BehaviourOnError Abort
NEW!BlockSize
NEW!Cipher
NEW!Compression
NEW!CompressionMethod gzip
NEW!ConnectionEstablishmentTimeout
NEW!ConnectionRetryCount
NEW!ConnectionRetryInterval
NEW!DailyDirectories no
NEW!DestinationHost
NEW!ExternalCommand
NEW!FlagFileSuffix
NEW!InputLinkDirectories Yes
NEW!InputLinksToSubDirectories CAS_COLLECTOR=RAW/&SOURCEID,COLLECTED=RAW/&SOURCEID
NEW!Kex
NEW!MaxDays 0
NEW!MaxFiles 0
NEW!MaximumReconnectionInterval
NEW!MaximumSustainedNetworkDowntime
NEW!OriginalFilenameMask
NEW!Password
NEW!Port
NEW!PrivateKey
NEW!PublicKey
NEW!RemoteOS
NEW!RemotePrefix
NEW!RemoteSuffix
NEW!Ruleset backup.pm
NEW!SourceIdDirectories no
NEW!TargetDirectory /backup/BACKUP_EL603/${StreamName}
NEW!TempDirectory
NEW!TempSuffix
NEW!TransferMode
NEW!TransferProtocol Local
NEW!TransferRetryCount
NEW!TransferRetryInterval
NEW!TransferType
NEW!Username
NEW?CAS_COLLECTOR 1
NEW?COLLECTED 1
NEW?ENCODED_AUDIT 1
NEW?ENCODED_DWH 1
NEW?ENCODED_KENAN 1
CH_PPV_CAS_COLLECTOR
NEW#UPC_CFG_CAS_COLLECTOR 2.1.0.
OLD#UPC_CFG_CAS_COLLECTOR 2.0.0.
NEW#UPC_APP_CAS_COLLECTOR 2.3.0.
OLD#UPC_APP_CAS_COLLECTOR 2.2.4.
NEW!BindTimeout 10s
NEW!CasHeartbeatInterval 5m
NEW!CloseFilesAtSchedule 0
NEW!CommitInterval 10s
NEW!MaxAgeOfFile 30m
NEW!NoDataTimeout 12h
NEW!OutputFileNameCreationMask ${Affiliate}CAS_&CREATIONTIME(%Y%m%d%H%M%S)
NEW!RecordLimit 10000
NEW!RestartInterval 1d
NEW!TransactionIdFile ${EventLinkUserHome}/delivery/${StreamName}/WORKING/id_file.txt
OLD!BindTimeout 10
OLD!CasHeartbeatInterval 300
OLD!CommitInterval 10
OLD!OutputFileNameCreationMask CHCAS_&CREATIONTIME(%Y%m%d%H%M%S)
OLD!TransactionIdFile ${EventLinkUserHome}/delivery/UPCCH_PPV/WORKING/id_file.txt
CH_PPV_DECODER
NEW#UPC_CFG_CAS_DECODER 1.0.0.
OLD#UPC_CFG_CH_PPV_CAS_DECODER 1.0.0.
NEW#EL_APP_ASCII_DECODER 3.0.3.upc5
OLD#EL_APP_MDS_ARM_FR_DECODER 1.1.5.
NEW!AuditFiltered 1
NEW!CSVFieldsContainDelimiters 0
NEW!DiscardEmptyFiles 0
NEW!DiscardOnRejection 0
NEW!EscapeWithBackslash 0
NEW!HeaderIdentifier
NEW!InputDescription cas.desc
NEW!InvalidLink INVALID
NEW!OutputLink DECODED
NEW!PercentOfRejectedRecordsAllowed
NEW!RecordIdentifier
NEW!RejectInvalid 0
NEW!SendInvalid 1
NEW!TrailerIdentifier
OLD!DiscardFailureThreshold 15
OLD!NetworkElementID CAS
OLD!SendDiscardedFilesToLink
OLD!UseRecordSequenceNumbers
NEW?DECODED 0
NEW?INVALID 0
OLD?DECODED_CAS 0
CH_PPV_DWH_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!CompressionMethod compress
NEW!OutputFileNameExtractionMask CDR\.(.+)
NEW!Password
NEW!PrivateKey ${EventLinkUserHome}/.ssh/id_rsa
NEW!PublicKey ${EventLinkUserHome}/.ssh/id_rsa.pub
NEW!Subdirectory UPC${Affiliate}_$1
NEW!TargetDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/DWH
NEW!Username ${EventLinkUser}
OLD!CompressionMethod nocompression
OLD!Password u1elkpe
OLD!PrivateKey
OLD!PublicKey
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/DWH
OLD!Username u1elkpe
CH_PPV_DWH_ENCODER
NEW#UPC_CFG_CAS_DWH_ENCODER 1.1.0.
OLD#UPC_CFG_CH_PPV_BRIO_ENCODER 1.1.0.
NEW!CreateEmptyOutputFileOnFlush 0
NEW!CreateEmptyOutputFileOnSeparation 0
NEW!CreateEmptyOutputFiles 0
NEW!MaxAgeOfFile
NEW!OutputDescriptionFileName output_description.xml
OLD!CreateEmptyOutputFileOnFlush
OLD!CreateEmptyOutputFileOnSeparation
OLD!CreateEmptyOutputFiles
OLD!MaxAgeOfFile 1M
OLD!OutputDescriptionFileName brio_description.xml
NEW@UPC_CFG_CAS_DWH_ENCODER REJ
OLD@UPC_CFG_CH_PPV_BRIO_ENCODER REJ
CH_PPV_FRAUD_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!OutputFileNameExtractionMask
NEW!Password
NEW!PrivateKey ${EventLinkUserHome}/.ssh/id_rsa
NEW!PublicKey ${EventLinkUserHome}/.ssh/id_rsa.pub
NEW!Subdirectory
NEW!TargetDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/FRAUD
NEW!TempDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/FRAUD/tmp
NEW!Username ${EventLinkUser}
OLD!Password u1elkpe
OLD!PrivateKey
OLD!PublicKey
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/FRAUD
OLD!TempDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/FRAUD/tmp
OLD!Username u1elkpe
CH_PPV_HORIZON_BLN
NEW#UPC_CFG_HORIZON_BLN 1.5.2.
OLD#UPC_CFG_HORIZON_BLN 1.5.1.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 3.1.0.
NEW!KenanCounter K_0_1_2
NEW!OracleDatabaseName ${EventLinkDatabase}
OLD!AuditOutType ${AuditOutType}
OLD!KenanCounter K_0_2_1
OLD!OracleDatabaseName U1ELKPE
OLD!StreamName ${StreamName}
CH_PPV_KENAN_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!OutputFileNameExtractionMask
NEW!Subdirectory
NEW!TargetDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/KENAN
NEW!Username ${EventLinkUser}
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/KENAN
OLD!Username u1elkpe
CH_PPV_KENAN_DWH_BLN
NEW#UPC_CFG_CH_PPV_KENAN_BRIO_BLN 2.0.0.
OLD#UPC_CFG_CH_PPV_KENAN_BRIO_BLN 1.0.5.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 1.11.2.
NEW!ReprocessLimit 100d
OLD!202_DEFAULT_TIME 000000
OLD!AuditOutType ${AuditOutType}
OLD!LsAdmin ${EventLinkUserHome}/EventLink/lookup_server3/bin/ls_admin
OLD!ReprocessLimit 100
OLD!StreamName ${StreamName}
NEW?RDC_OUT1 0
OLD?RDC_OUT 0
OLD@UPC_CFG_CH_PPV_KENAN_BRIO_BLN INVALID_TVOD
OLD@UPC_CFG_CH_PPV_KENAN_BRIO_BLN REJECTED
CH_PPV_KENAN_ENCODER
NEW#UPC_CFG_CAS_KENAN_ENCODER 1.1.0.
OLD#UPC_CFG_CH_PPV_KENAN_ENCODER 1.0.0.
NEW#EL_LIB_CML 1.5.5.
OLD#EL_LIB_CML 1.5.4.
NEW!CreateEmptyOutputFileOnFlush 0
NEW!CreateEmptyOutputFileOnSeparation 0
NEW!CreateEmptyOutputFiles 0
NEW!DataRecordType PV1
NEW!MaxAgeOfFile
NEW!OutputDescriptionFileName dpv_old.xml
NEW!OutputFileNameCreationMask PV1.&SEQ(7).&CREATIONTIME(%Y%m%d%H%M%S).&SOURCEID.00.&MIN(EL_PURCHASE_DATETIME) &MAX(EL_PURCHASE_DATETIME)
NEW!RecordLimit
NEW!SeparateSequenceNumbersInOutput 0
OLD!CreateEmptyOutputFileOnFlush
OLD!CreateEmptyOutputFileOnSeparation
OLD!CreateEmptyOutputFiles
OLD!MaxAgeOfFile 86400
OLD!OutputDescriptionFileName kenan_description.xml
OLD!OutputFileNameCreationMask PV1.&SEQ(7).&CREATIONTIME(%Y%m%d%H%M%S).&SOURCEID.00.&MIN(EL_PURCHASE_LOCALTIME) &MAX(EL_PURCHASE_LOCALTIME2)
OLD!RecordLimit 100
OLD!SeparateSequenceNumbersInOutput 1
NEW@UPC_CFG_CAS_KENAN_ENCODER REJ
OLD@UPC_CFG_CH_PPV_KENAN_ENCODER REJ
CH_PPV_LOCAL_COLLECTOR
NEW#EL_APP_FTP_COLLECTOR 2.1.4.upc3
OLD#EL_APP_FTP_COLLECTOR 2.1.4.upc1
NEW#EL_LIB_NODE_EXTENSIONS 1.1.1.
NEW#UPC_LIB_PERL_UTILS 1.11.2.
NEW!BehaviourOnError Abort
NEW!OnlyCollectFilesOlderThan
NEW!SourceDirectory ${EventLinkUserHome}/delivery/${StreamName}/INPUT/
OLD!BehaviourOnError Restart
OLD!SourceDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/INPUT/
CH_PPV_OUTPUT_BACKUP
DONOTEXISTANYMORE
OLD#UPC_CFG_BACKUP 1.1.1.
OLD#EL_APP_FTP_DISTRIBUTOR 2.1.4.upc1
OLD#EL_LIB_SFTP 1.1.0.
OLD!BehaviourOnError Restart
OLD!BlockSize
OLD!Cipher
OLD!Compression
OLD!CompressionMethod gzip
OLD!ConnectionEstablishmentTimeout
OLD!ConnectionRetryCount
OLD!ConnectionRetryInterval
OLD!DailyDirectories no
OLD!DestinationHost
OLD!ExternalCommand
OLD!FlagFileSuffix
OLD!InputLinkDirectories Yes
OLD!Kex
OLD!MaxDays 30
OLD!MaxFiles 0
OLD!MaximumReconnectionInterval
OLD!MaximumSustainedNetworkDowntime
OLD!OriginalFilenameMask
OLD!Password
OLD!Port
OLD!PrivateKey
OLD!PublicKey
OLD!RemoteOS
OLD!RemotePrefix
OLD!RemoteSuffix
OLD!Ruleset backup.pm
OLD!SourceIdDirectories no
OLD!TargetDirectory /backup/BACKUP_EL603/UPCCH_PPV
OLD!TempDirectory
OLD!TempSuffix
OLD!TransferMode
OLD!TransferProtocol Local
OLD!TransferRetryCount
OLD!TransferRetryInterval
OLD!TransferType
OLD!Username
OLD?ENCODED_AUDIT 1
OLD?ENCODED_DWH 1
OLD?ENCODED_KENAN 1
CH_PPV_PRODIS_COLLECTOR
NEW!DeleteSource No
NEW!RemotePrefix old_
NEW!SourceDirectory ${EventLinkUserHome}/delivery/${StreamName}/INPUT/PRODIS
NEW!Username ${EventLinkUser}
OLD!DeleteSource Yes
OLD!RemotePrefix
OLD!SourceDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/INPUT/PRODIS
OLD!Username u1elkpe
CH_PPV_PRODIS_COMMON_BLN
NEW#UPC_CFG_PRODIS_COMMON_BLN 1.2.1.
OLD#UPC_CFG_PRODIS_COMMON_BLN 1.2.0.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 2.0.0.
OLD!AuditOutType ${AuditOutType}
OLD!StreamName ${StreamName}
CH_PPV_PRODIS_DECODER
NEW#EL_APP_ASCII_DECODER 3.0.3.upc5
OLD#EL_APP_ASCII_DECODER 3.0.3.upc4
NEW!PercentOfRejectedRecordsAllowed
CH_PPV_PRODIS_FDC
NEW!SequenceChecking 0
NEW!TRS.DataStorage ${StreamName}
OLD!SequenceChecking 1
OLD!TRS.DataStorage UPCCH_PPV
CH_PPV_PRODIS_RDC
NEW#UPC_CFG_RECORD_CHECKER 8.2.0.
OLD#UPC_CFG_RECORD_CHECKER 8.1.0.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 2.0.0.
NEW!DuplicateCheckEnabled No
NEW!OutputCounter B_0_1_2
NEW!RestartInterval 1d
NEW!TRS.DataStorage ${StreamName}
OLD!DuplicateCheckEnabled Yes
OLD!NodeName CH_PPV_PRODIS_RDC
OLD!OutputCounter B_0_2_1
OLD!StreamName ${StreamName}
OLD!TRS.DataStorage UPCCH_PPV
NEW@UPC_CFG_RECORD_CHECKER INVALID_DATA
CH_PPV_RAW_BACKUP
DONOTEXISTANYMORE
OLD#UPC_CFG_BACKUP 1.1.1.
OLD#EL_APP_FTP_DISTRIBUTOR 2.1.4.upc1
OLD#EL_LIB_SFTP 1.1.0.
OLD!BehaviourOnError Restart
OLD!BlockSize
OLD!Cipher
OLD!Compression
OLD!CompressionMethod gzip
OLD!ConnectionEstablishmentTimeout
OLD!ConnectionRetryCount
OLD!ConnectionRetryInterval
OLD!DailyDirectories no
OLD!DestinationHost
OLD!ExternalCommand
OLD!FlagFileSuffix
OLD!InputLinkDirectories No
OLD!Kex
OLD!MaxDays 30
OLD!MaxFiles 0
OLD!MaximumReconnectionInterval
OLD!MaximumSustainedNetworkDowntime
OLD!OriginalFilenameMask
OLD!Password
OLD!Port
OLD!PrivateKey
OLD!PublicKey
OLD!RemoteOS
OLD!RemotePrefix
OLD!RemoteSuffix .&CREATIONTIME(%Y%m%d%H%M%S)
OLD!Ruleset backup.pm
OLD!SourceIdDirectories yes
OLD!TargetDirectory /backup/BACKUP_EL603/UPCCH_PPV/RAW
OLD!TempDirectory
OLD!TempSuffix
OLD!TransferMode
OLD!TransferProtocol Local
OLD!TransferRetryCount
OLD!TransferRetryInterval
OLD!TransferType
OLD!Username
OLD?CAS_COLLECTOR 1
OLD?COLLECTED 1
CH_PPV_RDC
NEW#UPC_CFG_RECORD_CHECKER 8.2.0.
OLD#UPC_CFG_RECORD_CHECKER 7.1.5.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
NEW!AuditFields AUD_UA=I_UA,AUD_STU_NUMBER=I_STU_NUMBER,AUD_IMS_PRODUCT_ID=I_IMS_PRODUCT_ID
NEW!DuplicateCheckEnabled No
NEW!DuplicateCounter P_0_1_1
NEW!DuplicateDurationCounter
NEW!DurationField
NEW!InputCounter
NEW!InputDurationCounter
NEW!Keys I_UA,I_IMS_PRODUCT_ID
NEW!OutputCounter
NEW!OutputDurationCounter
NEW!RestartInterval 1d
NEW!SeparationCondition
NEW!SeparationValue1
NEW!SeparationValue2
NEW!SeparationValue3
NEW!SeparationValue4
NEW!SeparationValue5
NEW!TRS.DataStorage ${StreamName}
OLD!DuplicateCheckEnabled Yes
OLD!Keys I_UA,I_PURCHASE_DATE,I_PURCHASE_TIME,I_IMS_PRODUCT_ID
OLD!StreamName ${StreamName}
OLD!TRS.DataStorage UPCCH_PPV
NEW?RDC_OUT1 0
NEW?RDC_OUT2 0
NEW?RDC_OUT3 0
NEW?RDC_OUT4 0
NEW?RDC_OUT5 0
NEW?VALID 0
OLD?RDC_OUT 0
OLD?VALIDATION_OUT 0
NEW@UPC_CFG_RECORD_CHECKER DUPLICATE_RECORDS
NEW@UPC_CFG_RECORD_CHECKER INVALID_DATA
OLD@UPC_CFG_RECORD_CHECKER PV_INVALID_DATA
CH_PPV_VALIDATOR
NEW#UPC_CFG_CAS_VALIDATOR 1.1.1.
OLD#UPC_CFG_VOD_VALIDATOR 1.1.1.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
NEW!InputCounter I_0_2_1
NEW!RejectedCounter P_0_1_1
NEW!RestartInterval 1d
NEW!Timezone ${Timezone}
OLD!AuditOutType ${AuditOutType}
OLD!StreamName ${StreamName}
NEW?AUDIT_REJECTED 0
NEW?DECODED 0
NEW?INVALID 0
NEW?VALID 0
OLD?DECODED_CAS 0
OLD?VALIDATION_AUDIT_OUT 0
OLD?VALIDATION_OUT 0
NEW@UPC_CFG_CAS_VALIDATOR INVALID_DATA
OLD@UPC_CFG_VOD_VALIDATOR INVALID_DATETIME

The script I want to show output as comparisons between the two same types of records;
NOTE: NEW#,OLD#,NEW!,OLD!,NEW?,OLD?,NEW@,OLD@ are my own defined ways to differentiate between the different types of outputs.

<Name>
<TechnicalName><TAB><OldValue><TAB><NewValue>

Also there are cases in which only NEW is present and OLD is not there leave compared value of OLD as NULL and vicaversa.

I hope i made the complete problem understood. I real;ly need your help as i have been stuck with this for long.

Thanks Again for your patience.

12 
awk -F "[# \!\?\@]" '{if($0~ /^NEW/){if(NF > 2){a[$2]=$2" "$3}else{print $2}}else if ($0 ~ /^OLD/){print $2" "$3,a[$2]}else{print}}' OFS=" --> " file

Hope this helps you:)

13

Does this give you what you want (from your original file)? If it does, format the output as per your requirement.

gawk -F: '{for(i=1;i<NF;i++)
{
 n=split($i,e,/[-;|]/)
 exists[e[1]]
 if(sub(/^OLD./,"",e[3]))
 {
  oldval[e[1],e[2],e[3]]=(e[4]~/^[[:blank:]]*$/?SUBSEP:e[4])
  newval[e[1],e[2],e[3]]
 }
 else if(sub(/^NEW./,"",e[3]))
 {
  newval[e[1],e[2],e[3]]=e[4]
  oldval[e[1],e[2],e[3]]
 }
}}
END{
no_of_names=asorti(exists)
for(i=1;i<=no_of_names;i++)
{
 print exists
 for(j in oldval)
 {
  split(j,temp,SUBSEP)
  if(temp[1]==exists)
   print temp[3],(length(oldval[j])?(oldval[j]==SUBSEP?" ":oldval[j]):"NULL"),newval[j]|"sort"
 }
 close("sort")
 printf "\n"
}
}' OFS='|' diffStreamVersions.txt
1 Like
14

Hi,

Thanks for the query. It seems to work but there are some small small correction required. If you can help me with then then it would be of great use.

The script generate almost corrrect output but please refer to below output:

CH_PPV_AUDIT_DISTRIBUTOR
BehaviourOnError|Restart|Abort
EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc1||
EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc2||
OutputFileNameExtractionMask||
Subdirectory||
TargetDirectory|${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/CH_PPV|${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/${Affiliate}_PPV
TempDirectory|${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/CH_PPV/temp|${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/${Affiliate}_PPV/temp

in input data:

CH_PPV_DWH_ENCODER;A;OLD#UPC_CFG_CH_PPV_BRIO_ENCODER-1.1.0.:CH_PPV_DWH_DISTRIBUTOR;A;OLD#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc1:CH_PPV_KENAN_DISTRIBUTOR;A;OLD#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc1:CH_PPV_AUDIT_DISTRIBUTOR;A;NEW#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc2:CH_PPV_AUDIT_ENCODER;A;NEW#UPC_CFG_AUDIT_REJECTED_ENCODER-1.1.0.:CH_PPV_HORIZON_BLN;A;NEW#UPC_CFG_HORIZON_BLN-1.5.2.:CH_PPV_DWH_ENCODER;A;NEW#UPC_CFG_CAS_DWH_ENCODER-1.1.0.:CH_PPV_DECODER;A;NEW#UPC_CFG_CAS_DECODER-1.0.0.:CH_PPV_KENAN_DWH_BLN;A;OLD#UPC_CFG_CH_PPV_KENAN_BRIO_BLN-1.0.5.:CH_PPV_RAW_BACKUP;A;OLD#UPC_CFG_BACKUP-1.1.1.:CH_PPV_PRODIS_RDC;A;OLD#UPC_CFG_RECORD_CHECKER-8.1.0.:CH_PPV_KENAN_DWH_BLN;A;NEW#UPC_CFG_CH_PPV_KENAN_BRIO_BLN-2.0.0.:CH_PPV_PRODIS_COMMON_BLN;A;NEW#UPC_CFG_PRODIS_COMMON_BLN-1.2.1.:CH_PPV_VALIDATOR;A;NEW#UPC_CFG_CAS_VALIDATOR-1.1.1.:CH_PPV_CAS_COLLECTOR;A;OLD#UPC_CFG_CAS_COLLECTOR-2.0.0.:CH_PPV_PRODIS_COMMON_BLN;A;OLD#UPC_CFG_PRODIS_COMMON_BLN-1.2.0.:CH_PPV_DECODER;A;OLD#UPC_CFG_CH_PPV_CAS_DECODER-1.0.0.:CH_PPV_RDC;A;NEW#UPC_CFG_RECORD_CHECKER-8.2.0.:CH_PPV_AUDIT_DISTRIBUTOR;A;OLD#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc1:CH_PPV_RDC;A;OLD#UPC_CFG_RECORD_CHECKER-7.1.5.:CH_PPV_HORIZON_BLN;A;OLD#UPC_CFG_HORIZON_BLN-1.5.1.:CH_PPV_BACKUP;A;NEW#UPC_CFG_BACKUP-1.2.1.:CH_PPV_DWH_DISTRIBUTOR;A;NEW#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc2:CH_PPV_OUTPUT_BACKUP;A;OLD#UPC_CFG_BACKUP-1.1.1.:CH_PPV_KENAN_ENCODER;A;OLD#UPC_CFG_CH_PPV_KENAN_ENCODER-1.0.0.:CH_PPV_FRAUD_DISTRIBUTOR;A;OLD#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc1:CH_PPV_CAS_COLLECTOR;A;NEW#UPC_CFG_CAS_COLLECTOR-2.1.0.:CH_PPV_FRAUD_DISTRIBUTOR;A;NEW#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc2:CH_PPV_AUDIT_ENCODER;A;OLD#UPC_CFG_CH_PPV_AUDIT_ENCODER-1.0.2.:CH_PPV_VALIDATOR;A;OLD#UPC_CFG_VOD_VALIDATOR-1.1.1.:CH_PPV_PRODIS_RDC;A;NEW#UPC_CFG_RECORD_CHECKER-8.2.0.:CH_PPV_KENAN_DISTRIBUTOR;A;NEW#EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc2:CH_PPV_KENAN_ENCODER;A;NEW#UPC_CFG_CAS_KENAN_ENCODER-1.1.0.:

the expected output was:

EL_CFG_FTP_DISTRIBUTOR-2.1.0.upc1|2.1.0.upc2|

Also if the OLD field is not exist i need to populate NULL
AND if OLD field exist and is blank then " "

Can you please check this. Really thanks a lot for putting soo much effort. Its really appreciated. I would never come up with such a script.

15

I've made slight modifications in the script in my earlier post. Try now.

16

Hi,

Thanks worked like a cream. Just one more thing i need to incorportae if you can tell me how to do it then my whole purpoose is solved.

if the OLD field is not exist i need to populate NULL
if OLD field exist and is blank then " "

For E.G:

NEW!DuplicateCheckEnabled No
NEW!DuplicateCounter P_0_1_1
NEW!DuplicateDurationCounter
NEW!DurationField
NEW!InputCounter
NEW!InputDurationCounter
NEW!Keys I_UA,I_IMS_PRODUCT_ID
NEW!OutputCounter
NEW!OutputDurationCounter
NEW!RestartInterval 1d
NEW!SeparationCondition
NEW!SeparationValue1
NEW!SeparationValue2
NEW!SeparationValue3
NEW!SeparationValue4
NEW!SeparationValue5
NEW!TRS.DataStorage ${StreamName}
OLD!DuplicateCheckEnabled Yes
OLD!Keys I_UA,I_PURCHASE_DATE,I_PURCHASE_TIME,I_IMS_PRODUCT_ID
OLD!StreamName ${StreamName}
OLD!TRS.DataStorage UPCCH_PPV

Current Output :

AuditFields||AUD_UA=I_UA, AUD_STU_NUMBER=I_STU_NUMBER, AUD_IMS_PRODUCT_ID=I_IMS_PRODUCT_ID
DuplicateCheckEnabled|Yes|No
DuplicateCounter||P_0_1_1
DuplicateDurationCounter||
DurationField||
InputCounter||
InputDurationCounter||
Keys|I_UA, I_PURCHASE_DATE, I_PURCHASE_TIME, I_IMS_PRODUCT_ID|I_UA, I_IMS_PRODUCT_ID
OutputCounter||
OutputDurationCounter||
RDC_OUT1||0
RDC_OUT2||0
RDC_OUT3||0
RDC_OUT4||0
RDC_OUT5||0
RDC_OUT|0|
RestartInterval||1d
SeparationCondition||
SeparationValue1||
SeparationValue2||
SeparationValue3||
SeparationValue4||
SeparationValue5||
StreamName|${StreamName}|
TRS.DataStorage|UPCCH_PPV|${StreamName}

Expected Output:

DuplicateCounter||P_0_1_1
SeparationValue1|NULL|
SeparationValue2|NULL|
SeparationValue3|NULL|
SeparationValue4|NULL|
SeparationValue5|NULL|

DuplicateCounter Value was present but was blank hence blank.

SperationValue is only new. Thats why OLD as NULL.

Thanks a lot for your help!!!!

---------- Post updated at 07:15 AM ---------- Previous update was at 03:24 AM ----------

Hi,

Can you please just help me out with a small modification as expected above.

Regards
Rajan

---------- Post updated at 09:39 AM ---------- Previous update was at 07:15 AM ----------

Hi Everyone,

Kindly help me in finding a way out of it.

Thanks
Rajan

17

try with this..

awk -F "[# \!\?\@]" '{if($0~ /^NEW/){if(NF > 2){a[$2]=$2" "$3}else{print $2,"NULL"}}
else if ($0 ~ /^OLD/){print $2" "$3,a[$2];delet a[$2]}else{print}}END{for(i in a){ print a}}' OFS="|" file
18

Hi,

This doesnot work. I just need a small correction in the awk published by elixir_sinari. Can you please try.
Error generated:

awk: cmd. line:2: fatal: cannot open file `file' for reading (No such file or directory)

[u1elkpe@ /u01/appl/u1elkpe/Rajan$]> awk -F "[# \!\?\@]" '{if($0~ /^NEW/){if(NF > 2){a[$2]=$2" "$3}else{print $2,"NULL"}}
else if ($0 ~ /^OLD/){print $2" "$3,a[$2];delet a[$2]}else{print}}END{for(i in a){ print a}}' OFS="|" diffStreamVersions.txt
awk: warning: escape sequence `\!' treated as plain `!'
awk: warning: escape sequence `\?' treated as plain `?'
awk: warning: escape sequence `\@' treated as plain `@'

Thanks for your help anyway.

Regards
Rajan

19

these are not errors. just warnings.. ignore it...

Are getting the output right..?

20

Hi,
have made some changes in the earlier code. Check now.