Extract data from a log file and put it in a file

Shell Programming and Scripting
1

Hi, I would like to seek your help for a script that will extract data from log file and put it in a file.

Sample log file

2018-10-23 12:33:21 AI ERROR -- tpid: SAMPLE_TH  account: 123456789  aiSessionNumber: 660640464  mapName: xxx to yyy  
errorDesc: Translation Error:ErrorNumber : 993   
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504 


  
sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/XXXXXX001314509_952834960.es2eg.4X
:
--------------------------------------------------------------------------------------------------
2018-10-23 12:33:21 AI ERROR -- tpid: SAMPLE_XX  account: 123654789  aiSessionNumber: 660640457  mapName: xxx to yyy  
errorDesc: Translation Error:ErrorNumber : 993   
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504 


  
sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/YYYYYY001314509_952834960.es2eg.3X
:
--------------------------------------------------------------------------------------------------
2018-10-23 12:39:15 AI ERROR -- tpid: XXXXX_TH  account: 321456789  aiSessionNumber: 660642348  mapName: xxx to yyy  
errorDesc: Translation Error:ErrorNumber : 993   
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504 


  
sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/ZZZZZZ001314509_952834960.es2eg.3X
:

I want the output to have something like this in order

TPID = SAMPLE_TH   |  Account =123456789  
Source filename =/apps/EGAPshared/Invoicing/gcs/large_file/XXXXXX001314509_952834960.es2eg.4X

TPID =  SAMPLE_XX  |  Account =123654789  
Source filename =/apps/EGAPshared/Invoicing/gcs/large_file/YYYYYY001314509_952834960.es2eg.3X
:

TPID = XXXXX_TH   |  Account =321456789  
Source filename =/apps/EGAPshared/Invoicing/gcs/large_file/ZZZZZZ001314509_952834960.es2eg.3X
:

Thanks in advance.... appreciate your time.

2

Welcome to the forum.

Please show us what you tried and where that failed. We're glad to help.

3 
C:\WINDOWS\system32>cat mytst
2018-10-23 12:33:21 AI ERROR -- tpid: SAMPLE_TH  account: 123456789  aiSessionNumber: 660640464  mapName: xxx to yyy
errorDesc: Translation Error:ErrorNumber : 993
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504



sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/XXXXXX001314509_952834960.es2eg.4X
:
--------------------------------------------------------------------------------------------------
2018-10-23 12:33:21 AI ERROR -- tpid: SAMPLE_XX  account: 123654789  aiSessionNumber: 660640457  mapName: xxx to yyy
errorDesc: Translation Error:ErrorNumber : 993
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504



sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/YYYYYY001314509_952834960.es2eg.3X
:
--------------------------------------------------------------------------------------------------
2018-10-23 12:39:15 AI ERROR -- tpid: XXXXX_TH  account: 321456789  aiSessionNumber: 660642348  mapName: xxx to yyy
errorDesc: Translation Error:ErrorNumber : 993
ErrorDescriptionSummary: Errors from sources other than XML Status File
ErrorDescriptionDetail : 504



sourceFileName: /apps/EGAPshared/Invoicing/gcs/large_file/ZZZZZZ001314509_952834960.es2eg.3X

C:\WINDOWS\system32>gawk  -F"[: ]" "/tpid/{print toupper($8) \" = \" $10 OFS $12 \" =\"  $14}/^sourceFileName/{p=toupper(substr($1,1,1));s=tolower(substr($1,2,5)\" \"substr($1,7));sub($1\":\",\" =\",$0); print p s $0}" OFS=" | " mytst
TPID = SAMPLE_TH | account =123456789
Source filename = /apps/EGAPshared/Invoicing/gcs/large_file/XXXXXX001314509_952834960.es2eg.4X
TPID = SAMPLE_XX | account =123654789
Source filename = /apps/EGAPshared/Invoicing/gcs/large_file/YYYYYY001314509_952834960.es2eg.3X
TPID = XXXXX_TH | account =321456789
Source filename = /apps/EGAPshared/Invoicing/gcs/large_file/ZZZZZZ001314509_952834960.es2eg.3X

C:\WINDOWS\system32>

So... with the unix syntax :

gawk -F"[: ]" '/tpid/{print toupper($8) " = " $10 OFS $12 " =" $14}
/^sourceFileName/{
p=toupper(substr($1,1,1))
s=tolower(substr($1,2,5)" "substr($1,7))
sub($1":"," =",$0)
print p s $0
}' OFS=" | " inputfile

Another approach:

gawk '{gsub(" ",RS)}1' inputfile | gawk '/(tpid|account|sourceFileName)/{f=$0;getline;print f,$0}'