Hi,
Our log4j file contents look like this:
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP1
AS SELECT name,dept
from employee
Where employee='Jeff'
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP2
AS SELECT name,dept
from employee
Where employee='Yung'
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182
As you can see the create statement is across many lines, and the number of lines can vary.
I need to have only one line per entry.
My output should look like this:
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP1 AS SELECT name,dept from employee Where employee='Jeff'
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP2 AS SELECT name,dept from employee Where employee='Yung'
2018-11-20T00:06:58,888 INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182
Any idea on how to achieve this?
I was trying sed and some regex patterns, but was unable to make it work
Thanks