Custom Report

Hi All,

Am getting the raw report from the source and need to prepare the custom report as per the requirement.

Requirement keep getting change according to the need.

Raw data is as below

/* ----------------- test_job_hu ----------------- */

insert_job: test_job_hu   job_type: CMD
command: sleep 600
machine: localhost
owner: account
permission: gx,wx
date_conditions: 0
std_out_file: "/tmp/$AUTO_JOB_NAME.out"
std_err_file: "/tmp/$AUTO_JOB_NAME.err"
alarm_if_fail: 1
group: P2
application: TBS_00_AXZCDE_Admin
send_notification: 1
notification_msg: "test_job_hu job completed in HU NP"
notification_emailaddress: pradeep.agarwal@in.com



/* ----------------- test_machine ----------------- */

insert_job: test_machine   job_type: CMD
box_name: test_machine_box
command: echo $HOME
machine: HULNXMACHINE
owner: svc_account@hu.europe.com
permission: gx,wx
date_conditions: 0
alarm_if_fail: 1



/* ----------------- test_machine ----------------- */

insert_job: test_web_cmd   job_type: CMD
box_name: test_web_box
command: dir
machine: HUWINMACHINE
owner: svc_account@hu.europe.com
permission: gx,wx
date_conditions: 0
alarm_if_fail: 0
application: my_web_application
group: P1

I want the custom report as below.

JOB_NAME;JOB_TYPE;box_name;command;machine;owner;permission;date_conditions;std_out_file;std_err_file;alarm_if_fail;group;application;send_notification;notification_msg,notification_emailaddress;
test_job_hu;CMD; ;sleep 600;localhost;account;gx,wx;0;/tmp/$AUTO_JOB_NAME.out;/tmp/$AUTO_JOB_NAME.err;1,P2;TBS_00_AXZCDE_Admin;1;test_job_hu job completed in HU NP;pradeep.agarwal@in.com;
test_machine;CMD;test_machine_box;echo $HOME;HULNXMACHINE;svc_account@hu.europe.com;gx,wx;0; ; ;1; ; ; ; ; ;
test_web_cmd;CMD;test_web_box,dir;HUWINMACHINE;svc_account@hu.europe.com;gx,wx;0; ; ;0;P1;my_web_application; ; ; ;

Details of System:

  • File comes as a plain text file
  • O/S: CentOS release 6.9 (Final) and Red Hat Enterprise Linux Server release 6.5 (Santiago)
  • Shell: BASH

Hello pradeep84in,

I have a few to questions pose in response first:-

  • What have you tried so far?
  • What output/errors do you get?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)

Most importantly, What have you tried so far?

Whilst there is lots of useful information in your post, there are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

We're all here to learn and getting the relevant information will help us all.

Kind regards,
Robin

Hi Robin,

Earlier we were getting this report in the Windows server and we had developed the VBS(VB Script) to perform the same.

Now it's coming to Linux System, so we have to start it from scratch.

I didn't started writing any code till now.

But as per my analysis "insert_job" is the 1st line for each attempt which is a catch here and if I can get the data from 1st "insert_job" to before the next insert_job then can grep each of them as per their 1st part of the line and get the 2nd part by using awk and then echo them to one single line.

Thanks & Regards,
Pradeep Agarwal

awk '
/:.*:/ {job=$2; jobs[$2]=$2; job_type=$3; sub(" *: *", "", job_type); cols[job_type]=job_type; data[job, job_type]=$NF; next}
/:/ {match($0, " *: *"); col=substr($0, 1, RSTART-1); val=substr($0, RSTART + RLENGTH); cols[col]=col; data[job, col]=val}
END {
   printf "JOB_NAME;JOB_TYPE;";
   for (j in cols) if (j !~ /^job_type$/) printf cols[j] ";";
   print "";
   for (i in jobs) {
      printf i ";" data[i, "job_type"] ";";
      for (j in cols) {
         if (j !~ /^job_type$/) printf (length(data[i, j]) ? data[i, j] : " ") ";";
      }
      print "";
   }
}
' infile

A flexible Perl program.

Save as formatter.pl , run as perl formatter.pl [file1 file2 ...]

#!/usr/bin/perl

use strict;
use warnings;

#
# An ordered list of item labels.
my @ordered_labels = (
   'insert_job',
   'job_type',
   'box_name',
   'command',
   'machine',
   'owner',
   'permission',
   'date_conditions',
   'std_out_file',
   'std_err_file',
   'alarm_if_fail',
   'group',
   'application',
   'send_notification',
   'notification_msg',
   'notification_emailaddress',);
#
# Buffer db for all values on each group.
my %db;

display_header();

#
# Parse and collect values.
while(<>) {
    #
    # Inside a group
    if(/^insert_job/../^\n/){
        if(/^insert_job/) {
            #
            # First line has two pairs and value does not have
            # spaces
            while(/(\w+):\s(\w+)/g) {
                $db{$1} = $2;
            }
        }
        else {
            #
            # Parse only one pair that may contain all kind
            # of characters as value, including spaces and quotes.
            /(\w+):\s(.*$)/ and $db{$1} = $2;
        }
        #
        # Display the data gathered and clear db for next time.
        if(/^\n/) {
            display_body();
            %db = ();
        }
    }
}
# Display any data still in the db buffer.
display_body() if %db;

sub display_header {
    print "JOB_NAME;JOB_TYPE;";
    for(@ordered_labels[2..$#ordered_labels]) {
        print "$_;";
    }
    print "\n";
}

sub display_body {
    for my $i (@ordered_labels) {
        if (defined $db{$i}){
            $db{$i} =~ s/"//g;
            print "$db{$i};";
        }
        else {
            print " ;";
        }
    }
    print "\n";
}

Output:

$ perl formatter.pl raw_file

JOB_NAME;JOB_TYPE;box_name;command;machine;owner;permission;date_conditions;std_out_file;std_err_file;alarm_if_fail;group;application;send_notification;notification_msg;notification_emailaddress;
test_job_hu;CMD; ;sleep 600;localhost;account;gx,wx;0;/tmp/$AUTO_JOB_NAME.out;/tmp/$AUTO_JOB_NAME.err;1;P2;TBS_00_AXZCDE_Admin;1;test_job_hu job completed in HU NP;pradeep.agarwal@in.com;
test_machine;CMD;test_machine_box;echo $HOME;HULNXMACHINE;svc_account@hu.europe.com;gx,wx;0; ; ;1; ; ; ; ; ;