Perl netbackup script?

Hi,

I'm currently working on adapting a netbackup script to run reports about the backup duration of clients and how many kb's written.
But I'm stuck at the time stamp of the backup.

I want to see in the sheet:
Sat Nov 07 00:00:26 2009 (1257548426)
But i get:
26 2009 (1257548426)

#!/usr/bin/perl


$DATA = system("/usr/openv/netbackup/bin/admincmd/bpimagelist -L -hoursago 24 > /tmp/Tapelog.txt"); # || die qq (Can't execute command bpimagelist.cmd);
#Tapelog.txt is made from bpimagelist -L -hoursago 24, this is just a rough list.  See Tapes_To_Be_Pulled for final output.

open(CSVFILE, ">./MK_Tapes_To_Be_Pulled.csv") || die qq(Can't open "Tapes_To_Be_Pulled.csv " for output.\n);
open (LINKLIST, "</tmp/Tapelog.txt") or die $!;

#creates Tapes_To_Be_Pulled.csv (output file)
print CSVFILE "Client,Policy,Schedule Type,Backup Time, Elapsed Time, Kilobytes,Media Date\n";
#Below parses Tapelog.txt for the listed fields
while (<LINKLIST>) {
    chomp;
     s{\d+:\d+:\d+\s*\([^)]*\)
    }{}x; #Takes C time off of the Media Date listing, or any other time listing in logfile.
    $_=~ s/^\s+//; $_ =~ s/\s+$//;
    my ($key, $value) = ($_=~ m!(.+):(.+)!);
    $value =~ s/^\s+//; $value =~ s/\s+$//;
    print "$key,$value\n";
    $client = $value if (/^Client:/);
    $policy = $value if (/^Policy:/);
    $schedule = $value if (/^Schedule Type/);
    $backuptime= substr($value if (/^Backup Time/));
    $elapsedtime = $value if (/^Elapsed Time/);
    $id = $value if (/^ID/);
    $mediatape = $value if (/^Media Date: /);
    $kilobytes = $value if (/^Kilobytes/);
    print CSVFILE  "$client,$policy,$schedule,$backuptime,$elapsedtime,$kilobytes,$mediatape\n" if /resume/ ;
}


close(CSVFILE) || die qq(Can't close output file "Tapes_To_Be_Pulled.csv".\n);

Post in english please

Can you post your "Tapelog.txt" file, or a sample if it is too big or confidential ?

tyler_durden

Tapelog.txt

Client:            babbage
Backup ID:         babbage_1262120428
Policy:            Unix
Policy Type:       Standard (0)
Proxy Client:      (none specified)
Creator:           NetBackup
Name1:             (none specified)
Sched Label:       Incrs
Schedule Type:     INCR (1)
Retention Level:   1 month (3)
Backup Time:       Tue Dec 29 22:00:28 2009 (1262120428)
Elapsed Time:      103 second(s)
Expiration Time:   Fri Jan 29 22:00:28 2010 (1264798828)
Compressed:        no
Client Encrypted:  no
Kilobytes:         32
Number of Files:   0
Number of Copies:  1
Number of Fragments:   1
Histogram:         -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
DB Compressed:     no
Files File Name:   Unix_1262120428_INCR.f
Previous Backup Files File Name:   (none specified)
Parent Backup Image File Name:   (none specified)
SW Version:        (none specified)
Options:           0x0
MPX:               1
TIR Info:          0
TIR Expiration:    Thu Jan 01 01:00:00 1970 (0)
Keyword:           (none specified)
Ext Security Info: no
File Restore Raw:  no
Image Dump Level:  0
File System Only:  no
Object Descriptor: (none specified)
Previous BI Time:  Thu Jan 01 01:00:00 1970 (0)
BI Full Time:      Thu Jan 01 01:00:00 1970 (0)
Request Pid:       0
Backup Status:     0
Stream Number:     5
Backup Copy:       Standard (0)
Files File size:     447
PFI type:     0
IMAGE_ATTRIBUTE:     0
Primary Copy:      1
Image Type:        0  (Regular)
Job ID:            89629
Num Resumes:       0
Resume Expiration: Thu Jan 01 01:00:00 1970 (0)
Data Classification:    (none specified)
Data_Classification_ID: (none specified)
Storage Lifecycle Policy:    (none specified)
STL_Completed:      0
Snap Time:      Thu Jan 01 01:00:00 1970 (0)
Copy number:       1
 Fragment:         1
 Kilobytes:        32
 Remainder:        512
 Media Type:       Media Manager (2)
 Density:          hcart2 (14)
 File Num:         28
 ID:               H445L2
 Host:             thor
 Block Size:       524288
 Offset:           23917
 Media Date:       Sat Dec 26 00:00:11 2009 (1261782011)
 Dev Written On:   0
 Flags:            0x0  
 Media Descriptor:        ?
 Expiration Time:  Fri Jan 29 22:00:28 2010 (1264798828)
 MPX:              1
 retention_lvl:    1 month (3)
 Try to Keep Time:  Thu Jan 01 01:00:00 1970 (0)
 Copy Creation Time:  Tue Dec 29 22:02:11 2009 (1262120531)
 checkpoint:       0
 resume num:       0
 Key tag:          *NULL*

Hi,

Thanks for posting the data file. I've made a few changes to your Perl script - changed the regex for splitting records, and removed some extraneous stuff.

Try this to see if it works for you:

$
$ cat netbkup.pl
#!/usr/bin/perl -w
$DATA = system("/usr/openv/netbackup/bin/admincmd/bpimagelist -L -hoursago 24 > /tmp/Tapelog.txt");
open (CSVFILE, ">MK_Tapes_To_Be_Pulled.csv") or die "Can't open MK_Tapes_To_Be_Pulled.csv for output: $!";
open (LINKLIST, "</tmp/Tapelog.txt") or die "Can't open /tmp/Tapelog.txt: $!";
print CSVFILE "Client,Policy,Schedule Type,Backup Time, Elapsed Time, Kilobytes,Media Date\n";
while (<LINKLIST>) {
  chomp;
  s/^\s+//;
  s/\s+$//;
  ($key, $value) = ($_=~ m!(.*?):(.+)!);
  $value =~ s/^\s+//;
  $value =~ s/\s+$//;
  printf("%-40s %-s\n",$key,$value);
  $client = $value if /^Client:/;
  $policy = $value if /^Policy:/;
  $schedule = $value if /^Schedule Type/;
  $backuptime = $value if /^Backup Time/;
  $elapsedtime = $value if /^Elapsed Time/;
  $mediatape = $value if /^Media Date: /;
  $kilobytes = $value if /^Kilobytes/;
  print CSVFILE  "$client,$policy,$schedule,$backuptime,$elapsedtime,$kilobytes,$mediatape\n" if /resume/ ;
}
close (CSVFILE)  or die "Can't close output file Tapes_To_Be_Pulled.csv: $!";
close (LINKLIST) or die "Can't close file /tmp/Tapelog.txt: $!";
$
$

tyler_durden

Thanks, I will check tomorrow iff the script will works now.

---------- Post updated 04-01-10 at 12:06 PM ---------- Previous update was 03-01-10 at 03:39 PM ----------

The script works, but there are errors at the beginning when I ran the script.

bash-3.00# ./test
Name "main::DATA" used only once: possible typo at ./test line 2.
Use of uninitialized value in substitution (s///) at ./test line 11, <LINKLIST> line 1.
Use of uninitialized value in substitution (s///) at ./test line 12, <LINKLIST> line 1.
Use of uninitialized value in printf at ./test line 13, <LINKLIST> line 1.
Use of uninitialized value in printf at ./test line 13, <LINKLIST> line 1.

---------- Post updated at 12:07 PM ---------- Previous update was at 12:06 PM ----------

Name "main:: DATA" used only once: possible typo at ./test line 2.
Use of uninitialized value in substitution (s///) at ./test line 11, <LINKLIST> line 1.
Use of uninitialized value in substitution (s///) at ./test line 12, <LINKLIST> line 1.
Use of uninitialized value in printf at ./test line 13, <LINKLIST> line 1.
Use of uninitialized value in printf at ./test line 13, <LINKLIST> line 1.

Those are warnings actually; set by the -w flag in the shebang.
Remove the assignment to $DATA. The warning is printed because $DATA is assigned but never used anywhere else. (As such, you don't have to assign it in the first place.)
My guess is that the uninitialized value warning is printed due to blank lines in your Tapelog.txt file. There's nothing to substitute at that point, so Perl prints a warning. On encountering a blank line, you may want to skip it and move on to the next iteration of the "while" loop.

tyler_durden

Ok, no problem. The script works and it gives me the output that I wan't to see thanks for the help!

Sure, here's the updated script with the changes suggested in the earlier post, in red color. I hope you were able to figure them out yourself.

$
$ cat netbkup.pl
#!/usr/bin/perl -w
system("/usr/openv/netbackup/bin/admincmd/bpimagelist -L -hoursago 24 > /tmp/Tapelog.txt");
open (CSVFILE, ">MK_Tapes_To_Be_Pulled.csv") or die "Can't open MK_Tapes_To_Be_Pulled.csv for output: $!";
open (LINKLIST, "</tmp/Tapelog.txt") or die "Can't open /tmp/Tapelog.txt: $!";
print CSVFILE "Client,Policy,Schedule Type,Backup Time, Elapsed Time, Kilobytes,Media Date\n";
while (<LINKLIST>) {
  chomp;
  next if /^\s*$/; # skip this line if it is blank or has nothing but whitespace
  s/^\s+//;
  s/\s+$//;
  ($key, $value) = ($_=~ m!(.*?):(.+)!);
  $value =~ s/^\s+//;
  $value =~ s/\s+$//;
  printf("%-40s %-s\n",$key,$value);
  $client = $value if /^Client:/;
  $policy = $value if /^Policy:/;
  $schedule = $value if /^Schedule Type/;
  $backuptime = $value if /^Backup Time/;
  $elapsedtime = $value if /^Elapsed Time/;
  $mediatape = $value if /^Media Date: /;
  $kilobytes = $value if /^Kilobytes/;
  print CSVFILE  "$client,$policy,$schedule,$backuptime,$elapsedtime,$kilobytes,$mediatape\n" if /resume/ ;
}
close (CSVFILE)  or die "Can't close output file Tapes_To_Be_Pulled.csv: $!";
close (LINKLIST) or die "Can't close file /tmp/Tapelog.txt: $!";
$
$

The alternative is to remove the "-w" warnings switch on first line. But I wouldn't recommend that.

tyler_durden

Late post but better late then never right.
For the system part in red i could have guessed it.
I add some code to the script also.

cat nbureport.pl
#!/usr/bin/perl -w
$bdate=$ARGV[0];
$edate=$ARGV[1];
system("/usr/openv/netbackup/bin/admincmd/bpimagelist -d $bdate -e $edate -L > /tmp/Tapelog.txt");
open (CSVFILE, ">MK_Tapes_To_Be_Pulled.csv") or die "Can't open MK_Tapes_To_Be_Pulled.csv for output: $!";
open (LINKLIST, "</tmp/Tapelog.txt") or die "Can't open /tmp/Tapelog.txt: $!";
print CSVFILE "Client,Policy,Schedule Type,Backup Time, Elapsed Time, Kilobytes\n";
while (<LINKLIST>) {
  chomp;
  next if /^\s*$/; # skip this line if it is blank or has nothing but whitespace
  s/^\s+//;
  s/\s+$//;
  s/ second\(s\)//;
  ($key, $value) = ($_=~ m!(.*?):(.+)!);
  $value =~ s/^\s+//;
  $value =~ s/\s+$//;
  printf("%-40s %-s\n",$key,$value);
  $client = $value if /^Client:/;
  $policy = $value if /^Policy:/;
  $schedule = $value if /^Schedule Type/;
  $backuptime = $value if /^Backup Time/;
  $backuptime = substr($backuptime,0,10);
  $elapsedtime = $value if /^Elapsed Time/;
  $kilobytes = $value if /^Kilobytes/;
  print CSVFILE  "$client,$policy,$schedule,$backuptime,$elapsedtime,$kilobytes\n" if /resume/ ;
}
close (CSVFILE)  or die "Can't close output file Tapes_To_Be_Pulled.csv: $!";
close (LINKLIST) or die "Can't close file /tmp/Tapelog.txt: $!";