Parse configuration file & add line in particular section

Greetings,

I recently built a replicated DRBD, Heartbeat, & iSCSI Target Initiator storage server on Ubuntu 10.04 to offer shared storage to server Vmware ESX and Microsoft Clusters. Everything works flawlessly, however I wanted to make a script to create, remove, grow volumes to offer ESX rather then having a technical diagram and instructions so that other technicians are able to do this process with limited Linux knowledge.

I know the basis to Linux scripting with variables and command scripting but I am not sure how to approach parsing a configuration file for iSCSI Target Initiator and adding lines between existing lines under a sub group. I figure it has to be done in an array but I am not sure how to keep it within a parameter as well as add a line adding the next sequential number available.

Let me give an example of my configuration file called ietd.conf:

Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1
	IncomingUser iqn.1998-01.com.vmware:esx001-3c494e40 password
	IncomingUser iqn.1998-01.com.vmware:esx002-7e5d5341 password
	IncomingUser iqn.1998-01.com.vmware:esx003-76a022ad password
        Lun 0 Path=/dev/replicated/resources,Type=blockio
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Lun 2 Path=/dev/replicated/volume2,Type=blockio
	Alias VMwareStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Target iqn.2011-03.stroageserver.workgroup.net:Microsoft.Storage.1
	IncomingUser MSiSCSIAdmin password
        Lun 1 Path=/dev/replicated/volume3,Type=blockio
	Alias MicrosoftStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

What I need to do is when my script asks the user which Server they wish to add storage too (lets say ESX) that it targets �Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1� and reads each �Lun #.....� and adds a line below LUN 2 (in example configuration file) as LUN 3? I will replace �/dev/replicated/<volume>� with a variable which I can do. I just do not know how to parse only the lines between �Target iqn� and the next �Target ign� looking for �LUN #� and adding a new line below the existing LUN with a higher sequential number.

I am also not sure how to approach gaps in LUN numbers when volumes are deleted. My assumption is to take the highest LUN # under the target and add one to it but I am not sure if this is correct?

Any insight would be great. Thanks.

How about this?

#!/bin/bash

servername=Microsoft.Storage.1 #to find appropriate paragraph

nums=( $( awk '
/'"$servername"'/,/^\s*$/  { #process only between $servername and empty line
  if(/Lun/){numLun=$2; numRec=FNR} #if found line with 'Lun', capture 2nd field and line number
}
END{
print numLun" "numRec
} ' ietd.conf ) ) 
#nums is an array; first element is the field right after 'Lun'
#and second element is its line number 

newEntry="Lun $((${nums[0]}+1)) and whatever else you enter" #what should be inserted

sed "${nums[1]} a \\\t$newEntry" ietd.conf   #'a' command (append) to the appropriate line 
1 Like

Thank you! I was able to utilize what you gave me with variables to make it work in my script perfectly.

I am working on my delete part of my script where now I am looking to gather all the LUN #'s under a particular field from the ietd.conf file (such as "Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1" and offer them to the user as a selection choice such as:

1) Lun 0 Path=/dev/replicated/resources,Type=blockio
2) Lun 1 Path=/dev/replicated/volume1,Type=blockio
3) Lun 2 Path=/dev/replicated/volume2,Type=blockio

So when the user selects 2, the line "Lun 1 Path=/dev/replicated/volume1,Type=blockio" is removed from the file. Any advice?

Could this help you ?

#!/usr/bin/perl

use strict;
my ($server,@flds,$answer,$i,@path,$ret);
system("clear");
{
local $/="\n\n";
$server='Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1';
open(FH,"ietd.conf") or die "$!\n";
while (<FH>){
chomp;
if(/$server/) {
@flds=map("$_\n",grep(/Lun/,split(/\n/)));
foreach (@flds) {s/^\s+//g; print ++$i.") ".$_;}
}
}
}
close(FH);
printf "Enter option:";
$answer=<STDIN>;
chomp;
die  "Invalid Option\n" if ($answer <= 0 ||  $answer > $i);
open(FH,"ietd.conf") or die "$!\n";
{
        local $/="\n\n";
        while(<FH>){
        if(/$server/) {
        @path=split(/=/,$flds[$answer-1]);
        $path[1]=~s/,.*$//g;
        system("./pass_variable.sh $path[1]");
        s/$flds[$answer-1]\s*//g;
        } print $_;
        }

}
close(FH);
#cat pass_variable.sh
#!/bin/sh
echo ""
echo "Bash script start"
echo $1
echo "Bash script end"
sleep 2

Please make bash script executable using

chmod +x pass_variable.sh

Thank you.

I have two questions as I am not familiar with the perl programming language.

1) When I execute what you gave me against the ietd.conf file I am prompted to "Enter option:" but nothing is printed on the screen telling me the available options or lines that it has found. No matter what I type I get the die "Invalid Option" and exit.

2) My current script is built in /bin/bash can I make perl and bash work together as my future plan is it extract Path="/dev/replicated/<name>" into a variable of the line being removed from ietd.conf for my second stage of deleting the lvm.

It's working fine at my end see the below o/p. I have modified my perl code in my previous post.You can see how to pass variable to bash script from perl.
ietd.conf file

#cat ietd.conf
Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1
        IncomingUser iqn.1998-01.com.vmware:esx001-3c494e40 password
        IncomingUser iqn.1998-01.com.vmware:esx002-7e5d5341 password
        IncomingUser iqn.1998-01.com.vmware:esx003-76a022ad password
        Lun 0 Path=/dev/replicated/resources,Type=blockio
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Lun 2 Path=/dev/replicated/volume2,Type=blockio
        Alias VMwareStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Target iqn.2011-03.stroageserver.workgroup.net:Microsoft.Storage.1
        IncomingUser MSiSCSIAdmin password
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Alias MicrosoftStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Perl script invocation

#perl DeleteLines.pl

Output

1) Lun 0 Path=/dev/replicated/resources,Type=blockio
2) Lun 1 Path=/dev/replicated/volume1,Type=blockio
3) Lun 2 Path=/dev/replicated/volume2,Type=blockio
Enter option:2

Bash script start
/dev/replicated/volume1
Bash script end
Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1
        IncomingUser iqn.1998-01.com.vmware:esx001-3c494e40 password
        IncomingUser iqn.1998-01.com.vmware:esx002-7e5d5341 password
        IncomingUser iqn.1998-01.com.vmware:esx003-76a022ad password
        Lun 0 Path=/dev/replicated/resources,Type=blockio
        Lun 2 Path=/dev/replicated/volume2,Type=blockio
        Alias VMwareStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Target iqn.2011-03.stroageserver.workgroup.net:Microsoft.Storage.1
        IncomingUser MSiSCSIAdmin password
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Alias MicrosoftStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C
1 Like

Thanks you for your assistance, I really do appreciate it. Everything seems to be working flawlessly.

Thanks again.