SED 4.1.4 - INI File Change Problem in Variables= in Specific [Sections] (Guru Help)

Shell Programming and Scripting
1

GNU sed version 4.1.4 on Windows XP SP3 from GnuWin32

I think that I've come across a seemingly simple text file change problem on a INI formatted file that I can't do with SED without side effects edge cases biting me. I've tried to think of various ways of doing this elegantly and quickly with SED but it seems to me that I'm going to have to try with AWK or finally learn PERL and move to it.

However, before I take these drastic measures let me present the problem here to the SED gurus.

The problem basically stems from the fact that INI format text files do not have a closing tag for the end of the [section], the start of the next section or end of file is the end of the section. This prevents me from using the AWK address range "/ADDR1/,/ADDR2/" feature for limiting the edits to only the section because of the annoying side effect that is the /ADDR2/ range is "inclusive" as per the SED info - 3.2 Selecting lines with `sed' page.

This /ADDR2/ inclusiveness causes problems since once it hits for the next section start then /ADDR1/ doesn't hit again so code for that address range is skipped entirely for the next section causing the SED script to hop-scotch across the INI file skipping every other match eligible section.

Here is an actual sample INI file to work on as an example of the format in case someone is unfamiliar with it. The idea is to change the values in the [supplies_res_*] sections without touching the [supplies_generic] or [trade_generic_sell] sections.

INI File


[supplies_generic]
ammo_12x70_buck            = 3,    1
ammo_12x76_zhekan        = 3,    0.5

ammo_9x18_fmj            = 2,    1


[supplies_res_3]:supplies_generic
ammo_12x76_zhekan        = 3, 1

wpn_ak74            = 1, 0.3



[supplies_res_6]:supplies_res_3
ammo_12x76_dart            = 3, 0.5

medkit_army            = 5, 1


[trade_generic_sell]
ammo_9x18_fmj            = 1.5, 1.5
ammo_9x19_fmj            = 1.5, 1.5

ammo_11.43x23_fmj        = 1.5, 1.5

ammo_12x70_buck            = 1.5, 1.5
ammo_12x76_zhekan        = 1.5, 1.5

ammo_5.45x39_fmj        = 1.5, 1.5

ammo_5.56x45_ss190        = 1.5, 1.5

grenade_rgd5            = 1.5, 1.5

The code below shows the problem with trying to create the address range using the beginning of the next section as the end range, since the inclusiveness of /ADDR2/ address makes the code skip any changes in that section.

SED Code Example - Broken = Hop-Scotch Section Skipping

# Section Address Range
/^[[:space:]]*\[[[:space:]]*%_section%[[:space:]]*\]/I , /^[[:space:]]*\[/

If I try other /ADDR2/ ranges such as "$" for end of file then all sections that contain the same variable will be changed to the end of the file, not something that I want. I cannot use blank line /^[[:space:]]*$/ for ADDR2 since the sections have blank spaces and lines inside them.

If I try to get really clever and do something like below then it doesn't work either no matter what I try to do or think of.

SED Code Example - Broken = Manual Line Reading Doesn't Work (Late Night Idea)

# INI Section
/^[[:space:]]*\[[[:space:]]*%_section%[[:space:]]*\]/I {

	# Read Line
	: read;
	n;

	# Next Section Jump Out
	/^[[:space:]]*\[[[:space:]]*%_section%[[:space:]]*\]/I! {
		/^[[:space:]]*\[/ b;
	};

	# Variable Line
	/^[[:space:]]*%_variable%[[:space:]]*=/I {

		# Value Replace
		# Note: Retain same whitespace after replacement except for value.
		s/^([[:space:]]*%_variable%[[:space:]]*=[[:space:]]*)(.*)[[:space:]]*$/\1%_value%/I;

	};

	# Loop
	b read;

};
2

Can you explain what you're trying to replace and post the expected output within code tags?

3

@OP, sed can definitely operate and edit a file, however, in your case, its not a suitable tool. You should, if possible, try to use available packages for parsing ini files with languages like Python or Perl. It will makes your sysadmin life easier. An example with Python

#!/usr/bin/env python
import ConfigParser
config = ConfigParser.ConfigParser()
config.read("file")
for section in config.sections():    
    print "SEction: [%s]" % section
    for options in config.options(section):            
        print "%s = %s" % (options, config.get(section, options))

output

# ./test.py
SEction: [trade_generic_sell]
grenade_rgd5 = 1.5, 1.5;x;xxx
ammo_9x19_fmj = 1.5, 1.5
ammo_9x18_fmj = 1.5, 1.5
ammo_11.43x23_fmj = 1.5, 1.5
ammo_12x76_zhekan = 1.5, 1.5
ammo_5.45x39_fmj = 1.5, 1.5
ammo_5.56x45_ss190 = 1.5, 1.5
ammo_12x70_buck = 1.5, 1.5
SEction: [supplies_res_3]
ammo_12x76_zhekan = 3, 1
wpn_ak74 = 1, 0.3
SEction: [supplies_generic]
ammo_12x76_zhekan = 3,    0.5
ammo_12x70_buck = 3,    1
ammo_9x18_fmj = 2,    1
SEction: [supplies_res_6]
medkit_army = 5, 1
ammo_12x76_dart = 3, 0.5

the above just prints out the values of the ini files, however, to change any value is just trivial.
anyway, its your call whether to use other tools besides sed or not.

4

Below is an example. Only change the values for the variables inside the supplies_res_* section without touching anything else.

INI Before Change

[supplies_generic]
ammo_12x76_zhekan        = 3,    0.5

[supplies_res_3]:supplies_generic
ammo_12x76_zhekan        = 3, 1

[supplies_res_6]:supplies_res_3
ammo_12x76_zhekan        = 3,    0.5

[trade_generic_sell]
ammo_12x76_zhekan        = 1.5, 1.5

INI After Change

[supplies_generic]
ammo_12x76_zhekan        = 3,    0.5

[supplies_res_3]:supplies_generic
ammo_12x76_zhekan        = 100, 1.0

[supplies_res_6]:supplies_res_3
ammo_12x76_zhekan        = 100, 1.0

[trade_generic_sell]
ammo_12x76_zhekan        = 1.5, 1.5

Sounds simple, like something that you can do with SED address ranges but it seems really hard or impossible.

You are correct that another scripting tool would make it easier and that using a dedicated INI file parser library would be the best solution. I think that this is the way to go at this point because I can't find a simple solution.

I chose SED because that is the tool that I know how to use and I thought it would be easy to do this with a single line of code for SED.

SED Script - One Line INI Variable Change in Section (Broken!, skips every other section when consecutive sections match)

/^\[%section%\]/I, /^\[/ s/^(%variable%=)(.*)$/\1%value%/I;

Whitespace Robust

/^[[:space:]]*\[[[:space:]]*%section%[[:space:]]*\]/I, /^[[:space:]]*\[/ s/^([[:space:]]*%variable%[[:space:]]*=[[:space:]]*)(.*)[[:space:]]*$/\1%value%/I;

SED Negative RegEx Modifier ! - Not The Solution

SED has a unary NOT "!" modifier available but it only applies to the entire address, singleton and range, and not for individual RegExp in the address. This modifier cannot be used for this solution because the match for ADDR2 has to be a positive match for the start of the next section but once you get a positive match then this line becomes inclusive in the address.

SED NonInclusive ADDR2 Modifier Required - New Feature Request

The real solution would be SED to have an option or a modifier available for the ADDR2 so that when it matches that line it becomes non-inclusive. This way address ranges for SED could be used to parse files that have a structure with markings only delimiting the beginning of sections but not the ending of sections.

If you think about it most of the human designed data formats for ad-hoc purposes such as documents or simple data files only delineate the beginning of a new section but never add any markings for the ending since they treat the beginning of a new section as that marking.

I think that this improvement to SED would be a worth while endeavor to include in the program. The problem is that I am not a programmer and I don't know how SED handles the stream input so I don't know how difficult it would be for ADDR2 to be made non-inclusive.

It could be simple with the internal line counter being made to not increment on ADDR2 match so that the next cycle of the script hits ADDR2 line again. But this could create a problem with stream processing or file editing since it is customary for SED to only hit each line once and not repeat it.

However, I always thought that the inclusiveness of ADDR2 was a weakness in the address range design since this is not the first time that I butted my head against this wall when trying to do seemingly simple file changes with SED only to realize that what I want to do just cannot be done, even if I go into SED Guru Mode and start doing branches with b, t, T.

After I hit this realization I usually boink myself and try to go next level up to AWK while silently cursing in my breath for never learning PERL in the first place to move all my file and line editing in it.

5

You can try this with awk. Suppose you want to change the value of the section [supplies_res_6] of ammo_12x76_zhekan with 100, 1.0:

awk -F":| " -v var="6" -v v1="100, 1.0" -v opt="ammo_12x76_zhekan" '
$1 == "[supplies_res_"var"]" && !f {f=1}
f && $1 == opt{sub("=.*","= "v1);f=0}
1' file
6

Thanks for the code but I won't be using it or needing it since I found the solution for SED. Your messy code reminded me of something that I did with SED a few years back when I had a similar problem. Your usage of the hard to understand cryptic "f=0" and "f=1" variable reminded me that. Thanks for the reminder though.

SED Solution Remembered and Found! - Hold Space As Variable

The idea is to use the hold space as a boolean variable to hold the line of the section that you are currently inside and then when you find the field in the file that you want to change, you check if you are in the proper section before making any changes to the field.

I used this with a previous problem and I just realized that I had the solution below. This works great and the code is below.

# Section Find and Save
/^[[:space:]]*\[/ h;

# Variable Line
/^[[:space:]]*%variable%[[:space:]]*=/I {

	# Section Recall
	x;

	# Section Check
	/^[[:space:]]*\[[[:space:]]*%section%[[:space:]]*\]/I {

		# Line Recall
		x;

		# Value Replace
		# Note: Retain same whitespace after replacement except for value.
		s/^([[:space:]]*%variable%[[:space:]]*=[[:space:]]*)(.*)[[:space:]]*$/\1%value%/I;

		# Section Recall
		x;
	};

	# Line Recall
	x;

};