Replace line with sed on OS X

Shell Programming and Scripting
1

So, I want to, in a script, do an in-line replacement of a line in a file. Piece of cake in Linux with GNU sed. With OSX and BSD sed, not so much. I've been Googling, but nothing is really helping... either an answer is for a use case different enough from what I want to do, or it's something like inserting a newline on the command line, and I need this in a script.

I'm not stuck on sed, but that really should be the right tool to use. I do want to use a tool that comes natively with OS X, so installing GNU sed is a last resort. I'm on Lion, but would like one solution that'll work for 10.5 - 10.8

For simplicity's sake, there's a file with a few lines, one of which starts with "flags". I want to just replace that line.

2

Try perl:

# cat file
hello
flags this line starts with flags
world
#
# perl -i.bak -pe 's/^flags.*/new line/' file
#
# cat file
hello
new line
world
#
3

Inserting a line is rewriting the file from that point, so just sed to a temp file and then cat the temp file content over the original. You can also drive ex, vi's foundation, from a script very like sed but with additional commands.

echo '2c
new line
.
w
q' | ex your_file

Really, to actually rewrite just from line 2, not the whole file, you might need a C/PERL routine that overwrites existing positions within the file, extending or shortening it at the end. I suspect most tools truncate and write data from temp files when you save. After all, they are not tolerant of other processes mmap()ing the file or seeking around and reading lower locations that might disappear momentarily. I will tusc a vi and see. Yep, vi close()s, opens with creat() and writes it all. So, back to "use sed with a temp file." If you dislike temp files and the file is smaller than max env size (a meg or so), You can store the file in the env:

z=`sed '...' your_file`
echo "$z" >your_file

I wrote a C tool once that overwrote $1 from stdin without truncating and trimmed the file length at the end if shorter using fcntl(), so I guess PERL can do that, too. But generally, update in place is a fantasy.

4

Thanks! I gotta learn me some perl...

5

Hi.

The in-place modification of files usually involves writing a scratch-temporary file and copying the scratch file back over the original (or renaming the file). As noted, not all versions of sed, etc., have this capability builtin.

One general solution is the addition of command sponge, a component of package moreutils. So for system OS, ker|rel, machine: FreeBSD, 8.0-RELEASE, i386 :

% mag show moreutils
moreutils is a growing collection of the unix tools that nobody thought to
write thirty years ago.

So far, it includes the following utilities:

 - sponge: soak up standard input and write to a file
 - ifne: run a program if the standard input is not empty
 - vidir: edit a directory in your text editor
 - vipe: insert a text editor into a pipe
 - ts: timestamp standard input
 - combine: combine the lines in two files using boolean operations
 - pee: tee standard input to pipes
 - zrun: automatically uncompress arguments to command
 - mispipe: pipe two commands, returning the exit status of the first
 - isutf8: check if a file or standard input is utf-8
 - lckdo: execute a program with a lock held
 - parallel: run multiple jobs at once

The 'ifdata' utility is not included in this FreeBSD port.

(The utility mag is a local generic repository scanner, so that we need not recall the details of apt-cache, yum, rpm, zypper, ports, etc.)

See thread Delete first 100 lines from a BIG File for a discussion of in-place editing and a few examples of a sponge-like perl utility.

Best wishes ... cheers, drl

6

If I understand GNU sed's -i option correctly, the following ex commands on OS X should be roughly equivalent to the corresponding sed commands:

# ex equivalent of sed -i '/flags/s/.*/new $text/' file
ex -s file <<-"EOF"
        /^flags/s/.*/new $text/
        w
        q
EOF

# ex equivalent of sed -i.bak '/flags/s/.*/new $text/' file
ex -s file <<-"EOF"
        w %.bak
        /^flags/s/.*/new $text/
        w
        q
EOF

text=something
# ex equivalent of sed -i.bak "/flags/s/.*/new $text/" file
ex -s file <<-EOF
        w %.bak
        /^flags/s/.*/new $text/
        w
        q
EOF
7

I usually just install a personal copy of gnu sed, namng it gsed. The old sed's had line length limits and EOF handling bugs that are aggravating, but often they were faster as the buffer was not so indirect.

8

Try:

sed -i.bak 's/^flags.*/new line/' file

--

OSX uses a more modern version of BSD sed that does not have the limitations of the seds of older Unices...

9

So, should any characters between the last new-line and EOF be the last line? hp 11.00 sed:

$ echo '123\c'|sed
$
10

BSD sed (OS X 10.8):

$ printf "123" | sed
123
$
11

So, closer but no -i for (probably pretend) inline editing.

I use ex on a pipe mostly for "diff -e" and "diff3 -e" output. It's cute little cheat is to do things from EOF back, so line numbers of work yet to be done (lower) never change!