Hi.
Here is a generalization of the perl script I posted earlier. This does not do any sed work. What this expects is text coming in. It saves it in memory and then writes to the file of one's choice. Not very exciting, however, the key idea is that it allows any utility to write in-place. It was inspired by sponge, part of moreutils:moreutils
#!/usr/bin/env bash
# @(#) s1 Demonstrate REAL re-write in place if enough memory, perl.
# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
edges() { local _f _n _l;: ${1?"edges: need file"}; _f=$1;_l=$(wc -l $_f);
head -${_n:=3} $_f ; pe "--- ( $_l: lines total )" ; tail -$_n $_f ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C sed perl absorb-memory
SACRED=${1-sacred}
FILE=data1
cat -n $SACRED > $FILE
pl " Short perl code, absorb-memory $( wc -l < ~/bin/absorb-memory) lines:"
cat ~/bin/absorb-memory | sanitize
pe "# --- end of perl code"
pl " Results sed:"
pl " Data file $FILE before (inode: $( stat -c " %i" $FILE )):"
specimen 3 $FILE
time sed -i '1,2d' $FILE
pl " Data file $FILE after (inode: $( stat -c " %i" $FILE )):"
specimen 3 $FILE
# perl
cat -n $SACRED > $FILE
pl " Results sed (no -i) into absorb-memory (perl):"
pl " Data file $FILE before (inode: $( stat -c " %i" $FILE )):"
specimen 3 $FILE
time sed '1,2d' $FILE | absorb-memory $FILE
pl " Data file $FILE after (inode: $( stat -c " %i" $FILE )):"
specimen 3 $FILE
rm -f $FILE
exit 0
producing:
./s2 /tmp/100-mb.txt
Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution : Debian GNU/Linux 5.0.8 (lenny)
bash GNU bash 3.2.39
sed GNU sed version 4.1.5
perl 5.10.0
absorb-memory - ( local: RepRev 1.7, ~/bin/absorb-memory, 2012-06-17 )
-----
Short perl code, absorb-memory 59 lines:
#!/usr/bin/perl
# @(#) absorb-memory Read STDIN to memory, write to file at EOF, work-alike for sponge.
# $Id: absorb-memory,v 1.7 2012/06/17 19:01:49 drl Exp drl $
## Modification history: when / who / what: most recent at top.
# Relocate to end if grows too long, or re-sequence.
#
# 2012.06.17 / drl / Version that uses memory only.
#
# 2011.07.15 / drl / Do initial test for write permission, abort
# if output file lacks it.
#
# 2010.11.09 / drl / Rename to absorb, avoiding conflict with
# sponge itself.
#
# 2009.04.06 / drl / original.
use warnings;
use strict;
use Carp;
my ($debug);
$debug = 1;
$debug = 0;
# Avoid hang for argument matching "-version","--version", etc.
exit(0) if @ARGV && $ARGV[0] =~ /-version/;
$/ = 0777;
my ( $file, $f, $memory );
if ( !$ARGV[0] ) { $ARGV[0] = "-"; }
$file = shift;
# Preliminary basic tests on output file.
if ( $file ne "-" ) {
if ( not -f $file ) {
croak("not a plain file, $file");
}
}
$memory = do { local $/; <> };
my ($len) = length($memory);
print STDERR " Length of file in memory variable: $len\n" if $debug;
if ( $file eq "-" ) {
open( $f, ">-" ) || die " Cannot open STDOUT for writing.\n";
}
else {
open( $f, ">", $file ) || die " Cannot open file \"$file\" for write.\n";
}
print " debug write - file is :$file:\n" if $debug;
print $f "$memory";
END { close(STDOUT) || die "can't close stdout: $!" }
exit(0);
# --- end of perl code
-----
Results sed:
-----
Data file data1 before (inode: 334060):
Edges: 3:0:3 of 1777700 lines in file "data1"
1 Preliminary Matter.
2
3 This text of Melville's Moby-Dick is based on the Hendricks House edition.
---
1777698 THEY GLIDED BY AS IF WITH PADLOCKS ON THEIR MOUTHS; THE SAVAGE SEA-HAWKS SAILE
1777699 D WITH SHEATHED BEAKS. +ON THE SECOND DAY, A SAIL DREW NEAR, NEARER, AND PIC
1777700 KED ME UP AT LAST. +IT WAS THE DEVIOUS-CRUISING +RACHEL, THAT IN HER RETRACIN
real 0m13.351s
user 0m1.384s
sys 0m11.189s
-----
Data file data1 after (inode: 334061):
Edges: 3:0:3 of 1777698 lines in file "data1"
3 This text of Melville's Moby-Dick is based on the Hendricks House edition.
4 It was prepared by Professor Eugene F. Irey at the University of Colorado.
5 Any subsequent copies of this data must include this notice
---
1777698 THEY GLIDED BY AS IF WITH PADLOCKS ON THEIR MOUTHS; THE SAVAGE SEA-HAWKS SAILE
1777699 D WITH SHEATHED BEAKS. +ON THE SECOND DAY, A SAIL DREW NEAR, NEARER, AND PIC
1777700 KED ME UP AT LAST. +IT WAS THE DEVIOUS-CRUISING +RACHEL, THAT IN HER RETRACIN
-----
Results sed (no -i) into absorb-memory (perl):
-----
Data file data1 before (inode: 334061):
Edges: 3:0:3 of 1777700 lines in file "data1"
1 Preliminary Matter.
2
3 This text of Melville's Moby-Dick is based on the Hendricks House edition.
---
1777698 THEY GLIDED BY AS IF WITH PADLOCKS ON THEIR MOUTHS; THE SAVAGE SEA-HAWKS SAILE
1777699 D WITH SHEATHED BEAKS. +ON THE SECOND DAY, A SAIL DREW NEAR, NEARER, AND PIC
1777700 KED ME UP AT LAST. +IT WAS THE DEVIOUS-CRUISING +RACHEL, THAT IN HER RETRACIN
real 0m2.890s
user 0m1.212s
sys 0m1.516s
-----
Data file data1 after (inode: 334061):
Edges: 3:0:3 of 1777698 lines in file "data1"
3 This text of Melville's Moby-Dick is based on the Hendricks House edition.
4 It was prepared by Professor Eugene F. Irey at the University of Colorado.
5 Any subsequent copies of this data must include this notice
---
1777698 THEY GLIDED BY AS IF WITH PADLOCKS ON THEIR MOUTHS; THE SAVAGE SEA-HAWKS SAILE
1777699 D WITH SHEATHED BEAKS. +ON THE SECOND DAY, A SAIL DREW NEAR, NEARER, AND PIC
1777700 KED ME UP AT LAST. +IT WAS THE DEVIOUS-CRUISING +RACHEL, THAT IN HER RETRACIN
This is a busy output, but there are some items of interest. First, one can use essentially the same sed command (omitting the "-i") as for a sed that knows about "-i". Second, the times are noticeably better for the in-memory version. Third, note that the inodes are different in the case of sed, proving that a temporary file was used, and was renamed as the main input file. For the absorb-memory case, the inode is that same.
The same perl code can make any standard utility(STDIN, STDOUT compatible) into a utility that can do in-place processing.
Best wishes ... cheers, drl