Replace String without using sed

Hi Team,

I have a file and need to replace string. Out of 20 rows, there is one row given below

$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z

in a Shell Script, I have a variable

HIST_DATE="1900-01-01T00:00:00.000Z"
INC_DATE="2018-10-04T09:18:43.000Z"

Now I need to replace HIST_DATE with INC_DATE.

That mean, the string in a file should get INC_DATE like below

$Paramsoqlfilter=Systemmodstamp > 2018-10-04T09:18:43.000Z

Issue: sed -i not available. I'm suing korn shell

Can someone please help me?

you don't need to have '-i' enabled sed to edit the file.
what's your file look like?
what do you have so far to edit the file so far (with or without sed )?

HIST_DATE="1900-01-01T00:00:00.000Z"
INC_DATE="2018-10-04T09:18:43.000Z"

ex input_file << EDIT
1,$ s/$HIST_DATE/$INC_DATE/g
w!
EDIT

You can also use perl, which should be pre-installed on any *nix disto:

perl -p -i -e 's/$HIST_DATE/$INC_DATE/g' /path/to/file

That's may be easier to commit to memory if you're used to sed -i.

Shell only:

while read -r line
do
  case $line in 
    (*"${HIST_DATE}") 
      line=${line#"${HIST_DATE}"}${INC_DATE}
    esac
  printf "%s\n" "$line"
done < file > file.new

--
bash / ksh93 / zsh:

while read -r line; do
  printf "%s\n" "${line/"${HIST_DATE}"/"${INC_DATE}"}"
done < file > file.new

----

You would need to use double quotes for this to work..

Maybe for the variable, but for straight substitution single quotes work just fine:

$ cat test
OK
$ perl -p -i -e 's/OK/NOTOK/' test
$ cat test
NOTOK

Hi, with sed without -i :
file to modify:

$ cat /tmp/file.tst 
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z

$ HIST_DATE="1900-01-01T00:00:00.000Z"
$ INC_DATE="2018-10-04T09:18:43.000Z"
$ exec 3</tmp/file.tst 
$ rm /tmp/file.tst 
$ sed -e "s/${HIST_DATE}/$INC_DATE/" <&3 >/tmp/file.tst
$ exec 3<&-

$ cat /tmp/file.tst 
$Paramsoqlfilter=Systemmodstamp > 2018-10-04T09:18:43.000Z
$Paramsoqlfilter=Systemmodstamp > 2018-10-04T09:18:43.000Z
$Paramsoqlfilter=Systemmodstamp > 2018-10-04T09:18:43.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 2018-10-04T09:18:43.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z

Regards.

Nice idea, delete and create!
One can also use a { code block } that holds the input file open.
In this case one can even use the stdin (descriptor 0) instead of a descriptor 3.

HIST_DATE="1900-01-01T00:00:00.000Z"
INC_DATE="2018-10-04T09:18:43.000Z"
fn=/tmp/file.tst
{
rm $fn
sed "s/$HIST_DATE/$INC_DATE/" >$fn
} <$fn

Nice trick , but it is not much different from or offers advantages over simply creating a new file and then renaming it to the old file.

sed "s/$HIST_DATE/$INC_DATE/" "${fn}" > "${fn}.tmp" && mv -- "${fn}.tmp" "${fn}"

--
What the trick with the file descriptor does is:

1. point a file descriptor (fd) to the inode that is pointed to by the file
2. remove the filename, but leave the inode (since it is still pointed to by the fd)
3. create a new file (and thus a new inode) using the old filename
4. operate sed reading from the fd (the old file) and writing to the new file
5. close fd and thus remove the old file inode

It is more risky, since if anything goes wrong with writing to the new file, the old file still gets deleted.

under linux, the risk can be reduced as long as the file descriptor is not closed:

$ exec 3</tmp/file.tst
$ rm /tmp/file.tst 
$ cat <&3
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$ file /tmp/file.tst
/tmp/file.tst: cannot open `/tmp/file.tst' (No such file or directory)
$ cat <&3
$ cat /proc/self/fd/3
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$ ls -l /proc/self/fd/3 
lr-x------ 1 disedorgue disedorgue 64 oct.  13 14:04 /proc/self/fd/3 -> /tmp/file.tst (deleted)

Regards.

With "anything goes wrong" Scrutinizer perhaps meant the system crashing, the process being aborted for some kind of error or something such. In this cases you'd end up with a deleted file or a zero-length results file.

In my experience it pays off to deleted the source only once you are satisfied with the target and not one iota sooner.

bakunin

Ok, but sed -i (or perl -i ) has the same problem, no ?

No, they don't - but they have other problems which is why their usage is discouraged anyway:

What sed -i does is in fact opening a second file, write to it and then deleting the original and moving its temporary file where the original one was - BEHIND YOUR BACK!

The "advantage" is manifold: first you have no control over this. If something happens (crash, aborted task, ...) then the temporary file remains and clutters up space. That might happen to you with writing to a target file "manually" too, but in this case you are aware of the fact and might take cleaning actions. In case of sed -i you are probably not aware of this temporary file because you have been lied to by the program about its existence.

Second, even if you are aware of that: where would start searching for it? If i have some code like this:

if sed '<something>' /path/to/input > /some/output
     mv /some/output > /path/to/input
else
     print -u2 "Error executing sed ..."
fi

I have control over where the temporary file goes. And because of that i know where to look at to clean up. Do you know where sed -i puts its temporary file? i don't and the man page is busy telling you lies about the non-existence of the file, so it won't tell you either.

Third, and worst, IMHO: because of the mechanism you are most probably not aware that the files inode changes! This does not only mean the inode number but it may also mean that the ownership and/or the filemode may change (in case of a directory with SUID bits set). If one uses an explicit temporary file one is perhaps aware of this and does (from the example above:

if sed '<something>' /path/to/input > /some/output
     mv /some/output > /path/to/input
     chown someuser:somegroup /path/to/input
     chmod ....
else
     print -u2 "Error executing sed ..."
fi

But with the original file still in place (wink wink) why should one do that?

Finally, and perhaps most importantly: scripts should make as few assumptions about their surroundings as possible. Only GNU-sed understands the -i-option, but POSIX does not. Should i willingly limit my script to systems with a GNU-sed available? I don't think so.

All these reasons are why i strongly suggest not to use the -i option even if the sed in question supports it.

I hope this helps.

bakunin

(GNU) sed -i and perl -i work differently as can be checked with strace .

GNU sed -i creates a new temporary file first and then moves it back to the original file if all is OK
perl -i opens the file, unlinks it and then open a new file with the same name..

So GNU sed -i works in a safer way whereas perl -i does not..

On Linux you can relink the deleted file with ln -L .

HIST_DATE="1900-01-01T00:00:00.000Z"
INC_DATE="2018-10-04T09:18:43.000Z"
fn=/tmp/file.tst
{
if rm $fn
then
  sed "s/$HIST_DATE/$INC_DATE/" >$fn ||
  ln -L /proc/$$/fd/1 $fn
fi
} <$fn