I am trying to automate a process of searching through a set of files and replace all occurrences of a formatted text with the next item in the list of a second file. Basically i need to replace all instances of T????CLK???? with an IP address from a list in a second file. the second file is one IP per line with no special formatting other then xxx.xxx.xxx.xxx
Thanks in advance Everyone!
How do you know which ip address in the second file goes with a given T???CLK???? pattern in file1?
The IP that gets inserted does not matter as long as each one is used only once. If its read out of order that would not matter to me.
Hope below can help you.
Below perl demo is to replace this with a array file by file. If reach out of the array, then stop replace.
@arr=(This,THis,THIS);
$n=0;
opendir(DIRECTORY, '.') or die "Can't open current directory.";
@files=readdir(DIRECTORY);
closedir DIRECTORY;
for($i=2;$i<$#files;$i++){
print "Begin change file:",$files[$i],"\n";
open FH,"<$files[$i]";
while(<FH>){
if($arr[$n]){
$_=~s/this/$arr[$n]/;
}
print $_;
}
close(FH);
print "End change file:",$files[$i],"\n\n";
$n++;
}
below awk demo is to replace this with value in another replace file file by file, if reach out of the end of replace file, will use the last one to replace the rest of file.
n=0
for i in *
do
temp=""
n=`expr $n + 1`
temp=`cat ../replace | head -"$n" |tail -1`
cat $i | nawk -v t="$temp" '{
gsub(/this/,t,$0)
print $0
}'
done
As a stylistic comment, you might want to avoid the Useless Use of Cat and Head | Tail.
n=0
for i in *
do
temp=""
n=`expr $n + 1`
temp=`sed -n "${n}p" ../replace`
nawk -v t="$temp" '{
gsub(/this/,t,$0)
print $0
}' "$i" >tmp
# Don't replace file if no change
cmp "$i" tmp >/dev/null && continue
mv "$i" "$i"~
mv tmp "$i"
done
rm -f tmp
I also added the storage of the results in a temporary file, and backing up of the original.
Thanks guys, I am still having an issue. OK a couple:
1) using the win32 versions of sed/awk do iI need to switch the / to \
2) I have never used awk before and don't understand is that a script to throw at awk or just a UNIX shell script?
sorry for the stupid questions, gotta learn somehow....
Thanks guys, I am still having an issue. OK a couple:
1) using the win32 versions of sed/awk do iI need to switch the / to \
2) I have never used awk before and don't understand is that a script to throw at awk or just a UNIX shell script?
sorry for the stupid questions, gotta learn somehow....
The slashes are part of awk syntax and not related to the directory separator character of your platform. So no, no need to change them to backslashes.
In the sed script, the file name containing the tags is supposed to be in ..\replace so yes, there, you'd perhaps change the directory separator (although I understand DOS can cope with a forward slash as well).
awk 'script in awk syntax here' file1 file2 file3 ... is how you typically invoke an awk script. On DOS you might need to use different quote marks, or simply put the script in a file and run it from there (with awk -f scriptfile.awk assuming your version of awk supports that).
The above script with sed and awk in it is a shell script, so you need /bin/sh or bash or something like that to execute it.
OK, i am REALLY lost now (I use SED for basic search/replace and AWK is a creature:
temp=\`sed -n "$\{n\}p" ../replace\`
I change the ../replace with the name of the files with the IPs in it?
Lets say the files are named dns.txt and ips.txt
dns.txt = the file with the T????CLK????
ips.txt = the file with the pool of IPs to use as the substitutes
script.awk has the awk command from above in it so i would type "awk -f script.awk dns.txt ips.txt" at the command prompt?
Sorry, no. It's still a shell script. You would do something like
n=0
for i in dns.txt
do
temp=""
# Grab the next line out of ips.txt
n=`expr $n + 1`
temp=`sed -n "${n}p" ips.txt`
# Replace all occurrences in current file with value $temp
nawk -v t="$temp" script.awk "$i" >tmp
# Don't replace file if no change
cmp "$i" tmp >/dev/null && continue
mv "$i" "$i"~
mv tmp "$i"
done
rm -f tmp
... and script.awk would contain the nawk script.
This is probably not directly useful, but shows where you would plug in the parts you had questions about.
The original script wanted to change all files in the current directory (that's * where now we have only dns.txt), and globally replace all occurrences of "this" in each file with a different value out of ../replace (now ips.txt), in a different directory because you were making substitutions in all files in the current directory (basically I imagine on the assumption that there is only one occurrence of "this" -- or actually T????CLK??? -- in each file, but a lot of files).
But what you seem to want is probably closer to
nawk 'NR == FNR { r[++i] = $0 ; next }
{ gsub (/T...CLK.../, r[++j]); print }' ips.txt dns.txt >dns.new
If there are multiple occurrences of T????CLK??? on a single line, they will both get the same value out of ips.txt -- if this is a problem, a slightly more complex script will be necessary. Also if there are more lines with matches in dns.txt than there are values in ips.txt the last ones in dns.txt will be replaced with an empty value. This is obviously not very newbie-friendly, but as long as you keep these constraints in mind, it's simple and understandable.
The output is stored in dns.new; you can move it to dns.txt when you're done. The earlier script has code for checking whether any substitutions took place, and only replacing the old file if the new file is different.
(Line wraps in the awk scripts are not significant; you can type it all on a single line if you like.)