Help needed with Content synchronization script using rsync

Shell Programming and Scripting
1

Hi All,

I need a .sh script which serves the below purpose. It would be of great help.
I am unable to perform the second need of mapping texts within the file and updating only those contents.
For Eg: There is a html file containing link abc.com. This should be replaced with xyz.com link in the destination folder.

##########################################
1)content sync from source to target machine, based on a extensive list of directories and hostnam/ip

2)search and replace of the mapped texts from source to target

3)replacement should be only in changed/updated files and to a configurable set of file extensions
#######################################

Below is my script workthrough as of now:

#############################################
#!/bin/sh
#!/usr/bin/perl

##################################################################################
#
#
# This script is used to sync the contents from source directory of local #machine
# to destination directory of target machine.
# 
##################################################################################

################################################################################
# Logging directory
################################################################################
LOGDIR="Path to logging directory"
################################################################################
# RSync binary location
################################################################################
rsync="/opt/mw/rsync-2.6.6/bin/rsync"

################################################################################
# RSync binary location on remote server
# this is only used when source and target directories are on different servers
# otherwise it should be same rsync value
################################################################################
rsync_path="/opt/mw/rsync-2.6.6/bin/rsync"

################################################################################
# For Content mapping below mentioned directory paths and hostnames are required
################################################################################
SOURCEDIR="Path to Source Directory on Local Machine"
TARGETDIR="Path to Target Directory on Remote Machine"
BACKUPDIR="Path to Backup Directory"
TARGETSERVER="Hostname"
TargetUser="Username"
LOGFILE="Path to Log file"
Rsync_Options = "--recursive --times --archive --verbose --update --links --checksum --progress --delete"
#-v, --verbose increase verbosity 
#-a, --archive archive mode
#-r, --recursive recurse into directories 
#-u, --update skip files that are newer on the receiver 
#-t, --times preserve modification times 
#-l, --links copy symlinks as symlinks
#-c, --checksum skip based on checksum, not mod-time & size 
#--dry-run
#rsync --recursive --times --archive --verbose --update --links --checksum --progress --delete


################################################################################
# RSync filters
# file/folders mentioned in the filter will be skipped for synchronization
# for more details on rsync filter read rsync documentation
################################################################################
rsync_filter="- .backup/"
#################################################################################
# Check if the log file exists
################################################################################
if [ ! -e $LOGFILE ]; then
     touch $LOGFILE
fi
if [ ! -d $TARGETDIR ]; then
     echo "Log destination directory doesn't seem to exist. Please investigate"
     exit 2
fi

echo $'\n\n' >> $LOGFILE

#####################################################################
#
# Gather a list of the files from source directory and the remote directory.
# It finds the file differences and copy the files to the destination directory.
# Only missing files are copied.
#
#####################################################################

#for i in `find $SOURCEDIR -type f`
#do
rsync $Rsync_Options -e ssh $SOURCEDIR/ $TargetUser@$TARGETSERVER :$TARGETDIR/ 
#done
echo "Sync Completed at:`/bin/date`" >> $LOGFILE
2

If you have some disk space available you could:

1- do a rsync to a temp dir.
2- Change all your files/links
3- rsync the temp dir to target server

Benefits would be that you don't mod the source files at the price of performance (since you do two rsync instead of one)

3

Hi Maverick,

Thanks for the reply. As i am not having much disk space, i want to include any command within the script which does a mapping of text.

4

Well you do not have much options then.

Best thing would be to do your rsync and change the files on the remote server:

  • Rsync on remote server
  • Find/edit/replace your files

You still don't touch your source files and you do not need double disk space on the other end since you will be editing the files directly.

Since you should probably have ssh keys on source and remove servers you could triger your script to ssh <command> to edit your files.

What are the conditions/changes/results your are targeting at the end?