Remote Copy Script

Hi All,

My first post on the Forums! I've just whipped up a quick script to slightly automate a log file retrieval process for work and to be completely honest my BASH scirpting knownledge is extremely limited as is my understanding of a lot of linux commands.

Basically the scripts purpose is to remove the hassle of having to type in several scp commands to retrieve logs from numerous boxes.

Fair warning the script could be completely laughable but this is why I'm asking for critique. Another massive no no is not having actually tested the script (this is why I'm asking the community to point out any majorily flawed coding).

Thank you in advance!

#!/bin/bash

# This bash script is used for the slightly automated process of exporting log files from the XXXXXXX Servers
# Version: 0.1
# Last Updated: 12/04/2011

ECHO THIS BASH SCRIPT IS TO BE USED FOR THE EXPORTATION OF ALL SERVER LOGS TO A USB
ECHO THE FOLLOWING CONDITIONS MUST HAVE BEEN COMPLETED BEFORE RUNNING THIS SCRIPT....
ECHO 1. YOU MUST BE LOGGED IN ON THE ROOT ACCOUNT OF XXXXX SERVER SERVER01. THIS SCRIPT WILL NOT WORK ON THE OTHER SERVERS!
ECHO 2. THE USB DRIVE MUST BE INSERTED AND MOUNTED DIRECTLY TO /mnt/ WITHOUT A FOLDER NAME PRIOR TO RUNNING THIS SCRIPT
ECHO If the drive is not mounted please type the following; mount -t vfat /dev/sdb1 /mnt/ (kill this script before attempting to run command)
ECHO NOTE: sdb1 may NOT be the correct dev for the USB refer to notes if an error occurs

$ read -p "If the above has been completed press any key....if it hasn't please exit the script"

ECHO Now creating directories on /mnt/

# This first command will first create a current date directory on the USB Drive
mkdir /mnt/`date +%Y%m%d`/

# The following commands will then create all relevant server folders within the current date of this backup process taking place
mkdir /mnt/`date +%Y%m%d`/SERVER01
mkdir /mnt/`date +%Y%m%d`/SERVER02
mkdir /mnt/`date +%Y%m%d`/SERVER03
mkdir /mnt/`date +%Y%m%d`/SERVER04
mkdir /mnt/`date +%Y%m%d`/SERVER05
mkdir /mnt/`date +%Y%m%d`/SERVER06
mkdir /mnt/`date +%Y%m%d`/SERVER07
mkdir /mnt/`date +%Y%m%d`/SERVER08
mkdir /mnt/`date +%Y%m%d`/SERVERR01
mkdir /mnt/`date +%Y%m%d`/SERVERR02
mkdir /mnt/`date +%Y%m%d`/SERVERR03

$ read -p "The directories have been created on the USB, press any key to start the copy process...."

# This first command is only using cp as you are already on the SERVER01 box and don't need to use scp (for remote copy)
cp /var/log/* /mnt/`date +%Y%m%d`/SERVER01

# The following commands will now execute Secure Remote Copy using the root account of all servers. It will prompt for the password for each instance
# We are not using an RSA Public Key and therefore cannot avoid the need to enter in the password for each server instance
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVER02
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVER03
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVER04
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVER05
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVER06
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVER07
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVER08
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVERR01
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVERR02
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/`date +%Y%m%d`/SERVERR03

$ read -p "The copying process is now completed, press any key to unmount the usb drive......"

# The change of directory is to ensure the drive can be unmounted (if the user ran the bash script within the /mnt/ directory itself 
cd $home

# Command to unmount the USB Drive
umount /mnt/

#The USB drive has been unmounted and can be removed
$ read -p "You can now remove the USB key....press any key to exit the script"
exit

Things i'm not sure on;

  • Using the read command purely as a pause process, will it work the way I intend it too?
  • Will the individual scp commands process, prompt for password, process the transfer and then move onto the next scp command?
  • If the script is run from within the USB directory will the cd and unmounting run into problems?

A few things:

  • Your uppercase ECHO won't work. Commands in Unix are case sensitive, and the ECHO command just isn't known.
  • The dollar sign in front of the read command will throw an error. What's the intent behind that?
  • read won't wait for any key, but only continue to run on newline (eg. Enter)
  • Instead of backticks ( `cmd` ) start using $(cmd) , as the former is considered deprecated.
  • A cd / will always succeed, so you don't have to rely on the $home variable being set
  • Instead of repeatedly calling date, why not just call it once, save the result in a variable, and use that from thereon?
  • And of course the script could be made shorter by using loops, but that's more or less optical for such a short script.
  • If you want to do a quick check of the syntax run bash -n script , which will do a syntax check but not run the script. Once it does that, you can run bash -x script to see a trace of what is done
1 Like

Hi pludi,

Thanks for your input, I've made adjustments to the script as suggested. I'm going to learn about looping to see if I can optimise the script a bit more.

The $ behind read was from what I found on a website discussing it's purpose, I did think it was a bit strange.

I'll run the checks now, if I've made a mistake, I'd appreciate it being pointed out!

Cheers,
effektd

#!/bin/bash

# This bash script is used for the slightly automated process of exporting log files from the XXXXXXX Servers
# Version: 0.2
# Last Updated: 13/04/2011

# Date variable
BKDATE=$(date +%Y%m%d)

echo THIS BASH SCRIPT IS TO BE USED FOR THE EXPORTATION OF ALL SERVER LOGS TO A USB
echo THE FOLLOWING CONDITIONS MUST HAVE BEEN COMPLETED BEFORE RUNNING THIS SCRIPT....
echo 1. YOU MUST BE LOGGED IN ON THE ROOT ACCOUNT OF XXXXX SERVER SERVER01. THIS SCRIPT WILL NOT WORK ON THE OTHER SERVERS!
echo 2. THE USB DRIVE MUST BE INSERTED AND MOUNTED DIRECTLY TO /mnt/ WITHOUT A FOLDER NAME PRIOR TO RUNNING THIS SCRIPT
echo If the drive is not mounted please type the following; mount -t vfat /dev/sdb1 /mnt/ (kill this script before attempting to run command)
echo NOTE: sdb1 may NOT be the correct dev for the USB refer to notes if an error occurs

read -p "If the above has been completed press ENTER to continue....if it hasn't please exit the script"

echo Now creating directories on /mnt/

# This first command will first create a current date directory on the USB Drive
mkdir /mnt/$BKDATE/

# The following commands will then create all relevant server folders within the current date of this backup process taking place
mkdir /mnt/$BKDATE/SERVER01
mkdir /mnt/$BKDATE/SERVER02
mkdir /mnt/$BKDATE/SERVER03
mkdir /mnt/$BKDATE/SERVER04
mkdir /mnt/$BKDATE/SERVER05
mkdir /mnt/$BKDATE/SERVER06
mkdir /mnt/$BKDATE/SERVER07
mkdir /mnt/$BKDATE/SERVER08
mkdir /mnt/$BKDATE/SERVERR01
mkdir /mnt/$BKDATE/SERVERR02
mkdir /mnt/$BKDATE/SERVERR03

read -p "The directories have been created on the USB, press ENTER to being the copy process..."

# This first command is only using cp as you are already on the SERVER01 box and don't need to use scp (for remote copy)
cp /var/log/* /mnt/$BKDATE/SERVER01

# The following commands will now execute Secure Remote Copy using the root account of all servers. It will prompt for the password for each instance
# We are not using an RSA Public Key and therefore cannot avoid the need to enter in the password for each server instance
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVER02
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVER03
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVER04
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVER05
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVER06
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVER07
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVER08
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVERR01
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVERR02
scp root@xxx.xxx.xxx.xxx:/var/log/* /mnt/$BKDATE/SERVERR03

read -p "The copying process is now completed, press ENETER to unmount the USB...."

# The change of directory is to ensure the drive can be unmounted (if the user ran the bash script within the /mnt/ directory itself 
cd /

# Command to unmount the USB Drive
umount /mnt/

#The USB drive has been unmounted and can be removed
read -p "You can now remove the USB key....press ENTER to exit the script"
exit

One thing I missed before: you should quote anything that you echo, since the text may contain special characters that the shell might try to interpret (for example the braces on line 14).

An (slightly advanced) example for a loop that you could use:

#!/usr/bin/bash

# Read the script, skip all lines up to #__DATA__, and remove the leading hash
# from the remaining lines
SERVERS=$( sed -e '1,/^#__DATA__/d; s/^#//' $0 )

# Echo the data and feed it into a loop reading each line. The line is split on
# the colon, and the data saved into 2 variables
echo "$SERVERS" | while IFS=':' read name ip
do
    echo "Server \"$name\" has the IP $ip"
done

# The servers are specified here. Since everything is quoted this won't be
# executed

#__DATA__
#SERVER01:1.2.3.4
#SERVER02:5.6.7.8
#SERVER03:9.8.7.6