Script to automate recovery process

Shell Programming and Scripting
1

Hello All!
First post...

I am working on a script that is used to recover a crashed drive from an rsync backup.

I'm down to the place where I need to create all of the directories in /mnt where I will then mount each of the volumes and begin the restore process to each volume...

I have scratched out my basic thoughts about what I need to happen, but am at a loss as to HOW to do it.

Any help would be greatly appreciated!!

Roger
***************************************************

/mnt/USBDrive/lvminfo/fstab_info was saved during the backup process...

vgname=$(grep ^/dev /mnt/USBDrive/lvminfo/fstab_info | cut -d'/' -f4 | cut -d'-' -f1 | sed '2,99d')

This results in vgname=fedora

read f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 <<< $(grep $vgname /mnt/USBDrive/lvminfo/fstab_info | cut -d'-' -f2 | cut -d'/' -f1 | cut -d" " -f1 | sort -d -t" " | awk '{printf("%s%s",$0,NR%9?" ":"\n")}')

This results in a sorted string in which each word is assigned to a variable:

 echo $f1 $f2 $f3 $f4 $f5 $f6 $f7 $f8 $f9
backup home opt root swap tmp usr usr_local var

So, from that I need:

for ($f1-$f9)
    mkdir /mnt/$vgname/($f1-$f9)
    mount /dev/mapper/$vgname-($f1-$f9)
    increment to next varname ($f1-$f9)
repeat until all filled variables have created required directories and mounted lvms
exit

Thank you again for any assistance!! :slight_smile:

---------- Post updated at 03:50 PM ---------- Previous update was at 02:40 PM ----------

For those interested, this is the script so far...
Any elegance suggestions are greatly appreciated! :slight_smile:

#!/bin/bash
#
# This scipt is to be run from a LiveCD environment in a root terminal
#
clear
echo
echo
echo "Mounting USB Hard Drive to mount point /mnt/USBDrive..."
# We'll create a mount point and mount the main directory structure of our backup drive to this point.
mkdir -p /mnt/USBDrive
mount -o defaults /dev/sdb2 /mnt/USBDrive
echo "Done!"
echo 
# We are going to need the physical volume name and UUID along with the Volume Group name later in this process
# We also want to determine the starting and ending sector (along with the partition number) of our luks partition
echo "Setting a few variables for later..."
pvname=$(grep Name /mnt/USBDrive/lvminfo/pvparse | cut -d'/' -f4)
pvuuid=$(grep UUID /mnt/USBDrive/lvminfo/pvparse | cut -b 25-62)
vgname=$(grep ^/dev /mnt/USBDrive/lvminfo/fstab_info | cut -d'/' -f4 | cut -d'-' -f1 | sed '2,99d')
startsec=$(grep ^/dev /mnt/USBDrive/info/fdisk_info | grep G | cut -d" " -f3)
endsect=$(grep ^/dev /mnt/USBDrive/info/fdisk_info | grep G | cut -d" " -f4)
partnum=$(grep $startsec /mnt/USBDrive/info/sgdisk_info | cut -b4)
# The following line initializes a series of variables from a string of alphabetically sorted lvnames created from fstab_info (with modification you can initialize as many variables as you desire -- just add more fX to the list) Here I know that I only have 9 lvm's so I chose to add only 9 f variables.
read f1 f2 f3 f4 f5 f6 f7 f8 f9 <<< $(grep $vgname /mnt/USBDrive/lvminfo/fstab_info | cut -d'-' -f2 | cut -d'/' -f1 | cut -d" " -f1 | sort -d -t" " | awk '{printf("%s%s",$0,NR%9?" ":"\n")}')
# Once we have gotten the volume group name from fstab we can create a mount point for the restore process 
mkdir -p /mnt/$vgname
echo
# We start the recovery process by writing our saved critical disk areas to the new drive
# This will give us a mountable drive with the partition table, EFI and boot partitions as they were on the original drive
echo "Restoring the first 2048 sectors of /dev/sda..."
dd if=/mnt/USBDrive/dd/partition-restore of=/dev/sdc bs=1 count=2047
echo
echo "Restoring the EFI partition as /dev/sda1..."
dd if=/mnt/USBDrive/dd/sda1EFI-restore of=/dev/sdc1 bs=8192
echo
echo "Restoring the Boot partition as /dev/sda2..."
dd if=/mnt/USBDrive/dd/sda2Boot-restore of=/dev/sdc2 bs=8192
echo
echo "Prepping to restore the luks partition, and VG/LVM structures..."
echo
# Just to ensure that there isn't any old luks data on our new drive that will present errors during the restortion of our saved luks partition header image, we need to wipe out any signs of any previous lvm info that might be there
echo "First we will ensure that no old crypt/lvm info remains on our recovery disk"
cryptsetup -v isLuks /dev/sdc$partnum
head -c 3145728 /dev/zero > /dev/sdc$partnum; sync
# Next, we will recreate the partition from scratch and format it to ext4
echo "Then we will create a new /dev/sda3 and restore our backup img"
sgdisk -d $partnum -n $partnum:$startsec:$endsec -t $partnum:8300 /dev/sdc
# We need to inform the kernel about the changes
partprobe
# And format the partition we just created
mkfs -t ext4 /dev/sdc$partnum
echo
echo "Restoring Cryptsetup header and key slots..."
cryptsetup -q luksHeaderRestore /dev/sdc$partnum --header-backup-file /mnt/USBDrive/lvminfo/cryptbackup.img
echo
echo "Opening crypt partition for writing..."
cryptsetup -vv open /dev/sdc$partnum --type luks $pvname
echo
echo "Restoring Physical Volume..."
pvcreate -f --uuid $pvuuid --restorefile /mnt/USBDrive/lvminfo/$vgname /dev/mapper/$pvname
echo
echo "Restoring & Checking Volume Group and Logical Volumes structures..."
vgcfgrestore $vgname
lvs -a -o +devices
vgchange -ay $vgname
pvck -vv
vgck -vv
mkfs -t ext4 /dev/mapper/$vgname-*
mkswap /dev/mapper/$vgname-swap
echo

---------- Post updated at 03:53 PM ---------- Previous update was at 03:50 PM ----------

All of the /dev/sdX's above are set to another external USB test drive -- wouldn't want to slam my /dev/sda! :slight_smile:

2 
for fs in $f{1..12}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs}
done

However, I would consider using an array instead of f1 .. f12 variables as you are not limited to fixed number of filesystems:

read -a farray <<<< $(grep $vgname ....)

for fs in ${farray[@]}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs}
done

And if you don't need access to these values again you avoid any array or variables all together simply like this:

for fs in $(grep $vgname ... )
do
    mkdir /mnt/$vgname/$fs
    ...
done
1 Like
3

Thank you very much for your reply.
I have been reading so much information in the last couple of days that when I got to doing a for loop, by brain just shut off! I feel embarrassed that it was such an "easy" piece of script!

The last piece of this will be to run rsync against the backups to the newly mounted VG/LVM's.

Thank you very much!!

Roger

---------- Post updated at 05:42 PM ---------- Previous update was at 05:19 PM ----------

Sorry to bother you again, but I forgot that I need a list of exclusions when it comes to restoring the root...

Is there a way to exclude root in the first section and then do just root in the second part?
Also, can we "variable-ize" the rsync line in the second part???

Thank you very much in advance!!
Roger

echo "Creating Mount Points, mounting individual LVM's from Volume Group and restoring data..."
for fs in ${farray[@]}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs} /mnt/$vgname/$fs
    rsync -aHvu /mnt/USBDrive/Recover/$fs /mnt/$vgname/$fs
done

# Restoring root requires all of the other lvm's as excludes...
# rsync -aHvu --exclude={/backup/*,/home/*,/opt/*,/tmp/*,/usr/*,/usr/local/*,/var/*}

---------- Post updated at 08:13 PM ---------- Previous update was at 05:42 PM ----------

Ok. I may have come up with a workaround...

By not sorting the list prior to reading it into the array, root will always be ${farry[0]}. Therefore, I can start my mkdirs and mounts with ${farry[1]} and then come back and do ${farry[0]} at the end.

How do we create and increment a variable to read in everything from 1 through an empty offset and perform our actions using their values in the commands?

---------- Post updated at 08:25 PM ---------- Previous update was at 08:13 PM ----------

Answer:

for fs in ${farray[@]:1}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs} /mnt/$vgname/$fs
    rsync -aHvu /mnt/USBDrive/Recover/$fs /mnt/$vgname/$fs
done

After that we can come back with

    mkdir /mnt/$vgname/${farray[0]}
    mount /dev/mapper/${vgname}-${farray[0]} /mnt/$vgname/${farray[0]}
    rsync -aHvu --exclude={/backup/*,/home/*,/opt/*,/tmp/*,/usr/*,/usr/local/*,/var/*} /mnt/USBDrive/Recover/${farry[0]} /mnt/$vgname/${farry[0]}

Last step:
To "auto-variable-ize" the rsync exclusions for / restoration...

4

UPDATE:
As it turns out, I am able to get the results I want by using this code:

for fs in ${farry[@]}
do
    mkdir -p /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-$fs /mnt/$vgname/$fs
    rsync -aHvu /mnt/USBDrive/Recover/$fs /mnt/$vgname/$fs
done
rsync -aHvu --ignore-existing /mnt/USBDrive/Recover/ /mnt/$vgname/

However, I discovered that since my drive geometry is different between the two drives (cylinder values are not the same) a simple dd of the first 2047 sectors does not effectively copy the MBR, GPT and Partition info correctly. So, I need to find a way to recreate the drive from scratch (using sgdisk?), format and copy over the EFI and Boot partition files.