Bash script to take cPanel backup in batches

Shell Programming and Scripting
1

I'm trying to resolve the below scenario by writing bash script.

On my managed linux server I have about 30 accounts and there is not enough space to generate full 30 accounts backup in one go and transfer it via SFTP to my Backup Synology Server. What I'm thinking of doing is breaking the backup into batches and transfer it over accordingly to prevent server choking:

Steps:

The logic that I'm trying to use is:

#Outer Loop to process mainList array elements
for i in "${mainList[@]}"
do
#Inner Loop to to process cPList array elements
for j in "${i[@]}" **

<<--I need help with this part, I was expecting the j variable to be accounts1, acocunts2 ...and so because it is supposed to be the elements of i variable. However, when I echo both i and j they hold same values i'e cPList1, cPList2 ....

SFTP 5 accounts backup to Synology
delete transferred files
done
done

Any input will be appreciated.

2

This should work in newer bash and ksh.
Additional error handling should be done against utilities used against accounts.

#!/usr/bin/ksh
begin=0
i=0
nump=5 # number of accounts per processing, increase or decrease

# plain text file with account1 to account39, each in new line is created named 'accs.txt'
# for test input, fill array 'accounts' with results

while read accs 
do
	accounts[$i]=$accs
	i=$(($i+1))
done < accs.txt

# number of iterations required for processing, we divide total number of elems in array with $nump
ITT="$(( ${#accounts[@]} / $nump ))"

for (( f=$begin; f<=$ITT; f++ ))
do
	# If utilities used support multiple accounts input, a for loop is not neccesary.
	# echo "${accounts[@]:$begin:$nump}"
	# this takes care of reminder as well
	for account in "${accounts[@]:$begin:$nump}"
	do
		echo "work (backup, sftp, cleanup) against for $account"	
	done
	begin=$(($begin+$nump))
	sleep 1 # we sleep here 1 second after 5 accounts have been processed, you might not.
done

Hope that helps
Regards
Peasant.

1 Like
3

Simplified: the outer loop increments by 5

#!/bin/bash
# bash or ksh93
nump=5 # number of accounts per processing, increase or decrease

# plain text file with account1 to account39, each in new line is created named 'accs.txt'
# for test input, fill array 'accounts' with results
na=0
while read accs 
do
  accounts[na]=$accs
  na=$((na+1))
done < accs.txt

for (( begin=0; begin<na; begin+=nump ))
do
  # If utilities used support multiple accounts input, a for loop is not neccesary.
  # this takes care of reminder as well
  for account in "${accounts[@]:begin:nump}"
  do
    echo "work on $account"
  done
  echo "group action for ${accounts[@]:begin:nump}"
  sleep 1 # we sleep here 1 second after 5 accounts have been processed, you might not.
done

Now a "classic" variant without an array

#!/bin/sh
# any Posix sh
nump=5 # number of accounts per processing, increase or decrease

# the final group action is after the loop, so we put it in a function
group_action(){
  echo "group action on
$accounts"
  sleep 1
}

# plain text file with account1 to account39, each in new line is created named 'accs.txt'
i=0 na=0 accounts=""
while read account 
do
  accounts=$accounts${accounts:+"
"}$account
  echo "work on $account"
  if [ $((i+=1)) -eq $nump ]
  then
    group_action
    accounts=""
    i=0
  fi
  na=$((na+1))
done < accs.txt
[ $i -ne 0 ] && group_action
4

@peasant,

Thank you for the solution, worked smoothly.

---------- Post updated 06-08-18 at 09:15 AM ---------- Previous update was 06-07-18 at 02:23 PM ----------

Also thanks to 2 other replies.