Copying large files in a bash script stops execution

Shell Programming and Scripting
1

Hello,

I'm new to this forum and like to first of all say hello to everyone.
I've got a really annoying problem at the moment.
I'm trying to rsync some files (about 200MB with one file of 120MB) from a Raspberry PI with raspbian to a debian server via rsync.
This procedure is stored in a bash-script. I'm generating some output before and after the rsync-command to see and check if everything went fine.
Now, when copying large files, rsync completes the task, gives out information about what and in which size was copied, but the shell script stops. The following calls to functions are not executed.
If I test the script with many small files, it works well and the script executes all commands and quits as expected.
First I thought this is a problem with rsync, but copying (via cp) the same files to a smb-share stops exactly at the same point.

What I found out so far (some ideas come from my PHP-Experience):

  1. Could it be related to some sort of timeout? Is there a timeout for a called function?
  2. When I watch the output of rsync, I see that after a large file is copied rsync sort of hangs for several seconds before it transmits the next file. When using small files this "lag" does not show up.
  3. Does it have something to do with the performance of the raspberry?
  4. Either using rsync over ssh or not (simply copying to smb) has the same effect.
  5. Also changing filenames (deleting whitespaces and special characters) had no effect.

Perhaps you have a clue?

Thank you very much

   wex_storm
2

Without seeing your shell script, it is difficult to give any useful advise on your issue. If you take the SMB share out of the picture and rsync or cp to a regular ext2/3/4 filesystem does your script work?

3

Hi.

My guess -- there is some computing done at both ends of the rsync transaction. From: rsync - Wikipedia, the free encyclopedia:

So the longer the file, the more time it takes to do the checksums.

Best wishes ... cheers, drl

4

Hi, first of all thank you for your answers.

Here is a shortened part of my script, where it "hangs" (after the rsync line, if big files are copied)

echo "Mounting SAMBA-Share..."
if [ "${SMB_OPTION}" = "" ];
then
    sudo mount -t cifs -o username="${SMB_USER}",password="${SMB_PASS}",rw,noserverino,nounix ${SMB_SHARE} ${SMB_MOUNT_POINT}
else
    sudo mount -t cifs -o ${SMB_OPTION} ${SMB_SHARE} ${SMB_MOUNT_POINT}
fi
echo "mounted"
gpio write 4 1
echo "Copying files to SAMBA"
mkdir ${SMB_MOUNT_POINT}${TIME_STAMP}
rsync -av --exclude="*/" --exclude=".*" --progress ${MOUNT_POINT}* ${SMB_MOUNT_POINT}${TIME_STAMP}
echo "Copying done"
echo "Unmounting SAMBA-Share..."
sudo umount -l ${SMB_MOUNT_POINT}
echo "unmounted"
gpio write 4 0

I just let the script run 20 minutes - it just does nothing after the lines:
sent 248566882 bytes received 829 bytes 2080064.53 bytes/sec
total size is 248533848 speedup is 1.00

Normally it echoes "Copying done" and so on. I'll just let it work for some time and let you know, if anything changed.

I tried copying the data to an internal folder (on the SD-Card) with absolutely the same result. Script hangs.

Appreciate your help.

wex_storm

---------- Post updated 02-03-14 at 02:04 AM ---------- Previous update was 02-02-14 at 02:33 PM ----------

I forgot to mention that the script is started by an udev-rule.

Thanks