While loop hangs in function running in background

Shell Programming and Scripting
1

Hello Everyone,

I am writing a shell script to fetch log files from remote servers within a time range. It copies log files to local server, grep for date and then compares the time stamp of each log entry with the once specified.
Below is the code.

# Get log and Parsing function

GetAndParse() {

    echo -e "\nWorking on server $host"

    # Looping through the log paths given in the argument.
    for logpath in $LOG_PATH
    do
        echo "Fetching log files from $logpath"

        # Looping through the log file names provided in the arguement.
		for logfile in $FILE_NAMES
        do
            # Finding the log file size on remote server
			logfilesize=$(ssh -n $host du -skLc $logpath/$logfile 2> /dev/null|awk '{print $1}'|tail -1)

			# Comapring the log file size with limit LOG_FILE_SIZE	            
			if [ ! -z $logfilesize ] && [ $logfilesize -le $LOG_FILE_SIZE ] && [ $logfilesize -gt 0 ]
			then
				# If the log file size is under limit copt it to local WORK_DIR
				scp $host:$logpath/$logfile $WORK_DIR/temp
				LOCAL_TEMP_FILE="$WORK_DIR"/temp/"$host"_"$logfile"_"$RANDOM".temp
				# Grep for dates in log entries
				cat $WORK_DIR/temp/$logfile| grep -E  "$START_TIME|$END_TIME" > $LOCAL_TEMP_FILE && rm $WORK_DIR/temp/$logfile
	

				while read line
				do
					line_date1=$(echo $line|grep -o "[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]")
					line_date=$(echo $line_date1|tr -s " "|cut -d " " -f1,2)
					UNIX_DATE=$(date -d"$line_date" +%s)

					if [ $UNIX_DATE -gt $START_UNIX_DATE -a $UNIX_DATE -lt $END_UNIX_DATE ]
					then
						echo "$host $line" >> $OUTPUT_UNSORTED_LOG
					fi

				done < $LOCAL_TEMP_FILE

			else
				echo -e "\e[0;33mNo file with name $logfile below $LOG_FILE_SIZE KB found under $logpath on $host\e[0m"
			fi

		done
        
    done

}

I call the above function as shown below

for host in ${ARR_HOSTS[@]}
do
    GetAndParse &
done

When i run this, the while loop which is reading lines one by one in log file is not terminating once it reaches EOF.

I have tried running it in foreground (removing '&' while calling function) which works perfect.

I want to run it in background because there are 20 servers i need to ssh, scp and read log files which are more than 10 MB each.

Please suggest me if there are any changes needed in this code.

2

I have a function with multiple while loops that fetch files from remote servers. I have paths in $paths variable and file names in $logs variable (Space separated).

Function () {
while read path from $paths
do
     while read log from  $logs
     do
           ssh -n $host du -skLc $logpath/$logfile   # get the size of remote file
           scp $host:$logpath/$logfile $localfile       # Download remote file

           while read line
           do
               grep $string $line > output file    # Grep for string in each line
           done < $localfile
    done
done

}

When i call this function as below it works fine

for host in $hosts
do
Function 
done

When i try to run it in background as below the inner most while loop which is reading log entries line by line does not exit after finishing the file.

for host $hosts
do
Function &
done

Please help me to fix this.