sftp user@hostname >filelist <<EOF
cd /home/data/
ls -l "ABC*"
quit
Add a couple of lines here to get rid of sftp> prompts in filelist
while read a b c d e f g h file
do
grep $file Files_Receievd_ABC.txt >/dev/null
if [ $? != 0 ] ; then
echo get /home/data/$file >> list_of_new_files_to_get
fi
done < filelist
echo quit >>list_of_new_files_to_get
#list_of_new_files_to_get >> Files_Receievd_ABC.txt
I would move this line to after the new files are received.
which is now giving the output as
sftp> cd /home/data
sftp> ls -l ABC*
-rw-r--r-- 0 13106 7745 0 Mar 29 07:34 ABC29073411
-rw-r--r-- 0 13106 7745 0 Mar 29 07:34 ABC29073412
-rw-r--r-- 0 13106 7745 0 Mar 29 07:34 ABC29073413
-rw-r--r-- 0 13106 7745 0 Mar 29 07:34 ABC29073415
sftp> exit
To get ris of first 2 and last line, I tried using all the below script but not a single record is getting copied to new file.
I tried executing the below scripts on unix prompt and they are working file but these script are not working inside the shell script.
#
1) /usr/bin/sed 's/sftp//g' filelist > filelist1
2) sed '1,2d' filelist > filelist1 and sed '$d' filelist >> filelist1
3) sed '/sftp/d' filelist > filelist1
4) grep -v 'sftp' filelist > filelist1
#
The filelist1 file does not contain any record.
please advise.
---------- Post updated at 05:09 PM ---------- Previous update was at 03:39 PM ----------
I think I found the issue.
The first sftp command is closing the complete script due to which the rest of the script is not executing.
I have created another script just to connect the host server and calling the new script from first script.
Hope this will work.
---------- Post updated at 05:36 PM ---------- Previous update was at 05:09 PM ----------
Done.
Finally I am able to create the script which will fetch all the latest files from remote server.
Thanks you very much all for your support and inputs.
*********************
# To copy latest files which were created in the last 15 min
# from upstream System Server to local Server
## changing the directory
cd /home/data/
## Removing all the the existing Files received from Upstream System
rm -rf ABC*
## Creating Empty 'list_of_new_files_to_get' File
echo '' > list_of_new_files_to_get
## Removing the confirmation File.
rm -rf ABC_READY.txt
## Creating batch to get the files from upstream System
echo "cd /data/cmv/GMM" > getFilelist
echo "ls -l ABC*" >> getFilelist
echo "quit" >> getFilelist
## Calling other shell script to get all the file list
getCompleteFileList.sh
## Removing String 'Sftp' from the filelist
sed '/sftp/d' filelist > filelist1
## comparing the new list with existing available list.
while read a b c d e f g h file
do
grep $file Files_Receievd_ABC.txt>/dev/null
if [ $? != 0 ] ; then
echo get /data/cmv/GMM/$file >> list_of_new_files_to_get
fi
done < filelist1
echo quit >> list_of_new_files_to_get
## Adding newly received files to the existing list
cat list_of_new_files_to_get >> Files_Receievd_ABC.txt
## Removing string 'quit' from the complete list and copying the data to Temp file.
sed '/quit/d' Files_Receievd_ABC.txt > Files_Receievd_ABC_TEMP.txt
## renaming the temp file to existing File.
mv Files_Receievd_ABC_TEMP.txt Files_Receievd_ABC.txt
## get all the new files from upstream System
sftp -b list_of_new_files_to_get username@hostname
## creating confirmation File.
touch ABC_READY.txt
**************************
Code of getCompleteFileList.sh
#!/bin/ksh
# To get the file list from upstrea Server
# The scrip will be called from ABC_Sync.sh script
sftp -b /home/data/getFilelist userName@hostname >/home/data/filelist <<EOF
************************
