I suspect what I'm trying to do isn't possible, but I'm hoping someone can either confirm this or point me in the right direction.
We have a third-party application which transfers a collection of files to our SFTP server ( Ubuntu 12.04 with OpenSSH ) . Once the app disconnects, we want to trigger the running of a script to organise the files into relevant subfolders. The server is only for this application so the script can be run on every disconnect.
From what I've seen, sftp doesn't seem to have any hooks to external sources. I've looked into using inotify but I don't think this will suffice. The number of files transferred and their size will change continuously so combining inotify-on-fileclose with a timeout would be unreliable.
So my question is: Is this possible at all or am I barking up the wrong tree?
Is that directory entirely dedicated to that specific application for it s dowload?
( Understand when you have moved the files found to relevant files, its can be considered as empty... and no one else but that application writes here...)
Is the deposit RANDOM or you have a littel idea of the schedule
The directory is dedicated to the sftp uploads. The script moves the files into relevant folders and then the folder is empty. There are no other apps that write to this directory.
The deposit is controlled by a user. They press a deploy button and the content is dumped into our server. so, logically we know roughly when it should be, but the system will not. The system is also aimed at untrained/partially-trained personnel and the fewer steps to go through, the easier it is to use.
If you have control of the sending function, send two files. First the data file and secondly a trigger file. Have the cron job only orocess data files for which there is a corresponding trigger file.
Yep, I think that's possibly my best option (or do it with cron/manually). We don't have control of the app but we are in communication with the developers so they may be able to accommodate.
You might want to watch the sftp-server process - once it disappears from the ps listing, the script can be run. Or you could look into /var/log/auth.log :
Apr 22 16:21:46 RFreeBSD sshd[1195]: subsystem request for sftp by user RUDIC
Apr 22 16:23:27 RFreeBSD sshd[1195]: Received disconnect from 10.1.1.1: 11: disconnected by user
Excellent. I've just tested ps -aux when the client is connected/disconnected and as you say, the process appears and disappears. The next question is how can I trigger a script off of that? The watch command is ringing bells but the documentation suggests it won't do what I think.
Good points. But - that wasn't meant as a solution but as a direction to think into. Unfortunately, unlike ftp , sftp is a handled as a subsystem of the sshd daemon. I'm not sure if it could be modified (e.g. replaced by a wrapper script) to detect/handle connects and disconnects.
I have used variations of this code for over 10 years.
put abcdefgh.ltr /u2/email/temp
put abcdefgh.mmo /u2/email/temp
put abcdefgh.job /u2/email/requests
#!/bin/ksh
BATCH=$1
proc=$$
if [ -r /u2/email/$BATCH.pid ]
then
echo Previous job still running `cat /u2/email/$BATCH.pid`
exit
fi
echo $proc >/u2/email/$BATCH.pid
echo $proc `date +"%Y%m%d %H%M%S"` >/u2/email/$BATCH.pidhistory
cd /u2/email
list=`ls requests|grep $BATCH`
if [ "$list" != "" ]
then
for request in $list
do
echo $request `date`
IFS="~"
cat requests/$request | read user email subject memo attachments
IFS=" "
cd /u2/email/temp
#do all the processing here
cd /u2/email
mv /u2/email/temp/$req* done
mv /u2/email/requests/$request done
if [ ! -r /u2/email/$BATCH.pid ]
then
echo Job terminated manually `cat /u2/email/$BATCH.pid`
exit
fi
done
fi
rm /u2/email/$BATCH.pid
We use 8.3 file names.
The file name extension for files in the request directory is used to create different queues: job for regular processing, ovn for overnight, rsh for rush. There is a separate cron job for each extension.
The process can be stopped by manually removing the pid file.
If a previous pid file exists, this does not necessarily mean that the previous invocation failed. It could be that there were more jobs to process than could be processed in the interval between invocations.