I'm quite new to unix and perl scripting and need to write a script to delete files older than 7 days from a remote ftp server. Unix or Perl script would do...
I wrote the following code:
#!/usr/local/bin/perl
use File::Basename;
use Net::FTP;
#use Net::FTP::File;
my $verbose = 0;
my $proxy = @ARGV[0];
my $host = @ARGV[1];
my $ftpuser = @ARGV[2];
my $ftppass = @ARGV[3];
my $remoteDir = @ARGV[4];
my $filetoDelete = @ARGV[5];
my $ftp=Net::FTP->new($host,
Debug => 0,
Passive => 0,
Hash => ($verbose ? \*STDOUT : undef),
FirewallType => 1,
Firewall => $proxy,
) or exit(1);
$ftp->login($ftpuser,$ftppass) or exit(2);
chomp($remoteDir);
if ( $remoteDir ne "/" )
{
$ftp->cwd($remoteDir) or exit(3);
print $remoteDir;
}
@files=$ftp->ls or exit(4);
foreach(@files) {
print "$_ \n";
$ftp->delete($_) ;
}
$ftp->quit;
exit(0);
I'm not able to find a way to identify files of pattern aaa* that are older than 7 days. Unix and Perl gurus, please help...
Even I am very new to scripting however i tried this in my local network and its working.
Iam not sure whether it will work with your requirement but it will give you an idea.
One of our application writes logs to the system as shown below:
bash-2.03$ pwd
/var/log/Application
bash-2.03$
bash-2.03$ ls -lrt Application*
-rw-r--r-- 1 root other 34752530 Jan 8 23:59 Application.log.Thu
-rw-r--r-- 1 root other 5179052 Jan 9 23:59 Application.log.Fri
-rw-r--r-- 1 root other 8420980 Jan 10 23:59 Application.log.Sat
-rw-r--r-- 1 root other 2993389 Jan 11 23:59 Application.log.Sun
-rw-r--r-- 1 root other 13932907 Jan 12 23:59 Application.log.Mon
-rw-r--r-- 1 root other 15685534 Jan 13 23:59 Application.log.Tue
-rw-r--r-- 1 root other 360387803 Jan 14 23:59 Application.log.Wed
bash-2.03$
Now in order to zip these files or delete it i set the following script as cron job at every 00:00 hrs.
#!/usr/bin/sh
INSTALL_DIR=/var/log/Application
Curr_Dir=`pwd`
cd $INSTALL_DIR
i=`ls |grep "\.log.[a-zA-Z]\{3,3\}$ |wc -l` //here iam grepping any char and minimum of 3 char as Mon, Tue etc..
if [ 0 -ne $i]
then
rm $INSTALL_DIR/*.log.???
#gzip -f9 $INSTALL_DIR/*.log.???
fi
Thanks for the reply, imas. This won't work for me as I'm trying to delete old files from a ftp server. ftp doesn't support grep and find command that's why I tried to write a PERL script for the same... But seems like PERL ftp also has these limitations...
Oh haha, I forgot to tell you.. that yeah perl does have a find command.
Perl is a great language (core), but mostly you have to use modules for a large amount of functionality.
Below are two links (even the servers I work on have these, so you should have them) to the Find::* type modules. The second one is pretty damn helpful.
Thanks for the reply, Rhije! But again I'm not sure if this would work for me... I need to do the following:
Connect to a remote FTP server.
Find all the files with names aaa* older than 7 days on the remote server.
Delete these old files.
Earlier when I tried some other FILE features, what it does is gives me list of files on my local machine... I wanted to know if there is a way to get the list of old files from the remote ftp server...
You can't tell the FTP server to only list the old files, no. You could tell it to list a directory and filter out any new files. The exact information ftp returns for a directory listing is implementation-dependent.
I wondered if you have had any success with this?
I am facing the exact same requirement, where I need to delete old log files from a remote server via FTP as I only have FTP access to the server in question.
In my case, we had a daily job that would download new files from the remote ftp server and then place it in an archive directory in local machine after processing those files. Since identifying old files from a remote ftp server was quite tricky, I wrote a script that deleted the old files from ftp server using the following logic:
Created a list of all the old files by going to the archive directory on my local machine.
Used this list to delete the old files from the remote ftp server.
remoteFtpServerCleanUp.ksh: Goes through the local archive directory and creates a list of all the old files that need to be deleted. Then it internally calls 'remoteFtpServerCleanUp.pl' for eac file that needs to be deleted.
remoteFtpServerCleanUp.pl: Connects to the remote ftp server and deletes the requested file.
You can customize the scripts as per your requirements.
USAGE:
remoteFtpServerCleanUp.ksh -f myremoteserver.net -u myuser -p mypswd -l /data/mylocaldirectory -m "myfilepattern*.txt" -r 2
-f Remote FTP server name
-u Remote FTP server user name
-p Remote FTP server password
-l Local archive directory
-m File pattern that needs to be deleted
-r Retention period
remoteFtpServerCleanUp.ksh:
#!/bin/ksh
#
####################################################################################
# #
# Script Name : remoteFtpServerCleanUp.ksh #
# Date : 26-Jan-2009 #
# Developer : Arun Kumar #
# Description : #
# #
####################################################################################
# Initialize all the variables
. /datapool/mlbdba/mlbdba_crontab_env
# Defined constant values
typeset UTIL_FILE_DIR="${HOME}/files/utilities"
typeset TEMP_DIR="${HOME}/remoteFtpServerCleanUp"
typeset PROXY_SERVER="my-proxy-vip.abc.com"
typeset DB_MAIL_GROUP="arun.kumar@abc.com"
typeset LOG_FILE=""
PGM_NAME=`basename $0`
BASENAME=`echo $PGM_NAME|cut -f1 -d "."`
LOG_FILE=${TEMP_DIR}/${BASENAME}.`date '+%m%d%Y'`.log.$$
OLD_LOG_FILE=${LOG_FILE}
>>${LOG_FILE}
USAGE="remoteFtpServerCleanUp.ksh -f <Remote FTP Server> -u <FTP User> -p <FTP password> -d [Remote FTP directory] -l <Local directory> -m <Local file match pattern> -r <Retention period in days>"
#Variable declaration
typeset -i exitStatus=0
typeset remoteFtpServer=""
typeset ftpUserNm=""
typeset ftpPassword=""
typeset remoteFtpDirectory=""
typeset localDirectory=""
typeset fileMatchPattern=""
typeset -i retentionPeriod=0
typeset -i cutOffPeriod=30
#********************************************************#
# Main program starts here
#********************************************************#
echo "Starting ${PGM_NAME}..." >> ${LOG_FILE}
#********************************************************#
# Initial Level Validations start here
#********************************************************#
echo "Starting initial level validations..." >> ${LOG_FILE}
# Check to make sure all the 5 mandatory options/arguments
# are passed in the command-line
if [[ $# -lt 5 ]]; then
echo "Incorrect number of arguments passed in the command-line."
echo "USAGE : $USAGE"
echo "Incorrect number of arguments passed in the command-line." >> ${LOG_FILE}
echo "USAGE : $USAGE" >> ${LOG_FILE}
exit 1;
fi
# Read the option argument values from the commandline.
while getopts :f:u:p:d:l:p:m:r: optVal
do case "$optVal" in
f) remoteFtpServer="$OPTARG";;
d) remoteFtpDirectory="$OPTARG";;
p) ftpPassword="$OPTARG";;
u) ftpUserNm="$OPTARG";;
l) localDirectory="$OPTARG";;
m) fileMatchPattern="$OPTARG";;
r) retentionPeriod="$OPTARG";;
esac
done
#Verify if the remote ftp server name was passed in the command-line
if [[ -z ${remoteFtpServer} ]];
then
echo "Remote FTP server name was not provided in command-line."
echo "USAGE : $USAGE"
echo "Exiting process"
echo "Remote FTP server name was not provided in command-line." >> ${LOG_FILE}
echo "USAGE : $USAGE" >> ${LOG_FILE}
echo "Exiting process" >> ${LOG_FILE}
exit 1;
fi
BASE_DIR=${TEMP_DIR}/${remoteFtpServer}
if [[ ! -d ${BASE_DIR} ]]; then
mkdir -p ${BASE_DIR}
fi
export LOG_FILE=${BASE_DIR}/${BASENAME}.`date '+%m%d%Y'`.log.$$
cat ${OLD_LOG_FILE} >> ${LOG_FILE}
rm -f ${OLD_LOG_FILE}
#Verify if the ftp user name was passed in the command-line
if [[ -z ${ftpUserNm} ]];
then
echo "FTP user name was not provided in command-line."
echo "USAGE : $USAGE"
echo "Exiting process"
echo "FTP user name was not provided in command-line." >> ${LOG_FILE}
echo "USAGE : $USAGE" >> ${LOG_FILE}
echo "Exiting process" >> ${LOG_FILE}
exit 1;
fi
#Verify if the ftp password was passed in the command-line
if [[ -z ${ftpPassword} ]];
then
echo "FTP password was not provided in command-line."
echo "USAGE : $USAGE"
echo "Exiting process"
echo "FTP password was not provided in command-line." >> ${LOG_FILE}
echo "USAGE : $USAGE" >> ${LOG_FILE}
echo "Exiting process" >> ${LOG_FILE}
exit 1;
fi
#Verify if the database name was passed in the command-line
if [[ -z ${remoteFtpDirectory} ]];
then
echo "FTP server directory name was not provided in command-line."
echo "Default FTP server directory name to /"
remoteFtpDirectory="/"
fi
#Verify if the ftp password was passed in the command-line
if [[ -z ${localDirectory} ]];
then
echo "Local directory was not provided in command-line."
echo "USAGE : $USAGE"
echo "Exiting process"
echo "Local directory was not provided in command-line." >> ${LOG_FILE}
echo "USAGE : $USAGE" >> ${LOG_FILE}
echo "Exiting process" >> ${LOG_FILE}
exit 1;
fi
#Verify if the ftp password was passed in the command-line
if [[ -z ${fileMatchPattern} ]];
then
echo "File matching pattern was not provided in command-line."
echo "USAGE : $USAGE"
echo "Exiting process"
echo "File matching pattern was not provided in command-line." >> ${LOG_FILE}
echo "USAGE : $USAGE" >> ${LOG_FILE}
echo "Exiting process" >> ${LOG_FILE}
exit 1;
fi
#Verify if the ftp password was passed in the command-line
if [[ -z ${retentionPeriod} || ${retentionPeriod} -le 0 ]];
then
echo "Valid Retention period was not provided in command-line."
echo "USAGE : $USAGE"
echo "Exiting process"
echo "Valid Retention period was not provided in command-line." >> ${LOG_FILE}
echo "USAGE : $USAGE" >> ${LOG_FILE}
echo "Exiting process" >> ${LOG_FILE}
exit 1;
fi
echo "Finished initial level validations..." >> ${LOG_FILE}
echo "remoteFtpServer : ${remoteFtpServer}" >> ${LOG_FILE}
echo "ftpUserNm : ${ftpUserNm}" >> ${LOG_FILE}
#echo "ftpPassword : ${ftpPassword}" >> ${LOG_FILE}
echo "remoteFtpDirectory : ${remoteFtpDirectory}" >> ${LOG_FILE}
echo "localDirectory : ${localDirectory}" >> ${LOG_FILE}
echo "fileMatchPattern : ${fileMatchPattern}" >> ${LOG_FILE}
echo "retentionPeriod : ${retentionPeriod}" >> ${LOG_FILE}
#********************************************************#
# Initial Level Validations end here
#********************************************************#
oldFileNameList=${BASENAME}.`date '+%m%d%Y'`.lst.$$
# Create a list of all the sql
echo "find ${localDirectory} -name '${fileMatchPattern}' -type f -mtime +${retentionPeriod} -mtime -${cutOffPeriod} | grep -v '${localDirectory}/.*/' > ${BASE_DIR}/${oldFileNameList}.tmp" >> ${LOG_FILE}
eval "find ${localDirectory} -name '${fileMatchPattern}' -type f -mtime +${retentionPeriod} -mtime -${cutOffPeriod} | grep -v '${localDirectory}/.*/' > ${BASE_DIR}/${oldFileNameList}.tmp"
exitStatus=$?
if (( ${exitStatus} != 0 )); then
echo "No files to be removed...." >> ${LOG_FILE}
echo "Exiting process" >> ${LOG_FILE}
rm -f ${BASE_DIR}/${oldFileNameList}.tmp
exit 0;
fi
# Create a new list file having only the filename and stripping the absolute path
# based on the temporary list file created in the step above
>${oldFileNameList}
while read myAbsFileName
do
basename ${myAbsFileName} >> ${BASE_DIR}/${oldFileNameList}
done < ${BASE_DIR}/${oldFileNameList}.tmp
cd ${BASE_DIR}
# Call the perl script to connect to the remote ftp server and delete the old files
# from the ftp server based on the list of files created above.
#echo "remoteFtpServerCleanUp.pl ${PROXY_SERVER} ${remoteFtpServer} ${ftpUserNm} ${ftpPassword} ${remoteFtpDirectory} ${oldFileNameList}" >> ${LOG_FILE}
${UTIL_FILE_DIR}/remoteFtpServerCleanUp.pl ${PROXY_SERVER} ${remoteFtpServer} ${ftpUserNm} ${ftpPassword} ${remoteFtpDirectory} ${oldFileNameList} >> ${LOG_FILE}
exitStatus=$?
case ${exitStatus} in
1)
echo "************************************************" >> ${LOG_FILE}
echo "Cannot connect to ${remoteFtpServer}..." >> ${LOG_FILE}
echo "Please check ${remoteFtpServer} availability!" >> ${LOG_FILE}
exit 1;;
2)
echo "************************************************" >> ${LOG_FILE}
echo "Cannot login to ${remoteFtpServer}..." >> ${LOG_FILE}
echo "Please check the ftp user and password info." >> ${LOG_FILE}
exit 1;;
3)
echo "************************************************" >> ${LOG_FILE}
echo "Cannot access ${remoteFtpDirectory} on server ${remoteFtpServer}..." >> ${LOG_FILE}
echo "Please check the ftp location name and permissions." >> ${LOG_FILE}
exit 1;;
*)
echo "Old files successfully deleted from ${remoteFtpServer}." >> ${LOG_FILE};;
esac
rm -f ${BASE_DIR}/${oldFileNameList}.tmp ${BASE_DIR}/${oldFileNameList}
echo "${PGM_NAME} finished processing..."
echo "${PGM_NAME} finished processing..." >> ${LOG_FILE}
#********************************************************#
# Main program ends here
#********************************************************#
exit 0
remoteFtpServerCleanUp.pl:
#!/usr/local/bin/perl
use File::Basename;
use Net::FTP;
my $verbose = 0;
my $proxy = @ARGV[0];
my $host = @ARGV[1];
my $ftpuser = @ARGV[2];
my $ftppass = @ARGV[3];
my $remoteDir = @ARGV[4];
my $lstOfFiletoDelete = @ARGV[5];
open (MYFILE, $lstOfFiletoDelete);
my $ftp=Net::FTP->new($host,
Debug => 0,
Passive => 0,
Hash => ($verbose ? \*STDOUT : undef),
FirewallType => 1,
Firewall => $proxy,
) or exit(1);
$ftp->login($ftpuser,$ftppass) or exit(2);
chomp($remoteDir);
if ( $remoteDir ne "/" )
{
$ftp->cwd($remoteDir) or exit(3);
print $remoteDir;
}
print "Starting deletion of files from server $host ... \n";
open (MYFILE, $lstOfFiletoDelete);
while (<MYFILE>)
{
chomp;
print "Deleting file: $_\n";
$ftp->delete($_) or print "$_ not deleted or not found... \n";
}
close (MYFILE);
$ftp->quit;
exit(0);
As per your requirement . Probably you can try the below code
{
echo "user userid Passwd" ( to provide the crendentials to login)
echo "cd locationofdirectorywherethe file recides"
echo "rm filname"
echo "bye"
} > pwd/filename (where pwd is present working directory
{
ftp -n "ipaddress" ( ip address of which you want to login)
sleep 3
} < pwd/filename
