[php] ftp get all files from a certain file type

Hi all,

I googled for this kind of function but didn't find anything. I have an FTP connection with a server, went to the dir. In this dir there are several TXT files. I would like to have function that downloads all this files, based on their .txt extension.

In bash, for example, simply: mget *.txt ...

Thanks in advance!

[solved] to slow... !

$list = ftp_nlist($conn_id, '.');

function is_txt($file) {
    return preg_match('/.*\.txt/', $file) > 0;
}

$filtered = array_filter($list, is_txt);

print_r($filtered);

ftp_close($conn_id);

Hi,

Another question, I need to have a certain functionality that jumps and combines data from 2 txt files. Earlier I had the same problem in bash:

line 1 = IP-address
line 2 = FTP-user
line 3 = FTP-pwd
line 4 = IP-address
line 5 = FTP-user
... etc ...
line 33 = FTP-user
line 34 = FTP-pwd
line 35 = IP-address
line 36 = FTP-user
line 37 = FTP-pwd
#-END

or the real file (little bit modified, for security reasons)

80.xx.xx.76              -> IP server
ftpmac28                 -> username
passO#lanL              -> password
212.xx.xx.52             -> IP server
ftpmac28                 -> username
passO#lanL              -> password
212.xx.xx.61
ftpmac28
passO#lanL
212.xx.xx.62
ftpmac28
passO#lanL
212.xx.xx.17
ftpmac28
passO#lanL
212.xx.xx.10
ftpmac28
passO#lanL
212.xx.xx.75
ftpmac28
passO#lanL

...etc...

In the same folder there are some other files named *IP*_folderlist. Where the first part is the IP address like xxx.xxx.xxx.xxx. These files contain the folders
The content of this files looks like this:

docshop.xx.be\files
extranetdocshop.xx.be\files
fleet.xx.be\doccshop\files

So what has to happen: a loop has to run through the ftpservers.txt and check if there is an IP_folderlist.txt in the currect dir, for the checked IP. If so it has to get them and put them in an array. After that, it should go to the 2nd line which is the $user, the 3rd line is the $passwd. If there is no IP_folderlist.txt, it has to skip to the next IP address line in the ftpserver.txt file, and check again, etc...

Thanks in advance!

No one? :confused:

you are already had a solution in a shell script that franklin helped you with. so why don't you show some code and show us where you are stuck?

This is the bash script I made with the kind help of Franklin, indeed.

set -xv
ftp -n -v immserv1.impactweb.imm.be << EOT
ascii
user ftpimpactstore VeneziA0211
prompt
cd BACKUP
#mget *.txt
ls
EOT
awk '{ sub("\r$", ""); print }' ftpservers.txt > unixfile.txt
awk '
NR%3==1{a[++i]=$0}
NR%3==2{u=$0}
NR%3==0{p=$0}
END{for(j=1;j<=i;j++){print a[j], u[j], p[j], a[j] "_folderlist.txt"}}' unixfile.txt |
while read ipaddress user passwd flist
do
while read dir
do
if [[ $ipaddress =~ ^192\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
echo "private IP - skipping"
else
ftp -n -v $ipaddress <<EOT
ascii
echo $user $passwd
user $user $passwd
prompt
echo $dir
mget $dir
EOT
fi
done < $flist
done
rm unixfile.txt

Although this could not really supply all the functionality we need, and no one here is really familiar with bash so in the end we choose to switch to PHP. I have to admit it programs better and there are lots of (build-in) functions available. Also the information on the internet is wider etc.

#!/usr/bin/php
<?php
// de server adressen, logins, paswoorden, paden, ... staan in een afzonderlijke config file
require "backupconfig.php";
require "backupfunctions.php";
// Het volgende levert ons een 
$files = directoryToArray($ipxbackup_path_dbfiles, true);
// Bereken datum van gisteren
$yesterday = date('d-m-Y', mktime(0, 0, 0, date("m") , date("d") - 1, date("Y")));
$arrFolders = array();
foreach ($files as $file) {
if (strpos($file, $yesterday . '.7z') !== false) {
//echo $file . "\n";
array_push($arrFolders, $file);
}
}
//print_r($arrFolders);
//---Voor FTP verbindingen: voorbeeld---//
// set up a connection or die
$conn_id = ftp_connect($impactweb_server) or die("Couldn't connect to $impactweb_server"); 
// try to login
if (@ftp_login($conn_id, $impactweb_user, $impactweb_pass)) {
echo "Connected as $impactweb_user@$impactweb_server\n";
} else {
echo "Couldn't connect as $impactweb_user\n";
}
// try to change the directory to backup dir
if (ftp_chdir($conn_id, $impactweb_path)) {
echo "Current directory is now: " . ftp_pwd($conn_id) . "\n";
} else { 
echo "Couldn't change directory\n";
}
//volgende lus overloopt de array van de 7z-bestanden en loadt ze up
/*
foreach($arrFolders as $folder) {
$stringArray = split('/',$folder);
$remote_file = $stringArray[4];
if (ftp_put($conn_id, $remote_file, $folder, FTP_BINARY)) {
echo "successfully uploaded $folder\n";
} else {
echo "There was a problem while uploading $folder\n";
}
}
*/
$conn_id = ftp_connect($ftpservers_server) or die("Couldn't connect to $ftpservers_server"); 
// try to login as impactstore
if (@ftp_login($conn_id, $ftpservers_user, $ftpservers_pass)) {
echo "Connected as $ftpservers_user@$ftpservers_server\n";
} else {
echo "Couldn't connect as $ftpservers_user\n";
}
if (ftp_chdir($conn_id, $ftpservers_path)) {
echo "Current directory is now: " . ftp_pwd($conn_id) . "\n";
} else { 
echo "Couldn't change directory\n";
}
$list = ftp_nlist($conn_id, '.');
function is_txt($file) {
return preg_match('/.*\.txt/', $file) > 0;
}
$filtered = array_filter($list, is_txt);
//print_r($filtered);
foreach($filtered as $txt) {
if (ftp_get($conn_id, $txt, $txt, FTP_BINARY)) {
echo "Successfully written to $txt\n";
} else {
echo "There was a problem\n";
}
}
ftp_close($conn_id);
exit;

// get contents of the current directory
$contents = ftp_nlist($conn_id, ".");
// output $contents
//var_dump($contents);

// close the connection
ftp_close($conn_id);
?>

This is what I have so far, this downloads files from another server, and it works. Now I have to download other files, those specified in the IP_folderlists, I wrote about.
I would like to have a funtion, in PHP, that does te same as:

awk '
NR%3==1{a[++i]=$0}
NR%3==2{u=$0}
NR%3==0{p=$0}
END{for(j=1;j<=i;j++){print a[j], u[j], p[j], a[j] "_folderlist.txt"}}' unixfile.txt |
while read ipaddress user passwd flist
do
while read dir
do
if [[ $ipaddress =~ ^192\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
echo "private IP - skipping"
else
ftp -n -v $ipaddress <<EOT
ascii
echo $user $passwd
user $user $passwd
prompt
echo $dir
mget $dir
EOT
fi
done < $flist
done

It checks every time the first line for an IP, and when it finds a folderlist, it jumps to that file and reads it out.

Thanks in advance !

in PHP, you can use file_get_contents() , fread() , fopen() to read files. Since you are more familiar with PHP, you can go to the manual to see examples of how to iterate a file.

Those concepts are -kind of- familiar to me, but it would be a lot clearer in form of a code example. Thx in advance

I have

$fh = fopen('ftpservers.txt','r');
while (!feof($fh)){
  $line = fgets($fh);
  echo $line;
}
fclose($fh);

ftp_close($conn_id);

How can I put line1 in $ip, line2 in $user and line3 in $pwd
Afterwards it have to pass on to line4 that gets to be $ip, line5 $user and line6 $pwd, line7 as $ip, etc...

Thanks in advance!

Without any experience with PHP (so there may be some syntax errors in the code ;)), I can give some idea to start with.
I use an array , a counter i and a mod operator to index the array:

$fh = fopen('ftpservers.txt','r');
$i=1
while (!feof($fh)){
  $line[$i % 3] = fgets($fh);
  if ($i % 3 == 0) {
    echo $line[1];
    echo $line[2];
    echo $line[0];
  }
  $i++;
}
fclose($fh);

ftp_close($conn_id);

Regards

I appreciate your effort Franklin, but unfortunatly I get...

Parse error: syntax error, unexpected T_WHILE in /home/backupscript.php on line 88

Can somebody check the syntax ?

Probably bash and PHP are kind of different :wink:

Thanks in advance