Query on scp

UNIX for Beginners Q & A
1

Hi ,
I am new to Shell script.

I got a requirement to get the filenames and timestamp into an array like below.

array=(Jul 19 06:10 file1-en-us_US-20170718T160150Z.json,Jul 19 06:10 file1-en-us_US-20170718T150157Z.json)

So as the first step I have downloaded the required files from remote server using scp .To preserve the server time have added -p flag.

Code

scp -p   username@hostname:/folder1/folder2 /home/username/json

Output

-rw-r--r-- 1 username domain users    17382 Jul 19 06:10 file1-en-us_US-20170718T160150Z.json
-rw-r--r-- 1 username domain users    20463 Jul 19 06:10 file1-en-us_US-20170718T150157Z.json
-rw-r--r-- 1 username domain users   493244 Jul 19 06:10 file1-en-us_US-20170718T163218Z.json
-rw-r--r-- 1 username domain users    36499 Jul 19 06:10 file1-en-us_US-20170718T170151Z.json

Following to this I tried by getting the required fields by awk command as follows

awk ' { print $7,$8,$9,$10,$11 } ' > /home/usrname/input.txt.

This works fine by shell commands.Is it possible to club the awk in the scp command and redirect the output to a file.I tried by piping but it is not giving any result.
Another way is to change to the required directory and put awk command.But cd is not working in shell script.
Please suggest a way to resolve this.

2

Having got the files, would a loop using stat and it's various flags be suitable? You don't tell us you OS, so I'm hoping it is available to you by default. The risk with trying to read the output of something like ls -l is that group names may contain multiple spaces, but maybe none, so that would cause a mismatch on your field counts.

Looking at my manual page for stat, I would use the -c flag with parameters to get the values like this:-

$ stat -c '%y' /etc/profile
2017-03-21 21:56:32.000000000 +0000

You could build on this with a loop, e.g.:-

for filename in *
do
   timestamp=$(stat -c %y $filename)
   ## Trim the timestamp as required or adjust it to your required format
   array_val="${array_val},${timestamp} ${filename}"
done

array_val="${array_val#,}"          # Trim off leading comma
array=("${array_val}")

Does that help? The timestamps from stat would be numeric so depending on your need, you might have to convert them. You might be better using %Y which will give you the value in seconds since the Epoch (1970-01-01 00:00:00) and then convert to the time format you want, perhaps with date if your system supports the -d flag.

Have a try and let us know if that helps or where you get stuck.

Kind regards,
Robin

3

Thanks for the approach, how can i get the array elements, say array[1],array[2] etc
When i gave echo ${array[1]}
it prints nothing.

4

Ah, I should not have quoted the line array=("${array_val}")

Try is with array=(${array_val}) for the last line instead. Does that help?

You might need to work with line 6 too, perhaps this would be better overall:-

array_val=""
for filename in *
do
   timestamp=$(stat -c %Y $filename)
   ## Trim the timestamp as required or adjust it to your required format
   array_val="${array_val} ${timestamp},${filename}"
done
array=(${array_val})

for item in 0 1 2 3 4 5
do
   echo "Value for item $item is ${array[$item]}"
done

The date/time value is in seconds from the Epoch, else the array would split up. is that a problem? We can work on fixing it if we need to.

Robin