AWS S3 with Bash Script

Shell Programming and Scripting
1

Hello All,

I am trying few stuff on Amazon S3 move files to EC2 (Ubuntu Server)

I am receiving daily 2 files in S# bucket

XYZ_2015-02-26_ 200.csv
ABC_2015-02-26_ 200.csv

I want to move daily received files to local EC2(Ubuntu server) and rename files to

XYZ_2015-02-26.csv
ABC_2015-02-26.csv

Instead of Renaming files after moving to EC2 , if possible i want to rename files on S3 itself.

and send an email notification with files received or not on daily basis

Is this possible using Bash? If yes, please guide.

Thanks
Krux

2

If rename command is available..

rename _200.csv .csv *2015-02-26_200.csv

will turn them into

ABC_2015-02-26.csv
XYZ_2015-02-26.csv
3

Hello Krux_rap,

Could you please try following.

ls -lhtr *200.csv | awk '{A=$(NF-1);gsub(/\_/,X,$(NF-1));print "mv " A $NF OFS $(NF-1)".csv"}'

It will print only the move command with new file and old file names, if happy with results you can try following.

ls -lhtr *200.csv | awk '{A=$(NF-1);gsub(/\_/,X,$(NF-1));print "mv " A $NF OFS $(NF-1)".csv"}' | sh

Thanks,
R. Singh

4

If you have s3cmd, it has a 'move' commmand.