Help with Command

Hi there,

I need some help with this.

I have large number of files in /edw2/source/emm directory and the files have names like:

CDRCCN_4004_120223-2342-DCCSCAP-GETSP2-120423-1612-230384_20120420161300

Actually, what they all have that can distinguish them is 20120423 which is a date - 23rd April 2012 and the trailing 6 numbers (161300) of each file is time 16H13:00 so, I'll have different files with different time but of same date.

E.g.:

1 - CDRCCN_4004_120223-2342-DCCSCAP-GETSP2-120423-1612-230384_20120420161300
2 - CDRCCN_4004_120223-2342-DCCSCAP-GETSP2-120423-1612-230384_20120420151324

and so on till 140,00 files

This is what I'm trying to achieve:

1 - I want to move all files according to their dates in a folder I'll name according to dates. E.g. folder 20apr2012 for all 20th April files.

2 - I want to zip this folder (20apr2012), compress it and move it to another folder on different mount point - /edw2/source/emm2

Any help is much appreciated.

Thanks

You could get a list of the files with a specific datenumber sequence by using something like the below:

for file in $(ls | nawk -F"_" 'substr($NF,1,8) == "20120420"') ; do
mv $file /path/to/new/folder/
done

assuming you start off in the directory containing all the files and you want to move all files from April 20th 2012

Thanks spynappels.

What if I'm starting in another directory?

Because, I want to run this as a cron job.

Try

for i in /source/dir/*20120420??????; do echo mv $i /target/dir/ ; done; tar czf  /edw2/source/emm2/archivefilename /target/dir

The 6 "?" are to make sure we use the pattern you mentioned, to my knowledge a repetition factor doesn't exist in shell globbing.

Hi,

I really appreciate all your efforts but the above 2 scripts are not working.

I just tested them.

Anyone with more help?

Thanks

You need to be more specific in how they did not work. Did you get error messages? What was the output if any?

I didn't get any errors of any sort.

I changed directories as specified and ran the code the way it was sent and nothing changed.

i.e no files were moved nor zipped

But I came across one that works but this is more of a manual process:

Step 1: This moved all files with _20120630 into target directory
$ find . -name "*_20120630*" -print -exec mv {} /home/oracle/work/target \;

Step 2: Tar the target directory
$ tar -cvf 30jun2012.tar /home/oracle/work/adams_de/target

Step 3: Compress the tar file
$ gzip 30jun2012.tar

This I just tested and it worked perfectly but I want a scenario where I can automate this task and also delete the files that have already been compressed. I want to run this via cron.

Thanks for your effort, I appreciate a lot.

Sorry, remove the echo in my command line (left in from "debugging", my fault); would it then mv files? If not, please give us the output of

for i in /source/dir/*20120420??????; do echo $i; done;

(I assume you replaced /source/dir with your respective source directory)

btw, the tar -z option will zip the archive on the fly.

Hi RudiC,

Here is the command I ran:

I'm simulating with this created directories. I have 3 files in /home/oracle/work/adams_de/source and /home/oracle/work/adams_de/target - the target directory.

Below is the error I get:

• but all those directories exist.

I just tested both directories earlier and I have access to them.

Thanks a lot.

---------- Post updated at 04:28 PM ---------- Previous update was at 04:23 PM ----------

And for your info I'm running on AIX 6.

option -z for tar doesn't work on AIX 6. And one more thing, I tried with both

and

• the trailing forward slash but same error.

---------- Post updated at 04:33 PM ---------- Previous update was at 04:28 PM ----------

RudiC,

Sorry my mistake, now they were moved to the target directory but not tar and compressed.

I didn't check as I got that error.

OK, so the

for i in /home/oracle/work/adams_de/source/*20120630??????; do mv $i  /home/oracle/work/adams_de/target ; done;

seems to work.
Next step: tar'ring. Your tar command

tar -cvf 30jun2012.tar  /home/oracle/work/adams_de/target/ gzip 30jun2012.tar

looks a bit strange to me, ar least the gzip should be separated with a ; . AFAIK tar writes to standard output if no archive file given and thus you can pipe that to gzip. Try the following:

tar -cv  /home/oracle/work/adams_de/target/|gzip > /dest/dir/30jun2012.tgz

You need to tell it - as the filename to make it write to stdout, and that's only for some implementations of tar.

That's what I remembered as well; but my linux version does it without - , in fact - gives an error. So Creems needs to test which one will work for him on his AIX. Another option would be to use tar's -O option, this would definitely tar to stdout.

*shrug* My linux version doesn't, my Linux version needs it, and it's just ordinary GNU tar. Maybe you're using something odd like busybox tar, or some sort of wrapper script. Most tars will complain if you don't give it some sort of output option like -f.

On systems which don't support -, you may have to do something kludgy like /proc/self/fd/1.

You got me confused. But, from this forum's linux man page:

which is what I get running tar.

No, it won't. AIX tar doesn't support -O . -f - will write to stdout when -c is also specified.

tar is a lot like ps in that it varies widely across platforms. Don't assume that anything beyond the very basics will work.

Regards,
Alister