Exclude certain folders

Hi Experts,
Below is my shell script and it will move the files older than 90 days to archive mount. Now my new requirement is , I need to move some of the directory files older than 365 days. How can I achieve this.

Simply I have
DIR1
DIR2
DIR3
DIR4
I need to exclude DIR 2 and DIR 2 directory older than 365 days files should be moved to another directory. Remaining directory will move the files older than 90 days as usual

#!/bin/ksh
#
#
# Backup Script to move older than 90 days files
DATE=`date +%Y%m%d`
#
#SRC Directories and Location
#-----------------------------------------------------
SRC_CDR_OUTBOUND=/omount/outbound
SRC_CDR_INBOUND=/omount/inbound
DEST_SRC_BILG=/omount/archive/
#-----------------------------------------------------
#
DATE=`date +%Y%m%d`
#Counting Number of files
#--------------------------------------------------------
BK_FILECOUNT_OUT=`find $SRC_CDR_OUTBOUND -type f -mtime +90 |wc -l`
BK_FILECOUNT_IN=`find $SRC_CDR_INBOUND -type f -mtime +90 |wc -l`
#--------------------------------------------------------
#
#Finding OUTBOUND files older than "90" days and Move the files to archive folder
#-----------------------------------------------------------------------------
find $SRC_CDR_OUTBOUND -type d -name '*' -exec mkdir -p $DEST_SRC_BILG{} \;
find $SRC_CDR_OUTBOUND -type f -name '*' -mtime +90 -exec mv {} $DEST_SRC_BILG{} \;
#-----------------------------------------------------------------------------
#
#Finding INBOUND files older than "90" days and Move the files to archive folder
#-----------------------------------------------------------------------------
find $SRC_CDR_INBOUND -type d -name '*' -exec mkdir -p $DEST_SRC_BILG{} \;
find $SRC_CDR_INBOUND -type f -name '*' -mtime +90 -exec mv {} $DEST_SRC_BILG{} \;
#-----------------------------------------------------------------------------
#Generating the log with date and file count for refernece
#-------------------------------------------------------------------------------------------
echo $BK_FILECOUNT_OUT "Outbound files moved to archive on"$DATE >>/omount/backup_logs.txt
echo $BK_FILECOUNT_IN "Inbound files moved to archive on"$DATE >>/omount/backup_logs.txt
#-------------------------------------------------------------------------------------------

Welcome arumugavelvelu ,

Firstly, you don't need -name '*' which says "only objects called anything" so leaving it out will ignore name matching and find objects that match the other criteria giving you the same result.

If you have a fixed set of input directories, then you could set your variables at the top to refer to them, e.g.:-

SRC_CDR_OUTBOUND="/omount/outbound/DIR1 /omount/outbound/DIR3 /omount/outbound/DIR4"
SRC_CDR_INBOUND="/omount/inbound/DIR1 /omount/inbound/DIR3 /omount/inbound/DIR4"

These will then get used by your find command as the directories to search.

If the directories may change over time and you don't want to recode them, you may need to do something a bit more elaborate like:

SRC_CDR_OUTBOUND="`ls -1d /omount/outbound/* | grep -v DIR2"
SRC_CDR_INBOUND="`ls -1d /omount/inbound/* | grep -v DIR2"

This will get you all the objects at that level, so if you want to exclude files, then you have to be more elaborate still:-

SRC_CDR_OUTBOUND="`find /omount/outbound -type d ! -name DIR2 | cut -f-4 -d\"\/\" |sort -u`"
SRC_CDR_INBOUND="`find /omount/inbound -type d ! -name DIR2 | cut -f-4 -d\"\/\" |sort -u`"

Do these achieve what you need, or have I missed the point somewhere?

Robin

Hi Robin
Thank so much for your explanation. That works for me ..
But what i am planning is i am going to put one text file named as 365.txt to the directory which i want to exclude.so my shell script will find whether 365.txt file is there or not in each directory. If it exist then it will move the file older than 365 days..... if not it will move the files older than 90 days.

So in future if new request comes to me to exclude this directory I will put 365.txt file to that directory then it will move older than 365 days files
I know its possible but i tried many ways .. Unfortunately could not achieve it. Could you please put some light on this.

Thanks in advance....

That makes it a little more tricky, but as they say, "Anything is possible."

You can test if a file exists with this:-

if [ -f filename ]
then
   echo "Found it!"
else
   echo "File is not there."
fi

Does that help? You could set a variable to use in the find as the value assigned to -mtime . I'm still not clear on the process you would want it to take and how you would know where to find the file to make your decision. Can you write out a set of logical steps so we can plan it. If you put the phrases one on a line, then highlight them and press a list button (same icon choice as MS word) it will format it a bit better. To show a loop, just highlight the section and again press the list button of your choice.

This:-

.... generates this:-

  • List line1
  • List line2
  • List line3
    [list]
  • 2nd line1
  • 2nd line2
    [/list]
  • List line4

If you can articulate what you need as a set of steps, then I'm sure someone here can help.

Regards,
Robin

Hi Robin,

Thank you for your reply.

Can you please tell me what the following command will do :confused:

cut -f-4 -d\"\/\" |sort -u

First I'm trying to get the directory name which has 365.txt file and accordingly, I will execute the command to move the files to Archive

Here I'm getting directory with file name as output, I would like to cut the 365.txt from the output.

Now I'll store this directory in a variable and execute the mv command.

[psoft@shdpsap1lql ~]$ find $src -type f -name 365.txt
/omount/zy_custom/outbound/archive/velu/365.txt
[psoft@shdpsap1lql ~]$

The partial command in question: cut -f-4 -d\"\/\" |sort -u splits the values from the preceding command based on a forward slash / . The backslashes \ are needed because the quotes " and forward slash / are special to the shell and might get interpreted as meaning something rather than just being plain text.

The sort -u is a sort-unique, so having cut off the first 4 fields (null and three layers of directory split by / ) there will be duplicates. This ensures that there is only one of each to be assigned to the variable in question.

I'm not sure if that makes it any clearer though. Maybe an example is better. Consider a directory structure like this:-

/a/b/c/d/e/1
/a/b/c/d/e/2
/a/b/c/d/e/3
/a/b/c/d/Z/1
/a/b/c/d/Z/2
/a/b/9/d/e/1
/a/b/9/d/e/2

If the first three layers of directories are significant, you want to get the result like this:-

/a/b/c
/a/b/9

To do this, my code would first get the first four fields (and delimiters / with the cut command to give you this:-

/a/b/c/d/e/1
/a/b/c
/a/b/c
/a/b/c
/a/b/c
/a/b/9
/a/b/9

..... then the sort would reduce it to the required list.

I hope that is a little better.

On the question of finding directories and the '365' file, you might need more steps:

  • Find all the significant directories as above (storing them in $SRC_CDR_OUTBOUND etc.)
  • Change your find to a loop for each directory in $SRC_CDR_OUTBOUND :-
    [list]
  • Check if the file 365.txt exists with if [ -f $directory/365.txt ] and if it does set the value of days as 365, else 90.
  • Run the find and mv for the specific directory using the value of days to define the -mtime limit
    [/list]
  • Go round the loop again for the next directory in $SRC_CDR_OUTBOUND etc.

Does this logic work for you, or have I missed the point?


Please remember to use CODE tags. Simply wrap code and data input/output in CODE tags, like this:-
[quote]
~~~
```text
This is my code
```
~~~
[/quote]

to produce the following \(fixed character width, space respected\):-
```text
This is my code
```


Not only does it make posts far easier to read, but CODE and ICODE sections respect multiple space and have fixed width characters, which is important for easily seeing input/output requirements.