Help with Logic - Copying the same files by 2 Linux scripts to same directory

Hi All,

Good Morning !

Can anyone help me understand the following scenario.

Scenario 1 : 2 LINUX Scripts(Process 1 & 2) are trying to copy the same file (lets call it FileA.txt) to the same directory (lets say, /home/abc/zyz) at the same time. If both copy is happening at the same time, will the 2 LINUX Scripts abort ?

Scenario 2 : Process 1 copied the fileA.txt to the directory & this file is being read by another script. Process 2 tries to copy the file again while it is being read by Process 1.

What will happen in these 2 scenarios ?

Thanks Much for your inputs,
Freddie

  1. No, they will not abort. In case of actual disk block copying the target will contain garbage.

  2. The same as 1. The reading proces will get garbage.

In some situations it could "work" - if all files are on the same filesystem and the blocks are
shared between the files.

With short files it could work most of the time - as in the target having the same content as
the source, you cannot rely on it though.

Thanks for the reply.

Is there a way to put a validation in the linux script which will do the following,

In the below FOR Loop, is there a way to put a logic to check whether the file *datevalue_filename* already exists in the $destinationpath.

If the date_filename is available, we shouldnt copy the file again.

Thanks Much,
Freddie

Because the "find" syntax is incorrect on my computer it not possible to guess your intention. The first parameter to "find" is usually a Directory Name.

Which files are you trying to copy ... and to where?

What is/are the actual name/names of "*datevalue_filename*" ? Is it a directory or a file?

Regarding two scripts potentially writing files of the same name to the same target directory.

Conventional technique is for each script to contain identical code to process a semaphore file in some mutally agreed directory (not the target directory) which means "target directory busy". Before accessing the target directory, check whether the semaphore file exists and either back off or exit. If the semaphore file does not exist, create the semaphore file, do your process on the target directory, then delete the semaphore file.

You may find a sempahore file referred to as a flag file or and interlock file. The meaning is the same.

Hi Methyl/Hipe,

Thanks for your reply.

I am currently doing a ls -l command based on the date value present in the file name & writing those filenames(into file_per_chg_indx.txt) that comes under 2 date range. (This is a while loop)

Then, using the below FOR loop, i move the files from one directory to another,

Is there a way I can put a logic inside this FOR loop that will check for the file availability in the $COMMON_INBOX & if it present then skip the copy, else copy the file?

Thanks much in advance.
Freddie

Please post the code which does this process. It is virtually impossible to do this based on the output from ls -l . The find command posted earlier is really dubious.

This is a very dubious script construct which will fail if any filename contains a space character or if there is a large number of files. I'd love to know where it comes from because I have never seen it in any scripting course, but it crops up frequently on unix.com.

Anyway, to answer your question and to correct some of the problems,

cat "${COMMON_TMP}/file_per_chg_indx.txt" | while read filename
do
      filebase=`basename "${filename}"`
      target="${COMMON_INBOX}/${filebase}" 
      if [ -f "${target}" ]
      then
              echo "Target file already exists: ${target}"  
      else
               cp -p "${filename}" "${target}"
               echo "Copied: ${filename} to ${target}"
      fi
done

Ps. Note that code is indented and that all string variables are within double quote characters.
Pps. Please just once post what Shell you are using. We might be able to eliminate the cat command in the script.

Hi Methyl,

Thanks for your reply.

Here is the code component which copies the 2 files from one directory to another based on 2 date ranges,

The file *_ABC_CUST_GS_NON_TRAN_CHANGE_INDEX_DLY* is copied to the same directory by 2 different scripts.

Also, can you pls let me know in-case of any coding/syntax/standards errors.

Thanks Much
Freddie

Please let us know what Shell you are running.

How do you match the value of $dynbusdate ? Is it part of the filename, or have you changed the default way that ls displays dates?
If it is to do with the filenames, please post sample filenames.
Earlier posts mention working with a range of dates. The script as posted only looks for yesterday's date (which simplifies the process).

On the subject of standards:
It is accepted practice to indent conditional code for readability.
It is normal to put double quotes round string variables (whether or not the command happens to work sometimes) because you can get undesirable effects if the variable contains space characters or contains a string sequence recognised by the command.
It is totally unacceptable to use for ... do with an open-ended list. I still don't know where the construct came from and have yet to find it in any Shell Manual, Shell book or Shell course notes. Kernighan himself recommends the while ... do construct.
There is never a reason to end a line with a semi-colon character (except with certain find ... -exec \; syntax where the escaped semi-colon is special to find and nothing to do with Shell syntax).

Thanks Methyl for your reply.:slight_smile:

The script is Korn Shell.

The $dynbusdate value is derived from a file that contains the date value, i.e. P_BUS_DATE=20120821 for e.g

I have one more date value named P_SUB_DATE (which is assigned to $dynsubdate) & the files that falls between these 2 ranges are copied from one directory to another.

The filenames are as follows,

20120821_ABC_CUST_GS_NON_TRAN_CHANGE_INDEX_DLY.dat.z

This file is copied to the same directory by 2 different scripts. (This is where i should put a logic to check whether the file(for a particular date) is already copied by the other shell script.

20120821_ABC_CUST_GS_NON_TRAN_PER_PROF_DLY.dat.z

Hope this helps,

Thanks,
Freddie