Hi,
TASK 1:
I have been using this code to print the information of files kept at "/castor/cern.ch/user/s/sudha/forPooja" in some text file name FILE.txt.
rfdir /castor/cern.ch/user/s/sudha/forPooja | grep data | awk '{print "rfio:///castor/cern.ch/user/s/sudha/forPooja/"$9}' > FILE.txt
Need to mention that "rfdir" is special command in my work. It works same as "ls".
Code, which I am working.
I have some script :
#!/bin/bash
SOURCEPATH=$1
TARGETPATH=$2
GREP=$3
echo "Copying \"$1 | grep $3\" to $2"
for FILE in `nsls $SOURCEPATH | grep $GREP`
do
if [ -f $TARGETPATH/$FILE ]; then
echo "Skipping $FILE."
else
echo "Copying $FILE ..."
awk '{print "$2"$9}' > $2
# rfcp $SOURCEPATH/$FILE $TARGETPATH
fi
done
TASK 2 :
In addition, I have a script which does the rest of the part
#!/bin/bash
split -10 **.txt chunk
i=0
for file in chunk*
do
((i=i+1))
new_file="datafilename"$i".list"
#sed '/^$/d' $file > $new_file
#or use the perl command
perl -0pe 's/\n$//' $file > $new_file
#to remove the processed file
rm -rf $file
echo 'succesful'
done
Here in this code, I am using that FILE.txt .
It would be really useful if I have single script doing both.
Thanks in advance
pooja.
Task 1:
Do you want to print the filenames having the string "data" in the filename with rfio:///castor/cern.ch/user/s/sudha/forPooja/ as prefix into FILE.txt?
And are the files in Task1 and Task2 related in anyway?
And what is the purpose of this?
awk '{print "$2"$9}' > $2
what are the input arguments to the script?
Task1 is not at all clear! 
--ahamed
---------- Post updated at 01:21 AM ---------- Previous update was at 01:17 AM ----------
Something like this?
#!/bin/bash
SOURCEPATH=$1
TARGETPATH=$2
search=$3
for file in $SOURCEPATH/*${search}*
do
file_name=$(basename $file)
test -f $TARGETPATH/$file_name && continue
echo "rfio://$SOURCEPATH/$file_name" >> FILE.txt
cp $SOURCEPATH/$file_name $TARGETPATH
done
split -10 **.txt chunk
i=0
for file in chunk*
do
((i=i+1))
new_file="datafilename"$i".list"
#sed '/^$/d' $file > $new_file
#or use the perl command
perl -0pe 's/\n$//' $file > $new_file
#to remove the processed file
rm -rf $file
echo 'succesful'
done
Everything based on assumptions!
--ahamed
Hi,
The files kept in the "/castor/cern.ch/user/s/sudha/forPooja" has field :
[lxplus415] /afs/cern.ch/user/p/pooja/test > rfdir /castor/cern.ch/user/s/sudha/forPooja
-rw-r--r-- 1 cms003 zh 969064642 Jul 29 17:08 data_10_0_dBw.root
-rw-r--r-- 1 cms003 zh 1021157382 Jul 29 18:46 data_11_0_rRG.root
-rw-r--r-- 1 cms003 zh 1136693129 Jul 30 09:33 data_13_0_Ljr.root
-rw-r--r-- 1 cms003 zh 1136693192 Jul 30 12:50 data_13_2_VEo.root
-rw-r--r-- 1 cms003 zh 458432186 Jul 30 04:58 data_14_0_5TU.root
-rw-r--r-- 1 cms003 zh 75970008 Jul 29 14:17 data_15_0_WKR.root
-rw-r--r-- 1 cms003 zh 1279818743 Jul 30 14:04 data_16_1_Ohw.root
-rw-r--r-- 1 cms003 zh 573268613 Jul 29 16:11 data_18_0_znC.root
-rw-r--r-- 1 cms003 zh 692655767 Jul 29 22:17 data_19_0_KMo.root
I want the information written in FILE.txt is like:
rfio:///castor/cern.ch/user/s/sudha/forPooja/data_10_0_dBw.root
rfio:///castor/cern.ch/user/s/sudha/forPooja/data_19_0_KMo.root
In awk comand
awk '{print "$2"$9}' > $2
$9 is the column for data**.root information and same for $2.
Yes you are right with this statement.
"Do you want to print the filenames having the string "data" in the filename with rfio:///castor/cern.ch/user/s/sudha/forPooja/ as prefix into FILE.txt?"
hope it is clear now.
Many thanks for reply..
pooja
Isn't this working?
SOURCEPATH=$1
TARGETPATH=$2
search=$3
for file in $SOURCEPATH/*${search}*
do
file_name=$(basename $file)
test -f $TARGETPATH/$file_name && continue
echo "rfio://$TARGETPATH/$file_name" >> FILE.txt
cp $SOURCEPATH/$file_name $TARGETPATH
done
--ahamed
Hi,
No, the code is not working
#!/bin/bash
SOURCEPATH=$1
TARGETPATH=$2
search=$3
for file in $SOURCEPATH/*${search}*
do
file_name=$(basename $file)
test -f $TARGETPATH/$file && continue
echo "rfio://$SOURCEPATH/$file_name" >> FILE.txt
cp $SOURCEPATH/$file_name $TARGETPATH
done
Error mesage is following
[lxplus427] /afs/cern.ch/user/p/pooja/test > ./pooja.sh /castor/cern.ch/user/s/sudha/forPooja FILE.txt data
cp: cannot stat `/castor/cern.ch/user/s/sudha/forPooja/*data*': No such file or directory
[lxplus427] /afs/cern.ch/user/p/pooja/test >
Actually, the problem is with the source directory. It is special one and normal command does not work there. As i mentioned "rfdir == ls" similarly, "rfcp == cp"
I have some hint, find below the working code (1.sh) which copy the root files kept in the "/castor/cern.ch/user/s/sudha/forPooja" to some target folder.
#!/bin/bash
SOURCEPATH=$1
TARGETPATH=$2
GREP=$3
echo "Copying \"$1 | grep $3\" to $2"
for FILE in `nsls $SOURCEPATH | grep $GREP`
do
if [ -f $TARGETPATH/$FILE ]; then
echo "Skipping $FILE."
else
echo "Copying $FILE ..."
rfcp $SOURCEPATH/$FILE $TARGETPATH
fi
done
I execute it following way
$./1.sh /castor/cern.ch/user/s/sudha/forPooja/ data /afs/cern.ch/user/p/pooja/
Thanks
pooja..
---------- Post updated at 10:37 AM ---------- Previous update was at 03:54 AM ----------
Hi,
I have a request, could you please explain me the meaning of these command lines.
for file in $SOURCEPATH/*${search}*
do
file_name=$(basename $file)
test -f $TARGETPATH/$file_name && continue
echo "rfio://$TARGETPATH/$file_name" >> FILE.txt
cp $SOURCEPATH/$file_name $TARGETPATH
done
I did not understand the usage of these 2 lines:
file_name=$(basename $file)
test -f $TARGETPATH/$file_name && continue
waiting..
pooja..
Can you tell me where the data files will be? And where you want to move it?
Following code will check if the data file is present at target folder, if so it will continue with the next file.
test -f $TARGETPATH/$file_name && continue
--ahamed
---------- Post updated at 09:10 AM ---------- Previous update was at 09:08 AM ----------
You will need to explain the task 1 more with more clarity.
What I have understood is that, there are data files at the source directory, you want take each data file from the source directory and check if it present in the target directory. If not present copy it to target and create a FILE.txt file with rfio://... entry.
--ahamed
Hi,
The source area from where I am trying to copy the file information is not regular one. As I already mentioned that usual terminal commands does nor work there.
Thanks for ur help..
pooja..