Hi Guys,
how to count number of files successfully copied while coping files from source to destination path
ex:10 files from source to target location copying
if 8 files copied successfully then
echo successfully copied=8
failure=2 files
if two files get error to coping files from source to destination directories
then print cause of error file
ex
if directory not exist
path not exist in target location when comparing with source path files and destination path target
There's umpteen ways to skin this cat. Any attempt from your side?
hi tried my side...
here my code is
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {id=m=mc=ct=" "} /\/Filename/ {id=$3} /\/DivisionCode/{m=$3} /\/ProductCode/ {mc=$3} /\/ContentType/ {ct=$3} /\/sequence/
{print id"\t"m"\t"mc"\t"ct;id=m=mc=ct=" "}' OFS=, $xm > XmlDataNew.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in `ls $src_dir{$fn,*/$fn}`
do
cp /home/oracle/arun/IRMS-CM/{$fn,*/$fn} /home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct
((Succ_Cnt++))
done
else
echo "$dest_dir Directory Does not Exist"
fi
done <<XmlDataNew.txt
here files coping from source to destination location we need to print successfully copied with in number...success copied=10
if failure occurred while coping files from source to destination
ex :failure=5
Well, on first sight there's quite some opportunities in your code sample.
grep (btw, the regex $*.xml is somewhat dubious)? ls -t | head -1 should give the most recent .xml file.IFS=$'\n' ? The file that you read has , as the field separator ( OFS=, ), or <TAB>, being explicitly printed.read fn dc pc ct REST from the input file and drop the four assignment lines? You'd need to define IFS corrrectly, then. -n (no-clobber) option to cp , as copying multiple files with identical file name (from different source dirs) to one single destination dir will overwrite the target files.Succ_Cnt without knowing if the cp was a success?dest_dir if it doesn't exist? << redirection operator), which will fail, instead redirect stdin ( < operator)?---------- Post updated at 12:26 ---------- Previous update was at 12:15 ----------
You may want to give this a try, although untested and needing some polishing:
gawk -F"[\"<>]" ' BEGIN {id=m=mc=ct=" "} /\/Filename/ {id=$3} /\/DivisionCode/{m=$3} /\/ProductCode/ {mc=$3} /\/ContentType/ {ct=$3} /\/sequence/
{print id"\t"m"\t"mc"\t"ct;id=m=mc=ct=" "}' OFS=, $(ls -t *.xml | head -1) > XmlDataNew.txt
src_dir="/home/oracle/arun/IRMS-CM"
while read -r fn dc pc ct REST
do dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ ! -d "$dest_dir" ]
then echo "$dest_dir Directory Does not Exist; creating ..."
mkdir "$dest_dir"
fi
for FILE in $src_dir{$fn,*/$fn}
do if [ ! -e "$dest_dir/$FILE" ]
then cp -n "$FILE" "$dest_dir" && ((Succ_Cnt++)) || ((Err_Cnt++))
else ((Err_Cnt++))
fi
done
done <XmlDataNew.txt
hi rudic
actually my intention....i explained my necessity...here step by step
step1. reading latest xml file from source and print the values in text.file
sample text.file out put
filename divisioncode product code contenttype
aldorzum.doc us vimzim template
alozyme.doc ca standard
aser.doc sa encloser template
step2;;reading text.file and copy the files from source to destination path
step3;check directory exist or not exist in target location
step4;if directory exist then copy the files from source to destination then count how many files copied according to text.file column $1 wise line by line
if 10 files copied out of 15
then print sccessfully copied =10
step5;if directory not exist (no need create directory)
just print how many files not copied successfully if 5 files failure to copy
then print failure =5
cause of failure; following destination folder or directory path not exist will print in to another text2.file
example i need to print text2.file
successfully copied=10
failure copied =5
cause of failure
/home/oracle/arun/livelink/irms-cm/us/vimizim/product directory does not exist
/home/oracle/arun/livelink/irms-cm/us/vimizim/enclosure directory does not exist
/home/oracle/arun/livelink/irms-cm/us/aldurazyme/product directory does not exist
/home/oracle/arun/livelink/irms-cm/us/ /template directory does not exist
/home/oracle/arun/livelink/irms-cm/us/naglazyme/product directory does not exist
step6;finally text2.txt file will attached in mail and send notification to me
using mailx or mail command with "subject status"
if all files copied success then subject status else status error
if copied 14 files out of 15 then we need to get error status
subject: irms-cm document import into livelink mm/dd/yyyy (status = ______)
if all files copied success
subject: irms-cm document import into livelink 23/07/2015 (status = success)
else
subject: irms-cm document import into livelink 23/07/2015 (status = error)
with attached file text2.txt
with require info
step7; after review the attached file we will create destination folders manually
Hello sravanreddy,
It's a request.
i- Please use code tags for commands/codes/inputs you are using in your posts as per forum rules.
ii- Go through the forum rules in following link for a better understanding please.
iii- Please do not use capital letters in posts it may be taken as offense. So request you to not to repeat this.
We all are here to help and learn from each other. We should respect each other as each one has their own identity and working/learning style.
Thanks,
R. Singh
thank you...RavinderSingh13
actually i'm new to this forum i will follow your instructions
if [ -d $dest_dir ]; then
for FILE in `ls $src_dir{$fn,*/$fn}`
do
cp /home/oracle/arun/IRMS-CM/{$fn,*/$fn} /home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct
if [[ $? -eq 0 ]]; then
((Succ_Cnt++))
else
((Err_Cnt++))
fi
done
if [[ ${Succ_Cnt} -ge 1 ]]; then
echo "Successfully copied files = ${Succ_Cnt}"
fi
if [[ ${Err_Cnt} -ge 1 ]]; then
echo "Failures = ${Err_Cnt}"
fi
else
echo "$dest_dir Directory Does not Exist"
fi
Hello SriniShoo,
Can you explain code we con't count succ_count and error count using below code
can you correct my code
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc"
for FILE in $src_dir/{$fn,*/$fn}
do
if [ -d $dest_dir ]; then
if [ -d $dest_dir/$pc ]; then
if [ -d $dest_dir/$pc/$ct ]; then
if [ -f $FILE ];then
cp $FILE $dest_dir/$pc/$ct && ((Succ_Cnt++)) || ((Err_Cnt++))
fi
else
echo "$dest_dir/$pc/$ct not exists"
fi
elif [ -d $dest_dir/$ct ];then
cp $FILE $dest_dir/$ct && ((Succ_Cnt1++))
fi
else
echo "$dest_dir/$pc/$ct not exists"
fi
done
done< XmlDataNew.txt
Succ=$[$Succ_Cnt1+$Succ_Cnt]
echo "Succ=$Succ"
echo $Err_Cnt
any body correct and clarify my doubt
how to count successfully fully copied files from source to target location
else error_count if directory not exist or file not exist at source location.
i want below manner
successcount=10 (out of 15)
error count=5(out of 15)
or
successcount=15(out of 15)
error_count=0
There's a done too many, and I'm not sure your if ... fi constructs are correctly built. Use a more intuitive, talking indentation and check the logical flow. And, why do you use two Succ_Cnt variables in lieu of incrementing just one on success? And, shouldn't Err_Cnt be incremented if no copy is done as the dest_dir doesn't exist?
Hi RudiC...
actually i'm using count as successfully copied files for directory exist and not exist position purpose...because i have problem with
destination path.....
source path files in current folder and with in folders....
so while coping files from source directory to destination directory..
check with source files and copying files from source to destination as well as how many files copied successfully count & remaining files not copied successfully information important.
error count if directory not exist destination path and file not exist at source location..
count error_count
i want below manner
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in $src_dir/{$fn,*/$fn}
do
if [ -f $FILE ];then
cp $FILE $dest_dir && ((Succ_Cnt++)) || ((Err_Cnt1++))
fi
done
else
echo " $dest_dir directory Does Not exists!" >> Test4.txt
Err_Cnt=$(wc -l<Test4.txt)
fi
done< XmlDataNew.txt
please check above my code and replay any one
Not sure I understand... did you run that code snippet? What error msgs were printed? There's again one done too many. And, the dest_dir would "not exist" exactly once, no matter how many files you want to copy there, so does counting the lines in Test4.txt really make sense?
Using TUI, this would be my apporach (untested, idh oracle):
....
touch copy.log
...
while read -r fn dc pc ct REST
do dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
tui-bol-dir "$dest_dir" || (echo "$dest_dir does not exist.. skipping.." >> copy.log ; continue )
echo "--- $dc/$pc/$ct ----" >> copy.log
tui-cp "$src_dir{$fn,*/$fn}" "$dest_dir" >> copy.log
echo >> copy.log
done<XmlDataNew.txt
When done, you can mail the copy log, and will not only see a list of files copied, but also each of which file has been copied successfull or not.
hth
hi All, Any one answer my requirement.
I have source location
src_dir="/home/oracle/arun/IRMS-CM"
My Target location
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
my source text files check with below example.text file content
$fn "\t" $dc "\t" $pc "\t" $ct
file1.doc US SANZA ENCLOSRE
FILE2.DOC CA SUPER
FILE3.DOC DA SLIDE SAVE
FILE4.DOC SA DOC
the above text file is source text file check with source location files and destination location files.
if i copied files from source to destination location then
files copied successfully then count successfully copied =?
if files not copied successfully count Error files not copied=?
(cause of error if destination folders not exist and source file not exist at source location)
in above source txt file while coping files to destination respective folders
ex; file1.doc---> $dc/$pc$ct
us/sanza/enclosure(file1.doc in enclosure folder )
as well as
ex; file2.doc -->$dc/$pc$ct
CA/ /SUPER(file2.doc in super folder directly if $pc flolder not exist then copy to $ct folder)
I developed below code
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in $src_dir/{$fn,*/$fn}
do
if [ -f $FILE ];then
cp $FILE $dest_dir && ((Succ_Cnt++)) || ((Err_Cnt1++))
fi
done
else
echo " $dest_dir directory Does Not exists!" >> Test4.txt
Err_Cnt=$(wc -l<Test4.txt)
fi
done< example.txt
please correct that code
count the successfullycopied files and error files count if not copied successfully
while using above code i'm unable to copy the files to $ct folder if $pc folder not exist then it shows error,but i need to copy files from source location to $ct if $pc folder not exist then jump to $ct folder
another issue is errorfiles count(we are not able to count suucessfully copied count and error files not copied files count at a once time with check from source location according example.txt file)
note :
No need to create destination folder if folders not exist at destionation just print cause of error in another error.txt file
then we check error.txt file and create folders manually.
Please suggest and help for in this code and replay with new code
coping files to source location if files copied successfully copied then count number of files counted and error files not copied successfully count
Sorry, I don't get the logics. Please describe every single step, provide more examples, and take more care to your orthography. Run this
src_dir="/home/oracle/arun/IRMS-CM"
while read fn dc pc ct REST
do ls "$src_dir"/{$fn,*/$fn}
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
echo $dest_dir
ls -d "$dest_dir"
done <example.txt
(or example.text?) and post the result.
above code not correct manner...
while coping files from source path of root folder files to destination path of root folders
if target folder exist then print error count file
if files copied successfully then count successfully copied
note must check once files present folder or with in folder of r0ot structure not count two times if target folder not exist or file not exist from source location count only once
one file if not copied successfully
Any body help i have urgent requirement
I was not saying that code snippet was a solution to your problem. It is meant to find out what your problem really is. If you don't give more details, I won't be in a position to help.
check my code here complete code posted here
#cnt_file="/home/oracle/arun/IRMS-CM/Control_File.txt"
#while [ -f cnt_file ];
#do
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {fn=dc=pc=ct=" "} /\/Filename/ {fn=$3} /\/DivisionCode/{dc=$3} /\/ProductCode/ {pc=$3} /\/ContentType/ {ct=$3}
/\/sequence/ {print fn"\t"dc"\t"pc"\t"ct;fn=dc=pc=ct=" "}' OFS=, $xm >XmlDataNew.txt
cp /dev/null Test4.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in $src_dir/{$fn,*/$fn}
do
if [ -f $FILE ];then
cp $FILE $dest_dir && ((Succ_Cnt++)) || ((Err_Cnt1++))
echo
fi
done
else
echo " $dest_dir directory Does Not exists!" >> Test4.txt
Err_Cnt=$(wc -l<Test4.txt)
fi
done< XmlDataNew.txt
err=$[$Err_Cnt1+$Err_Cnt]
echo $err
if $err=0
then
echo "Status = Success.
$Succ_Cnt files added/updated." | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Success)" xxxxxxx@gmail.com
else
echo "Status = Error.
$Succ_Cnt files added/updated.
$err are failed to copy" | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Error)" -a "/home/oracle/arun/IRMS-CM/Test4.txt" xxxxxxx@gmail.com
fi
above code developed with my spurce...
here problem is
while coping files from source to destination path if $pc folder not exist then check $ct folder exist or not exist and copy the related file to $ct folder.
second problem is
if files copied successfully count number files copied successfully
& if any files not copied count number of files not copied successfully
Several of us are trying to help you in this thread, but when we ask questions that will help us help you you show us a different copy of your code instead of answering the questions. We haven't seen your input files. We haven't seen the filesystem hierarchy you're using or the files you're trying to copy and whether or not the destination directory already contains files with those names. You have two different counts of successful and failed copies and have refused to show us why this matters in the output you want to produce. You have not explained why non-existent destination directories should be treated as failed copies instead of creating the missing directories.
I am guessing that you are using some type of Linux system (since you're using gawk instead of awk ), but, of course, gawk can be installed on non-Linux systems). And, you haven't told us what shell you're using.
You are using $* in your code, but you have never given us any indication of what operands you are passing to your script when you invoke it. (So there is no way for us to understand what you're trying to do nor for us to guess at how your code is going to behave.)
Please help us help you!
Please answer the following questions. Without answers to these questions, I don't think we're going to be able to help you get the results you want.
set -xv to your script and show us the tracing output you get when you run your script.XmlDataNew.txt ?Test4.txt ?.xml file that you processed with this invocation of your shell script?bash or ksh as your shell; what output do you get if you run the script RudiC provided in post #15 in this thread?Thanks for Replay Don Cragun,
My answers for your questions;
My requirement is ...following steps step by step...
1.Explore more on how to read an xml file using linux shell script.
2. Read and split the content of xml file for source and target by file and location.
3. store the values in text file below manner
$fn "\t" $dc "\t" $pc "\t" $ct
file1.doc US SANZA ENCLOSRE
FILE2.DOC CA SUPER
FILE3.DOC DA SLIDE SAVE
FILE4.DOC SA DOC
file5.doc IZ STANDS SRAV
file6.doc US SRAVAS
then source file are in folder and with in folders some files from source location
copy the files to target location as root folders
ex;
file1.doc copy to enclosure folder
as well as
file2.doc copy Super folder
file5.doc copy to srav folder(if folder srav not exist at target location just print /targetpath/srav not exist print in another textfile all the failed files info)
as well count if files copied to destination path ,count successfully copied files & failed files count and print failed file reason in another text file
ex;if files 6 from source
successfully copied =4
failed to copy=2
failed cause print in test4.txt file
/destination/srav/ (if folder folder not exist at destination location
/sourcepath/file4.doc (if file not exist at source location )
cause of above reasons files not copied and failed to copy
print failed files=2 with number count and cause failure issue in TEST4.txt file
and if files coped successfully just print number of files copied successfully only no need to print path loactions
no need to create destination path...after getting test4.txt file we will create folders manually
source path folders ex;/sourcepath/folders
US(FOLDER)
|
|---SANZA(FOLDER)
| |
| -------file1.doc
|
--------file6.doc
so files are from source location info
Below script is working almost but problem is
while error count counted one files check with folder and with in folder shows
if one file not exist source location it shows failed count=2(instead one)
i need to print success count and failed count
if all files coped successfully count success=6
and faild cunt=0
#cnt_file="/home/oracle/arun/IRMS-CM/Control_File.txt"
#while [ -f cnt_file ];
#do
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {fn=dc=pc=ct=" "} /\/Filename/ {fn=$3} /\/DivisionCode/{dc=$3} /\/ProductCode/ {pc=$3} /\/ContentType/ {ct=$3}
/\/sequence/ {print fn"\t"dc"\t"pc"\t"ct;fn=dc=pc=ct=" "}' OFS=, $xm >XmlDataNew.txt
cp /dev/null Test4.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in $src_dir/{$fn,*/$fn}
do
if [ -f $FILE ];then
cp $FILE $dest_dir && ((Succ_Cnt++)) || ((Err_Cnt1++))
fi
done
else
echo " $dest_dir directory Does Not exists!" >> Test4.txt
Err_Cnt=$(wc -l<Test4.txt)
fi
done< XmlDataNew.txt
err=$[$Err_Cnt1+$Err_Cnt]
echo $err
if $err=0
then
echo "Status = Success.
$Succ_Cnt files added/updated." | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Success)" arunasaagi81@gmail.com
else
echo "Status = Error.
$Succ_Cnt files added/updated.
$err are failed to copy" | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Error)" -a "/home/oracle/arun/IRMS-CM/Test4.txt" arunasaagi81@gmail.com
fi
as well as one more script have
#while [ -f cnt_file ];
#do
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {fn=dc=pc=ct=" "} /\/Filename/ {fn=$3} /\/DivisionCode/{dc=$3} /\/ProductCode/ {pc=$3} /\/ContentType/ {ct=$3}
/\/sequence/ {print fn"\t"dc"\t"pc"\t"ct;fn=dc=pc=ct=" "}' OFS=, $xm >XmlDataNew.txt
cp /dev/null Test4.txt
cp /dev/null Test.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc"
for FILE in $src_dir/{$fn,*/$fn}
do
if [ -d $dest_dir ]; then
if [ -d $dest_dir/$pc ]; then
if [ -d $dest_dir/$pc/$ct ]; then
if [ -f $FILE ];then
cp $FILE $dest_dir/$pc/$ct && ((Succ_Cnt++)) || ((Err_Cnt++))
fi
else
echo "$dest_dir/$pc/$ct not exists"
fi
elif [ -d $dest_dir/$ct ];then
cp $FILE $dest_dir/$ct && ((Succ_Cnt1++))
fi
else
echo "$dest_dir/$pc/$ct not exists"
fi
done
done< XmlDataNew.txt
Succ=$[$Succ_Cnt1+$Succ_Cnt]
echo "Succ=$Succ"
echo $Err_Cnt
if $Err_Cnt=
then
echo "Status = Success.
$Succ files added/updated." | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Success)" xxxxxx@gmail.com
else
echo "Status = Error.
$Succ files added/updated.
$Err_Cnt are failed to copy" | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Error)" -a "/home/oracle/arun/IRMS-CM/Test4.txt" xxxxxxxxxx@gmail.com
fi
plese help me...
count of files success count and failed count