Submitting multiple qsub jobs

UNIX for Advanced & Expert Users
1

Hi all,
Today, I want to ask how to submit multiple qsub jobs.

I want to submit 100 .sh files for the simulations.
The name of files is
run_001.sh ,
run_002.sh ,
run_003.sh ,
.....
.....
run_100.sh

Submitting each file manually are time-consuming, hence, I want to make another .sh file ( runAllFile.sh ) for just submitting those files at once.
The code of runAllFile.sh is as follows:

--------------------------------------------------------

#!/usr/bin/tcsh

set currDir = `pwd`

foreach i0 (run_*.sh)
	qsub -d ${currDir} ${i0}
end

--------------------------------------------------------

I think if I submit the runAllFile.sh file, it will submit all run_*.sh files to the machine.
However, I have received an error when I try to submit runAllFile.sh by " qsub runAllFile.sh ".

This is an error message:

-bash: /usr/bin/tcsh: bad interpreter: No such file or directory

Could you please help me solve this problem?

Thanks for your time and consideration.

2

Please use code tags as required by forum rules!

You seem to be running the bash shell. The script's shebang requests the tcsh shell. Does the file /usr/bin/tcsh exit?

Why don't you use bash to run all the .sh files?

3

You could also get an error message like that if you created runAllFile.sh with an editor that uses DOS (<carriage-return><newline>) line terminators instead of UNIX/Linux (<newline>) line terminators.

So, if tcsh on your system is in /usr/bin/tcsh (instead of /bin/tcsh where it is located on many systems) try:

od -c runAllFile.sh

and look for characters displayed as \r .

4

Thanks for your kind reply.
I can solve the issue by changing "

/usr/bin/tcsh

" to "

/bin/tcsh

"

However, I received another error message.

line 5: syntax error near unexpected token `('
line 5: `foreach i0 (run_*.sh)'

I think my code looks correct, so that it is hard to find the reason.

Thanks.

5

You dont need the () there, unless you encapsule ls like $(ls run_*.sh) , which would be unnecesary.
Not sure if it would require a leading ./run_*.sh .

hth

6

No no, if this is tcsh, foreach needs the brackets!
And the syntax looks correct to me.

tcsh --version

?

7

Thanks for your valuable comments.

I found that, if this is "bash", "foreach" does not work.
I use "for" instead.
Hence, I have modified the code as follows:

#!/bin/bash

set currDir = `pwd`

for i0 in 'run_*.sh'
do
	qsub -d ${currDir} ${i0}
done

Nonetheless, I have received an error:

line 7: qsub: command not found

Could you please help me solve this problem?

Thank you for your time and consideration.

8

I have no idea whether or not qsub is installed on your system or not. And, if it is installed, I have no idea what directory was specified when it was installed. But, in addition to that problem, bash (and any other shell that supports basic POSIX standard shell requirements) has a pre-initialized variable naming the current working directory, and you can't quote a filename pattern if you want the pattern to be expanded to a list of matching files. To protect yourself against whitespace characters in filenames and pathnames, you should also double-quote the variable expansions.

After you set and exported PATH to include the directory in which you have installed qsub , you might want to try something more like:

#!/bin/bash

for i0 in run_*.sh
do	qsub -d "$PWD" "$i0"
done
9

I have tried the code that you suggest, but it also has the same problem.
However, when I am trying to run a single simulation, "qsub" works well.

If I want to run the "runAll.sh" file, I use a command which is "qsub runAll.sh".
I think if I utilize appropriate options for qsub command, it can help to solve the issue.
But, it is hard to guess which option is appropriate for the simulation.

Could you suggest any qsub options for submitting this sh file?

Thanks a lot!

10

It appears that you completely ignored:

in my last response. I can't suggest anything on how to use qsub other than to try the code that you said is already working. I don't have qsub installed on my machine, and the qsub man page I found doesn't have the -d directory option that you're using. But, there is clearly no way that adding options to qsub is going to make any difference if qsub can't be found in your $PATH and you don't specify an absolute pathname to where you have installed qsub on your system. All I can do is help you with the bash shell syntax.

11

And, do you know how to find a path for qcommands?
Qcommands are already installed by the system manager so that I do not know where the path is.
So, I tried to find it, but it is hard to do it.

Thanks for your kind help.
I really appreciate it.

12

That might be an alias. Please post the output of alias qsub . Or, the outputs of type qsub and whereis qsub might help.