Something like:
#PBS -t 0-99
#PBS -d "~/$PBS_ARRAYID.output"
What I want to do here is to redefine the working directory of each individual job in the job array, using the job's array id. Is this valid code?
I need to know before I send to the cluster, because I can't run tests there.
Yes, you can use any of the environment variables listed here in -d, -o, or -e.
Related
In my main script, I call a function and within that function I call stata (with system command) where it takes input from matlab and then matlab calls the output of this stata command. I want each of these input and output files have a unique JobID. (I can give other unique IDs with datetime etc, but I don't want this.) I want JobIDs like 1,2,3...
I am using a cluster environment and PBS file to add details about my submit. I want this PBS file take the JOBID from my first step and then send it to the matlab script.
My steps are as follows:
1- write a .py script for multiple jobs (I first try for #2 jobs)
#!/bin/env python
#PBS -A open
import csv, subprocess
for i in range(1,3)
qsub_cmd = """qsub -F JOBID = {jobid} submit_m3.pbs""".format(jobid=i)
print(qsub_cmd)
exit = subprocess.call(qsub_cmd, shell=True)
2 - Write pbs file called submit_m3.pbs (I need to pass JOBID here)
#!/bin/bash
#PBS -l nodes=1:ppn=8
#PBS -l walltime=48:00:00
#PBS -l pmem=10gb
#PBS -A open
# Load in matlab
module purge
module load matlab
cd ~/code
matlab -nodisplay -nosplash < myScript.m > log.myLog
# JOBID must be placed somewhere in the above line.
3- Call the .py script in Linux terminal
python myPyFile.py
In the step 2, instead of the line starting with matlab, I also tried this (following other community questions)
matlab -nodisplay -nosplash -nodesktop -r "job_id=${JOBID};run('myScript.m');exit;"
but this code didn't pass JOBID to my script.
So, 2 things need to be solved. Passing JOBID to .pbs file and then to matlab file. Once it is in matlab, the rest (stata part) is OK!
I have seen similar questions, but not exactly the same as mine: Use Bash variable within SLURM sbatch script, because I am not talking about slurm parameters.
I want to launch a slurm job for each of my sample files, so imagine I have 3 vcfs and I want to run a job for each of them:
I created a script to loop through a file in which I wrote sampleIds to run another script with each sample, which would perfectly work if I wanted to run it directly with bash:
while read line
do
sampleID="${line[0]}"
myscript.sh $sampleID
The problem is that I need to run the script with slurm, so is there any way to indicate slurm the bash variable that it should include?
I was trying this, but it is not working:
sbatch myscrip.sh --export=$sampleID
Okay, I've solved it:
sbatch --export=sampleID=$sampleID myscript.sh
I'm running srun -n 100 python foo.py. Inside the python script how does it find out which task number/id/rank it is? Is there an environment variable set?
Have a look at man srun or man sbatch for a list of environment variables. $SLURM_PROCID might be the one you need.
I use -N option to specify the name for the job when submitting through qsub. The qsub, however, adds some numeric string after thejob name as described in the man page:
By default the file name for standard output has the
form job_name.ojob_id and job_name.ojob_id.task_id for
array job tasks (see -t option below).
Therefore, whenever I submit a new job with same job name, a new suffix .ojob_id is added to the job name and a new output file is created.
What I trying to achieve is to have same output file each time a job is submitted through qsub. How can I do that? I have to run a job several time and I want the output from a run to overwrite the output file generated in the previous run. How can I achieve that?
See the example below:
First time command is given to run script hello_world to output in log_hello_world:
qsub -cwd -N log_hello_world hello_world.sh
It creates two output files:
log_hello_world.e7584345
log_hello_world.o7584345
Second time the same command is given: It creates two more output files
log_hello_world.e7584366
log_hello_world.o7584366
What can I do to get the output in just one file log_hello_world.
I was able to resolve this issue by using options -o and -e to save the log and the error files respectively. With these options, the log and the errors from the job are written into same file every time from this command.
qsub -cwd -N job_hello_world -o log.hello_world -e error.hello_world hello_world.sh
You should append a file with a fixed name. This is done in the code which you run. You create a file in your directory, and then append it with the new results each time you run your code. So in your code (not in the qsub line), explicitly add a line of code which asks the results to be written to a file in your directory, in Mathematica this would be
str = OpenAppend["/home/my_name/Scratch/results.dat"];
Write[str,results];
Close[str];
Where results is a variable which contains the results of your computation. Then just run the job using qsub -N log_hello_world hello_world.sh. This will write the results to the same file every time you run the job without changing the name of the file.
(If you're looking to write both -o and -e files to the same file, you can just add -j y to the qsub after having specified the file path for the error file)
Is there a way to stop sbatch from copying the script to the compute node. For example when I run:
sbatch --mem=300 /shared_between_all_nodes/test.sh
test.sh is copied to /var/lib/slurm-llnl/slurmd/etc/ on the executing compute node. The trouble with this is there are other scripts in /shared_between_all_nodes/ that test.sh needs to use and I would like to avoid hard coding the path.
In sge I could use qsub -b y to stop it from copying the script to the compute node. Is there a similar option or config in slurm?
Using sbatch --wrap is a nice solution for this
sbatch --wrap /shared_between_all_nodes/test.sh
quotes are required if the script has parameters
sbatch --wrap "/shared_between_all_nodes/test.sh param1 param2"
from sbatch docs http://slurm.schedmd.com/sbatch.html
--wrap=
Sbatch will wrap the specified command string in a simple "sh" shell script, and submit that script to the slurm controller. When --wrap is used, a script name and arguments may not be specified on the command line; instead the sbatch-generated wrapper script is used.
The script might be copied there, but the working directory will be the directory in which the sbatch command is launched. So if the command is launched from /shared_between_all_nodes/ it should work.
To be able to lauch sbatch form anywhere, use this option
-D, --workdir=<directory>
Set the working directory of the batch script to directory before
it is executed.
like
sbatch --mem=300 -D /shared_between_all_nodes /shared_between_all_nodes/test.sh