I am trying to figure out how to get my PBS scripts to highlight in bash format with vim. PBS scripts are just bash scripts that dont start with the clasic shebang but are used on a queuing system on a cluster computer. There must be a way to alias anything that looks like this to :set syntax=sh-- without having to type that, right? Here is an example of a simple PBS script.
####Parameterized PBS Script ####
#PBS -S /bin/bash
#PBS -l walltime=00:05:00
#PBS -l nodes=1
#PBS -o /users/me/jobOuts/${PBS_JOBNAME}_${PBS_JOBID}.o
#PBS -j oe
#PBS -M my_email#my.org
#PBS -m abe
START=$(date +%s)
cd $PBS_O_WORKDIR
echo `pwd`
sleep 10
END=$(date +%s)
S=$(( $END - $START ))
((h=S/3600))
((m=S%3600/60))
((s=S%60))
TOTALTIME=`printf "%dh:%dm:%ds\n" $h $m $s`
Thanks!
Add these line to your .vimrc:
filetype on
syntax on
au BufRead,BufNewFile *.PBS set filetype=sh
And read :h autocommand, it worth the time.
Related
Hello I'm running this bash script here and the command "bwa index .." will generate several files, and this command doesn't have the "-o" flag for specifying output path. I want to redirect these files into a new folder. I've searched online and all the answers I came across are redirecting files generated to a specific file name, but how to generally just redirect the output files into a new folder? Thanks very much!
#!/bin/bash
### qsub file.name to run from anywhere
### first step of BMW, making indexed fasta file
#PBS -N bwa_index
#PBS -S /bin/bash
#PBS -l walltime=24:00:00
#PBS -l nodes=1:ppn=8
#PBS -l mem=64gb
#PBS -o /gpfs/data/mcnerney-lab/liuweihan/chip_seq/becker_lab/bwa_index.out
#PBS -e /gpfs/data/mcnerney-lab/liuweihan/chip_seq/becker_lab/bwa_index.err
date
module load gcc/6.2.0
module load bwa/0.7.17
bwa index -p mm10_bwa_idx -a bwtsw /gpfs/data/mcnerney-lab/liuweihan/chip_seq/becker_lab/mm10.fa
date
echo END
You can change to the target directory and then run the bwa command. Using pushd and popd will ensure that rest of the script runs in correct directory.
pushd some_directory
bwa index -p mm10_bwa_idx -a bwtsw /gpfs/data/mcnerney-lab/liuweihan/chip_seq/becker_lab/mm10.fa
popd
First
You can define a variable and use it in your script.
Second
You can pass arguments to your script and capture them via $1, $2, $3 and so on.
Third if you wanted to be fully dynamic by using option ( e.g. --output | -o ) you can use a simple parse as bellow:
#!/bin/bash
################################################################################
# main flags, both longs and shorts
################################################################################
ARGS=`getopt -o "o::" -l "output:" -- "$#"`
eval set -- "$ARGS"
declare -A _output;
_output['flag']=0;
_output['path']=0;
################################################################################
# extract options and their arguments into variables.
################################################################################
while true ; do
case "$1" in
-o | --output )
_output['flag']=1;
_output['path']=$2;
shift 2;
;;
--)
shift;
break;
;;
* )
echo "Internal error!" ; exit 1
;;
esac
done
echo "output: ${_output['path']}";
And usage is super simple, you can use --output <YOUR-PATH> or -o <YOUR-PATH>
# test
./cli.sh --output /path/to/dir
output: /path/to/dir
./cli.sh -o /path/to/dir
output: /path/to/dir
I am using WinSCP SSH Client to connect my Windows PC to a HPC system. I created a file called RUN1.sh including following code:
#PBS -S /bin/bash
#PBS -q batch
#PBS -N Rjobname
#PBS -l nodes=1:ppn=1:AMD
#PBS -l walltime=400:00:00
#PBS -l mem=20gb
cd $PBS_O_WORKDIR
module load mplus/7.4
mplus -i mplus1.inp -o mplus1.out > output_${PBS_JOBID}.log
It simply calls mplus1.inp file and save results of the analysis in the mplus1.out file. I can do it on Linux one by one. I can do the same for mplus2.inp file running RUN2.sh file below:
#PBS -S /bin/bash
#PBS -q batch
#PBS -N Rjobname
#PBS -l nodes=1:ppn=1:AMD
#PBS -l walltime=400:00:00
#PBS -l mem=20gb
cd $PBS_O_WORKDIR
module load mplus/7.4
mplus -i mplus2.inp -o mplus2.out > output_${PBS_JOBID}.log
However, I have 400 files like this (RUN1.sh, RUN2.sh, ......RUN400.sh). I was wondering if there is a way to create a single file in order to run all 400 jobs in linux.
I have a job called test.sh:
#!/bin/sh -e
#PBS -S /bin/sh
#PBS -V
#PBS -o /my/many/directories/file.log
#PBS -e /my/many/directories/fileerror.log
#PBS -r n
#PBS -l nodes=1:ppn=1
#PBS -l walltime=01:00:00
#PBS -V
#############################################################################
echo 'hello'
date
sleep 10
date
I submit it with qsub test.sh
It counts to 10 seconds, but it doesn't write hello to file.log or anywhere else. If I include a call to another script I need that I programmed (and runs outside the cluster), it just goes to Exiting status after said 10 seconds and plainly ignores the call.
Help, please?
Thanks Ott Toomet for your suggestion! I found the problem elsewhere. The .tschrc file had "bash" written in it. Don't ask me why. I deleted it and now the jobs happily run.
I am running a parallelised code on an SGE cluster, via the qsub command.
The code (which compiled successfully on the system on which it is supposed to run) is meant to take a file of input values, minimise some function of those values, and then output the new values to the same input file.
The job executes succesfully (code 0), and runs for about 40 minutes of walltime: but nothing is written to the input file.
This is my script to submit the jobs:
#!/bin/bash
#PBS -V
#PBS -l select=1:ncpus=20:mpiprocs=20,walltime=02:00:00
#PBS -o some/path
#PBS -e some/path
#PBS -q smp
#PBS -m ae
#PBS -M user#username.com
#PBS -P Name
#PBS -I
#PBS -N minMg-1
module load gcc/5.1.0
module load chpc/openmpi/1.10.2/gcc-5.1.0
mpirun -np 20 $SRCDIR/myexecutable args < inputfile.inp
I can't see why the thing executes successfully, but doesn't write to inputfile.inp. Strangely, I also don't get the standard ".o" and ".e" files, either. I am sure my mistake may be obvious to someone in the know! Any help would be deeply appreciated.
I'm trying to run some samtools commands from a qsub call (to run on a cluster). For some reason, the commands do not seem to be recognized. However, if I copy-paste the command and run it directly from the terminal cluster, it works fine. Has anybody experienced such issues or have an idea what I'm doing wrong?
Thanks,
Patrick
My qsub (this doesn't work):
#!/bin/bash
#./etc/sysconfig/pssc
#PBS -S /bin/bash
#PBS JOB_NAME="QSH_$(whoami)"
#PBS NODE_NUM="1"
#PBS NODE_PPN="${NODE_NCPUS}"
#PBS HOURS="24"
#PBS MINUTES="00"
#PBS SECONDS="00"
#PBS WALLTIME=${HOURS}:${MINUTES}:${SECONDS}
#PBS RES_LIST="nodes=${NODE_NUM}:ppn=${NODE_PPN}"
#PBS DIR_WORK="${PBS_O_WORKDIR}"
#PBS QUEUE="high"
#PBS cd ${DIR_WORK}
samtools index /data/test.bam /data/test.bai
If I run the command directly from the terminal, it works:
samtools index /data/test.bam /data/test.bai
Did you remember to cd into your working dir? I do not believe that qsub expands the $ variables in e.g. PBS cd ${DIR_WORK}.
Try with this script:
#!/bin/bash
#./etc/sysconfig/pssc
#PBS JOB_NAME=test
#PBS WALLTIME=24:00:00
cd ${PBS_O_WORKDIR}
echo `pwd`
dir