Basically, I want to let the system follow FIFO, but sometimes I want to change the priority of jobs by the administrator.
This is why I set the scheduler type to builtin, but in that case, I could not change the order of jobs using the scontrol top command.
I also tested the backfill scheduler type with default_queue_depth=0 option but it isn't worked.
If anyone have a good idea, please help me.
I could resolve this issue by using scontrol hold and scontrol release Command. The hold status takes precedence over the builtin scheduler type　setting.
Related
I am building a unix package in which there is a script, according to the client requirements, script should run only once (not on daily basis). I am not sure how will it work?
I mean, do i need to schedule it ? if yes , then I could not find any cron command for it.
If no, then how will script get execute when package is being installed?
Cron is used for repetitive executions of the same command or script. If you look for a way to schedule an only once script run i would suggest you to use "at" command
Schedule script to run in 10minutes from now:
at now +10 minutes
sh <path_to_my_script>
Then ctrl+d to save
Instead of minutes you can use hours|days|weeks and so on. For reference https://linux.die.net/man/1/at
List queued jobs:
atq
To remove a given scheduled job:
atrm <job_id>
Note: please ensure you have previleges to run the the scheduled script or run it with sudo.
There are two general ways to schedule a task in Unix/Linux:
cron is useful for repeatable tasks.
at is useful for single tasks.
So, in your case, I'd use at for schedule your task.
If it is only once at installing package, you do not need cron at all, just make the user execute it. Lets say you instruct the user to execute something like ./install, and include the script and that's it.
But as someone previously said, first define what is the meaning of once.
Regards.
as administrator I need to give the maximum priority to a given job.
I have found that submission options like: --priority=<value> or --nice[=adjustment] could be useful, but I do not know which values I should assign them in order to provide the job with the highest priority.
Another approach could be to set a low priority by default to all the jobs and to the special ones increase it.
Any idea of how I could carry it out?
EDIT: I am using sched/backfill policy and the default job priority policy (FIFO).
Thank you.
I found a solution that works without the need of using PriorityType=priority/multifactor (as suggested by Bub Espinja):
$ scontrol update job=<job-id> Priority=<any-integer>
The above command will update the priority of the job and update the queue accordingly.
The minimum priority needed to become the next one in line can be found by checking the priority of the next pending job and adding one to it. You can find the priority of a job using the following:
$ scontrol show job=<job-id>
(scontrol update can be used to change many aspects of a job, such as time limit and others.)
EDIT:
I just learned one can do
$ scontrol top <job-id>
to put a job at the top of their queue.
What I have done is to use the priority plug-in multifactor, with the default configuration, adding this line to slurm.conf:
PriorityType=priority/multifactor
Then, as all the jobs will have the priority 0, I must update the target job priority, in my case using the API:
job_desc_msg_t job_update;
slurm_init_job_desc_msg(&job_update);
job_update.job_id = target_job_id;
job_update.priority = 4294967295;
slurm_update_job(&job_update);
EDITED:
From the Slurm FAQ:
The job's priority is an integer that ranges between 0 and 4294967295. The larger the number, the higher the job will be positioned in the queue, and the sooner the job will be scheduled.
The requirement is simple, after creating a job in kue with a given priority, is it possible to change its priority (like renice in POSIX) before it's scheduled to run?
I had the same need.
It seems that job.priority(level).update(fn) works. Job#save(fn) could also be used as it calls Job#update(fn) if the job has already been saved.
I've been using Sun Grid Engine to run my jobs on a node of a cluster.
Usually I would wait for the job to complete before exiting and I use:
qsub -sync yes perl Script.pl
However now I don't use anymore Sun Grid Engine but PBS Pro 10.4
I'm not able to find a corresponding instruction to -sync.
Could someone help me?
Thanks in advance
PBSPro doesn't have a -sync equivalent but you might be able to use the
-I option combined with the use of expect to tell it what code to run in order to get the same effect.
The equivalent of -sync for PBS is -Wblock=true.
This prevents qsub from exiting until the job has completed. It is perhaps unusual to need this, but I found it useful when using some software that was not designed for HPC. The software executes multiple instances of a worker program, which run simultaneously. However, it then has to wait for one (or sometimes more) of the instances to complete, and do some work with the results, before spawning the next. If the worker program completes without writing a particular file, it is assumed to have failed. I was able to write a wrapper script for the worker program, to qsub it, and used the -Wblock=true option to make it wait for the worker program job to complete.
I'm trying to implement a JCL, in a JES2 environment, that launches a set of jobs with dependencies in it, for example:
JOB_A -> JOB_B )
JOB_C -> JOB_D ) -> JOB_E
In other words, JOB_E is only launched when JOB_B and JOB_D are finished.
I can launch JOB_B and JOB_D through job internal reader in JOB_A and JOB_C but I can't not create the dependencies for JOB_E.
I tried to explore JCL resource lock so that I could lock a data set in JOB_B and JOB_D that JOB_E needed so that JOB_E would only start when all data set's are available but the JCL only requests data set in STEP level and release them afterwards. If JCL could request all data set before start I could implement some sort of mutex in the JOBs, for example:
JOB_A locks data set DSN_A
JOB_B waits to get data set DSN_A
JOB_C locks data set DSN_C
JOB_D waits to get data set DSN_C
JOB_E waits to get data set DSN_A and DSN_C
How to do this?
I need this to test set of JCL's in a development environment without access to a scheduler.
Your comment that you need this to test in a development environment without access to a scheduler makes me wonder if your shop has a scheduler for the production environment. If it does, then your testing will not actually test what will be used in your production environment. Just something to think about if you haven't already.
In answer to your question, One technique is to use a utility such as IEBGENER in the last step of one job to submit a subsequent job.
For example, The last step of JOB_A would execute IEBGENER with SYSUT1 containing the execution JCL for JOB_B and SYSUT2 pointing at INTRDR. This is one technique you could use, though getting JOB_E to run so that it doesn't interfere with any of the other jobs might be tricky, as JOB_E needs to run after both JOB_B and JOB_D complete.
Another technique would be to use Rexx in batch mode to submit your jobs using the internal reader and then use the SDSF Rexx interface to watch for when they complete. Essentially you will be writing a special-purpose job scheduler, specific to your set of jobs.
Update, ten years later...
As of z/OS 2.2 IBM has added JES2 Execution Control Statements which "define the execution sequencing of a group of jobs and the jobs themselves". Prior to use of this feature, some configuration must be done my your z/OS Systems Programmer.
I'm wondering why to invest precious time to test a set of jobs, where the PROD set is entirely different and will be handled by some xyz scheduler. Don't mind, if I sound crazy but lemme propose mine too:
Assumption: Your jobs take manageable CPU and NO NEED to be run parallely.
A triggers B triggers C triggers D
triggers E (I know its not worthy but
your testing goes fine) I just put it
here by thinking what I would do if I
were you. I mainly need my testing to
go quick and fine. Lemme know your cliche.
Now, lemme appreciate you both for such resolution that we can manage submission of jobs by means of REXX that too creating a virtual and subjective scheduling.