Is it possible to expand the number of characters used in the JobName column of the command sacct in SLURM?
For example, I currently have:
JobID JobName Elapsed NCPUS NTasks State
------------ ---------- ---------- ---------- -------- ----------
12345 lengthy_na+ 00:00:01 4 1 FAILED
and I would like:
JobID JobName Elapsed NCPUS NTasks State
------------ ---------- ---------- ---------- -------- ----------
12345 lengthy_name 00:00:01 4 1 FAILED
You should use the format option, with:
sacct --helpformat
you'll see the parameters to show, for instance:
sacct --format="JobID,JobName%30"
will print the job id and the name up to 30 characters:
JobID JobName
------------ ------------------------------
19009 bash
19010 launch.sh
19010.0 hydra_pmi_proxy
19010.1 hydra_pmi_proxy
Now, you can to customize your own output.
To retrieve my list of SLURM jobs running I use the default format with 30 characters showing for job names using the bash command below:
squeue --format="%.18i %.9P %.30j %.8u %.8T %.10M %.9l %.6D %R" --me
If you want to show more job name characters simply change the number of %.30j.
Related
I have a data frame that shows me everyday call records and what happened to that call. I need to group the call records by caller ID, date, and description of the call, count the number of times that caller called in a day, and the time difference between them.
My current dataset is
Call record ID | DiscriptionCallEvent | CallDateTime |
14306187 | CallTerminated | 2021-02-08 17:48:04
14306189 | AbandonedQueue | 2021-02-08 18:18:44
14306189 | AbandonedQueue | 2021-02-08 18:19:20
14306189 | AbandonedQueue | 2021-02-08 18:20:00
14306189 | Caller termintaed | 2021-02-08 19:20:00
The second caller here called 2 times back to back. What I am trying to do is capture the information in this way.
Call record ID | DiscriptionCallEvent | CallDate | #Times | TimeDiff
14306189 AbandonedQueue 2021-02-08 3 2 minutes (say)
14306189 Caller terminated 2021-02-08 1 1 hour
I was able to group by record ID and description and date and count the number of times the call occurred. But I am not able to find the time difference within the group. I tried many StackOverflow solution but did not work.
Currently, I am using this query:
CallsGrouped=df.groupby(["CallRecordID","CallDate","DiscriptionCallEvent "],as_index=False)['Call Record ID'].count()
How do I modify the query such that I can find the time difference as well?
I run sacct with -j switch, for a specific job-id. Depending on other command line switches two completely different results are reported for the same job. Here are three examples. The second one shows different result than the other two.
attar#lh> sacct -a -s CA,CD,F,NF,PR,TO -S 2020-07-26T00:00:00 -E 2020-07-27T23:59:59 --format=JobId,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus -j 1401 JobID State Timelimit Start End Elapsed MaxRSS MaxVMSize NNodes NCPUS
------------ ---------- ---------- ------------------- ------------------- ---------- ---------- ---------- -------- ----------
1401 CANCELLED+ UNLIMITED 2020-07-26T20:45:31 2020-07-27T08:36:10 11:50:39 1 2
1401.batch COMPLETED 2020-07-26T20:45:31 2020-07-27T08:36:17 11:50:46 103856K 619812K 1 2
attar#lh> sacct -a -s CA,CD,F,NF,PR,TO -S 2020-07-26T00:00:00 -E 2020-07-26T23:59:59 --format=JobId,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus -j 1401
JobID State Timelimit Start End Elapsed MaxRSS MaxVMSize NNodes NCPUS
------------ ---------- ---------- ------------------- ------------------- ---------- ---------- ---------- -------- ----------
1401 NODE_FAIL UNLIMITED 2020-06-15T09:38:38 2020-07-26T00:17:26 40-14:38:48 1 2
attar#lh> sacct -a -s CA,CD,F,NF,PR,TO --format=JobId,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus -j 1401
JobID State Timelimit Start End Elapsed MaxRSS MaxVMSize NNodes NCPUS
------------ ---------- ---------- ------------------- ------------------- ---------- ---------- ---------- -------- ----------
1401 CANCELLED+ UNLIMITED 2020-07-26T20:45:31 2020-07-27T08:36:10 11:50:39 1 2
1401.batch COMPLETED 2020-07-26T20:45:31 2020-07-27T08:36:17 11:50:46 103856K 619812K 1 2
Why are the start/end times different for the same job? One reports 11 hours run-time and the other 40 days run-time!
Any of your insight is highly appreciated!
This would typically happen when two jobs have the same JobId. The sacct documentation says:
If Slurm job ids are reset, some job numbers will probably appear more than once in the accounting log file but refer to different jobs. Such jobs can be distinguished by the "submit" time stamp in the data records.
Try running the sacct command with the --duplicates option.
I have a question about Slurm sacct output.
JobID ReqCPUS ResvCPU AllocCPUS ResvCPURAW SystemCPU CPUTimeRAW TotalCPU UserCPU
------------ -------- ---------- ---------- ---------- ---------- ---------- ---------- ----------
6089463_1 10 00:03:30 10 210 00:02.727 4250 06:38.627 06:35.899
6089463_1.b+ 4 4 00:02.727 1700 06:38.627 06:35.899
Why one job has two different lines?
Why 2 line has different ReqCPUS value? A user requests 10 CPUs but used only 4 CPUs?
Thank you,
I have seen this question which indicates that the relationship between the wday and mday fields of a CRON schedule is an OR relationship. Say for example I want to schedule something for every Friday the 13th.
Rather than the expected result, the CRON
0 0 13 * 5
will give me all Fridays of every month, as well as every 13th of every month.
Is there any way to avoid this behavior and specify an AND relationship? (There seems to be mention of older versions using an AND relationship, however I would prefer to use a single tool with the ability to do both)
I guess, instead of specifying the wday (Friday=5), you'll just have to specify the months where the 13th is a Friday; so, for 2019:
0 0 13 9,12 * : do stuff, but avoid black cats
Or, eternally more elegant, create a small script:
$> cat /home/me/bin/test_friday_13
#!/bin/bash
if [ "$(date +'%A %d')" != "Friday 13" ]
then
exit 1
else
: do stuff, but avoid black cats
exit 0
fi
and make the crontab entry:
0 0 13 * * /home/me/bin/test_friday_13
The script approach has the added benefit that you can run it from the command line. (Note: do not forget to alter the weekday name in the script to reflect your environment.)
The cron-entry you are interested in is:
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7)
# | | | | |
# * * * * * command to be executed
0 0 13 * * [ $(date "+\%u") = "5" ] && command
This will execute the cronjob every 13th of the month. It will compute the day of the week, and test it if it is a Friday (5). If so, it will execute command.
Extended explanation:
If you want to have special conditions, you generally need to implement the conditions yourself with a test. Below you see the general structure:
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7)
# | | | | |
# * * * * * command to be executed
0 0 13 * * /path/to/testcmd && command
The command testcmd generally returns exit code 0 if the test is successful or 1 if it fails. If testcmd is successful, it executes command. A few examples that use similar tricks are in the following posts:
Linux crontab for nth Satuday of the month
Is it possible to execute cronjob in every 50 hours?
Cron expression to run every N minutes
how to set cronjob for 2 days?
The test you want to perform is written in /bin/sh as:
[ $(date "+%u") = 5 ]
Which makes use of the test command (see man test). This is POSIX compliant and will work with any shell that your cron might run in (sh,bash,dash,ksh,zsh). The command date "+%u" returns the weekday from 1 to 7 (Sunday is 7). See man date for more info.
So your cronjob would look like:
0 0 13 * * [ $(date "+\%u") = 5 ] && command
Note that we have to escape the <percent>-character.
A % character in the command, unless escaped with a backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input.
source: man 5 crontab
Unfortunately, CRON doesn't work the way you want
You can see all available options on below url
https://www.freeformatter.com/cron-expression-generator-quartz.html
So what you can do is the execute the cron on every 13th of the month but don't let the command run when its Friday
0 0 0 13 * ? * mybatchjob.sh
mybatchjob.sh
if [ `date +'%A'` = "Friday" ]; then
echo "Yep its Friday";
else
echo "Not Friday";
fi
This will make sure that the intended program only runs on Friday and the 13th
I have a markdown document I'm processing with the pandoc tool to generate HTML and PDF documents. I'm trying to include a table in the document. Regular markdown doesn't support tables, but pandoc does. I've tried copy-pasting the definition of a table from the pandoc documentation into my source document, but when running it through the pandoc program the resulting document is all crammed into one big table.
Can anyone show me a pandoc table that renders properly?
# Points about Tweedledee and Tweedledum
Much has been made of the curious features of
Tweedledee and Tweedledum. We propose here to
set some of the controversy to rest and to uproot
all of the more outlandish claims.
. Tweedledee Tweedledum
-------- -------------- ----------------
Age 14 14
Height 3'2" 3'2"
Politics Conservative Conservative
Religion "New Age" Syrian Orthodox
--------- -------------- ----------------
Table: T.-T. Data
# Mussolini's role in my downfall
--------------------------------------------------------------------
*Drugs* *Alcohol* *Tobacco*
---------- ------------- ----------------- --------------------
Monday 3 Xanax 2 pints 3 cigars,
1 hr at hookah bar
Tuesday 14 Adderall 1 Boone's Farm, 1 packet Drum
2 Thunderbird
Wednesday 2 aspirin Tall glass water (can't remember)
---------------------------------------------------------------------
Table: *Tableau des vices*, deluxe edition
# Points about the facts
In recent years, more and more attention has been
paid to opinion, less and less to what were formerly
called the cold, hard facts. In a spirit of traditionalism,
we propose to reverse the trend. Here are some of our results.
------- ------ ---------- -------
12 12 12 12
123 123 123 123
1 1 1 1
---------------------------------------
Table: Crucial Statistics
# Recent innovations (1): False presentation
Some, moved by opinion and an irrational lust for novelty,
would introduce a non-factual element into the data,
perhaps moving all the facts to the left:
------- ------ ---------- -------
12 12 12 12
123 123 123 123
1 1 1 1
---------------------------------------
Table: Crucial "Statistics"
# Recent innovations (2): Illegitimate decoration
Others, preferring their facts to be *varnished*,
as we might say, will tend to 'label' the columns
Variable Before During After
--------- ------ ---------- -------
12 12 12 12
123 123 123 123
1000 1000 1000 1000
----------------------------------------
# Recent innovations (3): "Moderate" decoration
Or, maybe, to accompany this 'spin' with a centered or centrist representation:
Variable Before During After
---------- ------- ---------- -------
12 12 12 12
123 123 123 123
1 1 1 1
-----------------------------------------
# The real enemy
Some even accompany these representations with a bit of leftwing
clap-trap, suggesting the facts have drifted right:
------------------------------------------------------
Variable Before During After
---------- ----------- ---------- -------
12 12 12 12
-- Due to
baleful
bourgeois
influence
123 123 123 123
-- Thanks
to the
renegade
Kautsky
1 1 1 1
-- All a
matter of
sound Party
discipline
-------------------------------------------------------
Table: *"The conditions are not ripe, comrades; they are **overripe**!"*
# The Truth
If comment be needed, let it be thus: the facts have drifted left.
------------------------------------------------------------------------
Variable Before During After
---------- ------------- ---------- ----------------------
12 12 12 12
(here's (due to (something to do
where the rot lapse of with Clinton and
set in ) traditional maybe the '60's)
values)
123 123 123 123
(too much (A=440?)
strong drink)
1 1 1 1
(Trilateral Commission?)
--------------------------------------------------------------------------
Table: *The Decline of Western Civilization*