I used apt to install sge in my ubuntu 16.04 server.
Howerver, when I use qsub to submit a job, qstat -j "*" can only show error jobs and running jobs information.
Does sge has some command or options to show all jobs info（including success finished jobs）
For terminated jobs, qacct should be used.
E.g. to show information about all jobs in the account file, use
qacct -j "*"
The qacct man page gives further information:
http://gridscheduler.sourceforge.net/htmlman/htmlman1/qacct.html
Related
Nutch jobs failing after second round that is in fetch stage i am using emr cluster it is not throwing any error.May i know the reason.May I know what can be reasons it is stopping second round.
The reason is because i did not run nohup command I was previously running with
sh filename.sh it stopped after some crawls now i am running by using the nohup sh filename.sh &.
Thanks # Sebastian Nagel
I'm playing around with Container Optimized OS on Google Cloud and found that the 'Auto Update' feature doesn't apply the updates until the system is restarted, and doesn't offer any functionality for scheduling a reboot after an update is applied.
I'm writing a simple startup script that schedules a shutdown when a reboot is needed, essentially:
#!/usr/bin/env sh
update_engine_client --block_until_reboot_is_needed
shutdown -r 02:00
My question is: how do I determine whether a shutdown has been scheduled? I have tried three methods so far that don't work in this OS:
$ ps -ef | grep shutdown - no shutdown process
$ systemctl status systemd-shutdownd.service - Unit systemd-shutdownd.service could not be found.
cat /run/systemd/shutdown/scheduled - no file found
Documentation on this OS, and what it's based on, are slim. What determines how shutdown is scheduled, and how does COS handle it?
In regard to your question: how do I determine whether a shutdown has been scheduled?
There's no shutdown taks configured by default, you have to configure it (daily, weekly, monthly, etc.), the easier way to do this is by using "crond" (OS Linux task scheduler) please follow this guide to know how to configure jobs in cron(COS usually use Ubuntu OS).
According to this GCP guide: " Container-Optimized OS instances are configured to automatically download weekly updates in the background; only a reboot is necessary to use the latest updates."
So, I suggest you to configure your cron jobs weekly on no peak production days (Saturday or Sunday).
Please let me know if you have further questions.
In Container-Optimized OS, the following command will display pending shutdown information in epoch time:
$ busctl get-property org.freedesktop.login1 /org/freedesktop/login1 org.freedesktop.login1.Manager ScheduledShutdown
I am curious why Google chose to use busctl instead of systemd - I was unfamiliar with busctl and had to do some reading on it to understand what the command is doing - busctl man page
Example:
$ sudo shutdown -r 02:00
Shutdown scheduled for Fri 2020-07-17 02:00:00 UTC, use 'shutdown -c' to cancel.
$ busctl get-property org.freedesktop.login1 /org/freedesktop/login1 org.freedesktop.login1.Manager ScheduledShutdown
(st) "reboot" 1594951200000000
$ sudo shutdown -c
busctl get-property org.freedesktop.login1 /org/freedesktop/login1 org.freedesktop.login1.Manager ScheduledShutdown
(st) "" 0
I want to know what is the user who stopped a docker container.
There are several user accounts on my server. I suspect that one of them sometimes stops the container.
How can I find the user that performed this operation?
You can use su -c history username to check command history of a user, I don't know how many users you have but you could loop through them and grep for commands taking docker containers down.
You can install GNU Accounting Utilities, to be able to see commands executed by users:
#centos
yum install psacct
# ubuntu:
apt-get install acct
#Also make sure that the cooresponding service is enabled:
/etc/init.d/psacct status
Then, after you realize that the container is stopped execute:
lastcomm --command docker
# or
lastcomm --command kill
to see which executed the above command(s).
You can use the above in combination with:
docker container logs <name-of-the-container>
to see what is the exact time on which the container was stopped. (E.g. you may see a message on the logs: "stopping service..") and match it with lastcomm output.
Other useful commands that come with the above package:sa, ac
I'm trying to get airflow working to better orchestrate an etl process. When I make changes to a dag in my dags folder, I often have to restart the scheduler with
airflow scheduler
before the changes are visible in the UI. I would like to run the scheduler as a daemon process with
airflow scheduler -D
but we I try to do so, I get a message saying
[2018-10-17 14:13:54,769] {jobs.py:580} ERROR -
Cannot use more than 1 thread when using sqlite. Setting max_threads to 1
I think this error pops up because the scheduler is already running as a daemon. However, when I try to find out where the scheduler is being run with
lsof -i
I don't get any results.
Question: Why am I not able to restart the scheduler with airflow scheduler -D. Why does the scheduler restart with airflow webserver? How do I successfully kill the process that is preventing me to run airflow scheduler -D?
Run ps aux | grep airflow and check if airflow webserver or airflow scheduler processes are running. If they are kill them and rerun using airflow scheduler -D
You need to clear out the airflow-scheduler.pid file at $AIRFLOW_HOME. The stale pid file from the daemon will prevent you to start another scheduler process.
If you just restart your webserver, the dag changes gets reflected in UI. No need to restart scheduler for the same.
Applicable for 1.8 and 1.10.3. Cant comment for latest 1.10.10.
I use ubuntu:latest image, cron is already installed, because my cron job doesn't run at all. I want to check if the cron deamon is running in my docker container.
so I type in
service cron status
results in:
Rather than invoking init scripts through /etc/init.d, use the service(8) utility, e.g. service cron status
Since the script you are attempting to invoke has been converted to an Upstart job, you may also use the status(8) utility, e.g. status cron
gives me no result of the status of cron, why is that?
The service name is wrong. Here is the fix with crond
service crond status