cron job randomly does not run until crond restarted - cron

I got similar cronjob in hundreds of VM with same enviornment and permission set by Chef.
On some VMs like 1% of total, Cronjob does not run or at least it does not log any entry in /var/log/cron or /var/log/messages if it was ran or not.
It is set to run each 3 minutes which ran without any issue on other VMs but randomly it need restart of cron to ran it, Even reload does not work.
It is strange and cant find any solution as to why it happens.
Any debugging option for cron on fly to use to check?
Tried to look into /var/log/messages no errors
tried to look into /var/log/cron no erros
tried to look into /var/log/sa no resource issue
tried to look into /var/log/zypper no updates
added new cron to the user and it ran without any issue and without even restarting/reloading.

Related

Run a command every 5 minutes and stop it at the end of that time

I have to run NodeJS application which has to run uninterruptedly, but I need to stop it and restart it every 10 minutes.
I'm Working on a dedicated Ubuntu 18.04 machine.
I have read something about cron but I don't know well how it works. Does it stop the command that was run with it?
You can use forever to run the application, and set up a cron job to restart it in forever every five minutes. As a bonus, forever will also restart it for you if it fails in between cron job executions.
In my knowledge, the cron doesn't stop the command that had run before, but you could add the logic to do that check in your application.
Here you can find an article that shows some solutions to prevent duplicate cron job executions.

Trying to write a shell script to monitor when a service stops in linux, and to automate the restart of this service

So I am relatively new to Centos, version 6.2. I have a service that needs to be mnonitored as a cron job, and if it stops needs to be restarted. I have a few ideas on how to monitor it, but when it comes to getting it restarted thats when I get stuck. I also know the PiD of the service I want to monitor.
You can use supervise for this: http://cr.yp.to/daemontools/supervise.html
Put it in your crontab to launch on system start:
#reboot supervise foo

Start and Stop script of ubuntu 12.04

I have a script (twoRules.sh) which add rules to ovs plugin bridge.
The rules gets deleted when someone does service neutron-plugin-openvswitch-agent restart or reboots the system. So where should I put my scripts so that after the restart of neutron-plugin-openvswitch-agent the (twoRules.sh) scripts get executed successfully and rules remain added.
I tried putting it in /etc/init.d/neutron-plugin-openvswitch-agent file as other people suggested but this file is only called on /etc/init.d/neutron-plugin-openvswitch-agent restart and not on service neutron-plugin-openvswitch-agent restart.
You have to convert the script to a a SysV-style init script. There are many documents out there explaining about this.
http://www.debian-administration.org/article/28/Making_scripts_run_at_boot_time_with_Debian
http://www.cyberciti.biz/tips/how-to-controlling-access-to-linux-services.html
https://wiki.debian.org/Daemon
This way you can configure the script to be executed after certain services start or stop or when runlevel changes.

Problems when executing services within cron in a Docker container?

I have some cronjobs that I always used and worked fine. But now, trying to move everything to Docker containers, I'm running into these errors:
/usr/bin/service: 127: /usr/bin/service: stop: not found
/usr/bin/service: 128: exec: start: not found
They occur when executing things like "service restart nginx" these cronjobs. Note that the same commands work fine outside the cronjobs.
The PATH is correctly set in /etc/crontab . Adding it to the individual cronfiles in /etc/cron.d doesn't work either. I also tried changing SHELL=/bin/sh to SHELL=/bin/bash (even though it's insecure, but wanted to try) in /etc/crontab, didn't work.
Any ideas?
I solved it changing the command from
"service start mysql"
to
"/sbin/start mysql &"
Good luck
Enric
I am not sure, but I think that the problem is in paths. I believe you want to run /usr/sbin/service and not /usr/bin/service. You may try to specify full path for service instead of just service.

Disable logging in a node.js script running with forever

I am continually running a few server scripts (on different ports) with nodejs using forever.
There is a considerable amount of traffic on some of these servers. The console.log commands I have for tracking connection anomalies result in bloated log files that I don't need all of the time - only for debugging. I have been manually stopping the scripts late at night, truncating the files, and restarting them. This won't do for long term, so we decided to find a solution.
Someone else on my system deleted the log files I had set up for each of the servers without my knowledge. Calling forever list on the command line shows that the server scripts are still running but now I can't tail the log files to see how the nodes are doing.
Node downtime should be kept to a bare minimum, so I'm hesitant to stop the servers during daylight hours for longer than a few minutes. Initial testing from the client side seems to indicate that the scripts are doing fine, but I can't be 100% sure there are no errors due to failed attempts at logging to a nonexistent file.
I have a few questions actually:
Is it ok to keep forever running like this?
If not, is there a proper way to disable logging? The github repository seems to indicate that forever will still log to a default file, which I don't want. Otherwise I may just write a cronjob that periodically stops scripts, truncates logs, then restarts the scripts.
What happens if I just create the logfile again with something like touch logfile_name.log while the script is still running - will this make forever freak out or is this a plausible solution?
Thanks for your time.
according to https://github.com/foreverjs/forever, try to pass -s to silent all log.
forever -s start YOURSCRIPT
Surely, before doing this, try to update forever to the latest:
sudo curl -L https://npmjs.com/install.sh | sudo sh
sudo npm update -g.
1) Just build in a periodic function or admin option to clear the forever logs. From the manual forever cleanlogs
2) At least for linux. Send each log file to /dev/null. Each log type is specified by options -l -o and -r. The -a option for append log, will stop it complaining about the log already existing.
forever start -a -l /dev/null -o /dev/null -r /dev/null your-server.js
Perhaps employ your own logging system, I use log4js, it doesn't complain if I delete the log file while the node process is still running.
There's a nifty tool that can help you that called logrotate. Have a look here
Especially the copytruncate option, it is very useful in your case.

Resources