I need to run my nodejs script for every second ,Similar to PHP cron jobs. I have tried some nodejs cron libraries like https://github.com/ncb000gt/node-cron but the issue was first run should be manual i:e I have to run the file with cron script for first time manually.
But in php cron jobs, they run by the server so if the apache server running script will automatically start and even if the script return an error for a cycle then script will run again from the beginning from the next cycle
So is there any way to achieve this in nodejs ?
You have two options:
using Node as a daemon, with something like Supervisord to run your node-cron script. This alternative is wasteful on resources such as RAM because Node and Supervisord are running all the time.
using the system's crontab, you can run your script like calling Node on the command line, such as * * * * node /path/to/your/script.js. This alternative is highly efficient but lacks some control, like being able to log the output in case of an error, although you could just pipe the output to a file: node script.js > logfile
Related
I am trying to build a shell script which can be called via a cronjob to trigger an npm nodejs application.
This is my start.sh shell script
#!/bin/bash
#!/usr/local/bin/npm
cd /home/lharby/sites/mysite
npm run start
If I cd to this folder and execute ./start.sh the command appears to run. (Path to bash and npm are both correct after checking which npm).
My cron job looks like this:
*/5 * * * * /home/lharby/sites/mysite/start.sh >> /home/lharby/sites/mysite/src/log/cron-errors.txt 2>&1
This is throwing an error and additionally I was lead to believe that using >> would append to the file, it seems to overwrite it each time.
My guess is that trying to run this command via cron it cannot access certain environment variables that are set up in my index.js
For example:
const config = {
access_token: process.env.NEXT_MASTODON_ACCESS_TOKEN,
client_key: process.env.NEXT_MASTODON_CLIENT_KEY,
client_secret: process.env.NEXT_MASTODON_CLIENT_SECRET,
timeout_ms: 60 * 1000,
api_url: 'https://botsin.space/api/v1/',
};
const M = new Mastodon(config);
I believe I see the same issue when I try to run node index.js from /home/sites/lharby/mysite/src/
As my package.json has this configuration:
"scripts": {
"start": "node ./src/index.js --experimental-modules",
"temp": "node ./src/temp.js --experimental-modules"
},
I was exploring looking at just trying to run the whole app passing in node index.js and passing in the argument flags but I need to be able to run the cron file invoking npm rather than node, as I guess that creates a wrapper and npm can access process.env variables.
From my cron-errors.txt file I am seeing this:
/home/lharby/sites/mysite/node_modules/mastodon-api/lib/mastodon.js:345
throw new Error('Mastodon config must include \'' + reqKey + '\' when using \'user_auth\'');
^
Error: Mastodon config must include 'access_token' when using 'user_auth'
at /home/lharby/sites/glyphbot/node_modules/mastodon-api/lib/mastodon.js:345:27
//
//
/home/lharby/sites/mysite/start.sh: line 4: npm: command not found
/bin/sh: 1: /home/sites/glyphbot/start.sh: not found
/bin/sh: 1: /home/sites/glyphbot/start.sh: not found
How can I ensure the crontab will invoke npm? I feel like I am doing everything correctly.
EDIT
My issues are:
How to run a node project using npm from a cron job?
Can the cronjob access environment variables from the npm command?
Should >> cron-errors.txt 2>&1 append the file rather than replace the content each time.
UPDATE 19.01.23
So I added exports for my variables to the .bashrc file. And when checking echo $NEXT_MASTODON_ACCESS_TOKEN I am seeing my string value.
Updated my .sh file so it now looks like this:
#!/bin/bash
node ./src/index.js --experimental-modules
And updated my cronjob to read this:
*/5 * * * * /home/lharby/sites/mysite/start.sh >> /home/lharby/sites/mysite/src/log/cron-errors.txt 2>&1
And I tried this also trying to bypass the shell script
*/5 * * * * /home/lharby/sites/mysite/src && /usr/local/bin/node index.js --experimental-modules >>/home/lharby/sites/mysite/src/log/cron-errors.txt 2>&1
It still failed with the same message being logged to the cron-errors.txt file.
However I am now able to run this command invoking node with argument flags (rather than using the npm command) So in the terminal I can type
node index.js --experimental-modules
As well as just running the .sh file I just don't understand why it is not passing this information to my cronjob.
I don't understand how if my code reads:
process.env.NEXT_MASTODON_ACCESS_TOKEN,
Will this get replaced or read by the bash export value instead?
Where are your environment variables being set up? It looks like your cron is at least running as expected, and your only problem is to get those env variables into the script correctly. My guess is that you need to use "export" (see bullet point 3)
I would debug this in a few different ways.
you should verify that it is the case that your index.js is not able to read your environment variables. I would recommend adding console.log(JSON.stringify(config, undefined, 2)) to check this.
you should verify that the SHELL has access to those variables before it runs the script. for this just run echo $NEXT_MASTODON_ACCESS_TOKEN (and similar for each variable) to verify if that is the case.
environment variables are a little funny. Assuming you set these values in your .bashrc with NEXT_MASTODON_ACCESS_TOKEN="whateverTheValueIs", just setting variables like this only affects your current process. In order for sub-processes to have the variable you need to export: export NEXT_MASTODON_ACCESS_TOKEN="insertValueHere"
Hope this helps!
Edit:
It's a little unclear what your current issues are, so it might be helpful if you bullet point each issue you are trying to solve
I have a job with Nodejs that I want to do it each 30 minutes to scan the Database and Update Products Data in An Ecommerce API with my Nodejs Program, note that the Nodejs Program is serving an REST API (Backend) for a react js web application
So I searched for that and I found that I can do that with Nodejs Cron Library like "node-schedule" but I know that will be more interesting to do it with Linux Cron
var j = schedule.scheduleJob('42 * * * *', function(){
console.log('The answer to life, the universe, and everything!');
});
Is there any library that can let me add Cron jobs to Linux using Nodejs Or would I do it with "fs" only? so I will open the cron job file and add my command?
The command crontab which is part of Vixie Cron allows you to create, edit and delete per-user cron entries.
Or if you are running as the root user, which you should not be doing, you can drop cron files into /etc/cron.d
This is not always supported, and if you're running in a Docker type container environment it is doubtful that you have any cron at all. In that environment you'd want your running Nodejs to handle scheduled jobs for you. Or use some other kind of distributed scheduled work system.
You can put your cron job to a nodejs script. Then adding to the crontab can be done with cronbee module, via API:
import { cronbee } from 'cronbee'
await cronbee.ensure({
taskName: 'do smth',
taskRun: `node my-script`,
cron: '42 * * * *'
})
or you can ensure the cron job via CLI, if the module is installed globally or from npm scripts:
$ cronbee ensure mytasks.json
I run one of my node.js command using the following command:
pm2 start sample.js --cron "0 1 * * *" -- SAMP
But the problem is this program run twice. First one is when I execute the command and send one is at 1:00 AM(Which we want).
So my query is how can set the cron so that this program runs at only once (At 1:00 AM only).
My suggestion to you is to use crontab for this task. This is exactly what cron was designed for, and not what pm2 was designed for.
As the comment above states, the --cron option only specifies when the app should be restarted, --cron cannot be used to schedule the running of an app. In the cases where we need to run a node app on a specific time table and don't need all of the fancy pm2 capabilities of auto-restarting and clustering, we simply use crontab.
I am working with Ubuntu 16.04 and I have two shell scripts:
run_roscore.sh : This one fires up a roscore in one terminal.
run_detection_node.sh : This one starts an object detection node in another terminal and should start up once run_roscore.sh has initialized the roscore.
I need both the scripts to execute as soon as the system boots up.
I made both scripts executable and then added the following command to cron:
#reboot /path/to/run_roscore.sh; /path/to/run_detection_node.sh, but it is not running.
I have also tried adding both scripts to the Startup Applications using this command for roscore: sh /path/to/run_roscore.sh and following command for detection node: sh /path/to/run_detection_node.sh. And it still does not work.
How do I get these scripts to run?
EDIT: I used the following command to see the system log for the CRON process: grep CRON /var/log/syslog and got the following output:
CRON[570]: (CRON) info (No MTA installed, discarding output).
So I installed MTA and then systemlog shows:
CRON[597]: (nvidia) CMD (/path/to/run_roscore.sh; /path/to/run_detection_node.sh)
I am still not able to see the output (which is supposed to be a camera stream with detections, as I see it when I run the scripts directly in a terminal). How should I proceed?
Since I got this working eventually, I am gonna answer my own question here.
I did the following steps to get the script running from startup:
Changed the type of the script from shell to bash (extension .bash).
Changed the shebang statement to be #!/bin/bash.
In Startup Applications, give the command bash path/to/script to run the script.
Basically when I changed the shell type from sh to bash, the script starts running as soon as the system boots up.
Note, in case this helps someone: My intention to have run_roscore.bash as a separate script was to run roscore as a background process. One can run it directly from a single script (which is also running the detection node) by having roscore& as a command before the rosnode starts. This command will fire up the master as a background process and leave the same terminal open for following commands to be executed.
If you could install immortal you could use the require option to start in sequence your services, for example, this is could be the run config for /etc/immortal/script1.yml:
cmd: /path/to/script1
log:
file: /var/log/script1.log
wait: 1
require:
- script2
And for /etc/immortal/script2.yml
cmd: /path/to/script2
log:
file: /var/log/script2.log
What this will do it will try to start both scripts on boot time, the first one script1 will wait 1 second before starting and also wait for script2 to be up and running, see more about the wait and require option here: https://immortal.run/post/immortal/
Based on your operating system you will need to configure/setup immortaldir, her is how to do it for Linux: https://immortal.run/post/how-to-install/
Going more deep in the topic of supervisors there are more alternatives here you could find some: https://en.wikipedia.org/wiki/Process_supervision
If you want to make sure that "Roscore" (whatever it is) gets started when your Ubuntu starts up then you should start it as a service (not via cron).
See this question/answer.
I currently have an api test script using supertest + mocha + grunt.
I would like to run this script every 1 minute. How is it possible
You can use linux cron job or node module schedular
https://github.com/node-schedule/node-schedule
TO run your script once in a minute.