I'm building an app with Nextjs and I'm using pages/api directory for my api endpoints. The server entry/root is hidden for me. How do I immediately run the scheduled task with node-cron when my app is deployed then?
I had the same problem. What you can do are scheduled tasks with the node-cron library. You have to put your task which you want to schedule in your next.config.js file like so:
/** #type {import('next').NextConfig} */
const cron = require('node-cron');
cron.schedule('* * * * *', function () {
console.log('Say scheduled hello')
});
const nextConfig = {
reactStrictMode: true,
}
module.exports = nextConfig
It is recommended that you keep the next.config.js pretty simple as stated here: https://github.com/vercel/next.js/issues/5318#issuecomment-540629772
So if using a custom server inside next.js is not recommended either, the best solution would be to use a third party or boot up a second server
You can use their Custom Server hook to include your task scheduling logic during the server initialization. But keep in mind that this is not recommended:
Before deciding to use a custom server please keep in mind that it should only be used when the integrated router of Next.js can't meet your app requirements. A custom server will remove important performance optimizations, like serverless functions and Automatic Static Optimization.
Related
I am making a web application that is scrapping the news from a news site and saves to my database (scrapping is done just for learning purpose). After the database is updated all the stored data is sent to the user frontend.
Here is the route responsible for the above action.
router.get('/news, postController.getNewsPost);
New news are added to the site being scrapped as the day passes. Lets say if no user logs in to my application my database does not updates because the route mentioned above does not fires.
I want my database to be updated periodically even when no users have logged into my web application.
I am new to backend development so please guide me on how i can achieve this, also let me know if more information is required.
The simplest solution would be to use a setInterval to execute a specific function every N seconds:
setInterval(function() {
// scrap news and save them to the database every 5 seconds
}, 5000)
More versatile and solid solutions can be implemented using an external library for scheduling task, something like Agenda
You can use node-cron module to schedule your scrapping job.
Install node-cron using npm:
$ npm install --save node-cron
Import node-cron and schedule your scrapping job:
var cron = require('node-cron');
cron.schedule('* * * * *', () => {
console.log('Scrapping....');
});
Here is the module link. https://www.npmjs.com/package/node-cron
You can use the following method
$ npm install --save node-cron
var cron = require('node-cron');
cron.schedule('second minute hour day month', () => {
//update database code
});
library link enter link description here
My question could be flagged as "opinion based" but I am wondering which approach is the best for my application as I am able to do it in both ways.
I am building chat application in which users and conversations are saved in MongoDB. I will have my react application consuming API/APIs. The question is - is it better to have REST API and Socket.io applications running separate? For example:
Have REST API running on port 3005
Have Socket.io running on port 3006
React Application consuming these 2 separately and basically they will not know about each other. My endpoints in REST API endpoints and socket.io will be invoked only in front-end.
On the other hand, I can have my socket.io application and REST API working together in 1 big application. I think it is possible to make it working without problems.
To sum up, at first glance I would take the first approach - more cleaner and easy to maintain. But I would like to hear other opinions or if somebody had a similar project. Usually how the things are made in this kind of projects when you have socket.io and REST API?
I would check the pros and cons for both scenario. For example code and resource reusability is better if you have a single application and you don't have to care about which versions are compatible with each other. On the other hand one error can kill both applications, so from security perspective it is better to have separate applications. I think the decision depends on what pros and cons are important to you.
you can make a separate file for socket.io logic like this:
// socket.mjs file
import { Server } from "socket.io"
let io = new Server()
const socketApi = {
io: io
}
io.on('connection',(socket)=>{
console.log('client connected:', socket.id)
socket.join('modbus-room')
socket.on('app-server', data=>{
console.log('**************')
console.log(data)
io.to('modbus-room').emit('modbus-client', data)
})
socket.on('disconnect',(reason)=>{
console.log(reason)
})
})
export default socketApi
and add it to your project like this:
// index.js or main file
//...
import socketApi from "../socket.mjs";
//...
//
/**
* Create HTTP server.
*/
const server = http.createServer(app);
socketApi.io.attach(server);
//
where is the start.js in the project which is generated by fastify-cli ?
i think its big different between the getting start example and the app generated by fastify-cli?
should i write the start function like this in the project created by fastify-cli?
const start = async () => {
try {
await sequelize.sync({})
app.log.info('database sync correctly')
await app.listen(PORT, '0.0.0.0')
app.swagger()
} catch (err) {
app.log.error(err)
process.exit(1)
}
}
start()
there are just a app.js in the project generated by fastify-cli.what a different!
where is the start.js in the project which is generated by fastify-cli ?
There is not, it is replaced by the CLI utility fastify your-file.js in the package.json (like mocha, jest etc.. does to run tests)
Usually the starter file is always the same, so it has been integrated in the cli and you can use the args to set the PORT or to reload the server automatically when you edit one file.
i think its big different between the getting start example and the app generated by fastify-cli?
The docs teach all you need to know about the framework, the plugins and utility around it want to ease the developer experience.. like manage a mongodb-connectio: it is one line with the official plugin.
should i write the start function like this in the project created by fastify-cli?
If you use fastify my-file.js you don't need it.
After some experience you will understand when you need the fastify-cli or not.
I think the cli is useful in the most use cases and it suggests good ways to implement configurations loading and encapsulation.
You won't need it for special use cases that need to run some async operation before the server is created
I have configured this cronjobe using node-cron in my node.js project
const cron = require('node-cron');
cron.schedule('0 0 * * *', () => {
console.log("CRON: Running");
// Do someting
});
and deployed project to Google APP engine, this cron normally runs on my local during testing at 24:00 no issue, but on Google APP engine it is not ...
Project itself normally deployed to server and running no issues (accessible through web) but cornjobe seems did not triggered at 24:00 by app engine server, i'm trying to understand why ?? Seems server was off at moment when no one is using it, or ??
This is because the App Engine instance is not necearily running at the time of the execution. (this happends more with Standard as it can scale down to 0)
Therefore a workaround for have Cronjobs in app engine is to have the schedule of this jobs managed by something else, in this case Google Cloud Scheduler which basically is a scheduler for tasks in app engine.
In This Guide You can see how to create cron job with the cloud Scheduler by the cloud console.
I have a route in my app that I've defined with tasks to be run in the style of a few cron jobs. I know that this can be triggered by a GET request from an external devise when necessary (and that's ideal). (FYI: I will be adding validations for security purposes to this route.)
router.get('/cron', function(req) {
/**
*
* Do cron things...
*/
task();
});
What I'm wondering is if I'd also be able to trigger this via a GET request from my own system when necessary?
What would be really helpful is to reuse the same route above with an npm module like node-crontab and simply make a request to the route every few hours.
var doEveryThirtyMinutes = crontab.scheduleJob("*/30 * * * *", function(){
/**
* Make GET request to '/cron' controller.
* Live a happy life.
*/
});
I can't find any information on how to make that request (to my same system), even in the npm request module documentation. Is there a reason not to do this? Am I missing something? Is this a bad practice?
The reason this setup would be incredibly beneficial is that I connect to my database via an extension of the req object and don't want to implement a new connection module. Also, I already have a logging procedure implemented for successful/ unsuccessful route executions, so I would be able to reuse that as well.
Thanks ahead of time for your help!
Yes, you can make a get request to your own application. You would make this request like any other request, just use your application's host and port.
If you want to grab the hostname from your os, you can do this with require('os').hostname
https://nodejs.org/api/os.html#os_os_hostname
The reason you wouldn't do this is you are already in your application, so you shouldn't need to communicate via it's network interface.