I'm running into issues with the node-schedule package.
I'm trying to run it with GCP cloud functions. The case where a job is scheduled and fired normally works fine. Cancelling a job also works.
However, when I try to reschedule it (by cancelling the job and scheduling a new one with the same id), it fires many times instead of just once.
Eg.: It is reschedule three times, but fires 15 times.
This is the code I have in my GCP function:
const functions = require('firebase-functions');
var schedule = require('node-schedule');
const {stringify} = require('flatted');
exports.jobScheduler = functions.https.onRequest(async (req, res) => {
try {
const jobId = req.body.id?.toString();
const type = req.body.type;
const date = new Date(req.body.date);
if (type === 'list_jobs') {
const scheduledJobs = schedule.scheduledJobs;
res.status(200).send(stringify(scheduledJobs));
} else {
if(type === 'schedule_job') {
const scheduledJob = schedule.scheduleJob(jobId, date, () => {
// API CALL
console.log(`SCHEDULED JOB IS RUNNING FOR ${jobId}`);
}
);
functions.logger.log(`Job scheduled for ${jobId}.`);
}
if (type === 'reschedule_job') {
const jobToReschedule = schedule.scheduledJobs[jobId];
if (jobToReschedule) {
jobToReschedule.cancel();
functions.logger.log(`Job canceled for a reschedule for ${jobId}.`);
const rescheduledJob = schedule.scheduleJob(jobId, date, () => {
//API CALL
console.log(`Rescheduled job is running for ${jobId}`);
}
);
functions.logger.log(`[Job rescheduled for ${jobId}.`);
} else {
functions.logger.log(`No pending job to reschedule for ${jobId}.`);
}
}
if (type === 'cancel') {
const jobToCancel = schedule.scheduledJobs[jobId];
if (jobToCancel) {
jobToCancel.cancel();
functions.logger.log(`Job canceled for ${jobId}.`);
} else {
functions.logger.log(`No pending job for ${jobId}.`);
}
}
res.status(200).send(`Job to ${type} notification for ${jobId} was successful.`);
}
} catch (error) {
functions.logger.log(error);
res.status(400).json(error);
}
});
I tried schedule.reschedule(), cancelling and creating a new schedule. I expect the job to be canceled and rescheduled for the new date, and for the job to fire only once (at the rescheduled date). There are some cases where a job might be rescheduled a lot of times. I expect it to only fire once (the last reschedule). However, the job fires more than once. In fact, it fires more than the number of times it was rescheduled.
I appreciate the help!
Related
I'm following this pattern in the docs https://docs.bullmq.io/patterns/process-step-jobs
But it seems like the example code throws a Missing lock for job <jobId> failed error. Following is a minimal reproducible version
import { Worker, Queue } from "bullmq";
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
enum Step {
Initial,
Second,
Finish,
}
type JobData = {
step: Step;
};
const worker = new Worker<JobData>("reproduce-error", async (job, token) => {
let step = job.data.step;
while (step !== Step.Finish) {
switch (step) {
case Step.Initial: {
await sleep(3 * 1000);
await job.moveToDelayed(Date.now() + 1000, token);
await job.update({
step: Step.Second,
});
await job.updateProgress(1);
step = Step.Second;
break;
}
case Step.Second: {
await sleep(3 * 1000);
await job.update({
step: Step.Finish,
});
await job.updateProgress(2);
step = Step.Finish;
return Step.Finish;
}
default: {
throw new Error("invalid step");
}
}
}
});
worker.on("error", (failedReason) => console.log(failedReason));
worker.on("progress", (job, progress) =>
console.log("progress", job.data, progress)
);
const queue = new Queue<JobData>("reproduce-error");
queue.add("Reproduce error", { step: Step.Initial });
Here's a stack trace of the error
at Scripts.finishedErrors (/Users/sumit/Coding/indiemaker/twips/reproduce-bullmq-error/node_modules/bullmq/src/classes/scripts.ts:356:16)
at Job.moveToFailed (/Users/sumit/Coding/indiemaker/twips/reproduce-bullmq-error/node_modules/bullmq/src/classes/job.ts:618:26)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at async handleFailed (/Users/sumit/Coding/indiemaker/twips/reproduce-bullmq-error/node_modules/bullmq/src/classes/worker.ts:642:11)
at async Worker.retryIfFailed (/Users/sumit/Coding/indiemaker/twips/reproduce-bullmq-error/node_modules/bullmq/src/classes/worker.ts:788:16)
at async Worker.run (/Users/sumit/Coding/indiemaker/twips/reproduce-bullmq-error/node_modules/bullmq/src/classes/worker.ts:385:34)
Here's what I've thought so far:
I suspect moveToDelayed moves the job to the delayed set but it keeps running the current execution. That's why it tries to move the same job to the delayed set twice and thus fails. But, changing the break to return in the Initial block should change that, but even that doesn't help and gives the same error. So there might be something more fundamental that's going wrong here.
Any help would be appreciated. Thanks in advance!
For the context, I'm new to Bull (https://github.com/OptimalBits/bull) and trying to execute a lot of jobs concurrently. But Bull seems it'll wait for one job to complete.
Here is my code
const Bull = require("bull");
const jobs = new Bull('jobs');
jobs.process(async (job) => {
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
console.log(`start job.id = ${job.id}`);
await wait(3000); // Wait 3 seconds.
console.log(`end job.id = ${job.id}`);
});
void async function main () {
for (let i = 0; i < 10; i++) {
jobs.add({});
}
}();
If I ran this code, it'll take 30 seconds to execute, one job after another. What am I doing wrong and how to make this concurrent?
I'm writing an HTTP API with expressjs in Node.js and here is what I'm trying to achieve:
I have a regular task that I would like to run regularly, approx every minute. This task is implemented with an async function named task.
In reaction to a call in my API I would like to have that task called immediately as well
Two executions of the task function must not be concurrent. Each execution should run to completion before another execution is started.
The code looks like this:
// only a single execution of this function is allowed at a time
// which is not the case with the current code
async function task(reason: string) {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
// call task regularly
setIntervalAsync(async () => {
await task("ticker");
}, 5000) // normally 1min
// call task immediately
app.get("/task", async (req, res) => {
await task("trigger");
res.send("ok");
});
I've put a full working sample project at https://github.com/piec/question.js
If I were in go I would do it like this and it would be easy, but I don't know how to do that with Node.js.
Ideas I have considered or tried:
I could apparently put task in a critical section using a mutex from the async-mutex library. But I'm not too fond of adding mutexes in js code.
Many people seem to be using message queue libraries with worker processes (bee-queue, bullmq, ...) but this adds a dependency to an external service like redis usually. Also if I'm correct the code would be a bit more complex because I need a main entrypoint and an entrypoint for worker processes. Also you can't share objects with the workers as easily as in a "normal" single process situation.
I have tried RxJs subject in order to make a producer consumer channel. But I was not able to limit the execution of task to one at a time (task is async).
Thank you!
You can make your own serialized asynchronous queue and run the tasks through that.
This queue uses a flag to keep track of whether it's in the middle of running an asynchronous operation already. If so, it just adds the task to the queue and will run it when the current operation is done. If not, it runs it now. Adding it to the queue returns a promise so the caller can know when the task finally got to run.
If the tasks are asynchronous, they are required to return a promise that is linked to the asynchronous activity. You can mix in non-asynchronous tasks too and they will also be serialized.
class SerializedAsyncQueue {
constructor() {
this.tasks = [];
this.inProcess = false;
}
// adds a promise-returning function and its args to the queue
// returns a promise that resolves when the function finally gets to run
add(fn, ...args) {
let d = new Deferred();
this.tasks.push({ fn, args: ...args, deferred: d });
this.check();
return d.promise;
}
check() {
if (!this.inProcess && this.tasks.length) {
// run next task
this.inProcess = true;
const nextTask = this.tasks.shift();
Promise.resolve(nextTask.fn(...nextTask.args)).then(val => {
this.inProcess = false;
nextTask.deferred.resolve(val);
this.check();
}).catch(err => {
console.log(err);
this.inProcess = false;
nextTask.deferred.reject(err);
this.check();
});
}
}
}
const Deferred = function() {
if (!(this instanceof Deferred)) {
return new Deferred();
}
const p = this.promise = new Promise((resolve, reject) => {
this.resolve = resolve;
this.reject = reject;
});
this.then = p.then.bind(p);
this.catch = p.catch.bind(p);
if (p.finally) {
this.finally = p.finally.bind(p);
}
}
let queue = new SerializedAsyncQueue();
// utility function
const sleep = function(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
// only a single execution of this function is allowed at a time
// so it is run only via the queue that makes sure it is serialized
async function task(reason: string) {
function runIt() {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
return queue.add(runIt);
}
// call task regularly
setIntervalAsync(async () => {
await task("ticker");
}, 5000) // normally 1min
// call task immediately
app.get("/task", async (req, res) => {
await task("trigger");
res.send("ok");
});
Here's a version using RxJS#Subject that is almost working. How to finish it depends on your use-case.
async function task(reason: string) {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
const run = new Subject<string>();
const effect$ = run.pipe(
// Limit one task at a time
concatMap(task),
share()
);
const effectSub = effect$.subscribe();
interval(5000).subscribe(_ =>
run.next("ticker")
);
// call task immediately
app.get("/task", async (req, res) => {
effect$.pipe(
take(1)
).subscribe(_ =>
res.send("ok")
);
run.next("trigger");
});
The issue here is that res.send("ok") is linked to the effect$ streams next emission. This may not be the one generated by the run.next you're about to call.
There are many ways to fix this. For example, you can tag each emission with an ID and then wait for the corresponding emission before using res.send("ok").
There are better ways too if calls distinguish themselves naturally.
A Clunky ID Version
Generating an ID randomly is a bad idea, but it gets the general thrust across. You can generate unique IDs however you like. They can be integrated directly into the task somehow or can be kept 100% separate the way they are here (task itself has no knowledge that it's been assigned an ID before being run).
interface IdTask {
taskId: number,
reason: string
}
interface IdResponse {
taskId: number,
response: any
}
async function task(reason: string) {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
const run = new Subject<IdTask>();
const effect$: Observable<IdResponse> = run.pipe(
// concatMap only allows one observable at a time to run
concatMap((eTask: IdTask) => from(task(eTask.reason)).pipe(
map((response:any) => ({
taskId: eTask.taskId,
response
})as IdResponse)
)),
share()
);
const effectSub = effect$.subscribe({
next: v => console.log("This is a shared task emission: ", v)
});
interval(5000).subscribe(num =>
run.next({
taskId: num,
reason: "ticker"
})
);
// call task immediately
app.get("/task", async (req, res) => {
const randomId = Math.random();
effect$.pipe(
filter(({taskId}) => taskId == randomId),
take(1)
).subscribe(_ =>
res.send("ok")
);
run.next({
taskId: randomId,
reason: "trigger"
});
});
I'm having this weird situation where my Cron Job is successfully executing a function that returns a promise, however, when it tries to execute a .find() on the Model, it never actually executes. I have this same function used elsewhere in my app and is called via an API call and returns no problem. Is there something I'm missing?
Here is my cron script:
var CronJob = require('node-cron');
var TradeService = require('../services/TradeService');
// Setup the cron job to fire every second
CronJob.schedule('* * * * * *', function() {
console.log('You will see this message every second');
TradeService.executePendingTrades();
}, null, true, 'America/Los_Angeles');
Here are the related functions that get called:
exports.executePendingTrades = () => {
// Get all pending trades
exports.getPendingTrades().then(results => {
console.log('results', results); // This never fires
})
}
exports.getPendingTrades = () => {
return new Promise((resolve, reject) => {
Trades.find({})
.where('is_canceled').equals('false')
.where('is_completed').equals('false')
.sort('-created_at')
.exec( (err, payload) => {
if (err) {
return reject(err); // This never fires
}
return resolve(payload); // This never fires
})
});
}
This is a shot in the dark, but make sure you are starting a database connection in your CRON job. Otherwise you won't be able to execute any queries.
I would like to test an application that uses Kue so that job queue is empty before each test and cleared after each test. Queue should be fully functional and I need to be able to check status of jobs that are already in the queue.
I tried mock-kue and it worked well until I had to get jobs from the queue and analyze them. I couldn't get it to return jobs by job ID.
Situations that I need to be able to test:
Something happens and there should be a job of a given type in the queue,
Something happens and produces a job. Something else happens and that job gets removed and replaced with another job (rescheduling or existing job).
Seams straightforward, but I have hard time wrapping my head around the problem. All pointers are welcome.
In my experience it's more straightforward to simply have redis running on localhost wherever you want to run your tests rather than dealing with a mocked version of kue.
First, to make sure kue is empty before each test it could be as simple as flushing redis, eg:
var kue = require('kue');
var queue = kue.createQueue();
queue.client.flushdb(function(err) {});
For #1, kue has a rangeByType() method that should solve your problem:
var getJobs = function(type, state, cb) {
kue.Job.rangeByType(type, state, 0, -1, 'asc', cb);
}
// After something happens
getJobs('myJobType', 'active', function(err, jobs) {});
For #2, you can use the same method and simply keep track of the job id to know that it has been replaced:
var jobId;
getJobs('myJobType', 'active', function(err, jobs) {
assert.lengthOf(jobs, 1);
jobId = jobs[0].id;
});
// After the thing happens
getJobs('myJobType', 'active' function(err, jobs) {
assert.lengthOf(jobs, 1);
assert.notEqual(jobId, jobs[0].id);
});
And if you ever need to query a job by ID you can do it like so:
kue.Job.get(jobId, function(err, job) {});
Take a look at the kue-mock lib, it is more likely for integration testing than unit.
The library doesn't hack on any kue's internals (replacing/overriding methods etc.). Instead, it creates the original queue instance with a separate redis namespace, then, when stubbing, it creates job process handlers on the fly, putting its own implementation that gives you the ability to control the job processing behaviour.
Example usage:
const expect = require('chai').expect;
const kue = require('kue');
const KueMock = require('kue-mock');
const $queue = new KueMock(kue);
const app = require('./your-app-file');
describe('functionality that deals with kue', () => {
before(() => $queue.clean());
afterEach(() => $queue.clean());
it('enqueues a job providing some correct data', () => {
let jobData;
$queue.stub('your job type', (job, done) => {
jobData = job.data;
done();
});
return yourJobRunnerFunction()
.then(() => {
expect(jobData).to.be.an('object')
.that.is.eql({ foo: 'bar' });
});
});
describe('when the job is completed', () => {
beforeEach(() => {
$queue.stub('your job type')
.yields(null, { baz: 'qux' });
});
it('correctly handles the result', () => {
return yourJobRunnerFunction()
.then((result) => {
expect(result).to.eql({ baz: 'qux' });
});
});
// ...
});
describe('when the job is failed', () => {
beforeEach(() => {
$queue.stub('your job type')
.yields(new Error('Oops!'));
});
it('correctly handles the job result', () => {
return yourJobRunnerFunction()
.catch((err) => {
expect(err).to.be.an('error')
.with.property('message', 'Oops!');
});
});
// ...
});
});