I'm having this weird situation where my Cron Job is successfully executing a function that returns a promise, however, when it tries to execute a .find() on the Model, it never actually executes. I have this same function used elsewhere in my app and is called via an API call and returns no problem. Is there something I'm missing?
Here is my cron script:
var CronJob = require('node-cron');
var TradeService = require('../services/TradeService');
// Setup the cron job to fire every second
CronJob.schedule('* * * * * *', function() {
console.log('You will see this message every second');
TradeService.executePendingTrades();
}, null, true, 'America/Los_Angeles');
Here are the related functions that get called:
exports.executePendingTrades = () => {
// Get all pending trades
exports.getPendingTrades().then(results => {
console.log('results', results); // This never fires
})
}
exports.getPendingTrades = () => {
return new Promise((resolve, reject) => {
Trades.find({})
.where('is_canceled').equals('false')
.where('is_completed').equals('false')
.sort('-created_at')
.exec( (err, payload) => {
if (err) {
return reject(err); // This never fires
}
return resolve(payload); // This never fires
})
});
}
This is a shot in the dark, but make sure you are starting a database connection in your CRON job. Otherwise you won't be able to execute any queries.
Related
I have not been able to test this function with Jest, I would appreciate your help. Thank you.
/**
* #function now
* #description When a Bash script is executed, it instantly displays the responses that appear on the screen.
* #param {string} script Bash script
* #example now('echo "Hello World!"')
*/
function now(script) {
let execute = exec(script)
execute.stdout.on('data', (data) => {
if (data.charAt(data.length - 1) === '\n') {
console.log(data.toString().slice(0, -1))
} else {
console.log(data.toString())
}
})
}
Since exec executes asynchronously, you'll need to change the now function a bit in order to test it. As it's written, you don't have a way for the caller to know when now is finished. The simplest ways to change this are:
Make it return a Promise that resolves when it's done
Make it take a callback that is called when it's done
Promise
function now(script) {
return new Promise((resolve, reject) => {
let execute = exec(script)
execute.stdout.on('data', (data) => {
if (data.charAt(data.length - 1) === '\n') {
console.log(data.toString().slice(0, -1))
} else {
console.log(data.toString())
}
})
// Resolve the Promise when the shell exits
execute.on('exit', resolve);
// Error handling omitted for brevity, you can call reject() on failure
});
}
Here's the test. There are various ways to check that console.log is called. This one uses a spy but you could also inject a log function and default it to console.log.
it('logs from shell command', async () => {
jest.spyOn(console, "log");
await greet('echo "foo"');
expect(console.log).toBeCalledWith('foo');
});
Callback
function now(script, callback) {
let execute = exec(script)
execute.stdout.on('data', (data) => {
if (data.charAt(data.length - 1) === '\n') {
console.log(data.toString().slice(0, -1))
} else {
console.log(data.toString())
}
})
// Resolve the Promise when the shell exits
execute.on('exit', callback);
// Error handling omitted for brevity, you can pass an error to the callback on failure
}
The expect needs to be in the callback so the test waits until the shell has exited. Note that you need to call done for the test to finish. Promises are generally more ergonomic, but callbacks have their uses as well so I included them both.
it('logs from shell command', (done) => {
jest.spyOn(console, "log");
greet('echo "foo"', () => {
expect(console.log).toBeCalledWith('foo');
done()
});
});
Using information from #helloitsjoe I got what I was looking for.
Modify the promise so that it correctly displays the messages on the screen.
function now(script) {
return new Promise((resolve, reject) => {
let execute = exec(script)
execute.stdout.on('data', (data) => {
console.log(data.toString().slice(0, -1))
process.stdout.cursorTo(0)
})
execute.on('exit', resolve);
})
}
I use the test with the promise of #helloitsjoe, for it to work I need to install npm i -D regenerator-runtime and import it with import 'regenerator-runtime/runtime'.
Thank you very much #helloitsjoe.
I'm testing my API with supertest package, I do have a business logic that when a property qrCodeDate is less than Date.now() the User status becomes inactive.
import { CronJob } from 'cron';
export default (Model) =>
new CronJob(
'0 1 * * *',
async function () {
await Model.updateMany(
{ qrCodeDate: { $lt: new Date(Date.now()) } },
{ $set: { status: 'INACTIVE', qrCodeDate: null } }
);
},
null
);
On userModel.ts:
// Cron Job to verify if Date.now() > qrCodeDate
const job = CronJob(UserModel);
job.start();
And in users.test.ts I doing something like this: expect(User['status']).toBe('INACTIVE');
On my app, I got those CronJob running everyday checking the condition previously mentioned, anyway, in my test I don't know how to implement, also need the commented line for TS types
import { CronJob } from 'cron';
jest.mock('cron');
xtest('Check if when QR Code expires, ordinary status back to INACTIVE', async () => {
// (CronJob as unknown as jest.Mock)
expect(User['status']).toBe('INACTIVE');
});
Thanks in advance.
You do not need to test the cron module in your project.
You just need to test the logic inside the function where you are calling Model.updateMany.
You can just declare that function separately not in the argument, give it a name and write tests for that function
I'm writing an HTTP API with expressjs in Node.js and here is what I'm trying to achieve:
I have a regular task that I would like to run regularly, approx every minute. This task is implemented with an async function named task.
In reaction to a call in my API I would like to have that task called immediately as well
Two executions of the task function must not be concurrent. Each execution should run to completion before another execution is started.
The code looks like this:
// only a single execution of this function is allowed at a time
// which is not the case with the current code
async function task(reason: string) {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
// call task regularly
setIntervalAsync(async () => {
await task("ticker");
}, 5000) // normally 1min
// call task immediately
app.get("/task", async (req, res) => {
await task("trigger");
res.send("ok");
});
I've put a full working sample project at https://github.com/piec/question.js
If I were in go I would do it like this and it would be easy, but I don't know how to do that with Node.js.
Ideas I have considered or tried:
I could apparently put task in a critical section using a mutex from the async-mutex library. But I'm not too fond of adding mutexes in js code.
Many people seem to be using message queue libraries with worker processes (bee-queue, bullmq, ...) but this adds a dependency to an external service like redis usually. Also if I'm correct the code would be a bit more complex because I need a main entrypoint and an entrypoint for worker processes. Also you can't share objects with the workers as easily as in a "normal" single process situation.
I have tried RxJs subject in order to make a producer consumer channel. But I was not able to limit the execution of task to one at a time (task is async).
Thank you!
You can make your own serialized asynchronous queue and run the tasks through that.
This queue uses a flag to keep track of whether it's in the middle of running an asynchronous operation already. If so, it just adds the task to the queue and will run it when the current operation is done. If not, it runs it now. Adding it to the queue returns a promise so the caller can know when the task finally got to run.
If the tasks are asynchronous, they are required to return a promise that is linked to the asynchronous activity. You can mix in non-asynchronous tasks too and they will also be serialized.
class SerializedAsyncQueue {
constructor() {
this.tasks = [];
this.inProcess = false;
}
// adds a promise-returning function and its args to the queue
// returns a promise that resolves when the function finally gets to run
add(fn, ...args) {
let d = new Deferred();
this.tasks.push({ fn, args: ...args, deferred: d });
this.check();
return d.promise;
}
check() {
if (!this.inProcess && this.tasks.length) {
// run next task
this.inProcess = true;
const nextTask = this.tasks.shift();
Promise.resolve(nextTask.fn(...nextTask.args)).then(val => {
this.inProcess = false;
nextTask.deferred.resolve(val);
this.check();
}).catch(err => {
console.log(err);
this.inProcess = false;
nextTask.deferred.reject(err);
this.check();
});
}
}
}
const Deferred = function() {
if (!(this instanceof Deferred)) {
return new Deferred();
}
const p = this.promise = new Promise((resolve, reject) => {
this.resolve = resolve;
this.reject = reject;
});
this.then = p.then.bind(p);
this.catch = p.catch.bind(p);
if (p.finally) {
this.finally = p.finally.bind(p);
}
}
let queue = new SerializedAsyncQueue();
// utility function
const sleep = function(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
// only a single execution of this function is allowed at a time
// so it is run only via the queue that makes sure it is serialized
async function task(reason: string) {
function runIt() {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
return queue.add(runIt);
}
// call task regularly
setIntervalAsync(async () => {
await task("ticker");
}, 5000) // normally 1min
// call task immediately
app.get("/task", async (req, res) => {
await task("trigger");
res.send("ok");
});
Here's a version using RxJS#Subject that is almost working. How to finish it depends on your use-case.
async function task(reason: string) {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
const run = new Subject<string>();
const effect$ = run.pipe(
// Limit one task at a time
concatMap(task),
share()
);
const effectSub = effect$.subscribe();
interval(5000).subscribe(_ =>
run.next("ticker")
);
// call task immediately
app.get("/task", async (req, res) => {
effect$.pipe(
take(1)
).subscribe(_ =>
res.send("ok")
);
run.next("trigger");
});
The issue here is that res.send("ok") is linked to the effect$ streams next emission. This may not be the one generated by the run.next you're about to call.
There are many ways to fix this. For example, you can tag each emission with an ID and then wait for the corresponding emission before using res.send("ok").
There are better ways too if calls distinguish themselves naturally.
A Clunky ID Version
Generating an ID randomly is a bad idea, but it gets the general thrust across. You can generate unique IDs however you like. They can be integrated directly into the task somehow or can be kept 100% separate the way they are here (task itself has no knowledge that it's been assigned an ID before being run).
interface IdTask {
taskId: number,
reason: string
}
interface IdResponse {
taskId: number,
response: any
}
async function task(reason: string) {
console.log("do thing because %s...", reason);
await sleep(1000);
console.log("done");
}
const run = new Subject<IdTask>();
const effect$: Observable<IdResponse> = run.pipe(
// concatMap only allows one observable at a time to run
concatMap((eTask: IdTask) => from(task(eTask.reason)).pipe(
map((response:any) => ({
taskId: eTask.taskId,
response
})as IdResponse)
)),
share()
);
const effectSub = effect$.subscribe({
next: v => console.log("This is a shared task emission: ", v)
});
interval(5000).subscribe(num =>
run.next({
taskId: num,
reason: "ticker"
})
);
// call task immediately
app.get("/task", async (req, res) => {
const randomId = Math.random();
effect$.pipe(
filter(({taskId}) => taskId == randomId),
take(1)
).subscribe(_ =>
res.send("ok")
);
run.next({
taskId: randomId,
reason: "trigger"
});
});
I'm using NodeJS Agenda module for job scheduling.
In agenda there is an event 'ready'
module.exports = function(agenda) {
agenda.define('test', function(job, done) {
});
agenda.on('ready', function() {
agenda.every('*/5 * * * * *', 'test');
agenda.start();
});
}
Here inside on ready event I'm using an anonymous function, but I don't want to use anonymous function, want to create a normal function.
for example
module.exports = function(agenda) {
agenda.define('test', function(job, done) {
});
agenda.on('ready', startJob());
}
function startJob(agenda) {
agenda.every('*/5 * * * * *', 'test');
agenda.start();
}
but it's not working getting
Cannot read property 'start' of undefined
The problem is that you're directly invoking the function without passing the agenda-object. One way to solve this, would be:
agenda.on('ready', () => { startJob(agenda); });
Or if you want to pass the startJob-function as the callback, you'd need to bind the agenda-object to it:
agenda.on('ready', startJob.bind(this, agenda));
I would like to test an application that uses Kue so that job queue is empty before each test and cleared after each test. Queue should be fully functional and I need to be able to check status of jobs that are already in the queue.
I tried mock-kue and it worked well until I had to get jobs from the queue and analyze them. I couldn't get it to return jobs by job ID.
Situations that I need to be able to test:
Something happens and there should be a job of a given type in the queue,
Something happens and produces a job. Something else happens and that job gets removed and replaced with another job (rescheduling or existing job).
Seams straightforward, but I have hard time wrapping my head around the problem. All pointers are welcome.
In my experience it's more straightforward to simply have redis running on localhost wherever you want to run your tests rather than dealing with a mocked version of kue.
First, to make sure kue is empty before each test it could be as simple as flushing redis, eg:
var kue = require('kue');
var queue = kue.createQueue();
queue.client.flushdb(function(err) {});
For #1, kue has a rangeByType() method that should solve your problem:
var getJobs = function(type, state, cb) {
kue.Job.rangeByType(type, state, 0, -1, 'asc', cb);
}
// After something happens
getJobs('myJobType', 'active', function(err, jobs) {});
For #2, you can use the same method and simply keep track of the job id to know that it has been replaced:
var jobId;
getJobs('myJobType', 'active', function(err, jobs) {
assert.lengthOf(jobs, 1);
jobId = jobs[0].id;
});
// After the thing happens
getJobs('myJobType', 'active' function(err, jobs) {
assert.lengthOf(jobs, 1);
assert.notEqual(jobId, jobs[0].id);
});
And if you ever need to query a job by ID you can do it like so:
kue.Job.get(jobId, function(err, job) {});
Take a look at the kue-mock lib, it is more likely for integration testing than unit.
The library doesn't hack on any kue's internals (replacing/overriding methods etc.). Instead, it creates the original queue instance with a separate redis namespace, then, when stubbing, it creates job process handlers on the fly, putting its own implementation that gives you the ability to control the job processing behaviour.
Example usage:
const expect = require('chai').expect;
const kue = require('kue');
const KueMock = require('kue-mock');
const $queue = new KueMock(kue);
const app = require('./your-app-file');
describe('functionality that deals with kue', () => {
before(() => $queue.clean());
afterEach(() => $queue.clean());
it('enqueues a job providing some correct data', () => {
let jobData;
$queue.stub('your job type', (job, done) => {
jobData = job.data;
done();
});
return yourJobRunnerFunction()
.then(() => {
expect(jobData).to.be.an('object')
.that.is.eql({ foo: 'bar' });
});
});
describe('when the job is completed', () => {
beforeEach(() => {
$queue.stub('your job type')
.yields(null, { baz: 'qux' });
});
it('correctly handles the result', () => {
return yourJobRunnerFunction()
.then((result) => {
expect(result).to.eql({ baz: 'qux' });
});
});
// ...
});
describe('when the job is failed', () => {
beforeEach(() => {
$queue.stub('your job type')
.yields(new Error('Oops!'));
});
it('correctly handles the job result', () => {
return yourJobRunnerFunction()
.catch((err) => {
expect(err).to.be.an('error')
.with.property('message', 'Oops!');
});
});
// ...
});
});