I can use KNEX=debug:query npm start so I get logs and time how long queries took.
I would like to know where these queries are defined (function name or file:line). Due to knexjs beeing ORM I can't find it in code easily.
I've usually have done it like this:
orm.on("query", (q) => try { throw new Error() }catch(e){ console.log(query, e.stack) }
First question - why it's not built-in for debugging? Or maybe it is? Second - how can I achieve this?
Related
The lambda's job is to see if a query returns any results and alert subscribers via an SNS topic. If no rows are return, all good, no action needed. This has to be done every 10 minutes.
For some reasons, I was told that we can't have any triggers added on the database, and no on prem environment is suitable to host a cron job
Here comes lambda.
This is what I have in the handler, inside a loop for each database.
sequelize.authenticate()
.then(() => {
for (let j = 0; j < database[i].rawQueries[j].length; j++) {
sequelize.query(database[i].rawQueries[j] => {
if (results[0].length > 0) {
let message = "Temporary message for testing purposes" // + query results
publishSns("Auto Query Alert", message)
}
}).catch(err => {
publishSns("Auto Query SQL Error", `The following query could not be executed: ${database[i].rawQueries[j])}\n${err}`)
})
}
})
.catch(err => {
publishSns("Auto Query DB Connection Error", `The following database could not be accessed: ${databases[i].database}\n${err}`)
})
.then(() => sequelize.close())
// sns publisher
function publishSns(subject, message) {
const params = {
Message: message,
Subject: subject,
TopicArn: process.env.SNStopic
}
SNS.publish(params).promise()
}
I have 3 separate database configurations, and for those few SELECT queries, I thought I could just loop through the connection instances inside a single lambda.
The process is asynchronous and it takes 9 to 12 seconds per invocation, which I assume is far far from optimal
The whole thing feels very very sub optimal but that's my current level :)
To make things worse, I now read that lambda and sequelize don't really play well together:
I am using sequelize because that's the only way I could get 3 connections to the database in the same invocation to work without issues. I tried mssql and tedious packages and wasn't able with either of them
It now feels like using an ORM is an overkill for this very simple task of a SELECT query, and I would really like to at least have the connections and their queries done asynchronously to save some execution time
I am looking into different ways to accomplish this and i went down the rabbit hole and I now have more questions than before! Generators? are they still useful? Observables with RxJs? Could this apply here? Async/Await or just Promises? Do I even need sequelize?
Any guidance/opinion/criticism would be very appreciated
I'm not familiar with sequelize.js but hope I can help. I don't know your level with RxJS and Observables but it's worth to try.
I think you could definitely use Observables and RxJS.
I would start with an interval() that will run the code every time you define.
You can then pipe the interval since it's an Observable, do the auth bit and do a map() to get an array of Observables (for each .query call, I am assuming all your calls, authenticate and query, are Promises so it's possible to transform them into Observables with from()). You can then use something like forkJoin() with the previous array to get a response after all calls are done.
In the .subscribe at the end, you would make the publishSns().
You can pipe a catchError() too and process errors.
The map() part might be not necessary and do it previously and have it stored in a variable since you don't depend on an authenticate value.
I'm certain my solution isn't the only one or the best but i think it would work.
Hope it helps and let me know if it works!
Most of Jest's expect(arg1).xxxx() methods will throw an exception if the comparison fails to match expectations. One exception to this pattern seems to be the toMatchSnapshot() method. It seems to never throw an exception and instead stores the failure information for later Jest code to process.
How can we cause toMatchSnapshot() to throw an exception? If that's not possible, is there another way that our tests can detect when the snapshot comparison failed?
This will work! After running your toMatchSnapshot assertion, check the global state: expect(global[GLOBAL_STATE].state.snapshotState.matched).toEqual(1);
Just spent the last hour trying to figure it out for our own tests. This doesn't feel hacky to me either, though a maintainer of Jest may be able to tell me whether accessing Symbol.for('$$jest-matchers-object') is a good idea or not. Here's a full code snippet for context:
const GLOBAL_STATE = Symbol.for('$$jest-matchers-object');
describe('Describe test', () => {
it('should test something', () => {
try {
expect({}).toMatchSnapshot(); // replace with whatever you're trying to test
expect(global[GLOBAL_STATE].state.snapshotState.matched).toEqual(1);
} catch (e) {
console.log(`\x1b[31mWARNING!!! Catch snapshot failure here and print some message about it...`);
throw e;
}
});
});
If you run a test (e.g. /Foobar.test.js) which contains a toMatchSnapshot matcher jest by default will create a snapshot file on the first run (e.g. /__snapshots__/Foobar.test.js.snap).
This first run that creates the snapshot will pass.
If you want the test to fail you need to commit the snapshot alongside with your test.
The next test builds will compare the changes you make to the committed snapshot and if they differ the test will fail.
Here is the official link to the Documentation on 'Snapshot Testing' with Jest.
One, less than ideal, way to cause toMatchSnapshot to throw an exception when there is a snapshot mismatch is to edit the implementation of toMatchSnapshot. Experienced Node developers will consider this to be bad practice, but if you are very strongly motivated to have that method throw an exception, this approach is actually easy and depending on how you periodically update your tooling, only somewhat error-prone.
The file of interest will be named something like "node_modules/jest-snapshot/build/index.js".
The line of interest is the first line in the method:
const toMatchSnapshot = function (received, testName) {
this.dontThrow && this.dontThrow(); const
currentTestName = ....
You'll want to split that first line and omit the calling of this.dontThrow(). The resulting code should look similar to this:
const toMatchSnapshot = function (received, testName) {
//this.dontThrow && this.dontThrow();
const
currentTestName = ....
A final step you might want to take is to send a feature request to the Jest team or support an existing feature request that is of your liking like the following: link
I'm trying to do a simple command line database transformation with node.js and sequelize. I've simplified my errant code down to the following, but it never returns:
// Set up database connection and models
var models = require('../models_sequelize');
models.User.findOne()
.then(a => {
console.log(a.name);
});
I get a name printed, but then the script hangs. What is wrong? How do I debug this to see what's stuck? I get the impression that there's an orphan promise that's not being fulfilled, but I don't understand where or why. I must be missing something obvious.
If I run the same interactively from the node console, it returns fine.
Sirko's comment re: close() gave me something to go on. I can stop the hanging with the following code:
var models = require('../models_sequelize');
models.User.findOne()
.then(a => {
console.log(a.name);
models.sequelize.close();
})
Alternatively, this seems to work too as I guess it's doing exactly the same thing:
var models = require('../models_sequelize');
models.User.findOne()
.then(a => {
console.log(a.name);
})
.finally(() => {
models.sequelize.close();
});
I also found something about connection pooling timeouts, but I don't think that affects my simple use case. I imagine it'll come into play in more complicated examples.
Would still like to find a good reference as to why this is necessary rather than just my guess.
When I first used MongoDB I managed to connect successfully, however then I wanted to carry out the most basic query such as:
db.users.find()
I got an error saying TypeError: Cannot read property 'find' of undefined
Basically meaning I cannot use a collection as a property to the object db.
So i tried this:
var user_col = db.collection('users');
user.col.find();
which works absolutely fine.
Over the last few days I have kept having to look up other ways of doing things as the standard documented way doesn't seem to work. Just now I wanted to get the total users on the app, so like it says in the documentation I should do this:
var count = db.runCommand( { count: 'users' } );
console.log(count);
however this gave the error:
TypeError: undefined is not a function
Is there a problem with MongoDB you have seen like this before or am I just being stupid? I do not want to have to keep finding other, less efficient ways of doing things so finally I ask here what is going on.
Thank you.
It appears you are confusing the Mongo shell API with the node.js native driver API. While both are JavaScript, the shell is sync while node.js is async so they're totally different.
The basic idea is something like this:
model.find({}, function(err, docs) {
docs.forEach(function(doc) {
// update it with whatever new object
model.update({"category":"cat1"});
});
});
Now my question is any sort of update/save doesn't work. I can find, count, etc with the model. I've tried doc.save(), model.findOneAndUpdate(), using the $set parameter for particular fields, they don't work. I went to mongo shell and typed the update command with the parameter, it worked.
One time it did work, was when I tried to put new documents using var m = new model({..}) and m.save() from outside the callback function of find(). But I need to find documents by their fields, and update those.
I feel like I'm missing something really elementary, could someone help me out?
edit: mongoose.disconnect() was being called later on in the code. So query did not execute.
Another possibility cause to this problem is when you use Mixed types, as in Schema.Types.Mixed. If you only update the Mixed field, then Mongoose doesn't know you've updated that field. So you need to call markModified
I ran into this recently and it was a pain in the neck to find out what was going on.
Unless it's just a typo, you're doing an update on the wrong object. You probably want to do
model.find({}, function(err, docs) {
docs.forEach(function(doc) {
doc.update({/fields here/});
});
});
Above answer is incorrect.
Edit
After thinking a little about it, why are you doing find at all? You should be able to update your docoments without it. Check the API docs for update
model.update({}, {field1: 'updated value'}, function(err) {});