I recently changed my queries on the SQL database from
Model.findOne().populate("Model2").exec(function(err, result) {
})
to :
Model.query(sqlQuery, parameters, function(err, result) {
})
The reason I changed the queries are performances problems : it goes much faster using the second way than the first one (when you join tables).
My question is the following : Using the Waterline syntax, I was able to retrieve only the elements I wanted (elements defined in the model), but using the query method ("Select * from model ...") gives me the result I ask to the database, but I'd like to be able to filter the fields that are defined in my model. Is that possible?.
Your model definitions are on the sails.models.MODEL object.
You could reference the available attributes there.
Object.keys(sails.models.MODEL._attributes).join(',')
Related
I recently started using Node.js + Express.js (generated with pug) + pg-promise for handling db.
My first target is to obtain data from Postgres (already set up) and display it pretty using render and pug. Let's say it is user list from Users table.
On this restful tutorial I have learned how to get data and return it as JSON - it worked.
Based on Mozilla's tutorial I seperated my code:
routes/users.js: where for '/' I call user_controller.user_list method (using router.get)
controllers/userController.js I have exported user_list where I would like to ask model for data and call render if I have results
queries.js which is kinda my model? But I'm not sure. It has API: connection to db with promises and one function for every query I am going to use in Controllers. I believe I should have like one Model file per table (or any logical entity) but where to store pgp connections?
This file is based on first tutorial I mentioned
// queries.js (connectionString is set properly to my postgres)
var pgp = require('pg-promise')(options);
var db = pgp(connectionString);
function getUsers(req, res, next) {
db.any('SELECT (user_id, username) FROM public.users ORDER BY user_id ASC LIMIT 1000')
.then(function (data) {
res.json({ data: data });
})
.catch(function (err) {
return next(err);
});
}
module.exports = {
getUsers: getUsers
};
Here starts my problem as most tutorials uses mongoose which is very model-db-schema-friendly and what I have is simple 'SELECT ...' string I pass to pg-promise's any() function.
Therefore I have no model class like User.
In userControllers.js I don't know how to call getUsers() to handle its data. Returning JS object from getUsers() would be nice.
Also: where should I call render? In controller or only in
db.any(...).then(function (data) { <--here--> })
Before, I also tried to embed whole Postgres handling into Controller but from db.any() I got this array for handling:
[{ row: '(1,John)' },{ row: '(2,Amy)' },{ row: '(50,Peter)' } ]
Didn't know how go from there as I probably lost my API functionality as well ;-)
I am browsing through multiple tutorials how to handle MVC but usually they handle MongoDB and
satisfy readers with res.send() not render().
I am not sure that I understand what your question is exactly about, but since I do not have enough reputation to comment, I'll do my best to help you with your interrogations. :)
First, regarding the queries.js file, it is IMO not exactly a model, but rather a DAO (Data Access Object) file. DAO comes between you Model (which is actually you database) and your Controller layers. There usually is a DAO file per object (User, Pet, whatever you want) in your data model.
When the data model is rather complex, it can be useful to use an Object Relational Mapping (ORM) such as Mongoose to map your database and execute complexe processes on your objects. In such a case, you might need a specific file per object so as to describe your model and store your queries. But since you don't need an ORM, you DAO can directly interact with your database. That is why you do not have a User.js file.
Regarding the way the db object should be used, I think you should refer directly to pg-promise documentation on the matter.
IMPORTANT: For any given connection, you should only create a single
Database object in a separate module, to be shared in your application
(see the code example below). If instead you keep creating the
Database object dynamically, your application will suffer from loss in
performance, and will be getting a warning in a development
environment (when NODE_ENV = development)
As a matter of fact, a db object in pg-promise sort of represents the database itself and is actually designed for the simultaneous use of several databases, which does not seem to be your case for the moment.
Finally, when it comes to the render function, I believe it should be in the controller, as your DAO is not supposed to know how the data it has gathered is going to be used.
Modularity is always a time-saving choice on the long-term.
Furthermore, note that you might later need a Business Layer between your DAO and your controller, in order to preprocess and postprocess data you are going to persist or to display. In such a case, if you need for instance to ask for data from your database, you will need to render data after it is processed by the Business layer. If the render is made in the DAO layer, it will not be possible.
In the link I provided earlier to pg-promise's db object connection, you will also find documentation on the any() method. You might already have looked it up.
It specifically states that it returns
A promise object that represents the query result:
When no rows are returned, it resolves with an empty array.
When 1 or more rows are returned, it resolves with the array of rows.
so your returned data is a JS Array. If you want to make it a JS object, just use
JSON.stringify(yourArray) to process your data before rendering it in your controller.
But I wonder if Pug is not able to use your data directly.
Also, if you cannot get any data out of your DAO, maybe you should check that your data object is not empty, as such a case is tolerated by the any() method. If you expect your query to always return something, you might want to consider using the many() or the one() methods.
I hope this helps you.
we are trying to update records in association though sequelize but its only updating the records of model on which it is fired or direct model no associated model affected by the query .
any pathway to perform this
return db.product.findById( 75759,{
include: [
{
model: db.productDescription
}
]})
.then(product => {
var rahulpro = {};
Object.assign(rahulpro,product);
// console.log(rahulpro.product_description.description);
rahulpro.name = "44444";
rahulpro.product_description.dataValues.description = "rahul";
product.updateAttributes(rahulpro);
});
What do you expect to happen?
we want to update data of description model
What is actually happening?
Nothing changed database of description
Database version: mysql 5.7
Sequelize version: 4.8.3
I've had a very similar issue using Sequelize. After hours of looking into it, I've seen there is no great way to handle data attached to through tables.
For what you're trying to do, you'll have to perform 2 queries, find the product instance from your product table, and then query the product_description table where description.product_id === product.id.
Then write the changes to the description instance, then to the product instance.
This was a shortcoming of ORM's in my project, and for more complex queries we started using raw sequel because it gave us the ability to do things like this.
I am using node-postgres for a production application and I am wondering if there is anything I should be concerned about? Is the data sanitized automatically by node-postgres?
I couldn't find anything about it on the github page: https://github.com/brianc/node-postgres
Absolutely! The parameterized query support in node-postgres is first class. All escaping is done by the postgresql server ensuring proper behavior across dialects, encodings, etc... For example, this will not inject sql:
client.query("INSERT INTO user(name) VALUES($1)", ["'; DROP TABLE user;"], function (err, result) {
// ...
});
This is from their documentation.
It basically depends on how you execute your queries as #vitaly-t described
Suppose you will define query in a string and execute as follows:
var query = `SELECT * FROM table where username='${username}' and password='${password}`;
pool.query(query, (error, results) => {
});
This case if i would pass username=' 'or 1=1; -- and password=' 'or 1=1; --
Then it will return all records from the table (means SQL injection works)
But if I would execute the following query
pool.query('SELECT * FROM table where username=$1 and password=$2', [username, password], (error, results) => {
});
Then SQL injection will never work because pg will sanitize the data.
So it's depends on how you execute the queries.
It depends on how you execute your queries:
Formatting via Prepared Statements is executed by the server, which in turn sanitizes your query from any SQL injection. But it has other restrictions, like you cannot execute more than one query at a time, and you cannot provide sanitizied entity names when needed.
Client-side query formatting, like the one implemented by pg-promise, sanitizes values, plus offers flexibility in formatting entity names and multiple queries.
I'm working on a custom adapter in sails#0.10.0-rc4 which will support associations but I am having trouble getting them working in conjunction with my adapter. My configuration is a one-to-many association between article and stats. My models and adapter are setup like this:
// api/models/article.js
module.exports = {
connection: ['myadapter'],
tableName: 'Knowledge_Base__kav',
attributes: {
KnowledgeArticleId: { type: 'string', primaryKey: true }
stats: {
collection: 'stats',
via: 'parentId'
}
}
// api/models/stats.js
module.exports = {
connection: ['myadapter'],
tableName: 'KnowledgeArticleViewStat',
attributes: {
count: 'integer',
ParentId: {
model: 'article'
}
}
}
// adapter.js
find: function(connectionName, collectionName, options, cb) {
console.dir(options)
// output
// {where: null}
db.query(options, function(err, res)) {
cb(err, res)
}
}
However, when I try to populate using Article.find().populate('stats').exec(console.log()), my adapter gets {where: null} as options when I would expect it to receive {where: {parentId: [<some-article-id>]}}. It will return a list of articles to me but the field which is supposed to be populated from another model (stats) is just an empty list.
I feel like this is related to the fact that my adapter is not getting the proper where param to search for the related model on the primary key. To test this further, I setup a test one-to-many relationship using the the sails-mongo adapter. In this case the adapter did receive params I expected and the association worked fine.
Does anyone have any idea on why .populate('stats') wouldn't be sending the proper "where" params to my adapter?
Update 3/7
So it seems like what happens in associations is that SomeModel.find() will hit the adapter once and then .populate('othermodel') hits the adapter again using the primary key of the first request. Then the results of both are joined together. In my case, the second hit against the adapter isn't happening for some unknown reason.
Update
The original issue was related to an attribute naming error that's mentioned in the comments below. However, there still appears to be some issue with the final population step mentioned by particlebanana:
Final step will: Take all of the query results from all the returned query operations
and combine them in-memory to build up a result set you can return in
the exec callback.
I'm seeing that all required queries are now firing but they are failing to actually populate the alias. Here's the call with some added debugging output in the form of a gist for easier consumption: https://gist.github.com/jasonsims/9423170
It looks like you are on the right track! The way the operation sets get built up, the .find() on the Article should run with the first log (empty where) and the second query should get run with the parentId criteria in the log. The second query isn't running because it can't build up that parentId array of primary keys when you don't return anything from the first query.
Short answer: you need to return something in the find callback to see the second log, which should match your expected criteria.
The query lifecycle looks something like this:
Check if all query pieces are on the same connection, if not break out which queries will run on which connections
For all queries on a single connection, check if the adapter supports native joins (has a .join() method, if so you can pass the criteria down and let the adapter handle the joins.
If no native join method is defined run the "parent" operation (in this case the Article.find())
Use the results of the parent operation to build up criteria for any populations that need to run. (The parentId array in your criteria) and run the child results.
Take all of the query results from all the returned query operations and combine them in-memory to build up a result set you can return in the exec callback.
I hope that helps some. Shoot me the url of your repo and I will look through it, if it's able to be open sourced, and can help some more if you come across any issues.
Just to summarize, there were multiple issues going on here which were causing associations not to populate:
Custom primary keys
There was a problem with waterline when joining data from models using custom primary keys. #particlebanana fixed this in 8eff54b and it should be included in the next rc of waterline (waterline#0.10.0-rc5).
Malformed SOQL query
When waterline queries the adapter for a second time in order to acquire the child rows, it does so using { foreignKey: [ value ] }. Since the value was a list, jsforce was incorrectly generating the SOQL query since it expected all list values to be accompanied by either $in or $nin operators. I addressed this issue in github/jsforce#9 and it's now included in jsforce#1.1.2.
Model attributes are case sensitive
The models in my project were defined in snakeCase but the json response from Salesforce was using EveryWordCapitalized. This causes 1-to-many joins in waterline to reduce the many child records to one when it runs _.uniq(childRows, pk). Since the model has defined pk == id but the actual value returned from Salesforce is pk == Id, this call to uniq blows away all child records but one. I'm not entirely sure if this should be a waterline bug or not but fixing the capitalization in the model attribute definitions resolved this.
I am migrating an application from mySQL to couchDB. (Okay, Please dont pass judgements on this).
There is a function with signature
getUserBy($column, $value)
Now you can see that in case of SQL it is a trivial job to construct a query and fire it.
However as far as couchDB is concerned I am supposed to write views with map functions
Currently I have many views such as
get_user_by_name
get_user_by_email
and so on. Can anyone suggest a better and yet scalable way of doing this ?
Sure! One of my favorite views, for its power, is by_field. It's a pretty simple map function.
function(doc) {
// by_field: map function
// A single view for every field in every document!
var field, key;
for (field in doc) {
key = [field, doc[field]];
emit(key, 1);
}
}
Suppose your documents have a .name field for their name, and .email for their email address.
To get users by name (ex. "Alice" and "Bob"):
GET /db/_design/example/_view/by_field?include_docs=true&key=["name","Alice"]
GET /db/_design/example/_view/by_field?include_docs=true&key=["name","Bob"]
To get users by email, from the same view:
GET /db/_design/example/_view/by_field?include_docs=true&key=["email","alice#gmail.com"]
GET /db/_design/example/_view/by_field?include_docs=true&key=["name","bob#gmail.com"]
The reason I like to emit 1 is so you can write reduce functions later to use sum() to easily add up the documents that match your query.