I just recently started to learn NodeJS/SailsJS and have few questions.
Please, note that I have strong Java background ( this affects my style of thinking and the architecture ).
How to prevent SQL injection in SailsJS?
Basically, I have:
User.query(query, function(err, result) {
if (err)
return next(err);
//
res.json({ data : stepsCount });
});
But where/how should I put parameters for SQL query?
In Java I used to make something like:
Query q = new Query("select * from User where id = :id");
q.setParameter("id", some-value);
What about Data Access Object?
I am feeling uncomfortable having SQL queries in Controller.
Do you have any best practices for that? Maybe some example projects?
All example projects I've found so far do not use some complex SQL queries.
They are more like school projects using some predefined methonds in Domain classes ( like User.create, User.find ) etc.
Thank you in advance.
Best Regards,
Maksim
Sailsjs use a DAO library written in javascript (of course). it is called waterlinejs The documentation is here
Basically, it means, if you want to find User with specific id, you only need
User.findOne({id:xxx}).then(function(data){
res.json(data);
})
This is like the main advantage of using sailsjs, because waterlinejs can have different adapters, but all of them can construct User model, and access them like this. So if you use sails-mysql-adapter, then it will create a user table, with id, name...etc as column, if you use memory adapter, it will be stored in memory.
It does provide User.query('select.....',callback) in case what you want cannot be achieved by this DAO methods. But because it is for last resort, for query building, sailsjs does not have native support, you can certainly use package like sprintf to build sql.
Since you are a java programmer, (just like me), as a side note, I'd like you to remember these findOne() methods provided by waterlinejs are asynchronous, and also in a promise manner. They are quite different from java. I spent a lot of time wrap my head around this, but as soon as I am comfortable with this idea, I start to love it right away.
Let me know if this is clear.
Related
I am using Knex.js to build SQL queries. It works well but I need to convert my query results into domain entities (a type representing an object from the domain) for my graphql resolvers. I used Knex to avoid using an ORM because a number of people online made it seem like an ORM will make queries more difficult. My current best idea is to follow the Repository pattern and have the ugly code for converting results to classes in the repo class. Better ideas are welcome :)
As I understood you want to just make a db-call based on GraphQL query (which is mean you already have db and want to use simple ORM instead of EF for example).
I don't know which platform do you have, but if you have .net, you can take a look NReco.GraphQL. It allows to set db-connection and define graphql schema in the json file (graphql schmea to db-table including relation between schemas), definately, it's worth take a look.
Does anybody have a hands-on experience with both DB-libraries — knex vs. mysql2?
After some googling (e.g. at NPMCompare), I'm still curious to know, based on real experience, what are the pros & contra of both options?
So far, the only real advantage of using knex over mysql2, that I clearly see, is its universal support of MSSQL, MySQL, PostgreSQL, SQLite3, and Oracle, while the latter supports MySQL only, but since currently I'm focusing on MySQL only, this knex's feature seems to be less relevant.
The parameters I would consider:
Performance & load resistance;
Stability (production ready);
Native ES8+ support (callback-hell-free, no extra Util.promisify wrappers, ESM/MJS support);
Short and clear, the less verbose the better.
I'm using knex on my primary project, I think that you are trying to compare apples with oranges, because Knex is a query builder that underline uses (mysql2) as the transport lib (in a case of MySql usage).
Benefits that I see in Knex are:
Prevents SQL injection by default.
Lets you build queries really easily without much on an effort
Lets you compose queries as you would compose javascript functions (this is a big big advantage in my opinion).
Since # 3 is a such big advantage in my opinion it is better to demonstrate it:
Think you have 2 endpoints
/users/list - which suppose to return a list of users ({id, name})
/users/:id - which suppose to return a single user with the same structure.
You can implement it like this.
async function getAllUsers() {
return db('users').columns('id', 'name'); //think that this can consist of many joins
}
async function getUserById(userId) {
return getAllUsers().where('id', userId);
}
Look how getUserById is re-uses the same query (may be really complex), and just "adding" the limitation that it requires.
Performance wise, I don't think that this abstraction has a big cost, (I didn't noticed any performance issues yet)
I'm not sure what do you refer as stability, but Knex has a really cool TS support which can make your queries strongly typed.
interface User {
id: number;
name: string;
}
const users = await db<User>('users').columns('id', 'name'); // it will autocomplete the columns names & users will be of type User[] automatically.
With a combination of auto generating these db type from the DB using #typed-code/schemats it makes the work & refactoring sooo much better.
As of ES6, Knex supports by default Promises & callbacks, so you can choose whatever suits you.
Other cool features that I'm using is auto converting between cases, my db has a snake case style as for tables & columns names but in my node I work with camel case, using knex-stringcase plugin.
Migrations, allow you to define how to build / upgrade your schema with code, which can help you to auto update your production schema from CI.
Mysql2 is a low level driver above the DB.
I am using mongodb and what I am trying to do is query a collection, and receive well filtered and ordered and projected result. Sadly the logic I want to implement is complex and even if we assume that it is possible to use db.collection.aggregate it will result in long, complex, hard to read aggregate descriptor, which I believe in most cases is unwanted.
So I was thinking - Mongodb understands javascript, therefor most likely I can pass a javascript function during the query itself, expecting that my mongo server will make the query, run the provided function passing the query result to it, then return the final result to me. Something like:
db.collection.find(myQuery, serverCallback).toArray(function(err, db) { ... });
Sadly it seems this is impossible. Investigating further I reached stored javascript and understood that I can define that serverCallback on the server instead of passing it. Which is good, but seems messy and wrong to me. So basically this is the reason, why i decided to ask here if someone with better mongodb experience can argument this approach.
[My understandings]
I beleive that not every case of filtering, aggregating, etc. can be achieved with db.collection.aggregate, which is pretty normal. For all the cases that need special way of filtering the query result, we have two options - to define stored javascript that we execute on the query result on the mongo server, or to fetch the information from the server and do the processing/filtering/etc. in the client.
If we choose to define stored javascript, it is very likely that we will define some project specific logic into the mongo server. I think that the project specifics should always belong to the project code instead of the database. That way we can version them with git and easily access them if we want to change them.
If we choose to apply the aggregation logic after the query we loose the capability to choose who will make the calculations - the server or the client. Which may be an opinion that we want to have.
[My question]
What is the reasoning behind not allowing serverCallback to be provided during the query? I believe that there must be reasons that I do not understand here.
[Edits]
First I want to say that I have resolve my problem and as it was way too complex to explain it easily. I will prefer to stick to something easier to explain and understand. I believe this example of MongoDB stored javascript provides great example so lets use it. Basically what i tried to ask above was is there a way to pass this sum function during db.collection.find (and why there isn't any). Something like this:
function sum(queryResultAsArray) {
//Do whatever we want with queryResultAsArray
//For the example we filter result rows with x + y == 6;
return queryResultAsArray.filter(function(row) {
return row.x + row.y == 6
});
}
db.test.find({}, queryResultAsArray);
And this to be equal to the examples:
db.test.find({$where: "sum(this.x, this.y) == 6"});
For reasoning on why would one can prefer passing function rather than stored javascript see the original post.
Mongoose (and MongoDB for that matter) seem to prefer the dbref/population idiom rather than traditional SQL relationships.
While I respect the simplicity of the solution proposed here: How to show related subdocument properties in Meteor
The MongoDB docs speak to DBRefs here:
http://docs.mongodb.org/manual/reference/database-references/#dbref-explanation
and Mongoose Populations are documented here: http://mongoosejs.com/docs/populate.html
While pulling an item from 1 query and passing it into another query is definitely one option the syntactic sugar of being able to pull a single query that provides all of the required data in one pull has it's advantages.
What is the Meteor philosophy on this?
The closest thing I know of to mongoose's populations is collection helpers. It works on both the client and the server and allows you to automatically transform documents into objects with useful methods.
As you'll see in the documentation, you can use these methods to, among other things, establish relationships between documents. For example if you have Books and Authors collections you can define a helper like this:
Books.helpers({
author: function() {
return Authors.findOne(this.authorId);
}
});
Which lets your write code this:
Books.findOne().author().firstName;
You still need to go through the effort of writing the join yourself, but once written you can use it everywhere.
Let's say I'm writing a log analysis application. The main domain object would be a LogEntry. In addition. users of the application define a LogTopic which describes what log entries they are interested in. As the application receives log entries it adds them to couchDB, and also checks them against all the LogTopics in the system to see if they match the criteria in the topic. If it does then the system should record that the entry matches the topic. Thus, there is a many-to-many relationship between LogEntries and LogTopics.
If I were storing this in a RDBMS I would do something like:
CREATE TABLE Entry (
id int,
...
)
CREATE TABLE Topic (
id int,
...
)
CREATE TABLE TopicEntryMap (
entry_id int,
topic_id int
)
Using CouchDB I first tried having just two document types. I'd have a LogEntry type, looking something like this:
{
'type': 'LogEntry',
'severity': 'DEBUG',
...
}
and I'd have a LogTopic type, looking something like this:
{
'type': 'LogTopic',
'matching_entries': ['log_entry_1','log_entry_12','log_entry_34',....],
...
}
You can see that I represent the relationship by using a matching_entries field in each LogTopic documents to store a list of LogEntry document ids. This works fine up to a point, but I have issues when multiple clients are both attempting to add a matching entry to a topic. Both attempt optimistic updates, and one fails. The solution I'm using now is to essentially reproduce the RDBMS approach, and add a third document type, something like:
{
'type':'LogTopicToLogEntryMap',
'topic_id':'topic_12',
'entry_id':'entry_15'
}
This works, and gets past the concurrent update issues, but I have two reservations:
I worry that I'm just using this
approach because it's what I'd do in
a relational DB. I wonder if there's
a more couchDB-like (relaxful?)
solution.
My views can no longer
retrieve all the entries for a
specific topic in one call. My
previous solution allowed that (if I
used the include_docs parameter).
Anyone have a better solution for me? Would it help if I also posted the views I'm using?
I cross-posted this question to the couchdb users mailing list and Nathan Stott pointed me to a very helpful blog post by Christopher Lenz
Your approach is fine. Using CouchDB doesn't mean you'll just abandon relational modeling. You will need need to run two queries but that's because this is a "join". SQL queries with joins are also slow but the SQL syntax lets you express the query in one statement.
In my few months of experience with CouchDB this is what I've discovered:
No schema, so designing the application models is fast and flexible
CRUD is there, so developing your application is fast and flexible
Goodbye SQL injection
What would be a SQL join takes a little bit more work in CouchDB
Depending on your needs I've found that couchdb-lucene is also useful for building more complex queries.
I'd try setting up the relation so that LogEntrys know to which LogTopics they belong. That way, inserting a LogEntry won't produce conflicts as the LogTopics won't need to be changed.
Then, a simple map function would emit the LogEntry once for each LogTopic it belongs to, essentially building up your TopicEntryMap on the fly:
"map": function (doc) {
doc.topics.map(function (topic) {
emit(topic, doc);
});
}
This way, querying the view with a ?key=<topic> argument will give you all the entries that belong to a topic.