Does pg (node-postgres) automatically sanitize data - node.js

I am using node-postgres for a production application and I am wondering if there is anything I should be concerned about? Is the data sanitized automatically by node-postgres?
I couldn't find anything about it on the github page: https://github.com/brianc/node-postgres

Absolutely! The parameterized query support in node-postgres is first class. All escaping is done by the postgresql server ensuring proper behavior across dialects, encodings, etc... For example, this will not inject sql:
client.query("INSERT INTO user(name) VALUES($1)", ["'; DROP TABLE user;"], function (err, result) {
// ...
});
This is from their documentation.

It basically depends on how you execute your queries as #vitaly-t described
Suppose you will define query in a string and execute as follows:
var query = `SELECT * FROM table where username='${username}' and password='${password}`;
pool.query(query, (error, results) => {
});
This case if i would pass username=' 'or 1=1; -- and password=' 'or 1=1; --
Then it will return all records from the table (means SQL injection works)
But if I would execute the following query
pool.query('SELECT * FROM table where username=$1 and password=$2', [username, password], (error, results) => {
});
Then SQL injection will never work because pg will sanitize the data.
So it's depends on how you execute the queries.

It depends on how you execute your queries:
Formatting via Prepared Statements is executed by the server, which in turn sanitizes your query from any SQL injection. But it has other restrictions, like you cannot execute more than one query at a time, and you cannot provide sanitizied entity names when needed.
Client-side query formatting, like the one implemented by pg-promise, sanitizes values, plus offers flexibility in formatting entity names and multiple queries.

Related

Do I have to write the reverse operation for migration rollback?

I use knex.js and it's good query builder for PostgreSQL. I haven't found any docs to explain how to do the migration rollback in a right way.
For now, I just write the reverse migrate operation in the down function to do the migration rollback. Is this a correct way?
import * as Knex from 'knex';
exports.up = async (knex: Knex): Promise<any> => {
await knex.schema.raw(`
ALTER TABLE IF EXISTS "GOOGLE_CHANNEL"
ADD COLUMN IF NOT EXISTS google_channel_ad_group_cpc_bid INTEGER NOT NULL DEFAULT 0;
`);
await knex.schema.raw(`
UPDATE "GOOGLE_CHANNEL" as gc
SET
google_channel_ad_group_cpc_bid = 7
FROM "CAMPAIGN_TEMPLATE" as ct
WHERE ct.campaign_channel_id = gc.campaign_channel_id;
`);
};
exports.down = async (knex: Knex): Promise<any> => {
// TODO: migration rollback
await knex.schema.raw(``);
};
I have two concerns:
If there are a lot of SQL statements in up function, I have to do write a lot of SQL statements in down function too in order to rollback the migration.
Why doesn't knex.js do the migration rollback without writing the reverse operation for us? I mean, knex.js can take a snapshot or record a savepoint of the database.
Yes, to rollback you use the down function of a migration script. When you run knex migrate:rollback the down function will run. Knex has meta tables in the database that are used to figure out what migrations that have run or not.
For example:
exports.up = function (knex, Promise) {
return knex.schema
.createTable('role', function (table) {
table.increments('role_id').primary();
table.string('title').notNullable().unique();
table.string('description');
table.integer('level').notNullable(),
})
.createTable('user_account', function (table) {
table.increments('user_id').primary();
table.integer('role_id').references('role_id').inTable('role').notNullable();
table.string('username').notNullable().unique();
table.string('passwordHashed').notNullable();
table.string('email', 50).notNullable().unique();
});
};
exports.down = function (knex, Promise) {
return knex.schema
.dropTable('user_account')
.dropTable('role');
};
Here I create two tables in the up function. The user_account has a foreign key constraint, and links with the role table, which means I have to drop the user_account table before the role table in the down function.
In your case, you use a update statement. In the down function you have to either make a new update with a hard-coded value (the old one before the migration), or make sure you store the old value in a history table.
As for your concerns:
Yes, if you add a lot of stuff, you also have to add a lot of code to reverse whatever you are doing. However, you can skip making the down scripts, but then you won't be able to rollback. Some (many?) choose to only go forward and never rollback. If they have to fix something they don't rollback but make a new migration script with the fix.
I would recommend you to create the down functions in the beginning. You can consider skipping making them when the time is right. People who don't make down functions usually have to test their migrations more thoroughly in a test or staging environment before deploying to production. This is to make sure it works, because they can't rollback after all.
I can't really answer for the Knex creators here. However, what you are describing as a potential solution is basically a backup of the database before a migration is done. After all, a migration does more than just change the layout of the tables, etc. A migration script will typically add or remove new rows as well. You can use the backup approach, but you have to take the backups yourself.
Knex is a fairly simple query builder. If you want the migration scripts to be written for you, you might want to go for a full-blown OR mapper.

What does the dollar sign ('$') mean when in the string to .query?

What does the $ sign mean in this statement:
// SQL Query > Update Data
client.query('UPDATE items SET text=($1), complete=($2) WHERE id=($3)',
[data.text, data.complete, id]);
Pretty universally in that context it's called a "placeholder". You got that code from this blog entry. You can see there that client is defined above in the callback to pg.connect,
pg.connect(connectionString, (err, client, done) => {
Looking up in that blog entry, pg is defined here
const pg = require('pg');
You can always find out what an npm-installed module name resolves to by doing a quick search. In this case though the blog openly says they're using node-postgres. Which documents this under,
Paramaterized Queries
Paramaterized means accepting of a parameter. That parameter's placement is defined with a "placeholder" as specified with $. The purpose of this is often to save time in planning and to avert SQL Injection attacks.

Should I seperate input validation and DB query?

I am now working on a API back-end. I've written a series of functions to query a database, like:
(yes, NodeJS)
function addGuys(guys, cb) {
var sql = 'INSERT INTO guys ...';
guys.ForEach(function(guy) {
// Construct sql according to keys given in `guy`
// Validate here?
// add col names, values to sql statment, etc
}
// Execute query
}
The problem now is, should I validate input in these DB functions, or should I validate input before sending it to the DB functions? The latter seems cleaner, but I am a novice and still not sure. Any advice will be welcomed. Thanks.
You should not construct SQL directly yourself at all. Instead you should use a library for that, which does its own validation and escaping. A popular example would be http://docs.sequelizejs.com/en/v3/

Delete multiple couchbase entities having common key pattern

I have a use case where I have to remove a subset of entities stored in couchbase, e.g. removing all entities with keys starting with "pii_".
I am using NodeJS SDK but there is only one remove method which takes one key at a time: http://docs.couchbase.com/sdk-api/couchbase-node-client-2.0.0/Bucket.html#remove
In some cases thousands of entities need to be deleted and it takes very long time if I delete them one by one especially because I don't keep list of keys in my application.
I agree with the #ThinkFloyd when he saying: Delete on server should be delete on server, rather than requiring three steps like get data from server, iterate over it on client side and finally for each record fire delete on the server again.
In this regards, I think old fashioned RDBMS were better all you need to do is 'DELETE * from database where something=something'.
Fortunately, there is something similar to SQL is available in CouchBase called N1QL (pronounced nickle). I am not aware about JavaScript (and other language syntax) but this is how I did it in python.
Query to be used: DELETE from <bucketname> b where META(b).id LIKE "%"
layer_name_prefix = cb_layer_key + "|" + "%"
query = ""
try:
query = N1QLQuery('DELETE from `test-feature` b where META(b).id LIKE $1', layer_name_prefix)
cb.n1ql_query(query).execute()
except CouchbaseError, e:
logger.exception(e)
To achieve the same thing: alternate query could be as below if you are storing 'type' and/or other meta data like 'parent_id'.
DELETE from <bucket_name> where type='Feature' and parent_id=8;
But I prefer to use first version of the query as it operates on key, and I believe Couchbase must have some internal indexes to operate/query faster on key (and other metadata).
The best way to accomplish this is to create a Couchbase view by key and then range query over that view via your NodeJS code, making deletes on the results.
http://docs.couchbase.com/admin/admin/Views/views-querySample.html
http://docs.couchbase.com/couchbase-manual-2.0/#couchbase-views-writing-querying-selection-partial
http://docs.couchbase.com/sdk-api/couchbase-node-client-2.0.8/ViewQuery.html
For example, your Couchbase view could look like the following:
function(doc, meta) {
emit(meta.id, null);
}
Then in your NodeJS code, you could have something that looks like this:
var couchbase = require('couchbase');
var ViewQuery = couchbase.ViewQuery;
var query = ViewQuery.from('designdoc', 'by_id');
query.range("pii_", "pii_" + "\u0000", false);
var myBucket = myCluster.openBucket();
myBucket.query(query, function(err, results) {
for(i in results) {
// Delete code in here
}
});
Of course your Couchbase design document and view will be named differently than the example that I gave, but the important part is the ViewQuery.range function that was used.
All document ids prefixed with pii_ would be returned, in which case you can loop over them and start deleting.
Best,

Private fields with sql query

I recently changed my queries on the SQL database from
Model.findOne().populate("Model2").exec(function(err, result) {
})
to :
Model.query(sqlQuery, parameters, function(err, result) {
})
The reason I changed the queries are performances problems : it goes much faster using the second way than the first one (when you join tables).
My question is the following : Using the Waterline syntax, I was able to retrieve only the elements I wanted (elements defined in the model), but using the query method ("Select * from model ...") gives me the result I ask to the database, but I'd like to be able to filter the fields that are defined in my model. Is that possible?.
Your model definitions are on the sails.models.MODEL object.
You could reference the available attributes there.
Object.keys(sails.models.MODEL._attributes).join(',')

Resources