I created a set of REST services based on Express.js to find some results stored in a Mongo Database. A very minimal version of the code for one of the services could be something like:
var express = require('express');
var app = express();
var bodyParser = require('body-parser');
var mongoClient = require('mongodb').MongoClient;
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.get('/results/:name', function(req, res){
var name = req.params.name;
mongoClient.connect('mongodb://localhost/test', function (err, db) {
var collection = db.collection('results');
collection.find({ name: name }).toArray( function (err, docs) {
res.json({results: docs});
});
});
});
app.listen(3000);
I'm coming from Java and I've been beaten by SQL injections in the past. So I'm not at all comfortable with using the user's input directly in the find request. With my very thin knowledge of the subject, I tried several special characters ( } ) " ' ; and so on) but I've not been able to produce any weird result.
What could go wrong here? What kind of validations or checks should I implement to make sure that it is not possible to inject code or to make the program fail?
Now, let's do something stupid and let's trust the user to input a correct record:
app.post('/results/', function(req, res){
var record = req.body;
if( record.name ) {
mongoClient.connect('mongodb://localhost/test', function (err, db) {
var collection = db.collection('results');
collection.insert( record, function(err, doc){});
res.json({message: 'ok'});
});
}
});
How can I validate the schema of the input? And apart from filling the DB with thousand of gigantic inputs, is it possible to exploit this code to inject some code? If yes, how to prevent that?
Thanks a lot!
I would have put this as a comment but since I'm not yet allowed to do that i'll just put it as a responce.
I go into the details since I'm not the expert here but here is an article I've found to be really interresting about vulnerability when using mongo and node.js.
For validating the model, I use mongoose as a client to my mongoDB, it helps a lot as it has its own validators and you can as well make your own.
I hope it helps you into your search.
Mongodb access is api-based, as oposed to SQL that is language-based. SQL is a language, and if you let inputs from users to be inserted in the language, then it is easy to make code injection and do almost everything to your database.
MongoDB has a different approach, when you are doing a search, you call an API function to do the search, and this API function can only do searchs.
If you let the user choose the fields and the values, then he can make searchs that you don't expect, but that's all.
The same applies for inserts, updates and deletes, be careful of not letting the user to choose the fields and the values, because he can choose ones that you do not expects nor wants.
Related
How can we transfer id data with node?
node index.js?id=1
var express = require('express');
var app = express();
app.get('/:id', function(req, res) {
res.send('id: ' + req.params.id);
console.log(req.params.id);
});
In nodejs.org there's a piece of documentation where the functionality you desire is explained in a clear and simple way (https://nodejs.org/en/knowledge/HTTP/clients/how-to-access-query-string-parameters/).
Basically, what you are using in the code you showed returns the part of the path where you use the ":[name]" (a subdirectory).
What you actually want to get are the query string parameters by using url.parse(req.url,true).query which returns an object with a set of key:value pairs with the name of each parameter and respective value.
Be aware that to use that line of code you'll need at least to require the url module.
I hope I made myself clear and that my answer helps you!
I have an existing Node.js/Express app which connects to 2 separate databases, it has a MySQL DB for all the relational and a MongoDB store for the non-relational vertical data.
It uses Sequelize and Mongoose and works absolutely swimmingly.
I've been looking at Next.js today and I'm pretty impressed, one of my pet peeves with React is actually how much bootstrapping there is and how much code it takes to achieve something simple. Next.js seems to solve some of those issues for me, so I'm willing to embrace it.
First issue - Is it possible to connect Next.js to existing DB's and read their objects directly in the view?
e.g. ./server.js:
const mongoDb = mongoose.connect(configDB.url); // MongoDB connection
const models = require('./models'); // Sequelize connection
app.prepare().then(() => {
server.use((req, res, next) => {
req.mongodb = mongoDb
req.mysqldb = models
// Logging req.mysqldb/req.mongodb at this point gives the correct result.
next()
});
server.get('*', (req, res) => {
return handle(req, res)
})
})
./pages/index.js:
Index.getInitialProps = async function(req) {
console.log(req.mongodb);
console.log(req.mysqldb)
// Example of what I want: req.mysqldb.users.findAll()....... to populate collection for this view
}
When the console statements are executed in the index.js page, they are logged as undefined.
Ideally I want to use the objects/ORM layer directly in the next.js templates, I do not want to have to call my own API internally, it seems like a huge waste of resources!
Any help, greatly appreciated.
Just for future reference. getInitialProps gets passed in an object with one of the keys being req. So you're meant to do something like the following instead
// add the curly braces around req
Index.getInitialProps = async function({ req }) {
// code
}
This is known as Function Parameter Destructuring and was introduced in ES6. What this accomplishes is similar to the following code
Index.getInitialProps = async function(_ref) {
var req = _ref.req;
}
Meaning, it takes the value of req of the object that gets passed and uses that value.
Well apparently by the time the request gets to the template it has changed a bit! Namely, it is nested within another request object.
req.req.mongodb and req.req.mysqldb both work fine :).
I have been writing a restful api in nodejs fairly succesfully for the most part. There are two collections in the MongoDB that I am accessing that return empty strings and happen to be the only collections that contain capital letters in their names. When I use MongoClient, I am able to access these collections just fine, so I know that it is not an out of date mongodb driver.
one example is when I try to access a collection called bulkBuds
//bulkBuds model
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
var BulkBudsSchema = new Schema({
sourceLicense: String,
quantity: Number,
strainName: String,
priceProfile: String
});
mongoose.model('bulkBuds', BulkBudsSchema);
The controller has a bit of excess logic in the query, but a simple find returns an empty string as well.
//bulkBuds controller
var express = require('express'),
router = express.Router(),
mongoose = require('mongoose'),
BulkBuds = mongoose.model('bulkBuds'),
Friends = mongoose.model('Api'),
config = require('../../config/config'),
jwt = require('express-jwt');
module.exports = function (app) {
app.use('/api/bulkBuds/', router);
};
router.get('/:license', jwt({secret: config.secret}), function (req, res, next) {
if(!req.user.friend){
res.status(401);
}
Friends.findById(req.user.id, function(err, friend){
if(err) throw err;
if(!friend) res.send("friend does not exist");
if(req.user.username != friend.username) res.send("invalid user");
console.log(req.params.license);
console.log(BulkBuds.find({}));
BulkBuds.find({'storeLicense': req.params.license, 'availableForSale': true},
"sourceLicense quantity strainName priceProfile", function (err, bulkBuds) {
if (err) return next(err);
console.log(bulkBuds);
res.send(bulkBuds);
});
})
});
Any suggestions would be greatly appreciated, thanks.
Very difficult to answer without being able to test against your database. But I would try a few things.
refactor {'storeLicense': req.params.license, 'availableForSale': true} to create the object outside of the query, and then console log that object prior to passing it to the query. That will ensure everything is as you expect.
Remove "sourceLicense quantity strainName priceProfile" as the second argument to BulkBuds.find, and replace with an empty object. I usually pass an object as the second param with the following syntax {_id:1,quantity:0} to modify the projection. Your syntax may work, but just in case I would try running the query without to see if that yields any results.
Confirm quantity in your db is indeed a Number and not a String. I know mongoose won't let you insert records that don't validate, not sure about querying. Most likely not the issue, but doesn't hurt to verify.
After creating the Bulkbirds schema try this:
mongoose.model('bulkBuds', BulkBudsSchema, 'bulkBuds');
Another long shot, but perhaps it has something to do with mongoose pluralizing the collection names. Using the above syntax will ensure it's querying the bulkBuds collection.
Once again, difficult to pinpoint without being able to test, but hopefully those ideas help.
Node.js is my first backend language and I am at the point where I am asking myself "where do I put the database connection information?".
There is a lot of good information regarding this issue. Unfortunately for me all the examples are in PHP. I get the ideas but I am not confident enough to replicate it in Node.js.
In PHP you would put the information in a config file outside the web root, and include it when you need database data.
How would you do this in Node.js? using the Express.js framework.
So far I have this:
var express = require('express'), app = express();
var mysql = require('mysql');
app.get('/', function(req,res) {
var connection = mysql.createConnection({
host: 'localhost',
user: 'root',
password: 'password',
database: 'store'
});
var query = connection.query('SELECT * from customers where email = "deelo42#gmail.com"');
query.on('error', function(err) {
throw err;
});
query.on('fields', function(fields) {
console.log('this is fields');
});
query.on('result', function(row) {
var first = row.first_name;
var last = row.last_name;
res.render('index.jade', {
title: "My first name is " + first,
category: "My last name is " + last
});
});
});
app.listen(80, function() {
console.log('we are logged in');
});
As you can see I have a basic express application with 1 GET route. This route sets off the function to go to the database and pull out information based on an email address.
At the top of the GET route is the database connection information. Where do I put that? How do I call it? How do I keep it out of web root, and include it like PHP ? Can you please show me in a working example. Thanks!
I use the Express Middleware concept for same and that gives me nice flexibility to manage files.
I am writing a detailed answer, which includes how i am use the config params in app.js to connect to DB.
So my app structure looks something this:
How i connect to DB? (I am using MongoDB, mongoose is ORM, npm install mongoose)
var config = require('./config/config');
var mongoose = require("mongoose");
var connect = function(){
var options = {
server: {
socketOptions:{
keepAlive : 1
}
}
};
mongoose.connect(config.db,options);
};
connect();
under the config folder i also have 'env' folder, which stores the environment related configurations in separate files such as development.js, test.js, production.js
Now as the name suggests, development.js stores the configuration params related to my development environment and same applies to the case of test and production. Now if you wish you can have some more configuration setting such as 'staging' etc.
project-name/config/config.js
var path = require("path");
var extend = require("util")._extend;
var development = require("./env/development");
var test = require("./env/test");
var production = require("./env/production");
var defaults = {
root: path.normalize(__dirname + '/..')
};
module.exports = {
development: extend(development,defaults),
test: extend(test,defaults),
production: extend(production,defaults)
}[process.env.NODE_ENV || "development"]
project-name/config/env/test.js
module.exports = {
db: 'mongodb://localhost/mongoExpress_test'
};
Now you can make it even more descriptive by breaking the URL's into, username, password, port, database, hostname.
For For more details have a look at my repo, where you can find this implementation, in fact now in all of my projects i use the same configuration.
If you are more interested then have a look at Mean.js and Mean.io, they have some better ways to manage all such things. If you are beginner i would recommend to keep it simple and get things going, once you are comfortable, you can perform magic on your own. Cheers
I recommend the 12-factor app style http://12factor.net which keeps all of this in env vars. You never should have this kind of information hard-coded or in the app source-code / repo, so you can reuse it in different environments or even share it publicly without breaking security.
However, since there are lots of environment vars, I tend to keep them together in a single env.js like the previous responder wrote - although it is not in the source code repo - and then source it with https://www.npmjs.org/package/dotenv
An alternative is to do it manually and keep it in, e.g. ./env/dev.json and just require() the file.
Any of these works, the important point is to keep all configuration information separate from code.
I agree with the commenter, put it in a config file. There is no ultimate way, but nconf is also one of my favourites.
The important best practise is that you keep the config separate if you have a semi-public project, so your config file will not overwrite other developers.
config-sample.json (has to be renamed and is tracked with for example git)
config.json (not tracked / ignored by git)
I am working on an API with MongoDB+Express+Node.js. So far so good, I can query for one record by ID, I can query for a specific field, delete, add, etc.
I am looking for a method that can be called to create a generic query for mongoDB, where i will pass it a json representation of the model with 'some' data to search for. If the field is not specified in the json object, then it will only search on those fields specified.
Here is the boilerplate code I had, done, but I am very new to JavaScript and Node, and I think i need to parse or sanitize the query object before running the .find method.
Basically I want to be able to perform a query similar to how it is done in parse.com where you send an object model with the data you want to search for and you get all the results that match.
Any help would be greatly appreciated!
--This code is in my custom module--
exports.findByQuery = function(req, res) {
var query = req.query;
db.collection('artists', function(err, collection) {
collection.find(query).toArray(function(err, docs) {
res.send(docs);
});
});
};
--This is the routes file--
var express = require('express'),
artist = require('./routes/artists');
var app = express();
app.configure(function () {
app.use(express.logger('dev')); /* 'default', 'short', 'tiny', 'dev' */
app.use(express.bodyParser());
});
app.get('/search/', artist.findByQuery);
app.get('/artist/:id', artist.findById);
app.post('/artist', artist.addArtist);
app.put('/artist/:id', artist.updateArtist);
app.delete('/artist/:id', artist.deleteArtist);
app.listen(3000);
I just found that the code above DOES work, however you cannot search for _id for some reason.. But any other field would work. Still looking how to incorporate _id in the search.