I've got my Mongoose code in a separate model.js file, while Express code for handling http requests is in app.js. I'm just practising creating APIs and testing them on Postman, for an imaginary wiki article site. The api I'm struggling to get it to work is deleting one article. (Note: for the sake of brevity, I've only included the code in question, i.e. app.delete('/articles/:id' .... from app.js, and the static method it calls from model.js - deleteOneArticleFromDB(articleID)
app.js:
const express = require('express');
const bodyParser = require('body-parser');
const model = require('./model');
const app = express();
app.use(bodyParser.urlencoded({ extended: true }));
app.delete('/articles/:id', async (req, res) => {
const articleID = req.params.id;
console.log(`req.params.id: ${req.params.id}`);
try {
const response = await model.DBUtility.deleteOneArticleFromDB(articleID);
res.status(200).json({message: response, app: 'wiki-api'});
} catch (err) {
res.json({message: err, app: 'wiki-api'});
}
});
const port = 3000;
app.listen(port, () => {
console.log(`Server started on port ${port}`);
});
model.js:
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/wikiDB', {useNewUrlParser: true, useUnifiedTopology: true, useFindAndModify: false });
const articleSchema = new mongoose.Schema({
title: String,
content: String
});
const Article = mongoose.model('Article', articleSchema);
class DBUtility {
static deleteOneArticleFromDB(articleID) {
return new Promise((resolve, reject) => {
Article.findByIdAndDelete(articleID, (err) => {
if (err) {
reject(err);
} else {
resolve(`Deleted article ${articleID} successfully`);
}
});
});
}
}
exports.DBUtility = DBUtility;
I have 5 articles (5 documents) in my Database:
{
"_id" : "5c139771d79ac8eac11e754a",
"title" : "API",
"content" : "API stands for Application Programming Interface. It is a set of subroutine definitions, communication protocols, and tools for building software. In general terms, it is a set of clearly defined methods of communication among various components. A good API makes it easier to develop a computer program by providing all the building blocks, which are then put together by the programmer."
}
/* 2 */
{
"_id" : "5c1398aad79ac8eac11e7561",
"title" : "Bootstrap",
"content" : "This is a framework developed by Twitter that contains pre-made front-end templates for web design"
}
/* 3 */
{
"_id" : "5c1398ecd79ac8eac11e7567",
"title" : "DOM",
"content" : "The Document Object Model is like an API for interacting with our HTML"
}
/* 4 */
{
"_id" : "5ea2c188fa57aa1b6453eda5",
"title" : "Node JS",
"content" : "Node.js is an open-source, cross-platform, JavaScript runtime environment that executes JavaScript code outside of a web browser. Node.js lets developers use JavaScript to write command line tools and for server-side scripting—running scripts server-side to produce dynamic web page content before the page is sent to the user's web browser. Consequently, Node.js represents a \"JavaScript everywhere\" paradigm,[6] unifying web-application development around a single programming language, rather than different languages for server- and client-side scripts.",
"__v" : 0
}
/* 5 */
{
"_id" : "5ea2d5304e19b11e0013a86a",
"title" : "EJS",
"content" : "EJS is a simple templating language that lets you generate HTML markup with plain JavaScript. No religiousness about how to organize things. No reinvention of iteration and control-flow. It's just plain JavaScript",
"__v" : 0
}
I'm trying to delete the last article (document) with title EJS. So in Postman I run the http request as follows:
As you can see, I get a success response. However, when I check my database, the document is still there (I have clicked refresh several times and also tested it with a GET request to return all articles, which showed the article is still there):
Here's the terminal output:
[nodemon] starting `node app.js`
Server started on port 3000
req.params.id: 5ea2d5304e19b11e0013a86a
I've been on this for two days. I've checked all previous SO posts with similar titles to mine, but I couldn't see one that applies to my issue. I don't understand where I'm going wrong!! Any help would be really appreciated.
UPDATE
As per the solution below by Mohammed Yousry, I realised that I added the _id field manually using strings, as I was following along to a tutorial. Hence not allowing MongoDB to create the _id field instead, as an ObjectId. Therefore, my _id field was of type String rather than ObjectId. So to resolve this, I deleted all documents from the database and re-added them, using POSTMAN with the POST method I created - for creating/adding a new article document to the database, providing only the title and content fields in the request body. Hence allowing MongoDB to create the _id field instead, for each article document. Now in the database the _id field is of type ObjectId. This still hasn't resolved my issue fully, but it's one step further. Still working on reaching a solution. Please refer to the discussions below in the solution section.
1- may be your mongoose version does not support findByIdAndDelete
try findByIdAndRemove instead
2- you can pass the id to that method as a string or as an objectId
make sure you are passing the right id without any spaces
const articleID = req.params.id.toString().trim(); // trim will remove any spaces before or after the id
Update
according to this image
it seems you have inserted your data manually, and these data contained the _id as a string, as the type of the _id is a string here as you can see in this image
I suggest you to let MongoDB to set the _id itself
If you could change the _id type to be ObjectId
or If you can't change the type of _id, you can take a backup of your articles, then remove all those articles manually and add them again without the _id, let mongoDB set that _id, and try your code again
hope this may solve the issue
Update 2
After checking the codes in github, I know the reason why you got this strange behavior
1- you have a route app.route('/articles/:articleTitle') defined before the route app.delete('/articles/:id'
so when you are trying to delete some article by passing the id, nodeJS an express will pass it to the first matching route, as it does not know the params you send, it's searching for the base url in the route, so it deals with the first route which matching the url : '/articles/something', take that something as the article title
then call the deleteOneArticleByTitleFromDB method
MongoDB is then searching for some article with title = the id you passed, it found nothing, then, it doesn't remove any article
2- I suggest you not to define some routes with the same base url
so you can define the delete by Id route like the following
app.delete('/arti/cles/:id', async (req, res) => // note the slash inside articles word
or anything you want except the current one
by this you avoid this contradiction between the routes
In MongoDB, the _id property is of type ObjectId.
I had the same problem and fixed it by passing over an ObjectId instead.
Have you tried the following ?
const Mongoose = require('mongoose');
const articleId = Mongoose.Types.ObjectId(req.params.id);
#Hazzaldo I was facing the same problem. But apparently, I didn't write await before findByIdAndDelete(). After writing await before it fixed it for me.
I saw your code, as you are wrapping the findByIdAndDelete inside a promise which in itself is a promise, I would recommend you to try to await the findByIdAndDelete first and then return the promise.
Related
I have an existing Node.js/Express app which connects to 2 separate databases, it has a MySQL DB for all the relational and a MongoDB store for the non-relational vertical data.
It uses Sequelize and Mongoose and works absolutely swimmingly.
I've been looking at Next.js today and I'm pretty impressed, one of my pet peeves with React is actually how much bootstrapping there is and how much code it takes to achieve something simple. Next.js seems to solve some of those issues for me, so I'm willing to embrace it.
First issue - Is it possible to connect Next.js to existing DB's and read their objects directly in the view?
e.g. ./server.js:
const mongoDb = mongoose.connect(configDB.url); // MongoDB connection
const models = require('./models'); // Sequelize connection
app.prepare().then(() => {
server.use((req, res, next) => {
req.mongodb = mongoDb
req.mysqldb = models
// Logging req.mysqldb/req.mongodb at this point gives the correct result.
next()
});
server.get('*', (req, res) => {
return handle(req, res)
})
})
./pages/index.js:
Index.getInitialProps = async function(req) {
console.log(req.mongodb);
console.log(req.mysqldb)
// Example of what I want: req.mysqldb.users.findAll()....... to populate collection for this view
}
When the console statements are executed in the index.js page, they are logged as undefined.
Ideally I want to use the objects/ORM layer directly in the next.js templates, I do not want to have to call my own API internally, it seems like a huge waste of resources!
Any help, greatly appreciated.
Just for future reference. getInitialProps gets passed in an object with one of the keys being req. So you're meant to do something like the following instead
// add the curly braces around req
Index.getInitialProps = async function({ req }) {
// code
}
This is known as Function Parameter Destructuring and was introduced in ES6. What this accomplishes is similar to the following code
Index.getInitialProps = async function(_ref) {
var req = _ref.req;
}
Meaning, it takes the value of req of the object that gets passed and uses that value.
Well apparently by the time the request gets to the template it has changed a bit! Namely, it is nested within another request object.
req.req.mongodb and req.req.mysqldb both work fine :).
I have been writing a restful api in nodejs fairly succesfully for the most part. There are two collections in the MongoDB that I am accessing that return empty strings and happen to be the only collections that contain capital letters in their names. When I use MongoClient, I am able to access these collections just fine, so I know that it is not an out of date mongodb driver.
one example is when I try to access a collection called bulkBuds
//bulkBuds model
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
var BulkBudsSchema = new Schema({
sourceLicense: String,
quantity: Number,
strainName: String,
priceProfile: String
});
mongoose.model('bulkBuds', BulkBudsSchema);
The controller has a bit of excess logic in the query, but a simple find returns an empty string as well.
//bulkBuds controller
var express = require('express'),
router = express.Router(),
mongoose = require('mongoose'),
BulkBuds = mongoose.model('bulkBuds'),
Friends = mongoose.model('Api'),
config = require('../../config/config'),
jwt = require('express-jwt');
module.exports = function (app) {
app.use('/api/bulkBuds/', router);
};
router.get('/:license', jwt({secret: config.secret}), function (req, res, next) {
if(!req.user.friend){
res.status(401);
}
Friends.findById(req.user.id, function(err, friend){
if(err) throw err;
if(!friend) res.send("friend does not exist");
if(req.user.username != friend.username) res.send("invalid user");
console.log(req.params.license);
console.log(BulkBuds.find({}));
BulkBuds.find({'storeLicense': req.params.license, 'availableForSale': true},
"sourceLicense quantity strainName priceProfile", function (err, bulkBuds) {
if (err) return next(err);
console.log(bulkBuds);
res.send(bulkBuds);
});
})
});
Any suggestions would be greatly appreciated, thanks.
Very difficult to answer without being able to test against your database. But I would try a few things.
refactor {'storeLicense': req.params.license, 'availableForSale': true} to create the object outside of the query, and then console log that object prior to passing it to the query. That will ensure everything is as you expect.
Remove "sourceLicense quantity strainName priceProfile" as the second argument to BulkBuds.find, and replace with an empty object. I usually pass an object as the second param with the following syntax {_id:1,quantity:0} to modify the projection. Your syntax may work, but just in case I would try running the query without to see if that yields any results.
Confirm quantity in your db is indeed a Number and not a String. I know mongoose won't let you insert records that don't validate, not sure about querying. Most likely not the issue, but doesn't hurt to verify.
After creating the Bulkbirds schema try this:
mongoose.model('bulkBuds', BulkBudsSchema, 'bulkBuds');
Another long shot, but perhaps it has something to do with mongoose pluralizing the collection names. Using the above syntax will ensure it's querying the bulkBuds collection.
Once again, difficult to pinpoint without being able to test, but hopefully those ideas help.
I created a set of REST services based on Express.js to find some results stored in a Mongo Database. A very minimal version of the code for one of the services could be something like:
var express = require('express');
var app = express();
var bodyParser = require('body-parser');
var mongoClient = require('mongodb').MongoClient;
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.get('/results/:name', function(req, res){
var name = req.params.name;
mongoClient.connect('mongodb://localhost/test', function (err, db) {
var collection = db.collection('results');
collection.find({ name: name }).toArray( function (err, docs) {
res.json({results: docs});
});
});
});
app.listen(3000);
I'm coming from Java and I've been beaten by SQL injections in the past. So I'm not at all comfortable with using the user's input directly in the find request. With my very thin knowledge of the subject, I tried several special characters ( } ) " ' ; and so on) but I've not been able to produce any weird result.
What could go wrong here? What kind of validations or checks should I implement to make sure that it is not possible to inject code or to make the program fail?
Now, let's do something stupid and let's trust the user to input a correct record:
app.post('/results/', function(req, res){
var record = req.body;
if( record.name ) {
mongoClient.connect('mongodb://localhost/test', function (err, db) {
var collection = db.collection('results');
collection.insert( record, function(err, doc){});
res.json({message: 'ok'});
});
}
});
How can I validate the schema of the input? And apart from filling the DB with thousand of gigantic inputs, is it possible to exploit this code to inject some code? If yes, how to prevent that?
Thanks a lot!
I would have put this as a comment but since I'm not yet allowed to do that i'll just put it as a responce.
I go into the details since I'm not the expert here but here is an article I've found to be really interresting about vulnerability when using mongo and node.js.
For validating the model, I use mongoose as a client to my mongoDB, it helps a lot as it has its own validators and you can as well make your own.
I hope it helps you into your search.
Mongodb access is api-based, as oposed to SQL that is language-based. SQL is a language, and if you let inputs from users to be inserted in the language, then it is easy to make code injection and do almost everything to your database.
MongoDB has a different approach, when you are doing a search, you call an API function to do the search, and this API function can only do searchs.
If you let the user choose the fields and the values, then he can make searchs that you don't expect, but that's all.
The same applies for inserts, updates and deletes, be careful of not letting the user to choose the fields and the values, because he can choose ones that you do not expects nor wants.
I am working on an API with MongoDB+Express+Node.js. So far so good, I can query for one record by ID, I can query for a specific field, delete, add, etc.
I am looking for a method that can be called to create a generic query for mongoDB, where i will pass it a json representation of the model with 'some' data to search for. If the field is not specified in the json object, then it will only search on those fields specified.
Here is the boilerplate code I had, done, but I am very new to JavaScript and Node, and I think i need to parse or sanitize the query object before running the .find method.
Basically I want to be able to perform a query similar to how it is done in parse.com where you send an object model with the data you want to search for and you get all the results that match.
Any help would be greatly appreciated!
--This code is in my custom module--
exports.findByQuery = function(req, res) {
var query = req.query;
db.collection('artists', function(err, collection) {
collection.find(query).toArray(function(err, docs) {
res.send(docs);
});
});
};
--This is the routes file--
var express = require('express'),
artist = require('./routes/artists');
var app = express();
app.configure(function () {
app.use(express.logger('dev')); /* 'default', 'short', 'tiny', 'dev' */
app.use(express.bodyParser());
});
app.get('/search/', artist.findByQuery);
app.get('/artist/:id', artist.findById);
app.post('/artist', artist.addArtist);
app.put('/artist/:id', artist.updateArtist);
app.delete('/artist/:id', artist.deleteArtist);
app.listen(3000);
I just found that the code above DOES work, however you cannot search for _id for some reason.. But any other field would work. Still looking how to incorporate _id in the search.
I have a problem testing my mongoose models
I have a structure like
app
models
Address
User
Organization
test
Both models User and Organization need to know the model Address. My models are structured like:
module.exports = function (mongoose, config) {
var organizationSchema = new mongoose.Schema({
name : {
type : String
},
addresses : {
type : [mongoose.model('Address')]
}
});
var Organization = mongoose.model('Organization', organizationSchema);
return Organization;
};
In my normal app i require Address before requiring User and Organization and everything is fine. I now wrote tests for User and Organization. In order to have the Address model registered i call require('../models/Address.js') This works fine if i run one test. But if i run all tests in a batch i get an error because i tried to register Address twice.
OverwriteModelError: Cannot overwrite Address model once compiled.
How do i solve this problem?
The problem is that you cant set mongoose model twice. The easiest way to solve your problem is to take advantage of node.js require function.
Node.js caches all calls to require to prevent your model from initializing twice. But you wrapping your models with functions. Unwrapping them will solve your problem:
var mongoose = require('mongoose');
var config = require('./config');
var organizationSchema = new mongoose.Schema({
name : {
type : String
},
addresses : {
type : [mongoose.model('Address')]
}
});
module.exports = mongoose.model('Organization', organizationSchema);
Alternative solution is to make sure that each model initialized only once. For example, you can initialize all you modules before running your tests:
Address = require('../models/Address.js');
User = require('../models/User.js');
Organization = require('../models/Organization.js');
// run your tests using Address, User and Organization
Or you can add try catch statement to your models to handle this special case:
module.exports = function (mongoose, config) {
var organizationSchema = new mongoose.Schema({
name : {
type : String
},
addresses : {
type : [mongoose.model('Address')]
}
});
try {
mongoose.model('Organization', organizationSchema);
} catch (error) {}
return mongoose.model('Organization');
};
Update: In our project we have /models/index.js file to handle everything. First, it calls mongoose.connect to establish connection. Then it requires every model in models directory and creates a dictionary of it. So, when we need some model (e.g. user) we requires it by calling require('/models').user.
Best solution (IMO):
try {
mongoose.model('config')
} catch (_) {
mongoose.model('config', schema)
}
This question already has an answer, but for a unique way to accomplish this check out https://github.com/fbeshears/register_models. This sample project uses a register_models.js that includes all models from an array of file names. It works really well and you end up with all your models pretty much wherever you need them. Keep in mind node.js's cache will cache objects in your project while it's running.
I use try/catch to tackle this problem and it works ok. But i think this not the best way of doing this.
try{
var blog = mongoose.model('blog', Article);
} catch (error) {}
I fixed this by patching r.js and making sure that all my modules used localRequire calls.
Checkout the thread here
https://github.com/jrburke/requirejs/issues/726