I'm new to node/express/mongoose and backend in general, and I'm building an API from the ground up.
I split my code into controllers and DAOs (mongoose models) and I'm not sure where validation should be taking place, what exactly should the controller and model each be doing?
For example, for the route GET /users/:id, I want it to:
Return 400 if the id given is not a valid ObjectId
Return 404 if the id is a valid ObjectId, but no documents exist with this id
Return 200 if a document is found, and remove some fields (password, __v, and _id (because I made a virtual field "id" without underscore)) before sending the response
Return 500 otherwise
Here's what I tried. It's currently doing everything I want above, but I'm not sure if it's the best implementation:
users.controller.js
const UserModel = require('../models/users.model')
const mongoose = require('mongoose')
exports.getById = async (req, res) => {
// Check that the id is a valid ObjectId
if (!mongoose.Types.ObjectId.isValid(req.params.id)) {
return res.status(400).json({ error: 'Invalid ObjectID' })
}
try {
const user = await UserModel.findById(req.params.id)
if (!user) return res.status(404).json({ error: 'No user with this ID' })
res.status(200).send(user)
} catch (err) {
res.status(500).send(err)
}
}
users.model.js
exports.findById = async (id) => {
let user = await User.findById(id)
if (!user) {
return null
}
user = user.toJSON()
delete user._id
delete user.__v
delete user.password
return user
}
Is this the best way to structure things? Please critique and suggest any improvements and best practices.
I prefer to do all validation and logic in Controllers file and helpers classes in separate small js files.
Also trying to keep the models as lean as possible. Actually you can create different middleware functions to validate some inputs.
It is also helpful to create ErrorResponse middleware so you can do such simple and easy to read validations as:
if (!userFromDB) throw new ErrorResponse({code: 404, msg: "User doesn't exist"})
than catch it and handle in separate file with different options of the answer.
Backend has 100+ ways of implementing it. You need to choose you own 😁
I have this code that check for me if the token provided is valid or not, the code is working but the problem that it is returning wrong id, => it returns the first Id that he found in the database
the code that I'm using is this
const checkToken = async (req, res ,next) => {
const token= req.body.token //the token is mix of number and letters lenght (6)
User.findOne(token,
(err, user) => {
if (err) {
const error = new Error('Token not Found');
error.status= 406;
return next(err, error);
}else
{
res.send('/api/users/'+ user.id +'/update') // u need to mention user.id from DB
}
})
this is my image of the database :
I don't want to use the id to search the token , what I want is use the provided token and search in DB if it is found so I retrieve Id
According to MongoDB documentation that is the usual behavior of findOne method. When multiple documents satisfy the condition the function returns the first document that is written first. Reference
To check use find() and see all the documents it returns for the correct id and to use findOne make sure the token is unique for all the document.
I'm doing a project using Express, & Sequelize and have run into an issue:
I'm trying to create an api route that, on a button click, will create a row in my 'member' & 'memberinstrument' tables. Those tables have an association in my models that: member 'hasMany' memberinstruments & memberinstruments 'belongsTo' member.
This is what I have right now:
router.post("/api/individual/signup", async (req, res) => {
try {
const member = await db.Member.create({
memberName: req.body.memberName,
location: `${req.body.city}, ${req.body.state}`,
profilePicture: req.body.profilePicture,
UserId: req.user.id
})
const memberInstrument = await db.MemberInstrument.create({
instrument: req.body.instrument,
MemberId: member.id
});
res.json({ member, memberInstrument });
} catch (error) {
console.log(error.message);
res.status(500).send('Sever Error');
}
})
I'm testing it out in postman and it simply says I have a 'server error' (which doesn't happen if I delete the whole memberInstrument posting section) and in node I get "Cannot read property 'id' of undefined". I know it must have something to do with the timing of trying to create member, and then trying to get that member id for memberinstrument, but I can't figure out how to resolve this.
Any help would be much appreciated!
(edit:
I commented out both UserId & MemberId and it successfully posts.
uncommenting just UserId creates the same error:
UserId is a nullable field so I don't know why it's doing a server error if I don't define it (or maybe I have to define it as null but I do not know how to do that in postman since it's coming from .user instead of .body
uncommenting just MemberId creates the same error)
Since I'm doing this in Postman, and don't know how to send a req.body with that, I was getting that id error. I changed req.user.id to req.body.id which will be populated with information from a front end state and it is now working correctly.
I think it's unnecessary for you to include all the fields in your .create()
try this
router.post("/api/individual/signup", async (req, res) => {
try {
const data = req.body;
const member = await db.Member.create(data);
const [registration, created] = member;
if(created){
//insert more of your code here on creating **MemberInstrument**, this is just a sample.
await db.MemberInstrument.create(registration.member_id);
}
res.json({
registration,
created
});
} catch (error) {
console.log(error);
res.status(500);
}
}
On your postman request, you will input the body or fields in json format, except the ID, IDs should not be included on the postman body.
I have a sever connected to a mongodb database. When I add a first level data and then save that, it works.
For example :
// this works fine
router.post('/user/addsomedata', async (req,res)=>{
try {
const user = await User.findOne({email : req.body.email})
user.username = req.body.username
await user.save()
res.send()
} catch(e) {
res.status(404).send(e)
}
})
BUT if I try to save the object with deeper level data, it's not getting saved. I guess the update is not detected and hence the user didn't get replaced.
Example :
router.post('/user/addtask', auth ,async (req,res)=>{
const task = new Task({
name : req.body.name,
timing : new Date(),
state : false,
})
try {
const day = await req.user.days.find((day)=> day.day == req.body.day)
// day is found with no problem
req.user.days[req.user.days.indexOf(day)].tasks.push(task)
// console.log(req.user) returns exactly the expected results
await req.user.save(function(error,res){
console.log(res)
// console.log(res) returns exactly the expected results with the data filled
// and the tasks array is populated
// but on the database there is nothing
})
res.status(201).send(req.user)
} catch(e) {
res.status(400).send(e)
}
})
So I get the tasks array populated on the console even after the save callback but nothing on the db image showing empty tasks array
You're working on the user from the request, while you should first find the user from the DB like in your first example (User.findOne) and then update and save that model.
Use .lean() with your find queries whenever you are about to update the results returned by mongoose. Mongoose by default return instance objects which are immutable by nature. lean() method with find returns normal js objects which can be modified/updated.
eg. of using lean()
const user = await User.findOne({email : req.body.email}).lean();
You can read more about lean here
Hope this helps :)
When sending a request to /customers/41224d776a326fb40f000001 and a document with _id 41224d776a326fb40f000001 does not exist, doc is null and I'm returning a 404:
Controller.prototype.show = function(id, res) {
this.model.findById(id, function(err, doc) {
if (err) {
throw err;
}
if (!doc) {
res.send(404);
}
return res.send(doc);
});
};
However, when _id does not match what Mongoose expects as "format" (I suppose) for example with GET /customers/foo a strange error is returned:
CastError: Cast to ObjectId failed for value "foo" at path "_id".
So what's this error?
Mongoose's findById method casts the id parameter to the type of the model's _id field so that it can properly query for the matching doc. This is an ObjectId but "foo" is not a valid ObjectId so the cast fails.
This doesn't happen with 41224d776a326fb40f000001 because that string is a valid ObjectId.
One way to resolve this is to add a check prior to your findById call to see if id is a valid ObjectId or not like so:
if (id.match(/^[0-9a-fA-F]{24}$/)) {
// Yes, it's a valid ObjectId, proceed with `findById` call.
}
Use existing functions for checking ObjectID.
var mongoose = require('mongoose');
mongoose.Types.ObjectId.isValid('your id here');
I had to move my routes on top of other routes that are catching the route parameters:
// require express and express router
const express = require("express");
const router = express.Router();
// move this `/post/like` route on top
router.put("/post/like", requireSignin, like);
// keep the route with route parameter `/:postId` below regular routes
router.get("/post/:postId", singlePost);
I have the same issue I add
_id: String .in schema then it start work
This might be a case of routes mismatch if you have two different routes like this
router.route("/order/me") //should come before the route which has been passed with params
router.route("/order/:id")
then you have to be careful putting the route that is using a param after the regular route that worked for me
Are you parsing that string as ObjectId?
Here in my application, what I do is:
ObjectId.fromString( myObjectIdString );
it happens when you pass an invalid id to mongoose. so first check it before proceeding, using mongoose isValid function
import mongoose from "mongoose";
// add this inside your route
if( !mongoose.Types.ObjectId.isValid(id) ) return false;
In my case, I had to add _id: Object into my Schema, and then everything worked fine.
As of Nov 19, 2019
You can use isValidObjectId(id) from mongoose version 5.7.12
https://mongoosejs.com/docs/api/mongoose.html#mongoose_Mongoose-isValidObjectId
if(mongoose.Types.ObjectId.isValid(userId.id)) {
User.findById(userId.id,function (err, doc) {
if(err) {
reject(err);
} else if(doc) {
resolve({success:true,data:doc});
} else {
reject({success:false,data:"no data exist for this id"})
}
});
} else {
reject({success:"false",data:"Please provide correct id"});
}
best is to check validity
You can also use ObjectId.isValid like the following :
if (!ObjectId.isValid(userId)) return Error({ status: 422 })
If above solutions do not work for you.
Check if you are sending a GET request to a POST route.
It was that simple and stupid for me.
All you have to do is change the parameter name "id" to "_id"
//Use following to check if the id is a valid ObjectId?
var valid = mongoose.Types.ObjectId.isValid(req.params.id);
if(valid)
{
//process your code here
} else {
//the id is not a valid ObjectId
}
I was faced with something similar recently and solved it by catching the error to find out if it's a Mongoose ObjectId error.
app.get("/:userId", (req, res, next) => {
try {
// query and other code here
} catch (err) {
if (err.kind === "ObjectId") {
return res.status(404).json({
errors: [
{
msg: "User not found",
status: "404",
},
],
});
}
next(err);
}
});
You could either validate every ID before using it in your queries (which I think is the best practice),
// Assuming you are using Express, this can return 404 automatically.
app.post('/resource/:id([0-9a-f]{24})', function(req, res){
const id = req.params.id;
// ...
});
... or you could monkey patch Mongoose to ignore those casting errors and instead use a string representation to carry on the query. Your query will of course not find anything, but that is probably what you want to have happened anyway.
import { SchemaType } from 'mongoose';
let patched = false;
export const queryObjectIdCastErrorHandler = {
install,
};
/**
* Monkey patches `mongoose.SchemaType.prototype.castForQueryWrapper` to catch
* ObjectId cast errors and return string instead so that the query can continue
* the execution. Since failed casts will now use a string instead of ObjectId
* your queries will not find what they are looking for and may actually find
* something else if you happen to have a document with this id using string
* representation. I think this is more or less how MySQL would behave if you
* queried a document by id and sent a string instead of a number for example.
*/
function install() {
if (patched) {
return;
}
patch();
patched = true;
}
function patch() {
// #ts-ignore using private api.
const original = SchemaType.prototype.castForQueryWrapper;
// #ts-ignore using private api.
SchemaType.prototype.castForQueryWrapper = function () {
try {
return original.apply(this, arguments);
} catch (e) {
if ((e.message as string).startsWith('Cast to ObjectId failed')) {
return arguments[0].val;
}
throw e;
}
};
}
I went with an adaptation of the #gustavohenke solution, implementing cast ObjectId in a try-catch wrapped around the original code to leverage the failure of ObjectId casting as a validation method.
Controller.prototype.show = function(id, res) {
try {
var _id = mongoose.Types.ObjectId.fromString(id);
// the original code stays the same, with _id instead of id:
this.model.findById(_id, function(err, doc) {
if (err) {
throw err;
}
if (!doc) {
res.send(404);
}
return res.send(doc);
});
} catch (err) {
res.json(404, err);
}
};
This is an old question but you can also use express-validator package to check request params
express-validator version 4 (latest):
validator = require('express-validator/check');
app.get('/show/:id', [
validator.param('id').isMongoId().trim()
], function(req, res) {
// validation result
var errors = validator.validationResult(req);
// check if there are errors
if ( !errors.isEmpty() ) {
return res.send('404');
}
// else
model.findById(req.params.id, function(err, doc) {
return res.send(doc);
});
});
express-validator version 3:
var expressValidator = require('express-validator');
app.use(expressValidator(middlewareOptions));
app.get('/show/:id', function(req, res, next) {
req.checkParams('id').isMongoId();
// validation result
req.getValidationResult().then(function(result) {
// check if there are errors
if ( !result.isEmpty() ) {
return res.send('404');
}
// else
model.findById(req.params.id, function(err, doc) {
return res.send(doc);
});
});
});
Always use mongoose.Types.ObjectId('your id')for conditions in your query it will validate the id field before running your query as a result your app will not crash.
I was having problems with this and fixed doing mongoose.ObjectId(id) without Types
ObjectId is composed of following things.
a 4-byte value representing the seconds since the Unix epoch
a 5-byte random value (Machine ID 3 bytes and Processor id 2 bytes)
a 3-byte counter, starting with a random
value.
Correct way to validate if the objectId is valid is by using static method from ObjectId class itself.
mongoose.Types.ObjectId.isValid(sample_object_id)
In my case, similar routes caused this problem.
Router.get("/:id", getUserById);
Router.get("/myBookings",getMyBookings);
In above code, whenever a get request to route "/myBookings" is made, it goes to the first route where req.params.id is equals to "myBookings" which is not a valid ObjectId.
It can be corrected by making path of both routes different.
Something like this
Router.get("/user/:id", getUserById);
Router.get("/myBookings",getMyBookings);
You are having the castError because the next route you called after the id route could not be attached to the id route.
You have to declare the id route as one last route.
The way I fix this problem is by transforming the id into a string
I like it fancy with the backtick:
`${id}`
this should fix the problem with no overhead
UPDATE OCT 2022
it would be best if you now used the :
{id: id} // if you have an id property defined
or
{_id: new ObjectId(id)} // and search for the default mongodb _id
OR you can do this
var ObjectId = require('mongoose').Types.ObjectId;
var objId = new ObjectId( (param.length < 12) ? "123456789012" : param );
as mentioned here Mongoose's find method with $or condition does not work properly
Cast string to ObjectId
import mongoose from "mongoose"; // ES6 or above
const mongoose = require('mongoose'); // ES5 or below
let userid = _id
console.log(mongoose.Types.ObjectId(userid)) //5c516fae4e6a1c1cfce18d77
Detecting and Correcting the ObjectID Error
I stumbled into this problem when trying to delete an item using mongoose and got the same error. After looking over the return string, I found there were some extra spaces inside the returned string which caused the error for me. So, I applied a few of the answers provided here to detect the erroneous id then remove the extra spaces from the string. Here is the code that worked for me to finally resolve the issue.
const mongoose = require("mongoose");
mongoose.set('useFindAndModify', false); //was set due to DeprecationWarning: Mongoose: `findOneAndUpdate()` and `findOneAndDelete()` without the `useFindAndModify`
app.post("/delete", function(req, res){
let checkedItem = req.body.deleteItem;
if (!mongoose.Types.ObjectId.isValid(checkedItem)) {
checkedItem = checkedItem.replace(/\s/g, '');
}
Item.findByIdAndRemove(checkedItem, function(err) {
if (!err) {
console.log("Successfully Deleted " + checkedItem);
res.redirect("/");
}
});
});
This worked for me and I assume if other items start to appear in the return string they can be removed in a similar way.
I hope this helps.
I had the same error, but in a different situation than in the question, but maybe it will be useful to someone.
The problem was adding buckles:
Wrong:
const gamesArray = [myId];
const player = await Player.findByIdAndUpdate(req.player._id, {
gamesId: [gamesArray]
}, { new: true }
Correct:
const gamesArray = [myId];
const player = await Player.findByIdAndUpdate(req.player._id, {
gamesId: gamesArray
}, { new: true }
In my case the parameter id length was 25, So I trimmed first character of parameter id and tried. It worked.
Blockquote
const paramId = req.params.id;
if(paramId.length === 25){
const _id = paramId.substring(1, 25);
}
To change the string object to ObjectId instance fromString() method is not exist anymore. There is a new method createFromHexString().
const _id = mongoose.Types.ObjectId.fromString(id); // old method not available
const _id = mongoose.Types.ObjectId.createFromHexString(id); // new method.
could happen if you are sending less or more then 24 characters string as id