I'm building a CRUD-style REST service with Node.js, Express and MongoDB using mongoose. This service is going to allow users of an already existing android application to upload/sync the contents of their individual databases online.
The data model for the already-existing application uses UUIDs (generated in Java) which clashes with the shorter, monotonic MongoDB style _id fields. Because the data model already exists and is populated with data from many users, I cannot convert the source data over to monotonic MongoDB-style _ids. This has left me with 2 options that I can think of: either 1) Make Mongo/Mongoose (or some other ODM) play nicely with full UUIDs instead of the monotonic _ids or 2) add a uuid field to the mongoose model in addition to the _id field and fight the pitfalls of this approach. I'm attempting to choose option #1 and running into issues with ObjectID references.
I originally stumbled upon mongoose-uuid, but unfortunately this isn't working for my use-case properly because it was overwriting my explicitly-set _id value when creating new Mongoose objects. Diving into the plugin code, it assumes that an object is new (by calling checking Mongoose's .isNew value) and thus overwrites the _id with a new uuid. Since I need to retain the original uuid when creating new documents in Mongo, this plugin isn't working for me.
Next, I found a post by Aaron Heckmann, creator of mongoose, on a similar topic. This has been helpful, however I am now encountering the problem where I cannot have my mongoose schemas reference each other by ObjectID, since they technically they are now referencing each other using String `_ids.
Schema example:
var mongoose = require('mongoose');
var uuid = require('node-uuid');
var Schema = mongoose.Schema;
var trackPassSchema = new Schema({
_id: { type: String, default: function genUUID() {
uuid.v1()
}},
//Omitting other fields in snippet for simplicity
vehicle: [
{type: Schema.Types.ObjectId, required: true, ref: 'Vehicle'}
]
});
module.exports = mongoose.model('TrackPass', trackPassSchema);
Referencing schema:
var mongoose = require('mongoose');
var uuid = require('node-uuid');
var Schema = mongoose.Schema;
var vehicleSchema = new Schema({
_id: { type: String, default: function genUUID() {
uuid.v1()
}},
//Omitting other fields in snippet for simplicity
description: {type: String},
year: {type: Number}
});
module.exports = mongoose.model('Vehicle', vehicleSchema);
When I attempt to call save() a trackPass that has been passed in from my application:
var trackPass = new TrackPass(req.body);
//Force the ID to match what was put into the request
trackPass._id = req.params.id;
trackPass.save(function (err) { ... }
I get the following error:
{ [CastError: Cast to ObjectId failed for value "b205ac4d-fd96-4b1e-892a-d4fab818ea2a" at path "vehicle"]
message: 'Cast to ObjectId failed for value "b205ac4d-fd96-4b1e-892a-d4fab818ea2a" at path "vehicle"',
name: 'CastError',
type: 'ObjectId',
value: ["b205ac4d-fd96-4b1e-892a-d4fab818ea2a"],
path: 'vehicle' }
I believe this error makes sense as I'm now using Strings which are longer than typical Mongo ObjectIDs. Without having the ObjectID reference, I don't believe I will be able to populate() referenced objects from other collections. I suppose I could simply not reference the other nested objects in my schema definitions, however I don't like this approach as I feel I will be losing a lot of the benefit of utilizing the ODM. Any other thoughts?
You can still use populate() with _id values of types besides ObjectID, but you do need to use the same type in the reference definition.
So your trackPassSchema would need to change to:
var trackPassSchema = new Schema({
_id: { type: String, default: function genUUID() {
return uuid.v1()
}},
vehicle: [
{type: String, required: true, ref: 'Vehicle'}
]
});
As Adam notes in the comments, you could simplify your default value to:
var trackPassSchema = new Schema({
_id: { type: String, default: uuid.v1 },
vehicle: [
{type: String, required: true, ref: 'Vehicle'}
]
});
Both JohnnyHK and Adam C answers are correct. But if you're using uuid in schema for an array of objects, it is good to use it like this
var trackPassSchema = new Schema({
_id: { type: String, default: () => uuid.v1 },
vehicle: [
{type: String, required: true, ref: 'Vehicle'}
]
});
Because, in one such scenario when i tried using like this _id: { type: String, default: () => uuid.v1 } multiple objects of the array had the same id.
It is not possible in this case as _id is unique field, but it can happen when you are using with fields that aren't unique.
Related
Suppose we have a schema that looks like this:
const RandomSchema = new Schema({
name: String,
randomField: String,
subDoc: {
name: String,
refDoc: {
type: Schema.Types.ObjectId,
ref: 'OtherModel',
required: true,
},
},
}, options);
Our OtherModel has a schema that looks like this:
const OtherModel = new Schema({
name: String,
funFact: String,
}, options);
From the front end of my application I'd like to query the RandomSchema model and return all instances of this model where subDoc.refDoc.funFact === someValue.
Is this possible? I know we have ways to populate those subdocs when return them but it happens only after matching docs have been returned, when in this case we'd need to know more than just the objectId of refDoc.
If multiple collections are involved, this task requires use of aggregation pipeline.
I am trying to set 2 fields to being unique to each other and not have duplicates.
The code is this :
const Connection = mongoose.model("Connection", new mongoose.Schema({
from_friend: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Friend'
},
to_friend: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Friend'
}
}))
exports.Connection = Connection;
You can do this using a unique index that includes both fields
const ConnectionSchema = mongoose.Schema({
from_friend: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Friend'
},
to_friend: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Friend'
}
});
ConnectionSchema.index({ from_friend: 1, to_friend: 1 }, { unique: true });
module.exports = mongoose.model('Connection', ConnectionSchema);
The unique Option is Not a Validator
A common gotcha for beginners is that the unique option for schemas is not a validator. It's a convenient helper for building MongoDB unique indexes. See the FAQ for more information.
From the FAQ:
Q. I declared a schema property as unique but I can still save duplicates. What gives?
A. Mongoose doesn't handle unique on its own: { name: { type: String, unique: true } } is just a shorthand for creating a MongoDB unique index on name. For example, if MongoDB doesn't already have a unique index on name, the below code will not error despite the fact that unique is true.
var schema = new mongoose.Schema({
name: { type: String, unique: true }
});
var Model = db.model('Test', schema);
Model.create([{ name: 'Val' }, { name: 'Val' }], function(err) {
console.log(err); // No error, unless index was already built
});
However, if you wait for the index to build using the Model.on('index') event, attempts to save duplicates will correctly error.
You will need to write your own custom validator.
If the built-in validators aren't enough, you can define custom validators to suit your needs.
Custom validation is declared by passing a validation function. You can find detailed instructions on how to do this in the SchemaType#validate() API docs.
I have a mongoose model that looks something like this
var LogSchema = new Schema({
item: {
type: ObjectId,
ref: 'article',
index:true,
},
});
But 'item' could be referenced from multiple collections. Is it possible to do something like this?
var LogSchema = new Schema({
item: {
type: ObjectId,
ref: ['article','image'],
index:true,
},
});
The idea being that 'item' could be a document from the 'article' collection OR the 'image' collection.
Is this possible or do i need to manually populate?
Question is old, but maybe someone else still looks for similar issues :)
I found in Mongoose Github issues this:
mongoose 4.x supports using refPath instead of ref:
var schema = new Schema({
name:String,
others: [{ value: {type:mongoose.Types.ObjectId, refPath: 'others.kind' } }, kind: String }]
})
In #CadeEmbery case it would be:
var logSchema = new Schema({
item: {type: mongoose.Types.ObjectId, refPath: 'kind' } },
kind: String
})
But I did't try it yet...
First of all some basics
The ref option says mongoose which collection to get data for when you use populate().
The ref option is not mandatory, when you do not set it up, populate() require you to give dynamically a ref to him using the model option.
#example
populate({ path: 'conversation', model: Conversation }).
Here you say to mongoose that the collection behind the ObjectId is Conversation.
It is not possible to gives populate or Schema an array of refs.
Some others Stackoverflow people asked about it.
Soluce 1: Populate both (Manual)
Try to populate one, if you have no data, populate the second.
Soluce 2: Change your schema
Create two link, and set one of them.
var LogSchema = new Schema({
itemLink1: {
type: ObjectId,
ref: 'image',
index: true,
},
itemLink2: {
type: ObjectId,
ref: 'article',
index: true,
},
});
LogSchema.find({})
.populate('itemLink1')
.populate('itemLink2')
.exec()
Dynamic References via refPath
Mongoose can also populate from multiple collections based on the value of a property in the document. Let's say you're building a schema for storing comments. A user may comment on either a blog post or a product.
body: { type: String, required: true },
on: {
type: Schema.Types.ObjectId,
required: true,
// Instead of a hardcoded model name in `ref`, `refPath` means Mongoose
// will look at the `onModel` property to find the right model.
refPath: 'onModel'
},
onModel: {
type: String,
required: true,
enum: ['BlogPost', 'Product']
}
});
const Product = mongoose.model('Product', new Schema({ name: String }));
const BlogPost = mongoose.model('BlogPost', new Schema({ title: String }));
const Comment = mongoose.model('Comment', commentSchema);
I am trying to write a schema that takes the parameters required by an activity. I am want to add a field 'activityParameters' that will be case specific depending on the activityType. Suppose if the activityType is email then the activityParameters should store details like'to:String, from:String, subject: String, body: String' and if the activity is "export" then it should store parameters like 'path:String' . Different types of activity will have different parameters. Please help me how to do this.
var activity_type = {
values: 'email export'.split(' '),
message: 'validation failed for path `{PATH}` with value `{VALUE}`'
};
var activitySchema = new Schema({
activityName: String,
activityDescription: String,
executionTime: {type: Date , default: null},
activityStartTime: {type: Date , default: null},
activityCompletionTime: {type: Date , default: null},
activityType: {type:String, enum: activity_type},
//activityParameters: ,
appName : String,
activityRetryCount: {type:Number,default:0},
createdOn: {type:Date , default:Date.now},
deletedOn: {type: Date , default: null},
updatedOn: {type: Date , default: null}
});
There's really no good answer for doing this with mongoose and maintaining a strongly typed schema.
You can specify all the fields for each type on the schema and then use them depending on type (export vs message).
var activitySchema = new Schema({
...
activityParameters: {
to:String,
from:String,
path:String
}});
You might consider having a key per subtype to be an improvement:
var activitySchema = new Schema({
...
emailActivityParameters:{
to:String,
from:String,
},
exportActivityParameters:{
path:String,
}
});
It would be pretty easy to access each "subdocument" depending on the activity type.
Finally, you can have a key called activityParameters and have that be freeform json:
var activitySchema = new Schema({
...
activityParameters: {}
});
In this case you can preserve your schema integrity using custom validators.
If these approaches don't appeal then maybe mongoose isn't the right tool you. You could use a lower level mongo driver and then something like Typescript or json schema to validate your models before you save them to mongoose. Check out this, for example: https://github.com/litixsoft/lx-valid.
Using the following line of code I can update a given Transaction (document), but when I try to update properties of its subdocument, the given value is not persisted.
Transaction.findById('55cf89abe148323e5368dcd5').populate('cryptocurrencies')
.then(function(transaction){
transaction.status = 'completed'; // this updates the transaction status correctly
transaction['cryptocurrencies'][0].status = 'ordered'; // this update is not persisted
return transaction.save()
.then(function(transaction){
console.log(transaction['cryptocurrencies'][0].status); // this shows the status as updated, but it's not persisted
})
})
I've also used the line transaction['cryptocurrencies'][0].markModified('status'); after I update the property to no avail. Can't find anything in the docs: What am I missing?
Update: I've tested this further and found that I have to use the .save() method on both the document and its subdocument. Is there any I can run a method that will save the document with its subdocument properties changed, or do I have to run two operations to save one document each time?
Update:
Here is my model code:
'use strict';
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var TransactionSchema = new Schema({
userId: { type: String, required: true },
status: { type: String, enum: ['unpaid', 'failed', 'paid', 'ordered', 'received', 'withdrawn', 'completed'] },
invoice: String,
saltStatus: String,
saltTransactionId : Number,
saltBank: String,
saltConfirmation: String,
saltAmount : Number,
saltDate : String,
saltResponseCode: String,
cryptocurrencies: [{ type: mongoose.Schema.Types.ObjectId, ref: 'CryptoCurrency' }]
});
var CryptoCurrencySchema = new Schema({
currencyName: String,
price: Number,
amount: Number,
total : Number,
walletAddress: String,
dollarsPaid : Number,
exchangeTransactionId: Number,
coinTransactionId : String,
status: { type: String, enum: ['ordered', 'received', 'withdrawn'] }
});
module.exports.Transaction = mongoose.model('Transaction', TransactionSchema);
module.exports.CryptoCurrency = mongoose.model('CryptoCurrency', CryptoCurrencySchema);
From your code example it seems that you are saving the sub-documents as references which means that once you update the sub-document, you only need to call .save() for it and not the parent document as well.
If you are saving your documents as sub-schemas, once updating the sub-document you can call .save() only for the parent document and it will persist the child document as well.
From the docs:
Sub-documents enjoy all the same features as normal documents. The
only difference is that they are not saved individually, they are
saved whenever their top-level parent document is saved.
http://mongoosejs.com/docs/subdocs.html