I've read a bunch about multitenancy setup options in Mongoose/MongoDB. The options are:
Separate databases. This is a no-no because of the overhead per database instance
Prefixed collections in the same database - I would have tenant1_Products and tenant2_Products which share the same schema.
Standard collections with a reference back to the tenant document.
It seems to be that #2 is the best option for scaling, since it allows for easy sharding by prefix. So, assuming that, I would implement something like this to dynamically retrieve the model for a certain tenant:
tenant.methods.getModel = function(name) {
return mongoose.model(this.uniqid + '_' + name);
}
Assuming I register all models for all tenants ahead of time with the correct schema, then I could do var productModel = myTenant.getModel('Product');, and then create/update from there.
The problem with this is when you need to implement document references to make use of Mongoose's populate method. E.g.,
var productSchema = new mongoose.Schema({
name:String,
categories:[
{
type:mongoose.Schema.Types.ObjectId,
// This will need to change to "tenant1_Category", etc
ref:'Category'
}
]
});
mongoose.model('tenant1_Product', productSchema);
tenant1.getModel('Product').find().populate('categories').exec(function(err, results) {
...
});
Essentially for this you'd need either need to make the ref property dynamic (IE, a function) that returns a different collection name depending upon some variable that defines the current tenant, or you could keep it static but find some way of telling the populate method to automatically add the appropriate prefix.
What is the best way of accomplishing this? Or, does it make more sense to go with option #3 above to avoid these issues? If so, does that have an effect on sharding/scaling methods?
For anyone interested in this, I went ahead and created a package that handles all of this, since I couldn't find an existing one.
https://www.npmjs.org/package/mongoose-multitenant
It uses separate collections per tenant and compiles the schema as needed, optionally modifying the ref attributes for related documents to keep it within the tenancy.
Related
It's my first time creating an application solo (back-end, front-end, design) and I could use some guidance on my back-end.
Let's say there is a table of Users, Jobs, and Applications (Job Applications). Right now I have it structured like so:
UserSchema {
// other attributes
_id: String,
application_ids: [String] // array of String id's
}
JobSchema {
// other attributes
_id: String,
application_ids: [String]
}
ApplicationSchema {
// other attributes
_id: String,
applicant_id: String, // the _id of a User
job_id: String // the _id of a Job
}
My current plan is like when a new Application is created (via POST /api/applications where I send the User's _id and Job's _id) I would then set the Application's applicant_id to the User's _id and then update the User's application_ids array as well as the Job's application_ids array.
Question: Am I going about this in a reasonable manner or am I touching too many tables for one POST request? There are other tables/schemas in my application that will follow a similar relationship structure. Then there's the matter of deleting Applications and then having to update application_ids again and etc, etc but that's another matter.
Note: I am using Express.js and Mongoose.js, if that helps
No, you shouldn't do it this way. By storing the ID of the user and job in the application, you can use a query to get all the applications by user or all applications for a given job. No need to touch both.
If you really want to have the relationship on both sides, at least set it up as an ObjectId and use the "ref" declaration. Check out the populate docs in the mongoose docs.
How, by using the 'findOne'/'find' functions of mongoose, can I find a specific document, where the results are filtered by a virtual field which does not appear physically in the db?
I'll try to further explain:
In my 'User' model, I have a field called 'email'. I want to create an alias for 'email' as 'userkey' for other functions in my nodejs app (mostly login validations and stuff).
To create the alias, I did this:
userSchema.virtual('userkey').get(function()
{
return this.email;
});
Now, after I created the alias, I want to filter my search result the following way:
restApi.post('/login', function (req, res) // (using expressjs)
{
User.findOne({'userkey': req.body.userkey}).exec(..);
}
How can I do this efficiently and without creating any more methods\field etc to save up time and memory?
The alias is very important, because in the future of the app I'll add more ways to login, and I want to be able to quickly switch between fields to be associated with 'userkey'. This alias saves the trouble of changing the entire app (which is a huge programming principle), and can help me to add more logic which is going to be necessary in the future.
You can't.
Virtuals do not exist on the document stored in mongodb, so there's nothing to query on.
I've been using Loopback to create an API. The documentation is generally really good but doesn't really answer my question about the following: how do I extend (not replace) a built in model?
The most promising piece of information came from this page - it specifies the way of basing a class from another class, via inheritance. This is useful but not ideal - I'd like to create relationships to custom models from the stock models, for example - "Role" should have many "Permission".
The page I mention also shows a Javascript file, located at common/models/<modelName>.js, where it states you can "extend" a model based on the properties and options you give it. The server never seems to hit the file... For example - I put a file in common/models/role.js with the following content:
var properties = {
exampleProperty: {type: String, required: true}
};
var user = loopback.Model.extend('Role', properties);
console.log('test');
First off, it doesn't seem to hit the file at all (no console.log output given). Second, obviously because of the first point, it doesn't extend the model with the properties I created.
Am I missing something obvious or is the documentation just plain wrong?
You should generate a new model via slc loopback:model named user. By default, the built in user is named User, which is why you can use lowercase user or even UserModel if you prefer. Then when you are prompted by the model generator for a base model, choose User. See https://github.com/strongloop/loopback-faq-user-management/blob/master/common/models/user.json#L3
I am currently playing around with node.js and MongoDB using the node-mongo-native driver.
I tested a bit around using the Mongo console storing and retrieving JS objects. I figured out, that if I store an object that contains functions/methods the methods and functions will also be stored in the collection. This is interesting since I thought that functions could not be stored in MongoDB (with the exception of the system.js collection, as suggested by the Mongo docs).
Also it will not only store the methods but actually each method and member of the object's entire prototype chain. Besides that I dont like this behaviour and think it's unintuitive I mustn't have it.
I was going to manage users in a Mongo collection. To do this I have a User object containing all of the users methods functioning as a prototype for each instance of an user. The user object itself would only contain the users attributes.
If I store a user in the Mongo collection I only want to store the own properties of the user object. No prototype members and especially no prototype methods. Currently I do not see how to cleanly do this. The options that I figured might work are:
creating a shallow copy using foreach and hasOwnProperty and storing this copy in the collection.
Add a data attribute to each user that contains all the object's attributes and can be stored in the collection.
This just came to my mind writing this: I could also set all the prototypes properties to not enumerable which should prevent them from being stored in the collection.
However, I do have the same issues the other way around: when loading a user from a collection. AFAIK there is no way to change an objects prototype in JavaScript after it was created. And there's also no way to specify a prototype to use when Mongo instantiates objects it retrieved from a collection. So basically I always get objects that inherit from Object using Mongo. As far as I can tell I have 2 options to restore a usable user object from this point on:
Create a fresh object inheriting from User and copying each attribute on the result object to the newly created object. (Compatible to storing mechanisms 1 & 3)
Create a fresh object inheriting from User and storing the result object as a data attribute on the newly created object. (Compatible to storing mechanism 2)
Are my assumptions, especially about the possibility to specify a prototype for query results, correct? What's the right way to do it, and why? I'm surely not the first person struggling to store and resurrect objects in/from MongoDB using node.js.
Currently I would go with the approach 2/2. I don't really like it, but it is the most efficient and the only one that works cleanly with the API. However, I'd much rather hear that actually the API does nothing wrong, but I do for not knowing how to use it correctly. So please, enlighten me :)
I just recently realized, that it actually is possible to change an objects prototype in V8/node. While this is not in the standard it is possible in various browsers and especially in V8/node!
function User(username, email) {
this.username = username;
this.email = email;
}
User.prototype.sendMail = function (subject, text) {
mailer.send(this.email, subject, text);
};
var o = {username: 'LoadeFromMongoDB', email: 'nomail#nomail.no'};
o.__proto__ = User.prototype;
o.sendMail('Hello, MongoDB User!', 'You where loaded from MongoDB, but inherit from User nevertheless! Congratulations!');
This is used all over various modules and plugins - even core modules make use of this technique, allthough it is not ECMAScript standard. So I guess it is safe to use within node.js.
I'm not sure I'm following you question exactly... but fwiw one thing came to mind: Have you checked out the Mongoose ORM? (http://mongoosejs.com/)
It gives you a lot of options when it comes to defining models and methods. In particular "Virtuals" might be of interest (http://mongoosejs.com/docs/virtuals.html).
Anyway, hope it helps some!
I've been exploring Sub Sonic 3's SimpleRepository and have been pretty happy with it but have a question regarding transactions. I am aware that using methods like 'AddMany' and 'DeleteMany' will automatically perform all of those operations within a single transaction, but was wondering if it's possible to force the SimpleRepository to perform the add or update of two different object types within the same transaction. For example, let's say I have the notion of two different but related entities in my application: a User and a Profile. Every user has to have a Profile and every Profile belongs to one and only one user. When a new user registers with my application I want them to provide the basic user information (credentials, name, e-mail) but also want some additional "profile" information (about me, gender, zip code, etc.)
I'd like to be able to perform the add of the User and the Profile object within a single transaction, but since it requires two distinct calls to the 'Add' method with different type parameters I'm not sure how to make this work.
You can do this using a transaction as follows:
using (TransactionScope transactionScope = new TransactionScope())
{
using (SharedDbConnectionScope connectionScope = new SharedDbConnectionScope())
{
// Add your user
// Add your profile
transactionScope.Complete();
}
}
adam - i think the code above is correct but is actually the wrong way around, it should be:
using (SharedDbConnectionScope connectionScope = new SharedDbConnectionScope())
{
using (TransactionScope transactionScope = new TransactionScope())
{
// Add your user
// Add your profile
transactionScope.Complete();
}
}
cheers
jimi