I currently have a posts table and a users table.
My Posts model:
id: {
type: Sequelize.INTEGER,
},
title: {
type: Sequelize.STRING,
},
images: {
type: Sequelize.STRING,
},
My Users model:
id: {
type: Sequelize.INTEGER,
},
name: {
type: Sequelize.STRING,
},
Each user can have many posts and each post can only have 1 user.
Association:
Users.hasMany(Posts, {
foreignKey: "id",
});
Posts.belongsTo(Users, {
foreignKey: "id",
});
In my posts table images is a string, a user can post many images in one post (hence an array of filename of images).
in my post method, I am using JSON.stringify to store the images filename as a string in the database as without JSON.stringify i get an error saying images cannot be an object or array. After hours of research i found that it is not best practise to use JSON.stringify on an array.
If that is correct, then how should i store my array of filename of images in Postgres using sequelize?
In my posts table images is a string, a user can post many images in one post (hence an array of filename of images).
Excuse me if I get preachy about this, but you should mend your evil ways and return to the golden path of normalized relational data modelling.
It may seem intuitive to store a relationship as an array of "foreign keys", but I promise you that you will suffer in the long run:
you cannot enforce the relationship with a constraint, which leads to inconsistencies
any attempt to join the tables via that relationship will lead to unnecessarily complicated SQL statements that won't perform and scale well
adding a new relationship means modifying (that is, rewriting) a potentially large table row
Use a "junction table" to model such relationships: such a table consists of two foreign keys, each pointing to one of the tables that are related. An entry means that the two referred rows are related.
If an image can only belong to a single post, the solution is even simpler: add a foreign key constraint to the images table.
Related
I am using Dynamodb with nodejs for my reservation system. And Dynamoose as ORM. I have two tables i.e Table and Reservation. To create relation between them, I have added tableId attribute in Reservation which is of type Model type (of type Table type), as mentioned in the dyanmoose docs. Using the document.populate I am able to get the Table data through the tableId attribute from Reservation table. But how can I retrieve all Reservation for a Table? (Reservation and Table has one to many relation)?
These are my Model:
Table Model:
const tableSchema = new Schema ({
tableId: {
type: String,
required: true,
unique: true,
hashKey: true
},
name: {
type: String,
default: null
},
});
*Reservation Model:*
const reservationSchema = new Schema ({
id: {
type: Number,
required: true,
unique: true,
hashKey: true
},
tableId: table, \\as per doc attribute of Table (Model) type
date: {
type: String
}
});
This is how I retrieve table data from reservation model
reservationModel.scan().exec()
.then(posts => {
return posts.populate({
path: 'tableId',
model: 'Space'
});
})
.then(populatedPosts => {
console.log('pp',populatedPosts);
return {
allData: {
message: "Executedddd succesfully",
data: populatedPosts
}
}
})
Anyone please help to retrieve all Reservation data from Table??
As of v2.8.2, Dynamoose does not support this. Dynamoose is focused on one directional simple relationships. This is partly due to the fact that we discourage use of model.populate. It is important to note that model.populate does another completely separate request to DynamoDB. This increases the latency and decreases the performance of your application.
DynamoDB truly requires a shift in how you think about modeling your data compared to SQL. I recommend watching AWS re:Invent 2019: Data modeling with Amazon DynamoDB (CMY304) for a great explanation of how you can model your data in DynamoDB in a highly efficient manner.
At some point Dynamoose might add support for this, but it's really hard to say if we will.
If you truly want to do this, I'd recommend adding a global index to your tableId property in your reservation schema. Then you can run something like the following:
async function code(id) {
const reservation = await reservationModel.get(id);
const tables = await tableModel.query("tableId").eq(id).exec(); // This will be an array of `table` entries where `"tableId"=id`. Remember, it is required you add an index for this to work.
}
Remember, this will cause multiple calls to DynamoDB and isn't as efficient. I'd highly recommend watching the video I linked above to get more information about how to model your data in an more efficient manner.
Finally, I'd like to point out that your unique: true code does nothing. As seen in the Dynamoose Attribute Settings Documentation, unique is not a valid setting. In your case since you don't have a rangeKey, it's not possible for two items to have the same hashKey, so technically it's already a unique property based on that. However it is important to note that you can overwrite existing items when creating an item. You can set overwrite to false for document.save or Model.create to prevent that behavior and throw an error instead of overwriting your document.
I have a standard mongoose User Scheme, and a 'Uniform' scheme that holds dress items in the following way:
mongoose.Schema({
user: { type: mongoose.Schema.Types.ObjectId, ref: 'User' },
items: { type: [String], unique: true }
})
Thing is, I want the items in the 'items' array to be unique only in the scope of the user. Meaning that I don't want a user to have serval uniforms with the same item. But I don't have any problem with other users having the same items as my user.
The current scheme prevents ANY uniform to have the same values inside 'items', I want it to be just within the scope of the user.
Can that even be done?
(In ruby on rails, the line that creates this behavior is: validates_uniqueness_of :items, scope: [:user])
If User and Uniform are different schemas (different collections) then having uniqueness achieved between keys of different collections cannot be done in mongodb.
We can have uniqueness achieved by combining different keys of a same collection using Compound Index
I'm building a Node/Express/Postgres version of an app that I already built in Rails. I'm learning Node, so I figured I'd rebuild something that I know works.
For now, I'm dumping everything in one file (set up my database, defined my models, etc.), just to make sure I have everything set up correctly before I divvy them up into different files.
I set up my postgres database at the very top of the file, like so:
var Sequelize = require('sequelize');
var db = new Sequelize('my_database_name', 'my_username', null, {
host: 'localhost',
dialect: 'postgres',
});
With regard to my models, I have a Politician model:
var Politician = db.define("politician", {
name: {
type: Sequelize.STRING,
},
politicalParty: {
type: Sequelize.STRING
}
});
A Category model:
var Category = db.define("category", {
name: {
type: Sequelize.STRING
},
keywords: {
type: Sequelize.ARRAY(Sequelize.TEXT)
},
});
And a join model of Politician and Category, called "Interest". Because Interest is a join model, it will have a "politicianId" and "categoryId" properties....but will those properties automatically generate in the database? And so, is this how I would define the Interest model, with no properties?
Interest Model:
var Interest = db.define("interest")
Or, will I have to be specific, and create "politicianId" and "categoryId" properties? Like so:
Interest Model:
var Interest = db.define("interest", {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
categoryId: {
type: Sequelize.INTEGER,
foreignKey: true
},
politicianId: {
type: Sequelize.INTEGER,
foreignKey: true
}
});
Also, do I need the "foreignKey: true" bit? Or will it automatically know that those properties are foreign keys? Also, do I need the "id" property? I know models automatically create their own primary key "id"...but again, I've been at this for hours, looking at docs, and trying everything.
I then defined my associations (again, all of this is the same file):
Politician.belongsToMany(Category, {through: "Interest"});
Category.belongsToMany(Politician, {through: "Interest"});
The Node/Sequelize docs seems to suggest that defining those 2 associations above will automatically "create a new model called Interest with the equivalent foreign keys politicianId and categoryId." So, do I even need to define a "Interest" model? Also, do I need the follow associations to describe that Interest belongs to Politician and Category?
Interest.belongsTo(Politician);
Interest.belongsTo(Category);
If I don't write the associations saying that Interest belongs to Politican and Catetory, I don't get the "politicianId" and "categoryId" columns in the Interest table. Just the "id" and createdAt/updatedAt columns.
I then created an instance of Politician, Category, and Interest, to persist everything to the database, to see if everything is there and set up correctly:
Politician Object:
var politician1 = Politician.sync({force: true}).then(function(){
return Politician.create(aPoliticianObjectDefinedInthisFile);
});
This works perfectly. I see this object in the politician table in the database.
Category Object:
var category1 = Category.sync({force: true}).then(function(){
return Category.create(aCategoryObjectDefinedInThisFile);
});
This works perfectly. I see this object in the category table in the database.
Here is what doesn't work. Creating an instance/object of Interest and synching it to the database. My thinking is, if I put integers as values, it will know that "politicianId: 1" means point to the politician object with an id of 1, and the same for "categoryId: 1". But when I write it as I have it below, the Interest table doesn't even show up in the Postgres database at all.
Interest Object:
Interest.sync({force: true}).then(function(){
return Interest.create(
{
politicianId: 1,
categoryId: 1
}
);
});
However, when I create the object of Interest like this, with no properties defined, the Interest table appears in the database, along with the "politicianId" and "categoryId" columns, however, those columns are empty. The object's primary id is in there at 1, and the "createdAt" and "updatedAt" columns have data too. But the foreign key columns are blank.
Interest Object:
Interest.sync({force: true}).then(function()
{
return Interest.create(
{
// No properties defined.
}
);
}
);
Sorry for this long post, lol, but, in all:
Am I creating the "Interest" model correctly?
Am I writing the associations for "Interest" correctly?
Do I even need to write associations for Interest, if I already have associations for its parent classes, Politican and Category defined?
In my Rails app, my associations for Politican and Category are like so:
Politician has_many interests
Politican has_many categories through interests
Category has_many interests
Category has_many politicians through interests
Interest belongs_to politician
Interest belongs_to category
But I use the "belongsToMay" association in Node because I got an error telling me to do so.
Basically, I need to create an instance of Politician, an instance of Category, and an instance of Interest that has "politicianId" and "categoryId" columns that point to those aforementioned instances of those classes.
politicanABC -- id: 1
categoryABC -- id: 1
instanceABC -- id: 1; politicanId: 1 (referring to politicanABC); categoryid: 1 (referring to categoryABC).
My app is set up like that in Rails and works wonderfully.
Help and thank you in advance :-)
You don't have to define the Interest model if you are not going to add any additional fields. Sequelize will internally define the model and add all required fields once you do following:
Politician.belongsToMany(Category, {through: "Interest"});
Category.belongsToMany(Politician, {through: "Interest"});
Sync needs to run on database level and not on tables since Interest model is implicit at this point.
db.sync({force: true});
Sequelize will add relationship build methods on both Politician and Category instances. Category will have methods addPolitician(), addPoliticians([]), setPoliticians([]), getPliticians(). Politician instances will have similar functions to associate categories to them. You can connect these after create option is performed on both objects successfully.
Politician.create({name: 'John Doe', politicalParty: 'Nice Party'})
.then(function(politician) {
Category.create({name: 'Nicers'})
.then(function(category) {
politician.addCategory(category);
});
});
You can also search and associate existing items using helper methods. Alternatively you can associate objects manually by accessing db.models.Interest model and running creates on it.
I have two collections in MongoDB: one saves post data of blog, the other saves comment data of blog with below schemas. How can I use nodejs and mongoose to query all posts with comment belong to it and respond to single page application?. Thanks!
var PostSchema = mongoose.Schema({
created: {
type: Date,
default: Date.now
},
content: {
type: String,
default: '',
trim: true
},
user: {
type: Schema.ObjectId,
ref: 'user'
}
});
var CommentSchema = mongoose.Schema({
created: {
type: Date,
default: Date.now
},
content: {
type: String,
default: '',
trim: true
},
ofpost: {
type: Schema.ObjectId,
ref: 'post' //which post this comment belong to
},
user: {
type: Schema.ObjectId,
ref: 'user'
}
});
var Post = mongoose.model('Post', PostSchema);
var Comment = mongoose.model('Comment', CommentSchema);
//example:the Comment1 and Comment2 belong to Post1
var Post1 = new Post({ content: 'good day', user: 'John' });
var Comment1 = new Comment({content: 'yeah', ofpost: Post1._id, user:'Tom'})
var Comment2 = new Comment({content: 'agree', ofpost: Post1._id, user:'Tina'})
As mongodb is NoSQL type of database and has no JOIN's or any sort of relationship between documents, you have to take care of such.
There are generally two ways to do so:
Caching
Consider storing comments data within blog document. You can have embedded documents without any problem. In reality it leads to some extra caches, like comments count, array of user id's of comments and other stuff that will make your queries indexed and more easy ways to search through collection.
Multiple Queries
If you still need separate collections, then you need to 'simulate' joins. Most efficient ways is to make temporary indexing arrays and multiple queries to different collections. Usually it should be just 2 queries for one Join (many to many), and small iteration to add second query documents to first array of documents.
Here is the flow that is suitable and performs well still, on example:
Two collections, first is posts, and second is comments which has id of post.
Make query to posts.
Iterate through each post and add its id into postIds array, as well make postMap object where key will be id of post and value will be specific post. - this is so called indexing posts.
Make query to comments collection with $in argument with postIds array of post id's. This collection should have indexing on post id field in order to make this query very efficient. As well this query can include sorting by date (additional compound indexing will speedup it).
Iterate through each comment and using postMap add it to comments array of post.
So we have only 2 queries, and one iteration through all comments to embed data into posts O(n). Without second step, adding to posts will be potentially O(p*c) where p - number of posts and c - number of comments. Which is obviously much slower as well on big queries can be potentially slow.
Summary
Second approach is more manageable approach from data point of view, as well is easier on writes, while is more complicated on reads.
Still will require some caching, like number of comments for blog posts.
I have two collections:
Users
Uploads
Each upload has a User associated with it and I need to know their details when an Upload is viewed. Is it best practice to duplicate this data inside the the Uploads record, or use populate() to pull in these details from the Users collection referenced by _id?
OPTION 1
var UploadSchema = new Schema({
_id: { type: Schema.ObjectId },
_user: { type: Schema.ObjectId, ref: 'users'},
title: { type: String },
});
OPTION 2
var UploadSchema = new Schema({
_id: { type: Schema.ObjectId },
user: {
name: { type: String },
email: { type: String },
avatar: { type: String },
//...etc
},
title: { type: String },
});
With 'Option 2' if any of the data in the Users collection changes I will have to update this across all associated Upload records. With 'Option 1' on the other hand I can just chill out and let populate() ensure the latest User data is always shown.
Is the overhead of using populate() significant? What is the best practice in this common scenario?
If You need to query on your Users, keep users alone. If You need to query on your uploads, keep uploads alone.
Another question you should ask yourself is: Every time i need this data, do I need the embedded objects (and vice-versa)? How many time this data will be updated? How many times this data will be read?
Think about a friendship request:
Each time you need the request you need the user which made the request, then embed the request inside the user document.
You will be able to create an index on the embedded object too, and your search will be mono query / fast / consistent.
Just a link to my previous reply on a similar question:
Mongo DB relations between objects
I think this post will be right for you http://www.mongodb.org/display/DOCS/Schema+Design
Use Cases
Customer / Order / Order Line-Item
Orders should be a collection. customers a collection. line-items should be an array of line-items embedded in the order object.
Blogging system.
Posts should be a collection. post author might be a separate collection, or simply a field within posts if only an email address. comments should be embedded objects within a post for performance.
Schema Design Basics
Kyle Banker, 10gen
http://www.10gen.com/presentation/mongosf2011/schemabasics
Indexing & Query Optimization
Alvin Richards, Senior Director of Enterprise Engineering
http://www.10gen.com/presentation/mongosf-2011/mongodb-indexing-query-optimization
**These 2 videos are the bests on mongoddb ever seen imho*
Populate() is just a query. So the overhead is whatever the query is, which is a find() on your model.
Also, best practice for MongoDB is to embed what you can. It will result in a faster query. It sounds like you'd be duplicating a ton of data though, which puts relations(linking) at a good spot.
"Linking" is just putting an ObjectId in a field from another model.
Here is the Mongo Best Practices http://www.mongodb.org/display/DOCS/Schema+Design#SchemaDesign-SummaryofBestPractices
Linking/DBRefs http://www.mongodb.org/display/DOCS/Database+References#DatabaseReferences-SimpleDirect%2FManualLinking