The issue is a self referencing schema such as employee has a supervisor that is an employee. This is tracked with a FK of supervisor_id in the employees table. The FKS is described below:
"fks": [
{
"referencedTableName": "employees",
"name": "supervisor_id_fks",
"label": "Supervisor",
"foreignKeyNames": [
"supervisor_id"
],
"referencedKeyNames": [
"id"
],
"reverseLabel": "Subordinate"
}
Once deployed, the FKS association and the reverse are available in the select options, however, these items are not selectable. Please let me know if there is something I am missing.
I'm afraid it is a bug/missing feature.
There is code to unravel joins when you get back to the same join table.
Please could you raise a ticket:
https://github.com/salk31/RedQueryBuilder/issues
?
Any more background to your use case would be helpful. e.g. Do you only ever want/need the user to go one level deep?
Related
Summary:
I am building my first large scale full stack application(MERN stack) that is trying to mimic a large scale clothing store. Each article of clothing has many 'tags' that represent its features, top/bottom/accessory/shoes/ect, and subcategories, for example on top there is shirt/outerwear/sweatshirt/etc, and sub-sub-categories within it, for example on shirt there is blouse/t-shirt/etc. Each article has tags for primary colors, hemline, pockets, technical features, the list goes on.
Main question:
how should I best organize the data in mongodb with mongoose schemas in order for it to be quickly searchable when I plan on having 50,000 or more articles? And genuinely curious, how do large clothing retailers typically design databases to be easily searchable by customers when items have so many identifying features?
Things I have tried or thought of:
On the mongoDB website there is a recommendation to use a tree structure with child references. here is the link: https://docs.mongodb.com/manual/tutorial/model-tree-structures-with-child-references/ I like this idea but I read here: https://developer.mongodb.com/article/mongodb-schema-design-best-practices/ that when storing over a few thousand pieces of data, using object ID references is no longer sufficient, and could create issues because of datalimits.
Further, each clothing item would fall into many different parts of the tree. For example it could be a blouse so it would be in the blouse "leaf" of the tree, and then if its blue, it would be in the blue "leaf" of the tree, and if it is sustainably sourced, it would fall into that "leaf" of the tree as well. Considering this, a tree like data structure seems not the right way to go. It would be storing the same ObjectID in many different leaves.
My other idea was to store the article information (description, price, and picture) seperate from the tagging/hierarchical information. Then each tagging object would have a ObjectID reference to the item. This way I could take advantage of the propogate method of mongoose if I wanted to collect that information.
I also created part of the large tree structure as a proof of concept for a design idea I had, and this is only for the front end right now, but this also creates bad searches cause they would look like taxonomy[0].options[0].options[0].options[0].title to get to 'blouse'. Which from my classes doesnt seem like a good way to make the code readable. This is only a snippet of a long long branching object. I was going to try to make this a mongoose schema. But its a lot of work and I wanna make sure that I do it well.
const taxonomy = [
{
title: 'Category',
selected: false,
options: [
{
title: 'top',
selected: false,
options: [
{
title: 'Shirt',
selected: false,
options: [
{
title: 'Blouse',
selected: false,
},
{
title: 'polo',
selected: false,
},
{
title: 'button down',
selected: false,
},
],
},
{
title: 'T-Shirt',
selected: false,
},
{
title: 'Sweater',
selected: false,
},
{
title: 'Sweatshirt and hoodie',
selected: false,
},
],
},
Moving forward:
I am not looking for a perfect answer, but I am sure that someone has tackled this issue before (all big businesses that sell lots of categorized products have) If someone could just point me in the right direction, for example, give me some terms to google, some articles to read, or some videos to watch, that would be great.
thank you for any direction you can provide.
MongoDB is a document based database. Each record in a collection is a document, and every document should be self-contained (it should contain all information that you need inside it).
The best practice would be to create one collection for each logical whole that you can think of. This is the best practice when you have documents with a lot of data, because it is scalable.
For example, you should create Collections for: Products, Subproducts, Categories, Items, Providers, Discounts...
Now, when you creating Schemas, instead of creating nested structure, you can just store a reference of one collection document as a property of another collection document.
NOTE: The maximum document size is 16 megabytes.
BAD PRACTICE
Let us first see what would be the bad practice. Consider this structure:
Product = {
"name": "Product_name",
"sub_products": [{
"sub_product_name": "Subpoduct_name_1",
"sub_product_description": "Description",
"items": [{
"item_name": "item_name_1",
"item_desciption": "Description",
"discounts": [{
"discount_name": "Discount_1",
"percentage": 25
}]
},
{
"item_name": "item_name_2",
"item_desciption": "Description",
"discounts": [{
"discount_name": "Discount_1",
"percentage": 25
},
{
"discount_name": "Discount_2",
"percentage": 50
}]
},
]
},
...
]
}
Here product document has sub_products property which is an array of sub_products. Each sub_product has items, and each item has discounts. As you can see, because of this nested structure, the maximum document size would be quickly exceeded.
GOOD PRACTICE
Consider this structure:
Product = {
"name": "Product_name",
"sub_products": [
'sub_product_1_id',
'sub_product_2_id',
'sub_product_3_id',
'sub_product_4_id',
'sub_product_5_id',
...
]
}
Subproduct = {
"id": "sub_product_1_id",
"sub_product_name": "Subroduct_name",
"sub_product_description": "Description",
"items": [
'item_1_id',
'item_2_id',
'item_3_id',
'item_4_id',
'item_5_id',
...
]
}
Item = {
"id": "item_1_id",
"item_name": "item_name_1",
"item_desciption": "Description",
"items": [
'discount_1_id',
'discount_2_id',
'discount_3_id',
'discount_4_id',
'discount_5_id',
...
]
}
Discount = {
"id": "discount_1_id",
"discount_name": "Discount_1",
"percentage": 25
}
Now, you have collection for each logical whole and you are just storing a reference of one collection document as a property of another collection document.
Now you can use one of the best features of the Mongoose that is called population. If you store a reference of one collection document as a property of another collection document, when performing querying of the database, Mongoose will replace references with the actual documents.
I'm super excited about CouchDB 3, especially the Partitioned Database feature. I would like to set up something similar to envoy, where you have one database that syncs to many smaller databases.
I was wondering, what's the best way to implement one partition for one user, and only enable users to sync their own partitions. Is it possible to have global changes split into partition changes, using that log for syncing, or is that overthinking on my part?
You can add a selector to the replication document.
"selector": { "_id": { "$regex": "^[PartitionName]:" } }
You are right , the brackets expression should be replace by the actual partition name - final expression should be without brackets..
You can add a selector to the replication document.
e.g.
"selector": { "_id": { "$regex": "^CANADA:" } }
or
"selector": { "_id": { "$regex": "^USA:" } }
Currently I am working on a mobile app. Basically people can post their photos and the followers can like the photos like Instagram. I use mongodb as the database. Like instagram, there might be a lot of likes for a single photos. So using a document for a single "like" with index seems not reasonable because it will waste a lot of memory. However, I'd like a user add a like quickly. So my question is how to model the "like"? Basically the data model is much similar to instagram but using Mongodb.
No matter how you structure your overall document there are basically two things you need. That is basically a property for a "count" and a "list" of those who have already posted their "like" in order to ensure there are no duplicates submitted. Here's a basic structure:
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3")
"photo": "imagename.png",
"likeCount": 0
"likes": []
}
Whatever the case, there is a unique "_id" for your "photo post" and whatever information you want, but then the other fields as mentioned. The "likes" property here is an array, and that is going to hold the unique "_id" values from the "user" objects in your system. So every "user" has their own unique identifier somewhere, either in local storage or OpenId or something, but a unique identifier. I'll stick with ObjectId for the example.
When someone submits a "like" to a post, you want to issue the following update statement:
db.photos.update(
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3"),
"likes": { "$ne": ObjectId("54bb2244a3a0f26f885be2a4") }
},
{
"$inc": { "likeCount": 1 },
"$push": { "likes": ObjectId("54bb2244a3a0f26f885be2a4") }
}
)
Now the $inc operation there will increase the value of "likeCount" by the number specified, so increase by 1. The $push operation adds the unique identifier for the user to the array in the document for future reference.
The main important thing here is to keep a record of those users who voted and what is happening in the "query" part of the statement. Apart from selecting the document to update by it's own unique "_id", the other important thing is to check that "likes" array to make sure the current voting user is not in there already.
The same is true for the reverse case or "removing" the "like":
db.photos.update(
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3"),
"likes": ObjectId("54bb2244a3a0f26f885be2a4")
},
{
"$inc": { "likeCount": -1 },
"$pull": { "likes": ObjectId("54bb2244a3a0f26f885be2a4") }
}
)
The main important thing here is the query conditions being used to make sure that no document is touched if all conditions are not met. So the count does not increase if the user had already voted or decrease if their vote was not actually present anymore at the time of the update.
Of course it is not practical to read an array with a couple of hundred entries in a document back in any other part of your application. But MongoDB has a very standard way to handle that as well:
db.photos.find(
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3"),
},
{
"photo": 1
"likeCount": 1,
"likes": {
"$elemMatch": { "$eq": ObjectId("54bb2244a3a0f26f885be2a4") }
}
}
)
This usage of $elemMatch in projection will only return the current user if they are present or just a blank array where they are not. This allows the rest of your application logic to be aware if the current user has already placed a vote or not.
That is the basic technique and may work for you as is, but you should be aware that embedded arrays should not be infinitely extended, and there is also a hard 16MB limit on BSON documents. So the concept is sound, but just cannot be used on it's own if you are expecting 1000's of "like votes" on your content. There is a concept known as "bucketing" which is discussed in some detail in this example for Hybrid Schema design that allows one solution to storing a high volume of "likes". You can look at that to use along with the basic concepts here as a way to do this at volume.
I'm trying to figure out the best way of sorting by rank using couch db. I have my documents setup in a players db like so:
{
"_id": "user2",
"_rev": "31-65e0e5bb1eba8d6a882aad29b63615a7",
"username": "testName",
"apps": {
"app1": {
"score": 1000
},
"app2": {
"score": 1000
},
"app3": {
"score": 1000
}
}
}
The player can have multiple scores for various apps. I'm trying to figure out the best way to pull say the top 50 scores for app1.
I think one idea could be to store the score of each user for each app seperately. Like so: -
{"app":"app1","user":"user_id","score":"9000"}
The you could write a map function
emit([doc.app,doc.score],{_id:doc.user,score:doc.score})
and query the view like
/view_name?include_docs=true&startkey=["app1",{}]&endkey=["app1"]&descending=true
With this arrangement you have a view sorted by the score and the name of the app. Here are the results that you get.
{"total_rows":2,"offset":0,"rows":[
{"id":"61181c784df9e2db5cbb7837903b63f5","key":["app1",10000],"value":
{"_id":"5002539daa85a05d3fab16158a7861c1","score":10000},"doc":
{"_id":"5002539daa85a05d3fab16158a7861c1","_rev":"1-8441f2f5dbaaf22add8969cea5d83e1b","user":"jiwan"}},
{"id":"7f5d53b2da8ae3bea8e2b7ec74020526","key":["app1",9000],"value":
{"_id":"336c2619b052b04992947976095f56b0","score":9000},"doc":
{"_id":"336c2619b052b04992947976095f56b0","_rev":"3-3e4121c1831d7ecafc056e71a2502f3a","user":"akshat"}}
]}
You have score in value. User in doc.
Edit
Oops! I mistyped the startkey and endkey :) Notice that it is not startKey but startkey same for endkey. Also note that since descending is true we reverse the order of keys. It should work as expected now.
For more help check out
This answer and
This answer
In a contact management app, each user will have his own database. When users wish to share certain categories of contacts with others, a backend will initiate a replication. Each contact is its own document, but also has various children documents such as notes and appointments.
Here is an example...
Contact:
{
"_id": 123,
"type": "contact",
"owner": "jimmy",
"category": "customer",
"name": "Bob Jones",
"email": "bob#example.com"
}
Note:
{
"_id": 456,
"type": "note",
"owner": "jimmy",
"contact_id": 123,
"timestamp": 1383919278,
"content": "This is a note about Bob Jones"
}
So let's say Jimmy wants to share his only his customers with sales manager Kevin, while his personal contacts remain private. When the note passes through the replication filter, is it possible to access the linked contact's category field?
Or do I have to duplicate the category field in every single child of a contact? I would prefer not to have to do this, as each contact may have many children which I would have to update manually every time the category changes.
Here is some pseudo-code for the filter function:
function(doc, req)
{
if(doc.type == “contact”) {
if(doc.category == req.query.category) {
return true;
}
}
else if(doc.contact_id) {
if(doc.contact.category == req.query.category) {
return true;
}
}
return false;
}
If this is possible, please describe how to do it. Thanks!
There are some other options.
There's a not-so-well-known JOIN trick in CouchDB. Instead of using replication, however, you'll have to share the results of a MapReduce View -- unfortunately you can use a view as a filter for replication. If you're using Cloudant (disclaimer: I'm employed by Cloudant) you can use chained-MapReduce to output the result to another database that you could then replication from...
Additionally, I think this SO post/answer on document structures and this join trick could be helpful: Modeling relationships on CouchDB between documents?
No, this is not possible. Each document must be consistent so it has no any explicit relations with others documents. Having contact_id value as reference is just an agreement from your side - CouchDB isn't aware about this.
You need to literally have category document be nested within contact one to do such trick e.g. have single document to process by filter function. This is good solution from point when you need to have consistent state of contact document.