PouchDB/CouchDB Group By Value in Array - couchdb

I am using PouchDB and I have a dataset representing a social network in a graph. People are documents, and the people they follow are in an array of the _id of the person followed. Here is a sample of the data:
[
{
"_id": "mc0001",
"name": "Jill Martin",
"joined": "2020-01-15",
"follows": []
},
{
"_id": "mc0002",
"name": "Elena Markova",
"joined": "2020-01-21",
"follows": ["mc0001"]
},
{
"_id": "mc0003",
"name": "Carlos Sanchez",
"joined": "2020-01-27",
"follows": ["mc0001", "mc0002"]
},
{
"_id": "mc0004",
"name": "Ai Sato",
"joined": "2020-02-21",
"follows": ["mc0001", "mc0003"]
},
{
"_id": "mc0005",
"name": "Ming Wu",
"joined": "2020-03-21",
"follows": ["mc0002", "mc0003", "mc0004"]
}
]
What I would like to do is query for each person, and get a list of followers. I am looking for something like this:
[
{
"_id": "mc0001",
"followers": ["mc0002", "mc0003", "mc0004"]
},
{
"_id": "mc0002",
"followers": ["mc0003", "mc0005"]
},
{
"_id": "mc0003",
"followers": ["mc0004", "mc0005"]
},
{
"_id": "mc0004",
"followers": ["mc0005"]
},
{
"_id": "mc0005",
"followers": []
}
]
Is there a way to do this without changing the data structure (e.g. moving the followers array into the doc of the person being followed)?

Create a Map/Reduce view that loops through the follows array in each document and emits those; like this:
function (doc) {
for(var i =0; i<doc.follows.length; i++) {
emit(doc.follows[i], null);
}
}
You end up with an index keyed on a user and where each row has the id of a follower of that user. You can then query the index, supplying the key of the user whose followers you want to find, like this:
$URL/users/_design/users/_view/by-follower?key="mc0001"&reduce=false
You will get something like this:
{"total_rows":8,"offset":0,"rows":[
{"id":"mc0002","key":"mc0001","value":null},
{"id":"mc0003","key":"mc0001","value":null},
{"id":"mc0004","key":"mc0001","value":null}
]}
This is not exactly the format of the data you have in your question, but you can see that the id field in each object contains a follower of your desired user, so you can go from there.

Related

Mongoose how to find specific value

After a lot of reading, I am stuck.
the code I posted here, is the implementation of a store database I am trying to make.
in every store, we have some fields. I am interested in doing something with the items array, that contains JSON variables.
I want to filter the items through three filters, firstly by the store ID, secondly by the category ID, and the last filter will be the semi category ID.
I want to send the data from the front end, meaning I supply STOREID, the CategoryID, and the SemiCategoryID.
after receiving the data at the back end side, I am expecting to receive only the relevant items according to the data supplied by the front end.
{
"_id": {
"$oid": "5a1844b5685cb50a38adf5bb" --> **ID of the STORE**
},
"name": "ACE",
"user_id": "59e4c41105d1f6227c1771ea",
"imageURL": "none",
"rating": 5,
"items": [
{
"name": "NirCohen",
"categoryID": "5a0c2d292235680012bd12c9",
"semiCatID": "5a0c2d5a2235680012bd12ca",
"_id": {
"$oid": "5a1958181cd8a208882a80f9"
}
},
{
"name": "he",
"categoryID": "5a0c2d292235680012bd12c9",
"semiCatID": "5a0c2d5a2235680012bd12ca",
"_id": {
"$oid": "5a1973c40e561e08b8aaf2b2"
}
},
{
"name": "a",
"categoryID": "5a0c2d292235680012bd12c9",
"semiCatID": "5a0c2d5a2235680012bd12ca",
"_id": {
"$oid": "5a197439bc1310314c4c583b"
}
},
{
"name": "aaa",
"categoryID": "5a0c2d292235680012bd12c9",
"semiCatID": "5a0c2d5a2235680012bd12ca",
"_id": {
"$oid": "5a197474558a921bb043317b"
}
},
],
"__v": 9
}
and I want the Backend to return the filtered items according to the query.
The problem is, I am not managing to get the CORRECT query.
your help will be much appreciated,
thank you in advance.
If I understand you correctly, you are doing something like this:
Store.find({}).then(..);
If you only want to find the stores where categoryID is equal to the variable myCategory, you could filter them out by using:
Store.find({semiCatID: myCategory}).then(..);
Please let me know if this is not what you are after, then we can keep trying to figure this out together.
EDIT: So you are sending the variables StoreID, CategoryID and SemiCategoryID from the frontend. Receive them in the backend, and want to filter your database collection matching all three fields?
If so.. then I think all you have to do is change your current query:
store.findOne({ _id: req.body.userID }, (err, store) => { console.log(store); });
To something like:
store.findOne({
_id: req.body.userID,
storeID: req.body.StoreID,
categoryID: req.body.CategoryID,
semiCategoryID: req.body.SemiCategoryID
}, (err, store) => { console.log(store); });
This way, the objects you get back from mongo must match all four criterias given from the frontend.
As far as I Understood your question here is my answer to it you can use findById
Store.findById({//store id}).then(..);
or
Store.findOne({_id:ObjectID(storeID)}).then(..);

MongoDB: Query model and check if document contains object or not, then mark / group result

I have a Model called Post, witch contains an property array with user-ids for users that have liked this post.
Now, i need to query the post model, and mark the returned results with likedBySelf true/false for use in by client - is this possible?
I dont have to store the likedBySelf property in the database, just modify the results to have that property.
A temporary solution i found was to do 2 queries, one that finds the posts that is liked by user x, and the ones that have not been liked by user x, and en map (setting likedBySelf true/false) and combine the 2 arrays and return the combined array. But this gives some limitations to other query functions such as limit and skip.
So now my queries looks like this:
var notLikedByQuery = Post.find({likedBy: {$ne: req.body.user._id}})
var likedByQuery = Post.find({likedBy: req.body.user._id})
(I'm using the Mongoose lib)
PS. A typical post can look like this (JSON):
{
"_id": {
"$oid": "55fc463c83b2d2501f563544"
},
"__t": "Post",
"groupId": {
"$oid": "55fc463c83b2d2501f563545"
},
"inactiveAfter": {
"$date": "2015-09-25T17:13:32.426Z"
},
"imageUrl": "https://hootappprodstorage.blob.core.windows.net/devphotos/55fc463b83b2d2501f563543.jpeg",
"createdBy": {
"$oid": "55c49e2d40b3b5b80cbe9a03"
},
"inactive": false,
"recentComments": [],
"likes": 8,
"likedBy": [
{
"$oid": "558b2ce70553f7e807f636c7"
},
{
"$oid": "559e8573ed7c830c0a677c36"
},
{
"$oid": "559e85bced7c830c0a677c43"
},
{
"$oid": "559e854bed7c830c0a677c32"
},
{
"$oid": "559e85abed7c830c0a677c40"
},
{
"$oid": "55911104be2f86e81d0fb573"
},
{
"$oid": "559e858fed7c830c0a677c3b"
},
{
"$oid": "559e8586ed7c830c0a677c3a"
}
],
"location": {
"type": "Point",
"coordinates": [
10.01941398718396,
60.96738099591897
]
},
"updatedAt": {
"$date": "2015-09-22T08:45:41.480Z"
},
"createdAt": {
"$date": "2015-09-18T17:13:32.426Z"
},
"__v": 8
}
#tskippe you can use a method like following to process whether the post is liked by the user himself and call the function anywhere you want.
var processIsLiked = function(postId, userId, doc, next){
var q = Post.find({post_id: postId});
q.lean().exec(function(err,res){
if(err) return utils.handleErr(err, res);
else {
if(_.find(doc.post.likedBy,userId)){ //if LikedBy array contains the user
doc.post.isLiked = true;
} else {
doc.post.isLiked = false;
}
});
next(doc);
}
});
}
Because you are using q.lean() you dont need to actually persist the data. You need to just process it , add isLiked field in the post and send back the response. **note that we are manuplating doc directly. Also you chan tweek it to accept doc containing array of posts and iterating it and attach an isLiked field to each post.
I found that MongoDB's aggregation with $project tequnique was my best bet. So i wrote up an aggregation like this.
Explanation:
Since i want to keep the entire document, but $project purpose is to modify the docs, thus you have to specify the properties you want to keep. A simple way of keeping all the properties is to use "$$ROOT".
So i define a $project, set all my original properties to doc: "$$ROOT", then create a new property "likedBySelf", which is marked true / false if a specified USERID is in the $likedBy set.
I think that this is more clean and simple, than querying every single model after a query to set a likedBySelf flag. It may not be faster, but its cleaner.
Model.aggregate([
{ $project: {
doc: "$$ROOT",
likedBySelf: {
$cond: {
"if": { "$setIsSubset": [
[USERID],
"$likedBy"
]},
"then": true,
"else": false
}
}
}}
]);

find object inside JSON using nested ID

i have a mongo collection like this
{"stores": [{"name": "foo",
"songs": [ {"id": "", "name": "", "artist": "", "category": "", "tags": []} ],
"songsSchedule": [
{
"song_id": "",
"date": ,
"user": "",
"help": ,
"partners": [{"user": ""}],
"likes":
}
]
}]}
and i want to get the songs name and artist from the songsSchedule song_id, i've tried this but it's not working
var query = { _id: fn.generateID(req.params.store_id), songsSchedule: { $exists: true } };
var select = { songsSchedule:1 };
var array = [];
client("stores", function(err, collection) {
if (err)
return res.json(fn.status("30"));
collection.findOne(query, select, function(err, store) {
if (err || !store)
return res.json(fn.status("31"));
for (var i in store.songsSchedule) {
var song = store.songsSchedule[i];
array.push(song.song_id);
}
collection.find({ _id: fn.generateID(req.params.store_id), "songs._id": { $in: array } }, function(err, songs) {
res.json(songs);
});
});
});
and i dont really know if it's the best way of doing it
I'm not entirely clear what you mean by "get the songs name and artist from the songsSchedule song_id" but it looks like that query will be messy.
If it were me, I'd consider splitting out songs and songSchedule into their own collections for easier querying.
from your document example, the "songs" field contains documents that do not contain an "_id" field.
"songs": [ {"name": "", "artist": "", "category": "", "tags": []} ]
But, your find() query is querying on the "songs._id" field.
Also, I'm not too familiar with the json() method, but does it handle cursors?
Regards,
Kay

M2M links in CouchDB with non-primary keys

I have a bunch of "meeting" documents and a bunch "user" documents. A
user may have multiple emails and a meeting may be between multiple
people identified by emails.
I need to lookup users by meeting ID and meeting by user ID by way of
the emails.
My documents look like this right now:
{
"type": "meeting"
"_id": "MEETINGID",
"emails": ["test1#example.com", "test2#example.com"]
// Lots of others things
}
{
"type": "user",
"_id": "USERID",
"emails": ["a#example.com", "test1#example.com"]
// Lots of others things
}
I need to keep them linked via email.
But I could certainly split them out like this if it would help:
{
"type": "user",
"_id": "USERID",
// Lots of others things
}
{
"type": "user-email",
"_id": "USERID",
"email": "a#example.com"
}
{
"type": "user-email",
"_id": "USERID",
"email": "test1#example.com"
}
I need views that produce something like this:
{
"key": "MEETING",
"document": {"_id": "USERID"}
}
{
"key": "USERID",
"document": {"_id": "MEETINGID"}
}
Is this going to be possible? I have a horrible feeling I'm going to
be making load of queries to do this :(
you can write a map like
"meetingByEmail":
function (doc) {
if(doc.type=="meeting") {
for (var curEmail in doc.emails) {
emit (doc.emails[curEmail],null);
}
}
}
call:
_view/byEmail?key="test1#example.com":
Result:
{"total_rows":4,"offset":0,"rows":[
{"id":"f2338c8e69d1da02c94a2104b6000e77","key":"test1#example.com","value":null},
{"id":"f2338c8e69d1da02c94a2104b6000e88","key":"test1#example.com","value":null}
]}
(f2338c8e69d1da02c94a2104b6000e77 is ID from a Meeting, user test1 has joined two meetings)
vice versa (for user)
function(doc) {
if(doc.type=="user") {
for (var curEmail in doc.emails) {
emit (doc.emails[curEmail],null);
}
}
}
the call
user/_view/byEmail?key="test1#example.com"
result:
{"total_rows":2,"offset":1,"rows":[
{"id":"user1","key":"test1#example.com","value":null}
]}
The simple answer is "you can't" unfortunately. CouchDB doesn't allow stuff like that.

CouchDB - Basic design for joins data

Just playing around with CouchDb and CouchApp, what amazing tech ! Very surprised, seams to be very powerful. After playing and reading a lot, as I'm a old relational database user, I still questioning myself about how to design some basic things.
Here is my question :
1/ I have a document of type 'user' and document of type 'item'
2/ My Couchdb contains the following documents :
{ "_id": "...", "type": "user", "email":"u1#gmail.com" ... }
{ "_id": "...", "type": "user", "email":"u2#gmail.com" ... }
{ "_id": "...", "type": "user", "email":"u3#gmail.com" ... }
{ "_id": "...", "type": "user", "email":"u4#gmail.com" ... }
{ "_id": "...", "type": "item", "title":"My title",
created_by:"u1#gmail.com", modified_by:"u3#gmail.com" }
3/ Now I want a view or something to fetch document by type=item and _id with informations for each users (creator & modifier)
I have seen a way to emulate a simple join here : http://www.cmlenz.net/archives/2007/10/couchdb-joins
But I can't adapt it for two joins, I'am playing around with key format since few hours, testing lots things, but nothing works.
I think, I'm missing something important with CouchDb map/reduce, if someone has some help I will appreciate it.
PS : Don't answer me to insert 'user' document inside 'item' document. This not my question.
Relax, relax ... :-)
I think you should put _ids in created_by and modified_by:
{ "_id": "u1", "type": "user", "email":"u1#gmail.com" ... }
{ "_id": "u2", "type": "user", "email":"u2#gmail.com" ... }
{ "_id": "u3", "type": "user", "email":"u3#gmail.com" ... }
{ "_id": "u4", "type": "user", "email":"u4#gmail.com" ... }
{ "_id": "anitem", "type": "item", "title":"My title",
created_by:"u1", modified_by:"u3" }
so you can use the following map function and query it with ?key="anitem"&include_docs=true:
function(doc) {
if (doc.type === "item") {
emit(doc._id, 1);
emit(doc._id, { _id: doc.created_by });
emit(doc._id, { _id: doc.modified_by });
}
}
You can read Jan Lehnardt's post about it for more details.
As a side note, I generally put the type in the _id so it is easier to get unique keys, and you do not need a view if you only want to filter by type:
{ "_id": "user/username1", "email":"u1#gmail.com" ... }
{ "_id": "user/username2", "email":"u2#gmail.com" ... }
{ "_id": "user/username3", "email":"u3#gmail.com" ... }
{ "_id": "user/username4", "email":"u4#gmail.com" ... }
{ "_id": "item/itemid1", "title":"My title",
created_by:"user/username1", modified_by:"user/username3" }
and the map function is
function(doc) {
if (doc._id.slice(0, 4) === "item/") {
emit(doc._id, 1);
emit(doc._id, { _id: doc.created_by });
emit(doc._id, { _id: doc.modified_by });
}
}
UPDATE: Due to bug COUCHDB-1229, use of / in doc._id may cause problems. Depending on your use-case, it may be better to use another separator, e.g. : or _.
What do you mean here "two joins"? Is this "tableA join tableB ON ... AND ..."?
Now I want a view or something to fetch document by type=item and _id with informations for each users
This can be done without two/more joins.
Anyway, my advise is to divide your data in 2 databases: items & users. The example above suits only few simple tasks. But when your data grows big (say 10K users & 100K items) it becomes very hard to process your data and it's kinda sucks that all your documents differ from each other with only one field.

Resources