I'm trying to figure out the best way of sorting by rank using couch db. I have my documents setup in a players db like so:
{
"_id": "user2",
"_rev": "31-65e0e5bb1eba8d6a882aad29b63615a7",
"username": "testName",
"apps": {
"app1": {
"score": 1000
},
"app2": {
"score": 1000
},
"app3": {
"score": 1000
}
}
}
The player can have multiple scores for various apps. I'm trying to figure out the best way to pull say the top 50 scores for app1.
I think one idea could be to store the score of each user for each app seperately. Like so: -
{"app":"app1","user":"user_id","score":"9000"}
The you could write a map function
emit([doc.app,doc.score],{_id:doc.user,score:doc.score})
and query the view like
/view_name?include_docs=true&startkey=["app1",{}]&endkey=["app1"]&descending=true
With this arrangement you have a view sorted by the score and the name of the app. Here are the results that you get.
{"total_rows":2,"offset":0,"rows":[
{"id":"61181c784df9e2db5cbb7837903b63f5","key":["app1",10000],"value":
{"_id":"5002539daa85a05d3fab16158a7861c1","score":10000},"doc":
{"_id":"5002539daa85a05d3fab16158a7861c1","_rev":"1-8441f2f5dbaaf22add8969cea5d83e1b","user":"jiwan"}},
{"id":"7f5d53b2da8ae3bea8e2b7ec74020526","key":["app1",9000],"value":
{"_id":"336c2619b052b04992947976095f56b0","score":9000},"doc":
{"_id":"336c2619b052b04992947976095f56b0","_rev":"3-3e4121c1831d7ecafc056e71a2502f3a","user":"akshat"}}
]}
You have score in value. User in doc.
Edit
Oops! I mistyped the startkey and endkey :) Notice that it is not startKey but startkey same for endkey. Also note that since descending is true we reverse the order of keys. It should work as expected now.
For more help check out
This answer and
This answer
Related
I have stored sentences in elasticsearch for autosuggestion.
format:
{
"text": "what is temperature in chicago"
}
it suggests correctly when w or wha or what typed. but I am wondering if there is any way I can fetch most search sentences from elasticsearch.
Sounds like what you need is terms aggregations:
Your request body should look something like this:
{
"query": {
//your query
},
"aggs": {
"common" : {
"terms" : { "field" : "text.keyword", "size": 20 }
}
}
}
If I get your question correctly you want most common searches done wrt to input query, a simple solution can be implemented.
Just track what user finally selects (document of ES) and then increment its counter by 1 keeping mapping of _id.
Running a batch system/sync/indexing this data in ES data will have counter value in your data.
Use this while giving suggestions i.e sort with count field.
This will start working properly as users start using.
Your ES document would look like.
{ "text":"what is temperature in chicago",
"count":10
}
I would suggest this is very raw solution there can be many, but nice to start with.
I've literally researched the entire web and couldn't find a satisfactory answer for this so thought I would ask here.
Basically what I'm trying to do is build a full text search query with pagination, which returns results sorted by time.
The problem is, a naive sort like the following doesn't perform at all:
db.collection
.find({ $text: { $search: "hello" } })
.sort({ created_at: -1 })
.limit(100)
.toArray(function(....
And yes, I've of course indexed it with created_at. And as you can see it's limited to 100 items.
So far what I gather is that the full text index in MongoDB doesn't let you sort by any arbitrary attribute in the collection AT ALL, and the only way to sort it is by adding some $meta attribute to sort it based on some internal scoring mechanism.
But that doesn't work for me, and i really want to sort this by created_at.
Maybe I'm misunderstanding the whole thing, but I refuse to believe that no one has come up with a solution for this very obvious use case. Am I missing something? Does anyone know how to sort a large text search result by a collection attribute? At this point I would appreciate ANY shine of light, even if it's a hack.
[EDIT] For example without the limit and sort, the response would look something like this:
[{
"msg": "hello world",
"created_at": 1000
}, {
"msg": "hello",
"created_at": 899
}, {
"msg": "hello hello",
"created_at": 1003
}, {
...
}]
But I want to limit it to only 100, sorted by created_at, AFTER having searched the database for the occurrrence of "hello". I don't care about relevance and I only want to sort so that it's ordered by time.
[{
"msg": "hello hello",
"created_at": 1003
}, {
"msg": "hello world",
"created_at": 1000
}, {
"msg": "hello",
"created_at": 899
}, {
...
}]
Just to be clear, the query DOES work, but it takes very long time even though I have indexed it with created_at. I don't have this issue when I do a similar find-sort-limit pattern with other queries (not full text search), and I think this is specific to full text search.
I am looking for a way to somehow make this query faster.
Currently I am working on a mobile app. Basically people can post their photos and the followers can like the photos like Instagram. I use mongodb as the database. Like instagram, there might be a lot of likes for a single photos. So using a document for a single "like" with index seems not reasonable because it will waste a lot of memory. However, I'd like a user add a like quickly. So my question is how to model the "like"? Basically the data model is much similar to instagram but using Mongodb.
No matter how you structure your overall document there are basically two things you need. That is basically a property for a "count" and a "list" of those who have already posted their "like" in order to ensure there are no duplicates submitted. Here's a basic structure:
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3")
"photo": "imagename.png",
"likeCount": 0
"likes": []
}
Whatever the case, there is a unique "_id" for your "photo post" and whatever information you want, but then the other fields as mentioned. The "likes" property here is an array, and that is going to hold the unique "_id" values from the "user" objects in your system. So every "user" has their own unique identifier somewhere, either in local storage or OpenId or something, but a unique identifier. I'll stick with ObjectId for the example.
When someone submits a "like" to a post, you want to issue the following update statement:
db.photos.update(
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3"),
"likes": { "$ne": ObjectId("54bb2244a3a0f26f885be2a4") }
},
{
"$inc": { "likeCount": 1 },
"$push": { "likes": ObjectId("54bb2244a3a0f26f885be2a4") }
}
)
Now the $inc operation there will increase the value of "likeCount" by the number specified, so increase by 1. The $push operation adds the unique identifier for the user to the array in the document for future reference.
The main important thing here is to keep a record of those users who voted and what is happening in the "query" part of the statement. Apart from selecting the document to update by it's own unique "_id", the other important thing is to check that "likes" array to make sure the current voting user is not in there already.
The same is true for the reverse case or "removing" the "like":
db.photos.update(
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3"),
"likes": ObjectId("54bb2244a3a0f26f885be2a4")
},
{
"$inc": { "likeCount": -1 },
"$pull": { "likes": ObjectId("54bb2244a3a0f26f885be2a4") }
}
)
The main important thing here is the query conditions being used to make sure that no document is touched if all conditions are not met. So the count does not increase if the user had already voted or decrease if their vote was not actually present anymore at the time of the update.
Of course it is not practical to read an array with a couple of hundred entries in a document back in any other part of your application. But MongoDB has a very standard way to handle that as well:
db.photos.find(
{
"_id": ObjectId("54bb201aa3a0f26f885be2a3"),
},
{
"photo": 1
"likeCount": 1,
"likes": {
"$elemMatch": { "$eq": ObjectId("54bb2244a3a0f26f885be2a4") }
}
}
)
This usage of $elemMatch in projection will only return the current user if they are present or just a blank array where they are not. This allows the rest of your application logic to be aware if the current user has already placed a vote or not.
That is the basic technique and may work for you as is, but you should be aware that embedded arrays should not be infinitely extended, and there is also a hard 16MB limit on BSON documents. So the concept is sound, but just cannot be used on it's own if you are expecting 1000's of "like votes" on your content. There is a concept known as "bucketing" which is discussed in some detail in this example for Hybrid Schema design that allows one solution to storing a high volume of "likes". You can look at that to use along with the basic concepts here as a way to do this at volume.
I want to store votes in CouchDB. To get round the problem of incrementing a field in one document and having millions of revisions, each vote will be a seperate document:
{
_id: "xyz"
type: "thumbs_up"
vote_id: "test"
}
So the actual document itself is the vote. The result I'd like is basically an array of: vote_id, sumOfThumbsUp, sumOfThumbsDown
Now I think my map function would need to look like:
if(type=="thumbs_up" | type =="thumbs_down"){
emit(vote_id, type)
}
Now here's the bit I can't figure out what to do, should I build a reduce function to somehow sum the vote types, keeping in mind there's two types of votes.
Or should I just take what's been emited from the map function and put it straight into an array to work on, ignoring the reduce function completely?
This is a perfect case for map-reduce! Having each document represent a vote is the right way to go in my opinion, and will work with CouchDB's strengths.
I would recommend a document structure like this:
Documents
UPVOTE
{
"type": "vote",
"vote_id": "test",
"vote": 1
}
DOWNVOTE
{
"type": "vote",
"vote_id": "test",
"vote": -1
}
I would use a document type of "vote", so you can have other document types in your database (like the vote category information, user information, etc)
I kept "vote_id" the same
I made the value field called "vote", and just used 1/-1 instead of "thumbs_up" or "thumbs_down" (really doesn't matter, you can do whatever you want and it will work just fine)
View
Map
function (doc) {
if (doc.type === "vote") {
emit(doc.vote_id, doc.vote);
}
}
Reduce
_sum
You end up with a result like this for your map function:
And if you reduce it:
As you add more vote documents with more vote_id variety, you can query for a specific vote_id by using: /:db/_design/:ddoc/_view/:view?reduce=true&group=true&key=":vote_id"
I am creating one application where for every product I have one database and I will create different document based on date. The keys in documents could be different and depend upon user, what he provides. Assumption is user will keep giving same key for tracking with changed value over time. In the end, I need to know all possible keys before creating automatic views on them.
Example:
If I had DB, say, test. It contains, say, two documents,
1. {
"_id":"1",
"_rev":"1-"
"type": "Note",
"content": "Hello World!"
}
2. {
"_id":"2",
"_rev":"1-"
"type": "Note",
"content": "Beyond Hello World!",
"extra":"Boom"
}
Then I want to list all keys in this DB. So, answer should be _id,_rev,type,content and extra.
These keys are dynamic and depend upon users. So, I couldn't assume that I knew them in advance.
I have never used stackoverflow before, I saw your question when trying to solve this problem myself so I have signed up. I think this solves your problem:
create a view where "views" includes this:
{
"keys": {
"map": "function(doc) { for (var thing in doc) { emit(thing,1); } }",
"reduce": "function(key,values) { return sum(values); }"
}
}
then query on that view with group=true e.g.:
http://localhost:5984/mydb/_design/myview/_view/keys?group=true
you should get back a list of all the keys in your database and a count of how often the occur.
does this help?