Lets say I have following schema in Azure Search Collection, with 100s of records.
{
"id": '1',
"Status" : "Available",
"name" : "demo 1"
},
{
"id": '2',
"Status" : "Available",
"name" : "demo 1"
},
{
"id": '3',
"Status" : "Removed",
"name" : "demo 1"
},
{
"id": '4',
"Status" : "Booked",
"name" : "demo 4"
}
In My Status field I can have three different values,
"Booked", "Available", "Removed".
Now I am getting the data by pagination using Skip and Top from the Azure Search.
However like in ElasticSearch Aggregation function, is there a way in Azure search to get total no of sites having status Booked , Available or not removed etc..
Because I cant do a count in Client Side because, I will have limited no of records, not all records from Azure Search.
If you are using a search index client object (Microsoft.Azure.Search.SearchIndexClient) to search for your documents you can provide an object as a parameter (Microsoft.Azure.Search.Models.SearchParameters) that contains a property IncludeTotalResultCount. Once you set this property to true and call the method Documents.Search(...) passing the parameter object, your response object (Microsoft.Azure.Search.DocumentSearchResult) will contain a value for the property Count considering the total count for the filter, regardless how many items selected by the pagination.
SearchParameters searchParameter = new SearchParameters
{
Filter = "organizationId eq '1'",
Skip = 0,
Top = 20,
IncludeTotalResultCount = true
};
using (var client = new SearchIndexClient(...)
{
DocumentSearchResult response = client.Documents.Search("*", searchParameter);
//pagination items
var collection = response.Results.ToArray();
//total items
var counter = response.Count;
}
When you create the index, make the status field facetable and filterable. Then issue a facets query on status, with $filter=status ne 'Booked'. This will give you the total counts of documents in each status category other than Booked, irrespective of pagination.
Related
I want to obtain documents (products in this case) using find() with two filters, the first one is state = true and the second one is if the product belong to a category received in the request.
I'm making the filter this way
let categories = req.body;
Product.find({ state: true, "category": {$in : categories} }...
This filter brings me the products that belong to a certain category but don't respect if the product has a state = true or false.
What I'm doing wrong?
Thx.
For example:
{
"_id" : ObjectId("601b9ef73faa662db0b29204"),
"state" : true,
"code" : "APE700JAGE",
"category" : ObjectId("601b98853faa662db0b291e5"),
}
{
"_id" : ObjectId("601b9ef73faa662db0b29204"),
"state" : false,
"code" : "PRU123FAKK",
"category" : ObjectId("601b98853faa662db0b291e5"),
}
Request:
{
categories: [ '601b98853faa662db0b291e5' ]
}
There are few fixes:
you are passing whole req.body in categories variable, you just need to pass only req.body.categories
let categories = req.body.categories;
the categories are array of string ids you need to convert it to object id using mongoose.Types.ObjectId because category is object id in document
let categories = req.body.categories.map(c => mongoose.Types.ObjectId(c));
I have a requirement to generate a unique number (ARN) in this format
DD/MM/YYYY/1, DD/MM/YYYY/2
and insert these in elastic search index.
The approach i am thinking of is to create an auto increment field in the doc and use it to generate a new entry and use the new auto generated number to create the ARN and update the doc.
doc structure that i am planning to use:
{ id: 1, arn: 17/03/2018/01 }
something like this.
How can i get auto increment field in elastic search?
It can't be done in a single step. First you have to insert the record into the database, and then update the ARN with it's id
There is no auto-increment equivalent, for example, to hibernate id generator. You could use the Bulk API (if you have to save multiple documents at a time) and increase the _id and the ending of your ARN value programmatically.
Note: if you want to treat your id as a number, you should implement it yourself (in this example, I added a new field "my_id", because the _id of the documents is treated as a string.
POST /bulk
{ "index" : { "_index" : "your_index", "_type" : "your_type", "_id" : "1" } }
{ "arn" : "2018/03/17/1", my_id: 1 }
{ "index" : { "_index" : "your_index", "_type" : "your_type", "_id" : "2" } }
{ "arn" : "2018/03/17/2", my_id: 2 }
Then, the next time that you want to save new documents, you query for the maximum id something like:
POST /my_index/my_type/_search?size=1
{
"query": {
"fields": ["my_id"],
"sort": [{
"my_id": { "order": "desc" } }
]
}
}
If your only requirement is that this ARN should be unique, you could also let elasticsearch calculate your _id by simply not setting it. Then you could relay at some unique token generator (UID.randomUUID().toString() if work with java). Pseudo code follows:
String uuid = generateUUID() // depends on the programming language
String payload = "{ \"arn\" : + uuid + "}" // concatenate the payload
String url = "http://localhost:9200/my_index" // your target index
executePost(url, payload) // implement the call with some http client library
I am trying to count the number of models in a collection based on a property:
I have an upvote model, that has: post (objectId) and a few other properties.
First, is this good design? Posts could get many upvotes, so I didn’t want to store them in the Post model.
Regardless, I want to count the number of upvotes on posts with a specific property with the following and it’s not working. Any suggestions?
upvote.count({‘post.specialProperty’: mongoose.Types.ObjectId(“id”), function (err, count) {
console.log(count);
});
Post Schema Design
In regards to design. I would design the posts collection for documents to be structured as such:
{
"_id" : ObjectId(),
"proprerty1" : "some value",
"property2" : "some value",
"voteCount" : 1,
"votes": [
{
"voter": ObjectId()// voter Id,
other properties...
}
]
}
You will have an array that will hold objects that can contain info such as voter id and other properties.
Updating
When a posts is updated you could simply increment or decrement the voteCountaccordingly. You can increment by 1 like this:
db.posts.update(
{"_id" : postId},
{
$inc: { voteCount: 1},
$push : {
"votes" : {"voter":ObjectId, "otherproperty": "some value"}
}
}
)
The $inc modifier can be used to change the value for an existing key or to create a new key if it does not already exist. Its very useful for updating votes.
Totaling votes of particular Post Criteria
If you want to total the amount for posts fitting a certain criteria, you must use the Aggregation Framework.
You can get the total like this:
db.posts.aggregate(
[
{
$match : {property1: "some value"}
},
{
$group : {
_id : null,
totalNumberOfVotes : {$sum : "$voteCount" }
}
}
]
)
I'm running a blog-style web application on AppFog (ex Nodester).
It's written in NodeJS + Express and uses Mongoose framework to persist to MongoDB.
MongoDB is version 1.8 and I don't know whether AppFog is going to upgrade it to 2.2 or not.
Why this intro? Well, now my "posts" are shown in a basic "paginated" visualization, I mean they're just picked up from mongo, sorted by date descending, a page at a time. Here's a snippet:
Post
.find({pubblicato:true})
.populate("commenti")
.sort("-dataInserimento")
.skip(offset)
.limit(archivePageSize)
.exec(function(err,docs) {
var result = {};
result.postsArray = (!err) ? docs : [];
result.currentPage = currentPage;
result.pages = howManyPages;
cb(null, result);
});
Now, my goal is to GROUP BY 'dataInserimento' and show posts like a "diary", I mean:
1st page => 2012/10/08: I show 3 posts
2nd page => 2012/10/10: I show 2 posts (2012/10/09 has no posts, so I don't allow a white page)
3rd page => 2012/10/11: 35 posts and so on...
My idea is to get first the list of all dates with grouping (and maybe counting posts for each day) then build the pages link and, when a page (date) is visited, query like above, adding date as parameter.
SOLUTIONS:
Aggregation framework would be perfect for that, but I can't get my hands on that version of Mongo, now
Using .group() in some way, but the idea it doesn't work in sharded environments does NOT excite me! :-(
writing a MAP-REDUCE! I think this is the right way to go but I can't imagine how map() and reduce() should be written.
Can you help me with a little example, please?
Thanks
EDIT :
The answer of peshkira is correct, however, I don't know if I need exactly that.
I mean, I will have URLs like /archive/2012/10/01, /archive/2012/09/20, and so on.
In each page, it's enough to have the date for querying for posts. But then I have to show "NEXT" or "PREV" links, so I need to know what's the next or previous day containing posts, if any. Maybe can I just query for posts with dates bigger or smaller than the current, and get the first one's date?
Assuming you have something similar as:
{
"author" : "john doe",
"title" : "Post 1",
"article" : "test",
"created" : ISODate("2012-02-17T00:00:00Z")
}
{
"author" : "john doe",
"title" : "Post 2",
"article" : "foo",
"created" : ISODate("2012-02-17T00:00:00Z")
}
{
"author" : "john doe",
"title" : "Post 3",
"article" : "bar",
"created" : ISODate("2012-02-18T00:00:00Z")
}
{
"author" : "john doe",
"title" : "Post 4",
"article" : "foo bar",
"created" : ISODate("2012-02-20T00:00:00Z")
}
{
"author" : "john doe",
"title" : "Post 5",
"article" : "lol cat",
"created" : ISODate("2012-02-20T00:00:00Z")
}
then you can use map reduce as follows:
Map
It just emits the date as key and the post title. You can change the title to the _id, which will probably be more useful to you. If you store the time of the date you will want to use only the date (without time) as the key, otherwise mongo will group by date time and not only date. In my test case all posts have the same time 00:00:00 so it does not matter.
function map() {
emit(this.created, this.title);
}
Reduce
It does nothing more, then just push all values for a key to an array and then the array is wrapped in a result object, because mongo does not allow arrays to be the result of a reduce function.
function reduce(key, values) {
var array = [];
var res = {posts:array};
values.forEach(function (v) {res.posts.push(v);});
return res;
}
Execute
Using db.runCommand({mapreduce: "posts", map: map, reduce: reduce, out: {inline: 1}}) will output the following result:
{
"results" : [
{
"_id" : ISODate("2012-02-17T00:00:00Z"),
"value" : {
"posts" : [
"Post 2",
"Post 1"
]
}
},
{
"_id" : ISODate("2012-02-18T00:00:00Z"),
"value" : "Post 3"
},
{
"_id" : ISODate("2012-02-20T00:00:00Z"),
"value" : {
"posts" : [
"Post 5",
"Post 4"
]
}
}
],
...
}
I hope this helps
I have created a collection in DB and updated with proper indexing to retrieve it back. Im getting with the folloging
Example:
db.car.save({ "name":"Toyota car", "affiliation":"Toyota", "loc":{"lon":55.93939251390387,"lat":-113.999}})
db.car.find({"loc" : {"$within" : {"$center" : [[50.93939251390,-114],5]}}})
I have 100K + records in Database when I search the above query returns 20K records.
Actually this has some duplicate values in the "name" column. How can I get the distinct value of "name".
You can't get the results you need using a query in MongoDB 2.0.x, so will need to manipulate the results in your application code.
If you are only after names you can limit the fields in the output:
db.car.find(
{"loc" : {"$within" : {"$center" : [[50.93939251390,-114],5]}}},
{"_id": 0, "name": 1}
)
If you want to find distinct names you could do something like:
// Save cars we've seen
> var cars_seen = {}
// The unique list we actually want
> var cars = []
// Find closest cars to given geo point
> db.car.find(
{"loc" : {"$within" : {"$center" : [[50.93939251390,-114],5]}}},
{"_id": 0, "name": 1}
).forEach(
function(doc) {
// Add names we haven't seen yet
if (!cars_seen[doc.name]) {
cars_seen[doc.name] = 1;
cars.push(doc.name);
}
}
)
> cars
[ "Skoda", "Benz", "Skoda SUV" ]