How to count a field if value is not null in elasticsearch - python-3.x

I have indexed some documents in ElasticSearch.
Short view looks like this:
{
"tenant_id":"abcd1234"
"provider_id":"3456"
.
.
.
"doctor_summary": ["line1", "line2", "line3"] OR it could be null.
}
I want to calculate this list as 1 if not null,
query_template = {
"bool":{
"must":[
{"term":{"tenant_id.keyword":tenant_id}},
{"term":{"provider_id.keyword":provider_id}},
{"range":{"visit_date":{"gte":from_date,"lte":to_date}}},
],
"filter":[]
}
}
aggs_query = {
"doctor_summary_count":{
"value_count":{"field":"doctor_summary.keyword"}
}
}
res = CLIENT.search(index = config['elasticsearch']['abcd'],
query = query_template,
size = 10000,
aggs = aggs_query)
After calling this aggregation query, it gives result as ( size of the list * total doctor_summary field).
For example: the result of above query on above document should be 1. (As it is not null).
But it gives as 3(because list contains 3 lines.)

You can use exist query in aggregation with filter aggregation.
Query:
{
"aggs": {
"count": {
"filter": {
"exists": {
"field": "doctor_summary.keyword"
}
}
}
}
}
Response:
"aggregations" : {
"count" : {
"doc_count" : 1
}
}

Related

is it possble to use lowdb to update an existing json value?

I'm using lowdb https://github.com/typicode/lowdb.
I have a small database that looks like this
{
"orders": [
{
"id": "0",
"kit": "not a real order"
},
{
"id": "1",
"kit": "kit_1"
}
],
"total orders": 21,
"216862330724548608": 1
}
is it possble to change the "kit": "x" to "kit": "y"
x and y are user input so I can't just use replace because I don't know what it will be equal to.
I did try to use some kind of replace but it didn't work
let = updateOrders = (items, id, newValue) => {
const {orders} = items;
orders.map((item) => {
item.kit = newValue;
//if you need id check uncomment below code and add id in arguments and pass id
// if (item.id === id) {
// item.kit = newValue;
// }
})
console.log(orders);
};
updateOrders(orders, 'updated');
Hopefully it would help.

couchDb map/reduce min&max + other data

I have a document list that consists of:
time
sessionId
appv
museum
I would like to use the couchDB map/reduce function to get the result:
key : sessionId, value : {begin : time, end: time, appv : appv, museum : museum}
for the begin value: the minimum time value.
for the end value: the maximum time value
Currently I can have the minimum and maximum value with this code:
MAP :
function(doc) {
if(doc.sessionId) {
emit(doc.sessionId, [doc.time])
}
}
REDUCE :
function(keys, values, rereduce) {
if (rereduce) {
return {
'min': values.reduce(function(a, b) { return Math.min(a, b.min) }, Infinity),
'max': values.reduce(function(a, b) { return Math.max(a, b.max) }, -Infinity),
}
} else {
return {
'min': Math.min.apply(null, values),
'max': Math.max.apply(null, values),
}
}
}
RESULT :
{"rows":[
{"key":"fromDev1548326238156","value":{"min":2,"max":999}}
]}
And when I use this map function:
function(doc) {
if(doc.sessionId) {
emit(doc.sessionId, [doc.time, doc.museum, doc.appv])
}
}
I can't find the reduce function that allows me to get the result I want
Can you help me?

Return object with dynamic keys in AQL

Can I return something like:
{
"c/12313" = 1,
"c/24223" = 2,
"c/43423" = 3,
...
}
from an AQL query? The idea is something like (this non-working code):
for c in my_collection
return { c._id : c.sortOrder }
where sortOrder is some property on my documents.
Yes, it is possible to have dynamic attribute names:
LET key = "foo"
LET value = "bar"
RETURN { [ key ]: value } // { "foo": "bar" }
An expression to compute the attribute key has to be wrapped in [ square brackets ], like in JavaScript.
This doesn't return quite the desired result however:
FOR c IN my_collection
RETURN { [ c._id ]: c.sortOrder }
[
{ "c/12313": 1 },
{ "c/24223": 2 },
{ "c/43423": 3 },
...
]
To not return separate objects for every key, MERGE() and a subquery are required:
RETURN MERGE(
FOR c IN my_collection
RETURN { [ c._id ]: c.sortOrder }
)
[
{
"c/12313": 1,
"c/24223": 2,
"c/43423": 3,
...
}
]

How do I create a query to count occurrences of tag words for each user in a dataset

I am pretty new to couchDB and having issues coming up with a query.
This is an example of the data set I am working with
{
"_id": "data",
"_rev": "3-b78ec99614827106f637148c73dbf876",
"data": [
{
"id": 0,
"tags": [
"cupidatat",
"mollit",
"labore",
"minim",
"pariatur",
"qui",
"ipsum"
]
},
{
"id": 1,
"tags": [
"ex",
"cillum",
"est",
"et",
"mollit",
"mollit",
"exercitation"
]
}
This is my map function
function(doc) {
for(var i in doc.data)
{
var person = doc.data[i];
for(var tag in person.tags)
{
emit(person.tags, 1);
}
}
}
This is the reduce function
function(keys, values)
{
return sum(values);
}
I am trying to produce results that will give me the number of occurrences of each tag for all the records like,
key value
"cupidatat" 1
"mollit" 3
How do I fix it so I can get the right results?
Looks like you are very close. Using your sample doc I got the results you are looking for using this map function:
function(doc) {
for (var i = 0; i < doc.data.length; i++) {
for (var j = 0; j < doc.data[i].tags.length; j++) {
emit(doc.data[i].tags[j], 1);
}
}
}
and used the built-in reduce:
_sum
The following request returns JSON in the format you specify:
curl -X GET http://host:5984/db/_design/words/_view/count?reduce=true&group_level=1

In couchdb How to get sorting on value with reduce _count

"myview": {
"map": "function(doc) { if (doc.type == 'user') emit(doc.city,null)}",
"reduce": "_count"
},
It return city with count in value. Using group=true
How to sort this data base on highest value? Want to get most popular city first...
You can use list to sort your resultset. Following function can sort anytype of a resultset on value.
"sort": "function() {
var row;
var rows=[];
while(row = getRow())
{
rows.push(row)
}
rows.sort(function(a, b)
{
return a.value - b.value;
});
rows.reverse();
send(JSON.stringify({\"rows\" : rows}))
}"

Resources