Reduce output must shrink more rapidly, what is this error about? - couchdb

An user can post multiple comments in a thread, and I try to get list of threads (distinct) that an user has make comment to it, like :-
// comment table (relation table)
id, thread_id, user_id
select comment.thread_id, count(*)
from user
inner join comment on user.id=comment.user_id
where user.id = ?
group by comment.thread_id;
This is pretty easy in MySQL.
But to convert to couchdb :-
// map
function(doc)
{
emit(doc.user_id, doc.thread_id);
}
// reduce
function (key, thread_id)
{
return thread_id;
}
If I using the above map function, I will hit into an error like :-
"error": "reduce_overflow_error",
"reason": "Reduce output must shrink more rapidly: Current output: ...
I think I have applied the reduce function in wrong manner.
If using another way, like :-
// map
function (doc)
{
emit([doc.user_id, doc.thread_id], 1);
}
// reduce
function(keys, values)
{
return sum(values);
}
The group=true result is look exactly what mysql group-by does.
However, I'm unable to get ALL the list of thread by an user (given I only have the user_id during query time)
Third way, I can discard use of map reduce, and directly apply :-
emit(doc.user_id, doc.thread_id);
And do an PHP array like
foreach ( ... )
{
$threads[$thread_id] = TRUE;
}
array_keys($threads);
However, this is quite bloated and less efficient.
Second method look more accurate :-
key=[user_id, *] <-- it does not work, believe only work on exact match
key=[user_id, thread_id] <-- return one row
Is there a way to get all result without knowing the thread_id ?
(ps: I new to couchdb, and I might have describe the scenario in a bad manner)
Some reference I gotten via #jasonsmith :- http://guide.couchdb.org/draft/cookbook.html
As a rule of thumb, the reduce function should reduce to a single scalar value. That is, an integer; a string; or a small, fixed-size list or object that includes an aggregated value (or values) from the values argument. It should never just return values or similar. CouchDB will give you a warning if you try to use reduce “the wrong way”:

Follow closely to what this docs saying :-
http://wiki.apache.org/couchdb/View_Snippets#Generating_a_list_of_unique_values
// map
function(doc)
{
emit([doc.user_id, doc.thread_id], null);
}
// reduce
function (keys, values)
{
return null;
}
Query :-
?startkey=["$uid"]&endkey=["$uid",{}]&group=true
And the result now is accurate,
so the problem is lied on the reduce function and how the query being construct.

Related

How to get a value from inside Anko's async? (KotlIn)

I have a function which uses Anko's async in order to call Google's Distance API and get the distance from one location to another. unfortunately i don't know how to get the data from inside the async and pass it to another function. the code looks something like this
fun getDistance(location1:LatLng,location2:LatLng){
async{
val result = URL(url).readtext()
uithread{
//Parser
//distance value
}
}
}
I'd like to also mention im really new to kotlin or android development in general, please be kind.
There are a number of ways to tackle this; pass an object to the function with your array in it that gets rearranged in your function, or go with something like:
fun getDistance(location1 : LatLng, location2 : LatLng, f: (Long) -> Unit){
doAsync{ // Anko is deprecated as I have been made aware
val result = URL(url).readtext()
val distance : Long = // parse result
uiThread{
f(distance)
}
}
}
and call that with
getDistance(loc1, loc2) { toast("The found distance was $it") }
This is by no means the only way to go; you could update a larger-scoped variable, call a listener, put your lat-longs in a class with updating functions that are called, or a bunch of other ways that I am too lazy to think about :)

Making Mongoose/MongoDB query helper plugin return zero results

I'm developing a Mongoose Query Helper plugin that provides the chainable method .search(query). On certain conditions, I want the query to return zero results, no matter how the other methods in the query builder chain behave. Turns out this isn't so easy as I have assumed.
Basically, I have the following code:
schema.query.search = function search(query) {
if ("query is invalid") {
// return no results => no easy way to achieve that?
}
return this.find(query);
};
Now, I want SomeModel.find({}).search(someQuery).exec() to return no results in case the query is invalid. I first tried to return this.limit(0), but turns out a limit of 0 is equivalent to setting no limit.
As a temporary solution, I do return this.find({ nonExistingField: 'something' }) which always results in no results, but this does seem a bit awkward and is probably also not so optimal in terms of performance as it triggers a search when no search is needed.
Thanks in advance for your help!

How to convert a DTO to Domain Objects

I'm trying to apply ubiquitous language to my domain objects.
I want to convert a Data Transfer Object coming from a client into the domain object. The Aggregate's Constructor only accepts the required fields, and the rest of parameters should be passed using aggregate's API even when the Aggregate is being created(by say CreateAggregate command).
But the DTO to Aggregate mapping code becomes a bit messy:
if(DTO.RegistrantType == 0){
registrantType = RegistrantType.Person()
}
elseif(DTO.RegistrantType == 1){
registrantType = RegistrantType.Company()
}
//.....
//.....
var aggregate = new Aggregate(
title,
weight,
registrantType,
route,
callNumber,
)
//look at this one:
if(DTO.connectionType == 0){
aggregate.Route(ConnectionType.InCity(cityId))
}
elseif(DTO.connectionType == 1){
aggregate.Route(ConnectionType.Intercity(DTO.originCityId,DTO.DestinationCityId)
}
//..........
//..........
One thing I should mention is that this problem doesn't seem a domain specific problem.
How can I reduce these If-Else statements without letting my domain internals leakage, and with being sure that the aggregate(not a mapping tool) doesn't accept values that can invalide it's business rules, and with having the ubiquitous language applied?
Please don't tell me I can use AoutoMapper to do the trick. Please read the last part carefully.'
Thank you.
A typical answer would be to convert the DTO (which is effectively a message) into a Command, where the command has all of the arguments expressed as domain specific value types.
void doX(DTO dto) {
Command command = toCommand(dto)
doX(command)
}
void doX(Command command) {
// ...
aggregate.Route(command.connectionType)
}
It's fairly common for the toCommand logic use something like a Builder pattern to improve the readability of the code.
if(DTO.connectionType == 0){
aggregate.Route(ConnectionType.InCity(cityId))
}
elseif(DTO.connectionType == 1){
aggregate.Route(ConnectionType.Intercity(DTO.originCityId,DTO.DestinationCityId)
}
In cases like this one, the strategy pattern can help
ConnectionTypeFactory f = getConnectionFactory(DTO.connectionType)
ConnectionType connectionType = f.create(DTO)
Once that you recognize that ConnectionTypeFactory is a thing, you can think about building lookup tables to choose the right one.
Map<ConnectionType, ConnectionTypeFactory> lookup = /* ... */
ConnectionTypeFactory f = lookup(DTO.connectionType);
if (null == f) {
f = defaultConnectionFactory;
}
So why don't you use more inheritance
for example
class CompanyRegistration : Registration {
}
class PersonRegistraiton : Registration {
}
then you can use inheritance instead of your if/else scenario's
public class Aggregate {
public Aggregate (CompanyRegistration) {
registantType = RegistrantType.Company();
}
public Aggregate (PersonRegistration p) {
registrantType = RegistrantType.Person();
}
}
you can apply simmilar logic for say a setRoute method or any other large if/else situations.
Also, i know you don't want to hear it, you can write your own mapper (inside the aggegate) that maps and validates it's business logic
for example this idea comes from fluentmapper
var mapper = new FluentMapper.ThatMaps<Aggregate>().From<DTO>()
.ThatSets(x => x.title).When(x => x != null).From(x => x.title)
It isn't too hard to write your own mapper that allow this kind of rules and validates your properties. And i think it will improve readability

Passing parameters to db.query with arangojs

I'm having problems sending parameters with the ArangoJS library and was wondering if anyone could help.
With the example below, it is possible to execute db.query if parameter values are in the query, but as soon as I try to use bindVars I get silent errors and I can't extract any error details.
var db = require('arangojs')("http://127.0.0.1:8529");
/*
The '_system' database contains a collection called 'test' that contains one document:
{
"a": 1,
"b": 2
}
*/
// This works
db.query('FOR t IN test FILTER t.a == 1 RETURN t')
.then((cursor) => {
cursor.all()
.then(vals => {
console.log("\nNo bindVars");
console.log(vals);
});
});
// This does not work
db.query("FOR t IN #first FILTER t.a == #second RETURN t", { first: "test", second: 1 })
.then((cursor) => {
cursor.all()
.then(vals => {
console.log("\nUsing bindVars");
console.log(vals);
});
});
I'm new to Node.js and ArangoDB and would love to be able to use properly parameterized queries.
I'm also assuming that this use of parameters protects you from SQL Injection style attacks?
Thanks!
The problem isn't with the JavaScript driver or Node, the problem is with the query itself:
FOR t IN #first FILTER t.a == #second RETURN t
In AQL collection names can't be injected with ordinary bind parameters. This is because you're not actually trying to use the parameter as a string value but to refer to a collection with that name. To quote the AQL documentation:
A special type of bind parameter exists for injecting collection names. This type of bind parameter has a name prefixed with an additional # symbol (thus when using the bind parameter in a query, two # symbols must be used).
In other words, in AQL it has to be called ##first (instead of #first) and in the bind parameters argument to db.query it has to be called #first (instead of just first).
When using arangojs it's actually possible to avoid this entirely by using the aqlQuery template handler:
var aqlQuery = require('arangojs').aqlQuery;
var first = db.collection('test');
var second = 1;
db.query(aqlQuery`
FOR t IN ${first}
FILTER t.a == ${second}
RETURN t
`).then(
cursor => cursor.all()
).then(vals => {
console.log('Using aqlQuery');
console.log(vals);
});
This way you don't have to think about bind parameter syntax when writing queries and can write more complex queries without having to mess with extremely long strings. Note that it will recognize arangojs collection instances and handle them accordingly. Using a string instead of a collection instance would result in the same problems as in your example.
Additionally note that the template handler also exists in the arangosh shell and in ArangoDB itself (e.g. when using Foxx).

need help removing an item from a collection of objects

I am working on a performance review application. I have a collection that tracks responses from various individuals. This particular closure will collect the scores from all the evaluations
def scores = responses.collectEntries { Category key, Collection<Response> value ->
[(key): value.score]
}
This is the structure of the value parameter, being passed in the closure
http://snag.gy/q4OEq.jpg
I am trying to remove a score from being added to the [(key):value.score]. More specifically, I'm trying to prevent someones self evaluation from being added.
I've tried several things. This being two of them.
def scores = responses.collectEntries { Category key, Collection<Response> value ->
for (i in value) {
if (i.evaluation.respondent == i.evaluation.performanceReview.employee) {
//didn't work
value.remove(i)
// also didn't work
value - i
}
}
[(key): value.score]
}
The if statement works fine, it will hit the inside conditions when there is a self evaluation. It's what's inside the if statment that doesn't work. What am I doing wrong?
Can't you get rid of the for loop and just have:
[ key,
value.findAll {
it.evaluation.respondent != it.evaluation.performanceReview.employeescore
}.score ]
As the body of the collectEntries call?

Resources