I'm trying to use aggregate functions to extract multiple different sums from a table using multiple different where clauses.
I'm trying to do something like this:
model.findAll({
attributes:[
[sequelize.fn('sum', sequelize.col('columnA')), 'sumA1'], // need to add where:{condition1}
[sequelize.fn('sum', sequelize.col('columnB')), 'sumB1'], // need to add where:{condition1}
[sequelize.fn('sum', sequelize.col('columnA')), 'sumA2'], // need to add where:{condition2}
[sequelize.fn('sum', sequelize.col('columnB')), 'sumB2'], // need to add where:{condition2}
...
]
)}
So far I've managed to make this work using Promise.all and making different calls to the database for each "where clause condition" or getting all data from the table and using node to calculate the sums. Is there a better way of doing this using sequelize?
If you prefer the literal way then it would be simple with subquery where
model.findAll({
limit: 1,
attributes:[
[sequelize.literal(`(select sum(columnA) from table where date = '2021-01-05')`), 'testSum'],
...and so on
]
})
you see from basic SQL concept the thing you need to perform is subquery or the over() clause using the window in either cases you need have a raw query first and then try to convert it over to sequelize syntax.
Related
So, the criteria are already quite powerful. Yet I came across a case I seem to not be able to replicate on the criteria object.
I needed to filter out all entries that were not timely relevant.
In a world, where you'd be able to mix SQL with the field definition, it would look like this:
...->addFilter(
new RangeFilter('DATEDIFF(NOW(), INTERVAL createdAt DAY)', [RangeFilter::LTE => 1])
)
Unfortunately that doesn't work in our world.
When i pass the criteria to a searchfunction, i only get:
"DATEDIFF(NOW(), INTERVAL createdAt DAY)" is not a field on xyz
I tried to do it with ->addExtensions and several other experiments, but i couldn't get it to work. I resorted to using the queryBuilder from Doctrine, using queryParts, but the data i'm getting is not very clean and not assigned to an ORMEntity like it should be.
Is it possible to write a criteria that incooperates native SQL filtering?
The DAL is designed in a way that should explicitly not accept SQL statements as it is a core concept of the abstraction. As the DAL offers extendibility for third party extensions it should be preferred to raw SQL in most cases. I would suggest writing a lightweight query that only fetches the IDs using your SQL query and then use these pre-filtered IDs to fetch complete data sets using the DAL.
$ids = (new QueryBuilder($connection))
->select(['LOWER(HEX(id))'])
->from('product')
->where('...')
->execute()
->fetchFirstColumn();
$criteria = new Criteria($ids);
This should offer the best of both worlds, the freedom of using raw SQL and the extendibility features of the DAL.
In your specific case you could also just take the current day, remove the amount of days that should have passed and use this threshold date to compare it to the creation date:
$now = new \DateTimeImmutable();
$dateInterval = new \DateInterval('P1D');
$thresholdDate = $now->sub($dateInterval);
// filter to get all with a creation date greater than now -1 day
$filter = new RangeFilter(
'createdAt',
[RangeFilter::GTE => $thresholdDate->format(Defaults::STORAGE_DATE_TIME_FORMAT)]
);
I'm trying to filter an influx DB query (using the nodeJS influxdb-client library).
As far as I can tell, it only works with "flux" queries.
I would like to filter out all records that share a specific attribute with any record that matches a particular condition. I'm filtering using the filter-function, but I'm not sure how I can continue from there. Is this possible in a single query?
My filter looks something like this:
|> filter(fn:(r) => r["_value"] == 1 and r["button"] == "1" ) and I would like to leave out all the record that have the same r["session"] as any that match this filter.
Do I need two queries; one to get those r["session"]s and one to filter on those, or is it possible in one?
Update:
Trying the two-step process. Got the list of r["session"]s into an array, and attempting to use the contains() flux function now to filter values included in that array called sessionsExclude.
Flux query section:
|> filter(fn:(r) => contains(value: r["session"], set: ${sessionsExclude}))
Getting an error unexpected token for property key: INT ("102")'. Not sure why. Looks like flux tries to turn the values into Integers? The r["session"] is also a String (and the example in the docs also uses an array of Strings)...
Ended up doing it in two queries. Still confused about the Strings vs Integers, but casting the value as an Int and printing out the array of r["session"] within the query seems to work like this:
'|> filter(fn:(r) => not contains(value: int(v: r["session"]), set: [${sessionsExclude.join(",")}]))'
Added the "not" to exclude instead of retain the values matching the array...
I have a type ORM query that returns five columns. I just want the company column returned but I need to select all five columns to generate the correct response.
Is there a way to wrap my query in another select statement or transform the results to just get the company column I want?
See my code below:
This is what the query returns currently:
https://i.stack.imgur.com/MghEJ.png
I want it to return:
https://i.stack.imgur.com/qkXJK.png
const qb = createQueryBuilder(Entity, 'stats_table');
qb.select('stats_table.company', 'company');
qb.addSelect('stats_table.title', 'title');
qb.addSelect('city_code');
qb.addSelect('country_code');
qb.addSelect('SUM(count)', 'sum');
qb.where('city_code IS NOT NULL OR country_code IS NOT NULL');
qb.addGroupBy('company');
qb.addGroupBy('stats_table.title');
qb.addGroupBy('country_code');
qb.addGroupBy('city_code');
qb.addOrderBy('sum', 'DESC');
qb.addOrderBy('company');
qb.addOrderBy('title');
qb.limit(3);
qb.cache(true);
return qb.getRawMany();
};```
[1]: https://i.stack.imgur.com/MghEJ.png
[2]: https://i.stack.imgur.com/qkXJK.png
TypeORM didn't meet my criteria, so I'm not experienced with it, but as long as it doesn't cause problems with TypeORM, I see an easy SQL solution and an almost as easy TypeScript solution.
The SQL solution is to simply not select the undesired columns. SQL will allow you to use fields you did not select in WHERE, GROUP BY, and/or ORDER BY clauses, though obviously you'll need to use 'SUM(count)' instead of 'sum' for the order. I have encountered some ORMs that are not happy with this though.
The TS solution is to map the return from qb.getRawMany() so that you only have the field you're interested in. Assuming getRawMany() is returning an array of objects, that would look something like this:
getRawMany().map(companyRecord => {return {company: companyRecord.company}});
That may not be exactly correct, I've taken the day off precisely because I'm sick and my brain is fuzzy enough I was making too many stupid mistakes, but the concept should work even if the code itself doesn't.
EDIT: Also note that map returns a new array, it does not modify the existing array, so you would use this in place of the getRawMany() when assigning, not after the assignment.
I am using knex.js
suppose we have three table :-
table1-- id,name,address
table2--id,city,sate,table1_id as fk
table3--id,housenumber,table1_id as fk
I want to join these three table using knex.js libraray of node and express
so that i want to get output json like this.
{
"id":1,
"name":"abc",
"address:"xyz",
"table2":{"id":1,"city":"ttt","state":"www" }//i want check if table1.id == table2.table1_id then put table details
"table3":[]//if no relation found between table1.id === table3.table1.id then kept it as an array
}
tl;dr knex is too low level tool for the thing you are trying to do, you should use an ORM for that kind of task
However you can do that with lots of manual work.
First you have to make the query with proper joins and creating aliases with table prefixes for each column of table to be able to get result data in a format where all data is in a flat array like:
knex('table1' as t1)
.join('table2 as t2', 't2.t1_id', 't1.id')
.select(
't1.id as t1_id',
't1.other_column as t1_other_column',
't2.id as t2_id', <more columns you want to extract>)
Results something like
[ { t1_id: 1, t1_other_column: 'foo', t2_id: 4}, ... more rows with flat data... }]
Then you need to write javascript code for restructuring flat data to nested objects.
But you should not do that kind of work manually. All knex based ORMs has already implemented general solutions for writing that kind of queries in easy manner.
I have defined a model like
Class Orders(Document):
orderAmount = fields.FloatField()
cashbackAmount = fields.FloatField()
meta = {'strict': False}
I want to get all orders where (orderAmount - cashbackAmount value > 500). I am using Mongoengine and using that I want to perform this operation. I am not using Django Framework so I cannot use solutions of that.
Let's approach this if you had to do this without Mongoengine. You would start by dividing this problem into two steps
1) How to get the difference between two fields and output it as the new field?
2) How to filter all the documents based on that field's value?
You can see that it consists of several steps, so it looks like a great use case for the aggregation framework.
The first problem can be solved using addFields and subtract operators.
{$addFields: {difference: {$subtract: ["$a", "$b"]}}}
what can be translated into "for every document add a new field called difference where difference=a-b".
The second problem is a simple filtering:
{$match: {difference:{$gt: 500}}}
"give me all documents where difference field is greater than 500"
So the whole query in MongoDB would look like this
db.collectionName.aggregate([{$addFields: {difference: {$subtract: ["$a", "$b"]}}}, {$match: {difference:{$gt: 500}}}])
Now we have to translate it into Mongoengine. It turns out that there is aggregate method defined, so we can easily make small adjustments to make this query work.
Diff.objects.aggregate({"$addFields": {"difference": {"$subtract": ["$a", "$b"]}}}, {"$match": {"difference":{"$gt": 500}}})
As a result, you get CommandCursor. You can interact with that object or just convert it to the list, to get a list of dictionaries.