I've traded node-DBI for knex because it has more function that I require.
So far I'd make the same choice again but only one thing is holding me back: writing abstract methods that take a options variable where params like where, innerjoin and such are contained in.
Using node-dbi I could easily forge a string using these variables but I can't seem to create the knex chain dymanicly because after using a switch, you'd get knex.method is not a function.
Any idea how to resolve this?
I'm looking for something as in
`getData(table,options){
var knex=knex
if(options.select)
/** append the select data using knex.select()
if(options.where)
/** append the where data using knex.where(data)*/
if(options.innerJoin)
/** append innerjoin data*/
}`
This way I can avoid having to write alot of DB functions and let my Business Logical Layers handel the requests
/*This function serves as the core of our DB layer
This will generate a SQL query and execute it whilest returning the response prematurely
#param obj:{Object} this is the options object that contain all of the query options
#return Promise{Object}: returns a promise that will be reject or resolved based on the outcome of the query
The reasoning behind this kind of logic is that we want to abstract our layer as much as possible, if evne the slightest
sytnax change occurs in the near future, we can easily update all our code by updating this one
We are using knex as a query builder and are thus relying on Knex to communicate with our DB*/
/*Can also be used to build custom query functions from a data.service. This way our database service will remain
unpolluted from many different functions and logic will be contained in a BLL*/
/* All available options
var options = {
table:'table',
where:{operand:'=',value:'value',valueToEqual:'val2'},
andWhere:[{operand:'=',value:'value',valueToEqual:'val2'}],
orWhere:[{operand:'=',value:'value',valueToEqual:'val2'}],
select:{value:['*']},
insert:{data:{}},
innerJoin:[{table:'tableName',value:'value',valueToEqual:'val2'}],
update:{data:{}}
}*/
/*Test object*/
/*var testobj = {
table:'advantage',
where:{operand:'>',value:'id',valueToEqual:'3'},
select:{value:['*']},
innerJoin:{table:'User_Advantage',value:'User_Advantage.Advantageid',valueToEqual:'id'}
}
var testobj = {
table:'advantage',
where:{operand:'>',value:'id',valueToEqual:'3'},
select:{value:['*']},
innerJoin:{table:'User_Advantage',value:'User_Advantage.Advantageid',valueToEqual:'id'}
}
queryBuilder(testobj)*/
function queryBuilder(options){
var promise = new Promise(function (resolve, reject) {
var query;
for (var prop in options) {
/*logger.info(prop)*/
if (options.hasOwnProperty(prop)) {
switch (prop) {
case 'table':
query = knex(options[prop]);
break;
case 'where':
query[prop](options[prop].value, options[prop].operand, options[prop].valueToEqual);
break;
/*andWhere and orWhere share the same syntax*/
case 'andWhere':
case 'orWhere':
for(let i=0, len=options[prop].length;i<len;i++){
query[prop](options[prop][i].value, options[prop][i].operand, options[prop][i].valueToEqual);
}
break;
case 'select':
query[prop](options[prop].value);
break;
/*Same syntax for update and insert -- switch fallthrough*/
case 'insert':
case 'update':
query[prop](options[prop].data);
break;
case 'innerJoin':
for(let i=0, len=options[prop].length;i<len;i++){
query[prop](options[prop][i].table, options[prop][i].value, options[prop][i].valueToEqual);
}
break;
}
}
}
return query
.then(function (res) {
return resolve(res);
}, function (error) {
logger.error(error)
return reject(error);
})
return reject('Options wrongly formatted');
});
return promise
}
Thanks to Molda I was able to produce the code above. This one takes a Object called options in as a parameter and will build the knex chain based on this value. See the comments for the syntax of Object
Not every knex query option has been included but this will serve as a good base for anyone trying to achieve a similar effect.
Some examples to use this:
/*Will return all values from a certain table
#param: table{String}: string of the table to query
#param: select{Array[String]}: Array of strings of columns to be select -- defaults to ['*'] */
function getAll(table,select) {
/*Select * from table as default*/
var selectVal=select||['*']
var options={
table:table,
select:{value:selectVal}
}
return queryBuilder(options)
}
or a more specific use case:
function getUserAdvantages(userid){
var options = {
table:'advantage',
innerJoin:[{table:TABLE,value:'advantage.id',valueToEqual:'user_advantage.Advantageid'}],
where:{operand:'=',value:'user_advantage.Userid',valueToEqual:userid}
}
return sqlService.queryBuilder(options)
}
Note: the sqlService is a module of node that I export containing the queryBUilder method.
Edit: I wanted to add that the only roadblock I had was using the .from / .insert from Knex. I no longer use these methods as they resulted in errors when using them. Ive used knex(table) as commented.
Related
I don't get this. I have a service that injects entity repositories and has dedicated methods to do some business logic and functions.
Beside that I expose a method that just returns QueryBuilder - to avoid injecting repositories all over the place - for a few occasions when other service needs just a quick query:
type EntityFields = keyof MyEntity;
entityQueryBuilder(alias?: string, id?: number, ...select: EntityFields[]) {
const q = this.entityRepository.createQueryBuilder(alias);
if (id) {
q.where({id});
}
if (select) {
q.select(select);
}
return q;
}
Now when I am trying to use this and call:
const r = await service.entityQueryBuilder('a', 1, 'settings').getOne();
the result is always empty although in the log the generated SQL is correct.
However when I do:
const r = await service.entityQueryBuilder('a', 1, 'settings').execute();
I get (almost) what I need. I get array instead of an entity object directly but the data are there.
I am unhappy though as I need to map the result to the object I wanted, which is something that getOne() should do on my behalf. getMany() does not return results either.
What did I do wrong?
Edit:
FWIW here is the final solution I came up with based on the hint in accepted reply:
entityQueryBuilder(id?: number, ...select: EntityFields[]) {
const q = this.entityRepository.createQueryBuilder('alias');
if (id) {
q.where({id});
}
if (select) {
q.select(select.map(f => `alias.${f}`));
}
return q;
}
Admittedly it has hardcoded alias but that I can live with and is OK for my purpose.
Hope this helps someone in the future.
It happens because you put no really proper select. In your case, you need a.settings instead of settings:
const r = await service.entityQueryBuilder('a', 1, 'a.settings').getOne(); // it should works
Using Knex to make queries to my Postgres DB. I have a function that provides a "base" query using the Knex QueryBuilder. This works fine until I need to add something raw to the SELECT statement. From what I can tell, running .raw always wants to return a result. I just need it to be added to the QueryBuilder, though, so it can be executed by a different part of my app.
const baseQuery = knex
.select(newUserFields)
.from('users')
.leftJoin('user_roles', 'users.id', 'user_roles.user_id')
.leftJoin('roles', 'user_roles.role_id', 'roles.id')
.leftJoin('role_permissions', 'roles.id', 'role_permissions.role_id')
.leftJoin(
'permissions',
'permissions.id',
'role_permissions.permission_id'
)
.groupBy(
'users.id',
'users.email',
'users.name',
'users.status',
'users.created_at',
'users.password_reset_expiration',
'users.password',
'users.password_reset_token'
)
.orderBy('users.created_at', 'desc');
I need to add the following to the select:
knex.raw('to_json(array_agg(distinct roles.name)) as roleNames')
knex.raw('to_json(array_agg(distinct permissions.name)) as permissionNames')
How can I add these raw selects to the base query so that the base query can then be passed to a different function as a QueryBuilder and added to?
The cool thing about knex is that it is a queryBuilder, which allows you to call the methods without any limits about the order of calls. That means that you can just construct your base query in a function, and then attach to it additional things (such as additional columns).
In your case you just can call another time to select (knex will join the select calls)
// base-query.js
export const getBaseQuery = () => knex
.select(newUserFields)
.from('users')
.leftJoin('user_roles', 'users.id', 'user_roles.user_id')
.leftJoin('roles', 'user_roles.role_id', 'roles.id')
.leftJoin('role_permissions', 'roles.id', 'role_permissions.role_id')
.leftJoin('permissions', 'permissions.id', 'role_permissions.permission_id')
.groupBy(
'users.id',
'users.email',
'users.name',
'users.status',
'users.created_at',
'users.password_reset_expiration',
'users.password',
'users.password_reset_token'
)
.orderBy('users.created_at', 'desc');
// other-file.js
import {getBaseQuery} from 'base-query';
const enhancedQuery = getBaseQuery().select([
knex.raw('to_json(array_agg(distinct roles.name)) as roleNames'),
knex.raw('to_json(array_agg(distinct permissions.name)) as permissionNames'),
]);
const results = await enhancedQuery;
Another cool way that I'm using heavily, solves the following requirement: Sometimes I need to change an internal query from the outside, I use the modify
For example, I have a getProducts method which execute a select query and do some data transformation.
In order to implement getProductById which needs to return the same data structure (it is just need to filter the base query) I pass a queryModifier method which modifies the original query.
async function getProducts(queryModifier) {
const products = await knex
.select('*')
.from('products')
.modify((queryBuilder) => {
if (typeof queryModifier === 'function') {
return queryModifier(queryBuilder);
}
});
return products.map(someDataTransformation);
}
async function getProductById(id) {
return getProducts((qb) => {
return qb.where('id', id);
});
}
I'm new to JavaScript and I'm struggling tying a couple of things together.
I need to set a global variable from within a function. The function is essentially querying a DB, getting an answer and doing a comparison to determine what variable to set.
Then, in a separate function, I need to set a value based on the value of the previous variable.
The problem I'm having is that the variables are only available within the function and I don't know how to set them to be available outside the function. I've tried to simplify the code so you can see what I'm attempting to do:
ddb.scan(scanparams, function(err, data) {
if (err) {
console.log(err);
} else {
var dbResp = data.key.value; //key is the name of the key, value is the value
if (Number(dbResp) === Number(sessionId)) { //sessionId is defined elsewhere
var result = '1';
} else {
var result = '0';
}
}
});
if result === '1' {
doThing1;
} else {
doThing2;
}
I can't move the logic for doThing into the previous function as it breaks other things. How can I expose the results of the DB query to other functions?
I want to create a dynamic menu bar by fetching data from two collections (supcat and cat) then combining the two to create a new array which i will access on page load for menu but the push() is not working.
ngOnInit() {
this.cattest();}
cattest(){
var x;
this.supcatobj.fetchsupcat().subscribe(
(res)=>
{
if(res.length!=0)
{this.supcat=res;
for(let x=0;x<this.supcat.length; x++)
{
this.catobj.fetchallcat(this.supcat[x]["_id"]).subscribe(
(resp)=>
{
this.allcat=resp;
for(let y=0;y<this.allcat.length;y++)
{
}
this.testarr[x].push(this.supcat[x]["supcatname"],this.allcat);
}
);
}
}
}
);}
Instead of nesting subscribe() calls, I would try to compose separate observables for your two different collections and then use the combineLatest() operator to combine those into your desired array. It is hard to see exactly what you are working for, but conceptually it would be something like this:
const supcat$ = this.supcatobj.fetchsupcat().pipe(filter(cat => cat.length > 0));
const allCat$ = this.catobj.fetchallcat();
const combinedCats$ = combineLatest(supcat$, allCat$);
const res$ = combinedCats$.pipe(map(res => {
// do operation that involves both data sets
});
Remember that map() will return a new array. This way you will only need to subscribe to the one variable, and if you put it at the class level you could use the async pipe (|) in your template so it will unsubscribe automatically.
I have two queries:
a) select id from ingredietns where name = my_param;
b) select word_id from synonyms where name = my_param;
Both return 0 or 1 row. I can also add limit 1 if needed (or in knex first()).
I can translate each into knex like this:
knex("ingredients").select('id').where('name', my_param) //do we need first()?
knex("synonyms").select('word_id').where('name', my_param) //do we need first()?
I need function called "ingredientGetOrCreate(my_param)". This function would
a) check if any of above queries return result
b) if any of these return, then return ingredients.id or synonyms.word_id - only one could be returned
c) if record doesn't eixst in any of tables, I need to do knex inesrt aand return newly added id from function
d) later I am not sure I also understand how to call this newly create function.
Function ingredientGetOrCreate would be used later as seperate function or in the following scenario (like "loop") that doesn't work for me either:
knex("products") // for each product
.select("id", "name")
.map(function (row) {
var descriptionSplitByCommas = row.desc.split(",");
Promise.all(descriptionSplitByCommas
.map(function (my_param) {
// here it comes - call method for each param and do insert
ingredientGetOrCreate(my_param)
.then(function (id_of_ingredient) {
knex('ingredients_products').insert({ id_of_ingredient });
});
...
I am stuck with knex and Promise queries because of asynchronouse part. Any clues, please?
I though I can somehow use Promise.all or Promise.some to call both queries.
P.S. This is my first day with nodejs, Promise and knex.
As far as I decode your question, it consists of two parts:
(1) You need to implement upsert logic (get-or-create logic).
(2) Your get part requires to query not a single table, but a pair of tables in specific order. Table names imply that this is some sort of aliasing engine inside of your application.
Let's start with (2). This could definitely be solved with two queries, just like you sense it.
function pick_name (rows)
{
if (! rows.length) return null
return rows[0].name
}
// you can sequence queries
function ingredient_get (name)
{
return knex('ingredients')
.select('id').where('name', name)
.then(pick_name)
.then(name =>
{
if (name) return name
return knex('synonyms')
.select('word_id').where('name', name)
.then(pick_name)
})
}
// or run em parallel
function ingredient_get (name)
{
var q_ingredients = knex('ingredients')
.select('id').where('name', name)
.then(pick_name)
var q_synonyms = knex('synonyms')
.select('word_id').where('name', name)
.then(pick_name)
return Promise.all([ q_ingredients, q_synonyms ])
.then(([name1, name2]) =>
{
return name1 || name2
})
}
Important notions here:
Both forms works well and return first occurence or JS' null.
First form optimizes count of queries to DB.
Second form optimizes answer time.
However, you can go deeper and use more SQL. There's a special tool for such task called COALESCE. You can consult your SQL documentation, here's COLASCE of PostgreSQL 9. The main idea of COALESCE is to return first non-NULL argument or NULL otherwise. So, you can leverage this to optimize both queries and answer time.
function ingredient_get (name)
{
// preparing but not executing both queries
var q_ingredients = knex('ingredients')
.select('id').where('name', name)
var q_synonyms = knex('synonyms')
.select('word_id').where('name', name)
// put them in COALESCE
return knex.raw('SELECT COALESCE(?, ?) AS name', [ q_ingredients, q_synonyms ])
.then(pick_name)
This solution guarantees single query and furthermore DB engine can optimize execution in any way it sees appropriate.
Now let's solve (1): We now got ingredient_get(name) which returns Promise<string | null>. We can use its output to activate create logic or return our value.
function ingredient_get_or_create (name, data)
{
return ingredient_get(name)
.then(name =>
{
if (name) return name
// …do your insert logic here
return knex('ingredients').insert({ name, ...data })
// guarantee homohenic output across get/create calls:
.then(() => name)
})
}
Now ingredient_get_or_create do your desired upsert logic.
UPD1:
We already got ingredient_get_or_create which returns Promise<name> in any scenario (both get or create).
a) If you need to do any specific logic after that you can just use then:
ingredient_get_or_create(…)
.then(() => knex('another_table').insert(…))
.then(/* another logic after all */)
In promise language that means «do that action (then) if previous was OK (ingredient_get_or_create)». In most of the cases that is what you need.
b) To implement for-loop in promises you got multiple different idioms:
// use some form of parallelism
var qs = [ 'name1', 'name2', 'name3' ]
.map(name =>
{
return ingredient_get_or_create(name, data)
})
var q = Promise.all(qs)
Please, note, that this is an agressive parallelism and you'll get maximum of parallel queries as your input array provides.
If it's not desired, you need to limit parallelism or even run tasks sequentially. Bluebird's Promise.map is a way to run map which analogous to example above but with concurrency option available. Consider the docs for details.
There's also Bluebird's Promise.mapSeries which conceptually is an analogue to for-loop but with promises. It's like map which runs sequentially. Look the docs for details.
Promise.mapSeries([ 'name1', 'name2', 'name3' ],
(name) => ingredient_get_or_create(name, data))
.then(/* logic after all mapSeries are OK */)
I believe the last is what you need.