I'm new to node-postgres and trying to pass in parameters connected to a table alias, but it's not working. How can I parameterize the simple query below? Take something like
const pgQuery = 'SELECT t1.YEAR as year, t1.CODE as code FROM data t1;';
and instead do something like
const pgQuery = 'SELECT t1.$1 as year, t1.$2 as code FROM data t1;';
const values = ['YEAR', 'CODE'];
Per the docs here:
https://node-postgres.com/features/queries
PostgreSQL does not support parameters for identifiers. If you need to have dynamic database, schema, table, or column names (e.g. in DDL statements) use pg-format package for handling escaping these values to ensure you do not have SQL injection!
Which takes you here:
https://www.npmjs.com/package/pg-format
Repo:
https://github.com/datalanche/node-pg-format
Related
I'm looking for a way to count with SQLAlchemy the number of rows that is returned from a given query (that potentially includes filters) but everything I find on the net makes explicit use of a model (example here). My problem is that I don't have a model, I only have a Table object (because I'm dealing with temporary tables that vary in format from time to time). For the moment I can do the following:
tbl = Table(mytablename,metadata,autoload=True, autoload_with=myengine, schema=myschemaname)
query = select([tbl])
filters = build_filters(...) #my function that build filters
query = query.where(and_(*filters))
conn = myengine.connect()
ResultProxy = conn.execute(query)
totalCount = len(ResultProxy.fetchall())
but it's very inefficient. Is there a way to do the count efficiently and without referring to any model?
Try the SQLAlchemy Core 'count' function documented here. I believe you can attach your filters on to that like you're doing now. So, (not guaranteeing my syntax here, but here's something to start you with)...
query = select([func.count()]).select_from(my_table).where(and_(*filters))
conn = myengine.connect()
ResultProxy = conn.execute(query)
totalCount = ResultProxy.fetchone()[0]
According to the documentation, I believe this will actually generate a SELECT COUNT from the database, not actually bring all the rows back from the DB and then count them.
I have a query (Update statement) wrapped in a function and will need to perform the same statement on multiple columns during the course of my script
async function update_percentage_value(value, id){
(async () => {
const client = await pool.connect();
try {
const res = await client.query('UPDATE fixtures SET column_1_percentage = ($1) WHERE id = ($2) RETURNING *', [value, id]);
} finally {
client.release();
}
})().catch(e => console.log(e.stack))
}
I then call this function
update_percentage_value(50, 2);
I have many columns to update at various points of my script, each one needs to be done at the time. I would like to be able to just call the one function, passing the column name, value and id.
My table looks like below
CREATE TABLE fixtures (
ID SERIAL PRIMARY KEY,
home_team VARCHAR,
away_team VARCHAR,
column_1_percentage INTEGER,
column_2_percentage INTEGER,
column_3_percentage INTEGER,
column_4_percentage INTEGER
);
Is it at all possible to do this?
I'm going to post the solution that was advised by Sehrope Sarkuni via the node-postgres GitHub repo. This helped me a lot and works for what I require:
No column names are identifiers and they can't be specified as parameters. They have to be included in the text of the SQL command.
It is possible but you have to build the SQL text with the column names. If you're going to dynamically build SQL you should make sure to escape the components using something like pg-format or use an ORM that handles this type of thing.
So something like:
const format = require('pg-format');
async function updateFixtures(id, column, value) {
const sql = format('UPDATE fixtures SET %I = $1 WHERE id = $2', column);
await pool.query(sql, [value, id]);
}
Also if you're doing multiple updates to the same row back-to-back then you're likely better off with a single UPDATE statement that modifies all the columns rather than separate statements as they'd be both slower and generate more WAL on the server.
To get the column names of the table, you can query the information_schema.columns table which stores the details of column structure of your table, this would help you in framing a dynamic query for updating a specific column based on a specific result.
You can get the column names of the table with the help of following query:
select column_name from information_schema.columns where table_name='fixtures' and table_schema='public';
The above query would give you the list of columns in the table.
Now to update each one for a specific purpose, You can store the result set of column name to a variable and pass that variable to the function to perform the required action.
I'd like to update the value of column A by applying a function to column B.
Is there a simple solution of the form:
knex('table')
.update({
colA: func(${colB})
})
Yes, there is a way to do this within Knex.
For SQL functions which don’t have explicit support in Knex you use knex.raw(SQLstring, parmArray) to encapsulate a SQL snippet or knex.schema.raw(...) to produce an entire SQL statement. And you use single question marks ? for value replacements, and double question marks ?? for field identifier replacements. (see link)
So the SQL: UPDATE table SET colA = func(colB)
... can be produced by including a SQL snippet: (you were close)
knex('table')
.update({
colA: knex.raw( 'func(??)', ['colB'] )
})
... or as full raw SQL:
knex.schema.raw( 'UPDATE table SET ?? = func(??)', ['colA', 'colB'] )
Cheers, Gary.
I have an Object that maps column names to values. The columns to be updated are not known beforehand and are decided at run-time.
e.g. map = {col1: "value1", col2: "value2"}.
I want to execute an UPDATE query, updating a table with those columns to the corresponding values. Can I do the following? If not, is there an elegant way of doing it without building the query manually?
db.none('UPDATE mytable SET $1 WHERE id = 99', map)
is there an elegant way of doing it without building the query manually?
Yes, there is, by using the helpers for SQL generation.
You can pre-declare a static object like this:
const cs = new pgp.helpers.ColumnSet(['col1', 'col2'], {table: 'mytable'});
And then use it like this, via helpers.update:
const sql = pgp.helpers.update(data, cs) + /* WHERE clause with the condition */;
// and then execute it:
db.none(sql).then(data => {}).catch(error => {})
This approach will work with both a single object and an array of objects, and you will just append the update condition accordingly.
See also: PostgreSQL multi-row updates in Node.js
What if the column names are not known beforehand?
For that see: Dynamic named parameters in pg-promise, and note that a proper answer would depend on how you intend to cast types of such columns.
Something like this :
map = {col1: "value1", col2: "value2",id:"existingId"}.
db.none("UPDATE mytable SET col1=${col1}, col2=${col2} where id=${id}", map)
I am trying to write a simple query using the pgp-promise library. My original implementation looks like:
var bar = function(orderBy){
var qs = 'select * from mytable order by ${orderBy};';
return db.many(qs,{orderBy:orderBy});
}
...
bar('id').then(...)
But this gives an error of non-integer constant in ORDER BY
I have also tried adding quotes aroung ${orderBy} and adding double quotes to the orderBy paramater to no avail. I have a working solution by doing var qs = 'select * from mytable order by "' + orderBy + '";' though it should be obvious why I don't want code like that in the project.
My question: Is there a way to get pg-promise to build a query with an order by clause that isn't vulnerable to sql injection?
Is there a way to get pg-promise to build a query with an order by clause that isn't vulnerable to sql injection?
The value for ORDER BY clause is an SQL name, and it is to be formatted using SQL Names:
const bar = function(orderBy) {
const qs = 'select * from mytable order by ${orderBy:name}';
return db.many(qs, {orderBy});
}
whereas :raw / ^ is injecting raw text, which is vulnerable to SQL injections when it comes from outside, and to be used only for strings that have been created and pre-formatted inside the server.