Working with KnexJS on a project and have been using the .raw() method throughout the project with out a single issue.
I now have one case where the .raw is just not being built into the SQL, and so I end up with the syntax error at end of input error. If I dump the SQL string, I can see why as it's failing
'update "mytable" set "a" = ? returning '
I can see why it's having an issue, the problem I'm having is that the returning is a Raw value as such, and so I just can't figure out why it's not being compiled with the SQL.
knexQuery.update(data).into('mytable').returning( knexQuery.raw('mytable::json') );`
If I use a string, in place of the raw in the returning() method, it will compile the string into it.
If I console out the knexQuery.raw('mytable::json') part, it shows as a Raw object, with the right object data...
Raw {
client:
Client_PG {
... },
sql: 'mytable::json',
bindings: undefined,
_wrappedBefore: undefined,
_wrappedAfter: undefined,
_debug: undefined } }
I know that the SQL works, as I've filled in the missing part and it works, but the SQL is not the issue, I just can't figure out why the raw() method is not being compiled with the SQL string.
I also have another piece of code (and INSERT one) somewhere else in my code that is using the same returning( knexQuery.raw(....) ) and that works perfectly fine.
I'm starting to think this is a bug in the code, but after spending an hour going through the KnexJS library code, I can't see any reason why it would not work.
So why does this code not build the raw into the query, while my other one has no problem and works?
I'm getting to the point where I just want to use something else to get around the problem, but it's just not possible without using this method.
This is bug in knex https://runkit.com/mikaelle/592412c3a631940012a51928 please open issue in knex github. Looks like handling raw input just haven't been implemented to returning builder method.
Anyways looks like this works as a workaround:
knex('TestTable')
.insert({ foo: 'bar' })
.returning([knex.raw('mytable::json')]) // raw wrapped in array
.toSQL();
Related
I use jooq to generate objects against a local database, but when running "for real" later in production the actual databases will have different names. To remedy this I use the <outputSchemaToDefault>true</outputSchemaToDefault> config option (maven).
At the same time, we have multiple databases (schemas), and are using a connection pool to the server like "jdbc:mysql://localhost:3306/" (without specifying a database here).
How do I tell jooq which database to use when running queries?
I have tried all config I can think of:
new Settings()
.withRenderSchema(true) // true/false seems to make no difference.
.withRenderCatalog(true) // true/false seems to make no difference.
.withRenderMapping(new RenderMapping()
.withDefaultSchema("my_database") // Seems to have no effect.
// The above 3 configs always give me an error saying "no database selected".
// Adding this gives me 'my_database.my_table' does not exist - while it actually does.
.withSchemata(new MappedSchema()
.withInputExpression(Pattern.compile(".*"))
.withOutput("my_database")
));
I have also tried using a database/schema name, as in not configuring outputSchemaToDefault. But then, adding the MappedSchema code above, but that gives me errors with "'my_databasemy_database.my_table' does not exist", which is correct. I have no clue why that code gives me the database/schema name twice?
Edit:
When jooq tells me that the db.table does not exist, if I put a break point in a good place and get the sql from jooq and run exactly that against my database it does work. But jooq fails to run it.
Also, I'm using version 3.15.3 of jooq.
I solved it. Instead of using .withInputExpression(Pattern.compile(".*")), it seems to work with .withInput("").
I'm still not sure why it works, or if this is the "correct" way of solving it. But at least it is a way forward.
No clue why using the pattern, I got the name twice though. But that one I'll leave alone.
The database is in Azure cloud and not being used in production currently. There are 80.000 rows and a uprn is a VARCHAR(100);
I'm already using JOI to validate each UPRN as well;
I'm using KNEX with a SQL Server database with the following whereIn query:
knex(LOCATIONS.table).whereIn(LOCATIONS.uprn, req.body.uprns)
but this takes 8-12s to complete and sometimes timesout. if I use .toQuery() on the same thing, SSMS will return the result within 1-2.
If I do a raw query, the resulting .toQuery() or toString() works in SSMS and returns results. But if I try to use the raw directly, it will return 0 results.
I'm looking to either fix what's making whereIn so slow or get the raw query working.
EDIT 1:
After much debugging and trying -- it seems that the bug is due to how knex deals with arrays, so I made a for-of loop to add ? ? ? for each array element and then inputed the array for all params.
This led me to realizing the performance issue is due to SQL server way of parameterising.
I ended up building a raw query string with all of the parameters and validating the input with Joi string/regex config:
Joi.string()
.min(1)
.max(35)
.regex(/^[a-z\d\-_\s]+$/i)
allowing only for alphanumeric, dashes and spaces which should prevent sql injection.
I'm going to look deeper into security issues with this and might make a separate login that can only SELECT data from that table and nothing more to run with these queries.
Needed to just handle it raw and validate separately.
I've read through every stack overflow I can find and I don't understand why this still isn't working.
I'm trying to construct a NodeJS Mongo find query and very simply want to use a variable as the values, the key does not need to be dynamic.
This is the code I was working with initially :
collection.find({project_id : project_id_val})
but this simply returns :
Found the following records
[]
I've also tried constructing my own javascript object and passing that in e.g.
Query = {}
Query["project id"] = project_id_val
collection.find(query)
But that doesn't work either, I know the key/value pair is correct because
project_id: "12345" works absolutely fine, and returns exactly what I want it to. I feel like this should be very simple so if someone could let me know where I'm going wrong that would be great.
Thanks.
So I have a NodeJS+KnexJS setup on a PostgreSQL DB, and am using the .whereRaw() method so I can use a CASE statement in my WHERE clause.
The query was tested in my CLI before migrating to code. Here is the code that is being used.
var qry = knex.select(....); // ignore the select, not important.
qry.with('daspecs', function(qy) {
qy.select('spec_id').from('drawings').where('uid', query.d);
}).whereRaw('CASE WHEN (select "spec_id" from "daspecs") IS NULL THEN true ELSE c.spec_id = (select "spec_id" from "daspecs") END');
The SQL that KnexJS is generating (output using qry.toString()) is correct, and I can even copy and paste this to my psql CLI and it returns the results I want (12 records), but for some wierd reason the KnexJS query seems to return a completely different set of results (1106 records).
Not sure where to go next, since KnexJS is giving me the right SQL, but seems like it's executing something else, and not sure how else to diagnose what it is actually doing (I've tried the knex.on('query'...) event).
Any alteration on the final SQL would result in an error (i've tested), so at the point of ruling out missing pieces.
Has anyone had any experience or issues with KnexJS saying one thing, but doing another, in particular, with the whereRaw command?
I am new to CouchDB / PouchDB and until now I somehow could manage the start of it all. I am using the couchdb-python library to send initial values to my CouchDB before I start the development of the actual application. Here I have one database with templates of the data I want to include and the actual database of all the data I will use in the application.
couch = couchdb.Server()
templates = couch['templates']
couch.delete('data')
data = couch.create('data')
In Python I have a loop in which I send one value after another to CouchDB:
value = templates['Template01']
value.update({ '_id' : 'Some ID' })
value.update({'Other Attribute': 'Some Value'})
...
data.save(value)
It was working fine the whole time, I needed to run this several times as my data had to be adjusted. After I was satisfied with the results I started to create my application in Javascript. Now I synced PouchDB with the data database and it was also working. However, I found out that I needed to change something in the Python code, so I ran the first python script again, but now I get this error:
couchdb.http.ResourceConflict: (u'conflict', u'Document update conflict.')
I tried to destroy() the pouchDB database data and delete the CouchDB database as well. But I still get this error at this part of the code:
data.save(value)
What I also don't understand is, that a few values are actually passed to the database before this error comes. So some values are saved() into the db.
I read it has something to do with the _rev values of the documents, but I cannot get an answer. Hope someone can help here.