With jooq 3.13.x, we are using
Field<Instant> midPointDueTime = TICKET.READY.plus(TICKET.DUE.minus(TICKET.READY).div(2));
where READY and DUE fields are of type java.time.Instant. They are DATETIME fields in the database (normally java.sql.Timestamp) but are converted to Instant with a javax.persistence.AttributeConverter. The database in question is Informix, but we are using the open source version of jooq for now with the DEFAULT dialect and trying to avoid cases where things would deviate from standard SQL syntax.
From that field declaration jooq 3.13.x creates the following SQL snippet which works as expected
TICKET.READY + ((TICKET.DUE - TICKET.READY) / 2))
This is the expected DATETIME arithmetic. We are looking for a timestamp halfway between READY and DUE.
But jooq 3.14 or 3.15 both throw a runtime exception.
org.jooq.exception.DataTypeException: Cannot convert from 2 (class java.lang.Integer) to class java.time.LocalDateTime
No SQL is generated, so I don't think this is an Informix compatibility issue. The error happens before any SQL statement is logged.
Is this possibly a bug, or is there something else I can do to achieve the same date arithmetic result?
Regarding dialects
From the SQLDialect.DEFAULT javadoc:
This dialect is chosen in the absence of a more explicit dialect. It is not intended to be used with any actual database as it may combine dialect-specific things from various dialects.
The purpose of this dialect is to be used in cases where no other dialect is available, and debug log information is needed, e.g. when writing:
DSL.abs(1).toString(); // This uses DEFAULT to render the abs(1) function call
You shouldn't use this dialect in any production situation, its behaviour may change at any time for various reasons. It's certainly not integration tested with any RDBMS. Use the SQLDialect.INFORMIX dialect instead
After further research
Thanks to your detailed bug report hree: https://github.com/jOOQ/jOOQ/issues/12544, it can be seen that this is a regression that will be fixed in jOOQ 3.16.0 and 3.15.4
Related
I am using Jooq version 3.17.0 and attempting to insert data into a table without codegen.
At the minute, I am designing a system that allows data to be imported into multiple tables (one at a time, and starting with just one), yet I do not want to write specific code for each table and as of now, I haven't had a need for codegen.
The code currently works for importing data via JSON, with json being a String formatted in the 'Jooq' format. This imports data correctly into the database. This also allows us to send json data of table updates from one system to our main system that uses Jooq. Yet it gives me an error when I try to update.
I am using MYSQL as my database.
The original code for insertion is :
Result<Record> convertedJson = dslContext.fetchFromJSON(json);
Loader<Record> res1 = dslContext.loadInto(table(tableName)).loadJSON(json).fields(convertedJson.fields()).execute();
However, if we try to update data by sending in the same json, but with one field changed, jooq gives an error org.jooq.exception.DataAccessException stating that there is a duplicate entry for key.
I tried to use :
Loader<Record> res2 = dslContext.loadInto(table(tableName)).onDuplicateKeyUpdate().loadJSON(json).fields(convertedJson.fields()).execute();
But then this throws an error ON DUPLICATE KEY UPDATE only works on tables with explicit primary keys. Table is not updatable : <tableName> since in LoaderImpl.onDuplicateKeyUpdate():220 since table.getPrimaryKey() is null which technically makes sense since table(tableName) returns a Table that does not know it's fields.
My question is probably two-fold.
Is there a way to have a table that is aware of it's fields without codegen?
Is there a way for me to allow jooq to update rows this way.
My preferences is to steer clear of codegen, unless it's really needed. I probably could switch to codegen if needed, but again I would still need to be able to execute SQL without writing specific code for each table. Using JSON is still very much desired, as that allows me to send data from one application to another for import.
Using code generation
You've run into one of those many reasons why code generation is very helpful with jOOQ. If your various tables are known at compile time, and all you're doing is switch table names, then I would go with generated code, making the lookup of the table dynamic. That would solve the problem easily.
From experience with various similar support cases, I've always recommended this first, because as soon as these kinds of troubles start, it's a good idea to re-think the code generation strategy as you will run into other, similar problems, having to work around the lack of ubiquitously available meta data all the time. There are many other benefits to using the code generator.
Emulating code generation
If for some reason you cannot (e.g. the tables aren't known at compile time) or do not want to use the code generator, then you can do the code generator's work yourself at runtime, by building CustomTable types as documented here.
Using other means of providing meta information
Another way to provide jOOQ with meta data is to use one of various forms of implementing org.jooq.Meta, which include:
Looking up meta data from the JDBC driver's DatabaseMetaData (this can be slow, depending on your schema)
Letting jOOQ interpret some DDL scripts
Using jOOQ's XML representation of the standard SQL INFORMATION_SCHEMA
Using generated code
I'm using jOOQ to generate queries to run against Athena (AKA PrestoDB/Trino)
To do this, I am using SQLDialects.DEFAULT, and it works because I use very basic query functionalities.
However, jOOQ renders queries like this:
select *
from "Artist"
limit 10
offset 10
God knows why, but the order of limit and offset seem to matter, and the query only works if written with the order swapped:
select *
from "Artist"
offset 10
limit 10
Is there a class I can subclass, to modify the statement render function so that the order of these are swapped? Or any other way of implementing this myself?
A generic solution in jOOQ
There isn't a simple way to change something as fundamental as the SELECT clause order (or any other SELECT clause syntax) so easily in jOOQ, simply, because this was never a requirement for core jOOQ usage, other than supporting fringe SQL dialects. Since the support of a SQL dialect is a lot of work in jOOQ (with all the integration tests, edge cases, etc.) and since market shares of those dialects are low, it has simply never been a priority to improve this in jOOQ.
You may be tempted to think that this is "just" about the order of keywords in this one case. "Only this one case." It never is. It never stops, and the subtle differences in dialects never end. Just look at the jOOQ code base to get an idea of how weirdly different vendors choose to make their dialects. In this particular case, one would think that it seems extremely obvious and simple to make this clause MySQL / PostgreSQL / SQLite compatible, so your best chance is to make a case with the vendor for a feature request. It should be in their own best interest to be more compatible with the market leaders, to facilitate migration.
Workarounds in jOOQ
You can, of course, patch your generated SQL on a low level, e.g. using an ExecuteListener and a simple regex. Whenever you encounter limit (\d+|\?) offset (\d+|\?), just swap the values (and bind values!). This might work reasonably well for top level selects. It's obviously harder if you're using LIMIT .. OFFSET in nested selects, but probably still doable.
Patching jOOQ is always an option. The class you're looking for in jOOQ 3.17 is org.jooq.impl.Limit. It contains all the rendering logic for this clause. If that's your only patch, then it might be possible to upgrade jOOQ. But obviously, patching is a slippery slope, as you may start patching all sorts of clauses, making upgrades impossible.
You can obviously use plain SQL templates for simple cases, e.g. resultQuery("{0} offset {1} limit {2}", actualSelect, val(10), val(10)). This doesn't scale well, but if it's only about 1-2 queries, it might suffice
Using the SQLDialect.DEFAULT
I must warn you, at this point, that the behaviour of SQLDialect.DEFAULT is unspecified. Its main purpose is to produce something when you call QueryPart.toString() on a QueryPart that is not an Attachable, where a better SQLDialect is unavailable. The DEFAULT dialect may change between minor releases (or even patch releases, if there's an important bug in some toString() method), so any implementation you base on this is at risk of breaking with every upgrade.
The most viable long term solution
... would be to have support for these dialects in jOOQ:
#5414 Presto
#11485 Trino
Since JOOQ 3.6+, they no longer ship with SQL 2 jOOQ Parser. Search on the internet, I can't find the tool SQL 2 jOOQ Parser anywhere.
Just wonder is there any similar tool like SQL 2 jOOQ Parser so we can generate the JOOQ code from native sql?
There's a feature request for this:
https://github.com/jOOQ/jOOQ/issues/6277
From the feature request:
This was already implemented in the past by the https://github.com/sqlparser/sql2jooq third party module, but it suffered from several flaws:
It didn't produce very good jOOQ code
It worked only for MySQL and PostgreSQL
It depended on a third party parser (by Gudu Soft), which was proprietary and not under our control
It was hard to use
The product got zero (!) user feedback over quite some time, which is never a good sign.
Eventually, we'll re-iterate the idea, but it's a lot of work, and there are probably more interesting things that can be done first. The approach most people will choose when writing jOOQ queries is they:
Choose a test driven approach where the feedback cycle is tight, such that executing a query to test if it's correct is done relatively easily
Use views (seriously, use views! Why don't people use views more often?) for your very complex static SQL and query the views from jOOQ
I am getting Caused by: java.lang.IllegalArgumentException: Invalid code: UNRECOGNIZED while trying to access a spanner NUMERIC column from Java application using spanner repository.
I am getting the above error .I have tried with Long,float and bigdecimal on the entity.Can anyone share the thoughts on this .
NUMERIC data types in Cloud Spanner should be used with BigDecimal.
Support for the NUMERIC data type was introduced in Cloud Spanner Java client library in version 1.59.0. The most probable reason for this error is that you are using an older version of the client library. Your question mentions that you tried it 'on the entity', which might indicate that you are using Hibernate.
Could you please share more information about your specific situation, which should at least include:
Any relevant frameworks you are using (Hibernate, Spring, ...)
Versions of these frameworks
Version of the Cloud Spanner Java client library
Please also always include:
A snippet of your actual code, or otherwise a small code sample that will actually reproduce the problem.
The entire (anonymized) stacktrace of your error, instead of only the error message.
Numeric will refer to BigDecimal in java. You could always check from Spring Data Spanner framework code that contains all datatype mapping from java to spanner.
SpannerTypeMapper.java
https://github.com/spring-cloud/spring-cloud-gcp/blob/master/spring-cloud-gcp-data-spanner/src/main/java/org/springframework/cloud/gcp/data/spanner/core/convert/SpannerTypeMapper.java
this is because of older version of spanner, numeric type was introduced later, you might be using bigdecimal which is not a part of TYPECODE.class
BigDecimal works perfectly fine with NUMERIC fields.
TYPE_CODE_UNSPECIFIED(0),
BOOL(1),
INT64(2),
FLOAT64(3),
TIMESTAMP(4),
DATE(5),
STRING(6),
BYTES(7),
ARRAY(8),
STRUCT(9),
NUMERIC(10)
if your TYPECODE.class has all of these. if not Please update dependency and your issue must be resolved.
I am on jooq queries now...I feel the SQL queries looks more readable and maintainable and why we need to use JOOQ instead of using native SQL queries.
Can someone explains few reason for using the same?
Thanks.
Here are the top value propositions that you will never get with native (string based) SQL:
Dynamic SQL is what jOOQ is really really good at. You can compose the most complex queries dynamically based on user input, configuration, etc. and still be sure that the query will run correctly.
An often underestimated effect of dynamic SQL is the fact that you will be able to think of SQL as an algebra, because instead of writing difficult to compose native SQL syntax (with all the keywords, and weird parenthesis rules, etc.), you can think in terms of expression trees, because you're effectively building an expression tree for your queries. Not only will this allow you to implement more sophisticated features, such as SQL transformation for multi tenancy or row level security, but every day things like transforming a set of values into a SQL set operation
Vendor agnosticity. As soon as you have to support more than one SQL dialect, writing SQL manually is close to impossible because of the many subtle differences in dialects. The jOOQ documentation illustrates this e.g. with the LIMIT clause. Once this is a problem you have, you have to use either JPA (much restricted query language: JPQL) or jOOQ (almost no limitations with respect to SQL usage).
Type safety. Now, you will get type safety when you write views and stored procedures as well, but very often, you want to run ad-hoc queries from Java, and there is no guarantee about table names, column names, column data types, or syntax correctness when you do SQL in a string based fashion, e.g. using JDBC or JdbcTemplate, etc. By the way: jOOQ encourages you to use as many views and stored procedures as you want. They fit perfectly in the jOOQ paradigm.
Code generation. Which leads to more type safety. Your database schema becomes part of your client code. Your client code no longer compiles when your queries are incorrect. Imagine someone renaming a column and forgetting to refactor the 20 queries that use it. IDEs only provide some degree of safety when writing the query for the first time, they don't help you when you refactor your schema. With jOOQ, your build fails and you can fix the problem long before you go into production.
Documentation. The generated code also acts as documentation for your schema. Comments on your tables, columns turn into Javadoc, which you can introspect in your client language, without the need for looking them up in the server.
Data type bindings are very easy with jOOQ. Imagine using a library of 100s of stored procedures. Not only will you be able to access them type safely (through code generation), as if they were actual Java code, but you don't have to worry about the tedious and useless activity of binding each single in and out parameter to a type and value.
There are a ton of more advanced features derived from the above, such as:
The availability of a parser and by consequence the possibility of translating SQL.
Schema management tools, such as diffing two schema versions
Basic ActiveRecord support, including some nice things like optimistic locking.
Synthetic SQL features like type safe implicit JOIN
Query By Example.
A nice integration in Java streams or reactive streams.
Some more advanced SQL transformations (this is work in progress).
Export and import functionality
Simple JDBC mocking functionality, including a file based database mock.
Diagnostics
And, if you occasionally think something is much simpler to do with plain native SQL, then just:
Use plain native SQL, also in jOOQ
Disclaimer: As I work for the vendor, I'm obviously biased.