I'm investigating an issue where we are seeing strange exceptions related to jooq trying to populate a generated Record class, where it gets data type errors because it uses java.sql.ResultSet::getXXX(int) (column index based) to fetch data.
The part of the stacktrace that I can share looks like:
Caused by: java.sql.SQLDataException: Value 'ABC' is outside of valid range for type java.lang.Byte
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:114)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:73)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:92)
at com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1423)
at com.mysql.cj.jdbc.result.ResultSetImpl.getByte(ResultSetImpl.java:710)
at com.zaxxer.hikari.pool.HikariProxyResultSet.getByte(HikariProxyResultSet.java)
at org.jooq.tools.jdbc.DefaultResultSet.getByte(DefaultResultSet.java:124)
at org.jooq.tools.jdbc.DefaultResultSet.getByte(DefaultResultSet.java:124)
at org.jooq.impl.CursorImpl$CursorResultSet.getByte(CursorImpl.java:688)
at org.jooq.impl.DefaultBinding$DefaultByteBinding.get0(DefaultBinding.java:1783)
at org.jooq.impl.DefaultBinding$DefaultByteBinding.get0(DefaultBinding.java:1755)
at org.jooq.impl.DefaultBinding$AbstractBinding.get(DefaultBinding.java:871)
at org.jooq.impl.CursorImpl$CursorIterator$CursorRecordInitialiser.setValue(CursorImpl.java:1725)
Which is definitely a column mismatch caused using wrong column index.
The issue comes up because we are using the record on an evolving schema so the underlying table contains columns not available in the record definition.
Note that the actual code that triggers this is:
jooq.insertInto(TABLE)
.set(TABLE.COL, "ABC")
.returning(TABLE.asterisk())
.fetchOne();
What scares me a bit here is, that if it does indeed use column indexes by design, that will make schema evolution somewhat hard (how would you delete a column from a running application).
Long story (sorry), question is: does jooq use column index in records generated by jooq-generator and is there a way to use column names instead?
One thing I have noticed is that when I compare the documentation at https://www.jooq.org/doc/3.14/manual/code-generation/codegen-records/ the shown generated records does not match what the generator actually generates. The documentation show methods like:
// Every column generates a setter and a getter
#Override
public void setId(Integer value) {
setValue(BOOK.ID, value);
}
But in reality the generated code looks like (taken from jOOQ-examples):
/**
* Setter for <code>PUBLIC.BOOK.ID</code>.
*/
public void setId(Integer value) {
set(0, value);
}
Btw we are using jooq 3.14.15.
Ok, this was a local error. What actually caused the issue was that our code was written as:
jooq.insertInto(TABLE)
.set(TABLE.COL, "ABC")
.returning(TABLE.asterisk())
.fetchOne();
And the TABLE.asterisk() is what messes it up (since on the database that contains extra columns it does not return what jooq expects). Fortunately removing it solves the problem so our code now looks like:
jooq.insertInto(TABLE)
.set(TABLE.COL, "ABC")
.returning()
.fetchOne();
Related
When you start to use gpath processing for collections in groovy, exactly what type of data structure you are dealing with can get somewhat confusing (at least for me)--typically in groovy you just use "Duck typing" and everything works, but Duck Typing actually harms the coders understanding of what a variable might contain at any point in time.
In order to keep it straight I started breaking out variables as types to improve understanding. The types however are very cryptic and unhelpful:
Map<String, List<Map<String, String>>>
is actually a pretty common (and immensely useful) intermediate structure, but someone looking at that in code isn't really helped. (It acts much like an indexed/grouped sql table if it's not obvious, which I bet it's not)
While explaining my code to a co-worker (and trying to understand it myself) I created a few new types that more clearly state my intentions:
class Row extends LinkedHashMap<String, String>{} // Record--like a row in a table
class Table extends ArrayList<Row>{} // A table--like an SQL table. A list of rows
class GroupedTable extends LinkedHashMap<String, Table> // A map of tables indexed by some name using groupBy()
This works surprisingly well. After defining these I could say things like:
Table t=sql.rows(someQuery) as Table
Row r=t.get(0)
GroupedTable grouped= t.groupBy{it.an_sql_column} as GroupedTable
Sometimes I don't have to manually coerce the values (as in the Row) sometimes I do (as in the GroupedTable) which is a little confusing but it generally works (even with #TypeChecked turned on)
The biggest confusion I'm having is that even though grouped was "Converted" to GroupedTable, this closure signature doesn't work:
grouped.collect{String modelName, Table subTable->…}
It will break at runtime because even though I converted the grouped object using "as GroupedTable", it didn't convert the values returned from groupBy from LinkedLists to a tables.
It works if I use the closure signature:
grouped.collect{String modelName, LinkedList subTable->…}
but everywhere else things seem to automatically convert nicely. I tried to specify ", subTable as Table" but that is not valid syntax for a closure.
So I'm wondering if there is a way to either supply an "Automatic" conversion from LinkedList to table or to easily process a GroupedTable so that it contains Tables for it's values instead of LinkedLists
A small set of automatic conversions that made Row, Table and GroupedTable work more smoothly would be my ideal solution, but I don't think that asType() is going to do it since I'd need to apply that to LinkedList instead of Table (and it would probably require explicit "as" conversion anyway.)
Update--I just made a "fix" method in GroupedTable that seems to fix the problem but is clunky!
def fix(){keyset().each{put(it, get(it) as Table)}}
I don't know why auto-type conversion does not work
however, you could #Override the groupBymethod like this:
(Edit by billk--this is how I implemented Daggett's suggestion)
class Table extends ArrayList<Row> {
GroupedTable groupBy(Closure closure)
def gt=DefaultGroovyMethods.groupBy(this as Iterable, closure) as GroupedTable
gt.changeTypeOfAllValuesToTable()
return gt
static Table build(Iterable list){
return list.collect{it as Row} as Table
}
}
class GroupedTable extends LinkedHashMap<String, Table> {
GroupedTable changeTypeOfAllValuesToTable()
this.keySet().each{String k->
Table t=Table.build(get(k))
put(k, t)
return this
}
With a little extra effort (I could post it if anyone is interested) I got these all to work with the #TypeChecked annotation. I will probably clean these up and make more useful classes out of them.
I am attempting to use the Roslyn SDK and StackExchange.Precompilation (thank you!) to implement aspect-oriented programming in C#6. My specific problem right now is, starting with an IdentifierNameSyntax instance, I want to find the "member type" (method, property, field, var, etc.) that the identifier refers to. (How) can this be done?
Background:
The first proof-of-concept I am working on is some good old design-by-contract. I have a NonNullAttribute which can be applied to parameters, properties, or method return values. Along with the attribute there is a class implementing the StackExchange.Precompilation.ICompileModule interface, which on compilation will insert null checks on the marked parameters or return values.
This is the same idea as PostSharp's NonNullAttribute, but the transformation is being done on one of Roslyn's syntax trees, not on an already compiled assembly. It is also similar to Code Contracts, but with a declarative attribute approach, and again operating on syntax trees not IL.
For example, this source code:
[return: NonNull]
public string Capitalize([NonNull] string text) {
return text.ToUpper();
}
will be transformed into this during precompilation:
[return: NonNull]
public string Capitalize([NonNull] string text) {
if (Object.Equals(text, null))
throw new ArgumentNullException(nameof(text));
var result = text.ToUpper();
if (Object.Equals(result, null))
throw new PostconditionFailedException("Result cannot be null.");
return result;
}
(PostconditionFailedException is a custom exception I made to compliment ArgumentException for return values. If there is already something like this in the framework please let me know.)
For properties with this attribute, there would be a similar transformation, but with preconditions and postconditions implemented separately in the set and get accessors, respectively.
The specific reason I need to find the "member type" of an identifier here is for an optimization on implementing postconditions. Note in the post-compilation sample above, the value that would have been returned is stored in a local variable, checked, and then the local is returned. This storage is necessary for transforming return statements that evaluate a method or complex expression, but if the returned expression is just a field or local variable reference, creating that temporary storage local is wasteful.
So, when the return statement is being scanned, I first check if the statement is of the form ReturnKeyword-IdentifierSyntaxToken-SemicolonToken. If so, I then need to check what that identifier refers to, so I avoid that local variable allocation if the referent is a field or var.
Update
For more context, check out the project this is in reference to on GitHub.
You'll need to use SemanticModel.GetSymbolInfo to determine the symbol an identifier binds to.
Use SemanticModel.GetTypeInfo.Type to obtain the TypeInfo and use it to explore the Type
In my plugin code, I use early bound entities (generated via the crmsvcutil). Within my code, I am using MemberExpression to retrieve the name of the property. For instance, if I want the full name of the user who initiated the plugin I do the following
SystemUser pluginExecutedBy = new SystemUser();
pluginExecutedBy = Common.RetrieveEntity(service
, SystemUser.EntityLogicalName
, new ColumnSet(new string[] {Common.GetPropertyName(() => pluginExecutedBy.FullName)})
, localContext.PluginExecutionContext.InitiatingUserId).ToEntity<SystemUser>();
The code for GetPropertyName is as follows
public static string GetPropertyName<T>(Expression<Func<T>> expression)
{
MemberExpression body = (MemberExpression)expression.Body;
return body.Member.Name.ToLower();
}
The code for RetrieveEntity is as follows
public static Entity RetrieveEntity(IOrganizationService xrmService, string entityName, ColumnSet columns, Guid entityId)
{
return (Entity)xrmService.Retrieve(entityName, entityId, columns);
}
My solution architect's comments:
Instead of writing the code like above, why not write it like this (hardcoding the name of the field - or use a struct).
SystemUser pluginExecutedBy = null;
pluginExecutedBy = Common.RetrieveEntity(service
, SystemUser.EntityLogicalName
, new ColumnSet(new string[] {"fullname"})
, localContext.PluginExecutionContext.InitiatingUserId).ToEntity<SystemUser>();
Reason:
Your code unnecessarily creates an object before it requires it (as you instantiate the object with the new keyword before the RetrieveEntity in order to use it with my GetProperty method) which is bad programming practice. In my code, I have never used the new keyword, but merely casting it and casting does not create a new object. Now, I am no expert in C# or .NET, but I like to read and try out different things. So, I looked up the Microsoft.Xrm.Sdk.dll and found that ToEntity within Sdk, actually did create a new Entity using the keyword new.
If the Common.Retrieve returns null, your code has unnecessarily allocated memory which will cause performance issues whereas mine would not?
A managed language like C# "manages the memory" for me, does it not?
Question
Is my code badly written? If so, why? If it is better - why is it? (I believe it is a lot more cleaner and even if a field name changes as long as as the early bound class file is regenerated, I do not have to re-write any code)
I agree that cast does not create a new object, but does my code unnecessarily create objects?
Is there a better way (a completely different third way) to write the code?
Note: I suggested using the GetPropertyName because, he was hard-coding attribute names all over his code and so in a different project which did not use early bound entities I used structs for attribute names - something like below. I did this 3 weeks into my new job with CRM 2011 but later on discovered the magic of MemberExpression. He was writing a massive cs file for each of the entity that he was using in his plugin and I told him he did not have to do any of this as he could just use my GetPropertyName method in his plugin and get all the fields required and that prompted this code review comments. Normally he does not do a code review.
public class ClientName
{
public struct EntityNameA
{
public const string LogicalName = "new_EntityNameA";
public struct Attributes
{
public const string Name = "new_name";
public const string Status = "new_status";
}
}
}
PS: Or is the question / time spent analyzing just not worth it?
Early Bound, Late Bound, MemberExpression, bla bla bla :)
I can understand the "philosophy", but looking at your code a giant alarm popup in my head:
public static Entity RetrieveEntity(IOrganizationService xrmService, string entityName, ColumnSet columns, Guid entityId)
{
return (Entity)xrmService.Retrieve(entityName, entityId, columns);
}
the Retrieve throws an exception if the record is not found.
About the other things, the GetPropertyName is ok, but are always choices, for example I try to use always late bound in plugins, maybe in a project I prefer to use early bound, often there is more than one way to resolve a problem.
Happy crm coding!
Although GetPropertyName is a quite a clever solution I don't like it, and that's entirely to do with readability. To me its far easier to understand what is going on with: new ColumnSet(new string[] {"fullname"}).
But that's pretty much personal preference, but its important to remember that your not just writing code for yourself you are writing it for your team, they should be able to easily understand the work you have produced.
As a side a hardcoded string probably performs better at runtime. I usually hardcode all my values, if the entity model in CRM changes I will have to revisit to make changes in any case. There's no difference between early and late bound in that situation.
I don't understand the point of this function,
public static Entity RetrieveEntity(IOrganizationService xrmService, string entityName, ColumnSet columns, Guid entityId)
{
return (Entity)xrmService.Retrieve(entityName, entityId, columns);
}
It doesn't do anything (apart from cast something that is already of that type).
1.Your code unnecessarily creates an object before it requires it (as you instantiate the object with the new keyword before the
RetrieveEntity in order to use it with my GetProperty method) which is
bad programming practice. In my code, I have never used the new
keyword, but merely casting it and casting does not create a new
object.
I believe this refers to; SystemUser pluginExecutedBy = new SystemUser(); I can see his/her point here, in this case new SystemUser() doesn't do much, but if the object you were instantiating did something resource intensive (load files, open DB connections) you might be doing something 'wasteful'. In this case I would be surprised if changing SystemUser pluginExecutedBy = null; actually yielded any significant performance gain.
2.If the Common.Retrieve returns null, your code has unnecessarily allocated memory which will cause performance issues
I would be surprised if that caused a performance issue, and anyway as Guido points out that function wont return null in any case.
Overall there is little about this code I strongly feel needs changing - but things can be always be better and its worth discussing (e.g. the point of code review), although it can be hard not to you shouldn't be precious about your code.
Personally I would go with hardcoded attribute names and dump the Common.RetrieveEntity function as it doesn't do anything.
pluginExecutedBy = service.Retrieve(SystemUser.EntityLogicalName, localContext.PluginExecutionContext.InitiatingUserId, new ColumnSet(new String[] {"fullname"} ));
After spending a year working with the Microsoft.Xrm.Sdk namespace, I just discovered yesterday the Entity.FormattedValues property contains the text value for Entity specific (ie Local) Option Set texts.
The reason I didn't discover it before, is there is no early bound method of getting the value. i.e. entity.new_myOptionSet is of type OptionSetValue which only contains the int value. You have to call entity.FormattedValues["new_myoptionset"] to get the string text value of the OptionSetValue.
Therefore, I'd like to get the crmsrvcutil to auto-generate a text property for local option sets. i.e. Along with Entity.new_myOptionSet being generated as it currently does, Entity.new_myOptionSetText would be generated as well.
I've looked into the Microsoft.Crm.Services.Utility.ICodeGenerationService, but that looks like it is mostly for specifying what CodeGenerationType something should be...
Is there a way supported way using CrmServiceUtil to add these properties, or am I better off writing a custom app that I can run that can generate these properties as a partial class to the auto-generated ones?
Edit - Example of the code that I would like to be generated
Currently, whenever I need to access the text value of a OptionSetValue, I use this code:
var textValue = OptionSetCache.GetText(service, entity, e => e.New_MyOptionSet);
The option set cache will use the entity.LogicalName, and the property expression to determine the name of the option set that I'm asking for. It will then query the SDK using the RetrieveAttriubteRequest, to get a list of the option set int and text values, which it then caches so it doesn't have to hit CRM again. It then looks up the int value of the New_MyOptionSet of the entity and cross references it with the cached list, to get the text value of the OptionSet.
Instead of doing all of that, I can just do this (assuming that the entity has been retrieved from the server, and not just populated client side):
var textValue = entity.FormattedValues["new_myoptionset"];
but the "new_myoptionset" is no longer early bound. I would like the early bound entity classes that gets generated to also generate an extra "Text" property for OptionSetValue properties that calls the above line, so my entity would have this added to it:
public string New_MyOptionSetText {
return this.GetFormattedAttributeValue("new_myoptionset"); // this is a protected method on the Entity class itself...
}
Could you utilize the CrmServiceUtil extension that will generate enums for your OptionSets and then add your new_myOptionSetText property to a partial class that compares the int value to the enums and returns the enum string
Again, I think specifically for this case, getting CrmSvcUtil.exe to generate the code you want is a great idea, but more generally, you can access the property name via reflection using an approach similar to the accepted answer # workarounds for nameof() operator in C#: typesafe databinding.
var textValue = entity.FormattedValues["new_myoptionset"];
// becomes
var textValue = entity.FormattedValues
[
// renamed the class from Nameof to NameOf
NameOf(Xrm.MyEntity).Property(x => x.new_MyOptionSet).ToLower()
];
The latest version of the CRM Early Bound Generator includes a Fields struct that that contains the field names. This allows accessing the FormattedValues to be as simple as this:
var textValue = entity.FormattedValues[MyEntity.Fields.new_MyOptionSet];
You could create a new property via an interface for the CrmSvcUtil, but that's a lot of work for a fairly simple call, and I don't think it justifies creating additional properties.
I need to query a table using FreeTextTable (because I need ranking), with SubSonic. AFAIK, Subsonic doesn't support FullText, so I ended up creating a simple UDF function (Table Function) which takes 2 params (keywords to search and max number of results).
Now, how can I inner join the main table with this FreeTextTable?
InlineQuery is not an option.
Example:
table ARTICLE with fields Id, ArticleName, Author, ArticleStatus.
The search can be done by one of more of the following fields: ArticleName (fulltext), Author (another FullText but with different search keywords), ArticleStatus (an int).
Actually the query is far more complex and has other joins (depending on user choice).
If SubSonic cannot handle this situation, probably the best solution is good old plain sql (so there would be no need to create an UDF, too).
Thanks for your help
ps: will SubSonic 3.0 handle this situation?
3.0 can do this for you but you'd need to make a template for it since we don't handle functions (yet) out of the box. I'll be working on this in the coming weeks - for now I don't think 2.2 will do this for you.
I realize your question is more complex than this, but you can get results from a table valued function via SubSonic 2.2 with a little massaging.
Copy the .cs file from one of your generated views into a safe folder, and then change all the properties to match the columns returned by your UDF.
Then, on your collection, add a constructor method with your parameters and have it execute an InlineQuery.
public partial class UDFSearchCollection
{
public UDFSearchCollection(){}
public UDFSearchCollection(string keyword, int maxResults)
{
UDFSearchCollection coll = new InlineQuery().ExecuteAsCollection<UDFSearchCollection>("select resultID, resultColumn from dbo.udfSearch(#keyword, #maxResults)",keyword,maxResults);
coll.CopyTo(this);
coll = null;
}
}
public partial class UDFSearch : ReadOnlyRecord<UDFSearch>, IReadOnlyRecord
{
//all the methods for read only record go here
...
}
An inner join would be a little more difficult because the table object doesn't have it's own parameters collection. But it could...