The JOOQ Java code generation tools uses regular expressions defined in the includes and excludes elements to control what is generated. I cant find an explanation of what the schema structure is that these expressions are run against.
I want to have the ability to exclude specific databases in the server as well as tables by prefix or specifically.
Simple examples:
Given a SQL server with two DBs 'A' and 'B', how do I instruct JOOQ to only generate for tables in DB 'A'?
How do in instruct JOOQ to only generate for tables starting with the prefix "qtbl"?
It would be great if there were some example use cases available showing some simple common configurations.
The jOOQ manual section about includes and excludes, as well as a few other sections that explain the code generator's usage of regular expressions to match identifier establishes that the code generator will always try to:
Match fully qualified identifiers
Match unqualified identifiers
Or, if you're using jOOQ 3.12+ and did not turn off <regexMatchesPartialQualification/>:
Match partially qualified identifiers (see #7947)
For example:
<excludes>
(?i: # Using case insensitive regex for the example
database_prefix.*?\. # Match a catalog prefix prior to the qualifying "."
.*?\. # You don't seem to care about schema names, so match them all
table_prefix.*? # Match a table prefix at the end of the identifier
)
</excludes>
In addition to the above, if you want to exclude specific databases ("catalogs") from being generated without pattern matching, you'll get even better results if you specify your <inputCatalog>A</inputCatalog>. See also the manual's section about schema mapping.
Benefits include a much faster code generation, because only that catalog will be searched for objects to generate, prior to excluding them again using regular expressions. So, your configuration could be this:
<!-- Include only database A -->
<inputCatalog>A</inputCatalog>
<!-- Include only tables with this (unqualified) prefix -->
<includes>qtbl.*</includes>
Related
I am working on a function that analyzes data (based on some domain-specific logic) in protobufs. When the function finds an issue, I want to include the path to the offending field, including the indexes for the repeated fields.
For example, given the protobuf below:
proto = ECS(
service=[
Service(),
Service(
capacity_provider_strategy=[
ServiceCapacityProviderStrategyItem(base=1),
ServiceCapacityProviderStrategyItem(base=2),
]
)
]
)
Let's assume that the offending field is field = proto.service[1].capacity_provider_strategy[0].
How would I, given only the field produce ecs.service[1].capacity_provider_strategy[0] in a general way?
Please, note that I am looking for a way to produce the path mentioned above solely based on the supplied field since the logic of producing the error message is de-coupled from the analyzing logic. I realize, that (in the analyzing logic) I could keep track of the indexes of the repeated fields, but this would put more overhead on the analyzing function.
When I applied pattern in JDL, the generated entity classes has #Patternannotation, but the value for that annotation is not the exact pattern which applied in JDL.
For example, if I've defined patterns as pattern('/[^\\s]+.*[^\\s]+/') and in java
it reflects as
#Pattern(regexp = "[^\\\\s]+.*[^\\\\s]+")
If you noticed in java class, there are 4 (slash) which indeed should be 2 only. Because of this functionality is getting failed.
It looks to me like you are trying to use regex control characters in your pattern, which do not need to be doubled up in your JDL: see https://www.jhipster.tech/jdl/entities-fields, especially the part under "Regular Expressions" where it says: "/.../ the pattern is declared inside two slashes... \ anti-slashes needn’t be escaped"
So it's acting correctly. Since you have double-backslants in your JDL, Java is correctly interpreting it with quadruple-backslants. Your solution is just to use single backslants in your JDL, as per the documentation.
I am fairly new to U-SQL and trying to run a U-SQL script in Azure Data Lake Analytics to process a parquet file using the Parquet extractor functionality. I am getting the below error and I don't find a way to get around it.
Error - Change the identifier to use at least one lower case letter. If that is not possible, then escape that identifier (for example: '[ACTIVITY]'), or embed it in a CSHARP() block (e.g CSHARP(ACTIVITY)).
Unfortunately all the different fields generated in the Parquet file are capitalized and I don't want to to escape these identifiers. I have tried if I could wrap the identifier with CSHARP block and it fails as well (E_CSC_USER_RESERVEDKEYWORDASIDENTIFIER: Reserved keyword CSHARP is used as an identifier.) Is there anyway I could extract the parquet file? Thanks for your help!
Code Snippet:
SET ##FeaturePreviews = "EnableParquetUdos:on";
#var1 =
EXTRACT ACTIVITY string,
AUTHOR_NAME string,
AFFLIATION string
FROM "adl://xxx.azuredatalakestore.net/Abstracts/FY2018_028"
USING Extractors.Parquet();
#var2 =
SELECT *
FROM #var1
ORDER BY ACTIVITY ASC
FETCH 5 ROWS;
OUTPUT #var2
TO "adl://xxx.azuredatalakestore.net/Results/AbstractsResults.csv"
USING Outputters.Csv();
Based on your description you try to say
EXTRACT ALLCAPSNAME int FROM "/data.parquet" USING Extractors.Parquet();
In U-SQL, we reserve all caps identifiers so we can add new keywords in the future without invalidating old scripts.
To work around, you just have to quote the name (escape it) like in any other SQL dialect:
EXTRACT [ALLCAPSNAME] int FROM "/data.parquet" USING Extractors.Parquet();
Note that this is not changing the name of the field. It is just the syntactic way to address the field.
Also note, that in most SQL communities, it is considered a best practice to always quote identifiers to avoid reserved keyword clashes.
If all fields in the Parquet file are all caps, you will have to quote them all... In a future update you will be able to say EXTRACT * FROM … for Parquet (and Orc) files, but you still will need to quote the columns when you refer to them explicitly.
I am processing http logs and converting querystring parameters to fields.
kv
{
source => "uriQuerystring"
field_split => "&"
target => "uriQuerystringKeys"
}
However because callers are using mixed case parameters, I end up with numerous duplicates.
eg: uriQuerystringKeys.apiKey, uriQuerystringKeys.ApiKey, uriQuerystringKeys.APIKey
What do I need to do in my logstash configuration to convert all these field names to lowercase?
I see there's an open issue for this feature to be implemented in Logstash, but it's incomplete. There's a suggestion for some ruby code to be directly executed, but it looks like this converts all fields (not just ones of a certain prefix).
Here's a prior answer that contains the basic code you would need.
You can see a conditional inside the loop, which you could use to enforce the prefix limitations on the fields.
I see that in the cobertura-maven-plugin, I can configure my instrumentation with both excludes and ignores. What is the difference between these two?
Really sorry, but parroting the horrible documentation is not helpful. You quoted:
You can tell Cobertura to ignore certain classes by passing in "ignore" regular expressions. The ignore pattern can be any valid perl 5 regular expression. This will ignore any calls to any method that matches the ignore regular expression. It will NOT skip over these classes during instrumention. To exclude classes from being instrumented, either exclude them from your fileset or use the alternative method below and specify an excludeClasses pattern.
This inconsistent paragraph starts by stating "...to ignore certain classes by passing in 'ignore' regular expressions.", then a second sentence later, "This will ignore any calls to any method that matches the ignore regular expression."
Well, so which is it? Does "ignore" ignore classes or methods? The command-line reference seems to imply ignore is ignoring a line of code based on RegEx matching (I wish that was true):
None of these interpretations works for <ignore> in the maven plugin.
Experimentally I have found that <exclude> is the way to exclude at the class-level. I have not found a way to exclude at a granularity less then class. However, what they also don't tell you is that <exclude> expects a relative filesystem path, not a Java package.class expression, which means <exclude> expects e.g. one/two/three/Myclass.class, rather then "one.two.three.Myclass".
Ignore and exclude really function nothing alike. Exclude will remove a file from Cobertura's scan. Ignore will remove all usages of a class from contributing to line coverage.
Consider the following code snippet:
public class Foo {
public void someMethod() {
Bar bar = new Bar();
}
}
If you configure <exclude>Foo.class</exclude>, then the entirety of the Foo class will not show up on your reporting.
Semi-confusingly, if you configure <ignore>Foo.class</ignore>, then Foo will still show up on your reporting, but uses of Foo in other classes will be ignored.
If you configure <ignore>Bar.class</ignore>, then the scan of Foo will ignore the line in which Bar is initialized.
To really make that stick, let's talk about why you would use ignore. Let's say that there is an object (such as a Logger) that you do not want to show up in the code coverage scan of any classes. Ignore allows you to do this.
The maven plugin does not contain any documentation for this configuration options (but they are included in the docs). The ant cobertura doc contains the following documentation:
You can tell Cobertura to ignore certain classes by passing in "ignore" regular expressions. The ignore pattern can be any valid perl 5 regular expression. This will ignore any calls to any method that matches the ignore regular expression. It will NOT skip over these classes during instrumention. To exclude classes from being instrumented, either exclude them from your fileset or use the alternative method below and specify an excludeClasses pattern.
See also: https://github.com/cobertura/cobertura/wiki/Ant-Task-Reference#cobertura-instrument-task