I'm using scalaxb with scalaxb-sbt to generate the hl7 FHIR specification classes from the provided XSD files.
I get the following errors:
[error] Could not write class hl7/org/fhir/XMLProtocol$DefaultHl7orgfhir_BackboneElementableFormat because it exceeds JVM code size limits. Method reads's too large!
[error] Could not write class hl7/org/fhir/XMLProtocol$DefaultHl7orgfhir_ElementableFormat because it exceeds JVM code size limits. Method reads's code too large!
Is there a way to tell scalaxb that methods should be broken up upon reaching this limit?
Related
I'm trying to upload a lot of files to S3. This cannot be done with the standard AWS CLI because of the translation required between file names on disk and object names in S3. Indeed may of the objects don't exist at all on disk.
I keep getting an error:
botocore.exceptions.ClientError: An error occurred (SlowDown) when calling the PutObject operation (reached max retries: 0): Please reduce your request rate
It doesn't seem to make a difference wether I use boto3 / botocore / aioboto3 / aiobotocore. I've tried various configurations of retry logic as described here. Nothing seems to fix the problem. That includes all three retry modes and retry counts ranging everything from 0 to 50.
I could add custom retry logic to every method that calls the client but that's going to be a lot of work and feels like the wrong approach.
Is it possible to customize the retry logic used by boto3 or aiobotocore?
I have unloaded more than 100 CSV files in a folder. When I try to load those files to cassandra using DSBULK load and specifying the the folder location of all these files, I get the below error
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "unVocity-parsers input reading thread"
I wanted to see if anyone else has faced it and how it has been resolved.
Here are a few things you can try:
You can pass any JVM option or system property to the dsbulk executable using the DSBULK_JAVA_OPTS env var. See this page for more. Set the allocated memory to a higher value if possible.
You can throttle dsbulk using the -maxConcurrentQueries option. Start with -maxConcurrentQueries 1; then raise the value to get the best throughput possible without hitting the OOM error. More on this here.
I have a pipeline that reads data from an MS SQL Server and stores them into a file in a BLOB container in Azure Storage. The file has Parquet (or Apache Parquet, as it is also called) format.
So, when the “sink” (output) file is stored in a compressed way (snappy, or gzip – does not matter) AND the file is large enough (more than 50 Mb), the pipeline failed. The message was the following:
"errorCode": "2200",
"message": "Failure happened on 'Sink' side.
ErrorCode=UserErrorJavaInvocationException,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An error occurred when invoking java, message: java.lang.OutOfMemoryError:Java heap space\ntotal entry:11\r\njava.util.ArrayDeque.doubleCapacity(Unknown Source)\r\njava.util.ArrayDeque.addFirst(Unknown Source)\r\njava.util.ArrayDeque.push(Unknown Source)\r\norg.apache.parquet.io.ValidatingRecordConsumer.endField(ValidatingRecordConsumer.java:108)\r\norg.apache.parquet.example.data.GroupWriter.writeGroup(GroupWriter.java:58)\r\norg.apache.parquet.example.data.GroupWriter.write(GroupWriter.java:37)\r\norg.apache.parquet.hadoop.example.GroupWriteSupport.write(GroupWriteSupport.java:87)\r\norg.apache.parquet.hadoop.example.GroupWriteSupport.write(GroupWriteSupport.java:37)\r\norg.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:123)\r\norg.apache.parquet.hadoop.ParquetWriter.write(ParquetWriter.java:292)\r\ncom.microsoft.datatransfer.bridge.parquet.ParquetBatchWriter.addRows(ParquetBatchWriter.java:60)\r\n,Source=Microsoft.DataTransfer.Common,''Type=Microsoft.DataTransfer.Richfile.JniExt.JavaBridgeException,Message=,Source=Microsoft.DataTransfer.Richfile.HiveOrcBridge,'",
"failureType": "UserError",
"target": "Work_Work"
}
The "Work_Work" is the name of a Copy Data activity in the pipeline.
If I turn the compression off (the generated BLOB file is uncompressed), the error does not happen.
Is this the error described in link the
“…If you copy data to/from Parquet format using Self-hosted
Integration Runtime and hit error saying "An error occurred when
invoking java, message: java.lang.OutOfMemoryError:Java heap space",
you can add an environment variable _JAVA_OPTIONS in the machine that
hosts the Self-hosted IR to adjust the min/max heap size for JVM to
empower such copy, then rerun the pipeline….”?
If it is, have I understood correctly that I have to do the following:
To go to a server where the “Self-hosted Integration Runtime” (still have no idea what it is) and increase the max heap size for JVM. Is this correct?
If it is, my next question is: how large the max heap size should be? My pipeline can generate a file whose size will be 30 GB.
What “max heap size” can guarantee that such a file will not cause the fail?
If you copy data to/from Parquet format using Self-hosted Integration Runtime and hit error saying "An error occurred when invoking java, message: java.lang.OutOfMemoryError:Java heap space", you can add an environment variable _JAVA_OPTIONS in the machine that hosts the Self-hosted IR to adjust the min/max heap size for JVM to empower such copy, then rerun the pipeline.
Example: set variable _JAVA_OPTIONS with value -Xms256m -Xmx16g. The flag Xms specifies the initial memory allocation pool for a Java Virtual Machine (JVM), while Xmx specifies the maximum memory allocation pool. This means that JVM will be started with Xms amount of memory and will be able to use a maximum of Xmx amount of memory. By default, ADF use min 64MB and max 1G.
Reference: https://learn.microsoft.com/en-us/azure/data-factory/format-parquet
Hope this helps.
I am trying to do functional testing with JMeter for a Web service. I am trying to build a tractability metrics using JMeter itself for which I need the count of the samplers in a thread, no of passed samplers, failed samplers in a thread.
Is there any way to collect no of the samplers, failed or passed samplers in a thread?
Any help is much appreciated.
Thanks
By adding Aggregate Report listener to your test, you will be able to get the number of passed and failed requests details. Below is the purpose of aggregate report.
The aggregate report creates a table row for each differently named
request in your test. For each request, it totals the response
information and provides request count, min, max, average, error rate,
approximate throughput (request/second) and Kilobytes per second
throughput. Once the test is done, the throughput is the actual
through for the duration of the entire test.
Also, you can leverage View Results Tree, View Results in Table and Debug Sampler to debug your samplers.
To enable functional test mode in JMeter, click on the Test Plan and then check Functional Test Mode check box as shown below.
While executing one Proc, I am geting a 'GDG Roll In Error'. The Error Message says 'IGD07001I GDG ROLL IN ERROR -RETURN CODE 20 REASON CODE 0 MODULE IGG0CLEG'. The proc is supposed to create 19 generations of a GDG. This error occurs after creating first 6 Generatons. The parameters of the GDG are Limit=100, NOEMPTY,SCRATCH. What could be the reason.?
Experts, Please help.
If you look up IGD07001I it says, among other things, to look at IDC3009I for an explanation of the return and reason codes. For return code 20 reason code 0, IDC3009I says
RETURN CODE 20 Explanation: There is insufficient space in the
catalog to perform the requested update or addition.
The catalog cannot be extended for one of the following reasons:
There is no more space on the volume on which the catalog resides
The maximum number of extents has been reached
The catalog has reached the 4GB limit
There is not enough contiguous space on the volume (required when the catalog's secondary allocation is defined in tracks).
Programmer Response: Scratch unneeded data sets from the volume.
Delete all unnecessary entries from the catalog. The catalog may need
to be reallocated and rebuilt if these steps do not resolve the space
shortage.
I suggest contacting your DFSMS Administrator. I also suggest bookmarking the z/OS documentation for your version/release.