I'm having an issue with a roxie query when being called via an API. I've tested the query in HThor and the web service on 8002 and it runs perfectly. When calling it from an API I'm getting memory pool exhausted. The only clue I have in the logs is
Pool memory exhausted: pool id 0 exhausted, requested 1 heap(4095/4294967295) global(4096/4096) WM(1..128) (in Index Read 40)"
How can I find out which index read this is referring to?
Thanks
David
The index read is indicated in the brackets - it is activity 40 in the graph.
The error message is indicating that all of the memory has been exhausted (4096 pages of 256K each). It is likely to be an index read that has a poor filter condition, returns a large number of rows and is feeding into an activity that needs all the input rows to process it (e.g. a sort).
Related
I have a KLOG statement used to extract some data from Azure Log Analytics. My problem is related to the fact that Azure Log Analytics seems to truncate the SQL statements longer than 4000 characters. For the audited server, I have more queries written by the users longer than 4000 characters. Can I increase the size of the column "Statement" somehow?
Thank you
Can I increase the size of the column "Statement" somehow?
Azure has a limitation of collecting log messages size.
If you want to add a custom property message with large no of length you can use the trackTrace. It has the capacity to have max length up to 8192 characters.
# it will add more than 4000 character in AI Logs
telemetryClient.TrackTrace(<telemetry with more than 4000 character >);
The highest max allowed message limit is 32768 characters for items in the property collection have limit of 8192. (Max key length 150 value length 8192 characters)
Refer MS-DOC for length limits of Application Insights Data model with respective to the type of telemetry.
Reference
Follow the Steps given by #cijothomas to add large length of message (more than 8K) to application Insights
I have a Spring CM folder that has 1000s of small files in it. I'm doing retrieval this way:
GET /v201411/folders/{id}/documents
but when it executes, I get back only 20 files. The sum of all of their sizes is: 1.8 MB and the Content Length of the response -> content -> headers is only 3.8 MB.
I didn't find anything in their documentations that mentions the limit of retrieving documents via the api.
Is that really the limitation of spring CM?
From the documentation on API Collections:
Limit (integer): The maximum number of elements retrieved per request.
Default limit is 20. Maximum limit is 100
When there are more items in the collection than the specified limit,
the application can page through the collection, retrieving the
objects in chunks by specifying the limit and/or offset on the query
string when the collection is requested. The first, previous, next,
and last properties are added as a convenience by appending the
appropriate limit and offset to the URI and a GET request can be done
to this URIs specified by these properties to navigate the collection.
To minimize the number of sequential calls you need to make, you can adjust the limit property up to the max, 100.
TableStorage & Nodejs
Using the function "queryEntities" sometimes result.entries.length is 0, even when I am pretty sure there are a lot of entries in the database. The "where" parameters are ok, but sometimes (maybe one every 100) it returns 0 entries. Not error returned. Just 0 entries.
And in my function that's causing troubles.
My theory is that the database sometimes is saturated because this function executes every 10 seconds and maybe sometimes before one finish another one starts and both operate over the same table, and instead of error it returns a length 0 , what is something awful.
There is any way to resolve this? Shouldn't it return error?
This is expected behavior. In this particular scenario, please check for the presence of continuation tokens in the response. Presence of these tokens in the response indicate that there may be entities available matching the query and you should execute the same query again with the continuation token you received.
Please read this document for explanation: https://learn.microsoft.com/en-us/rest/api/storageservices/query-timeout-and-pagination.
From this link:
A query against the Table service may return a maximum of 1,000 items
at one time and may execute for a maximum of five seconds. If the
result set contains more than 1,000 items, if the query did not
complete within five seconds, or if the query crosses the partition
boundary, the response includes headers which provide the developer
with continuation tokens to use in order to resume the query at the
next item in the result set.
Couchdb keeps crashing whenever I try to build the index of the views of a design document emitting values for large documents. The total size of the database is 40 MB and I guess the documents are about 5 MB each. We're talking about large JSON without any attachment.
What concerns me is that I have 2.5 GB of free ram before trying to access the views but as soon as I try to access them, the CPU usage raises to 99% and all the free RAM gets eaten by erl.exe before the indexing fails with exit code 1.
Here is the log:
[info] 2016-11-22T22:07:52.263000Z couchdb#localhost <0.212.0> -------- couch_proc_manager <0.15603.334> died normal
[error] 2016-11-22T22:07:52.264000Z couchdb#localhost <0.15409.334> b9855eea74 rexi_server throw:{os_process_error,{exit_status,1}} [{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,56}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
Views skipping these documents can be accessed without issue. Which general guidelines could you provide me to help with this kind of situation? I am using couchdb 2.0 on windows.
Many thanks
Update : I tried to limit the number of view server instances to 1 and vary the max RAM allowed for couchjs, but it keeps crashing. Also I noticed that even though CouchDb is supposed to pass only one document at a time to the view server, erl.exe keeps eating all the available RAM (3GB used for three 5mb docs to update...). Initially I thought this could be because of the multiple couchjs instances but apparently this isn't the case.
Update : Made some progress, now it looks like the indexing is progressing well for just less than 10 minutes then erl.exe crashes. I have posted the dump here (just to clarify "well" means, 99% CPU usage and computer screen completely frozen).
While executing one Proc, I am geting a 'GDG Roll In Error'. The Error Message says 'IGD07001I GDG ROLL IN ERROR -RETURN CODE 20 REASON CODE 0 MODULE IGG0CLEG'. The proc is supposed to create 19 generations of a GDG. This error occurs after creating first 6 Generatons. The parameters of the GDG are Limit=100, NOEMPTY,SCRATCH. What could be the reason.?
Experts, Please help.
If you look up IGD07001I it says, among other things, to look at IDC3009I for an explanation of the return and reason codes. For return code 20 reason code 0, IDC3009I says
RETURN CODE 20 Explanation: There is insufficient space in the
catalog to perform the requested update or addition.
The catalog cannot be extended for one of the following reasons:
There is no more space on the volume on which the catalog resides
The maximum number of extents has been reached
The catalog has reached the 4GB limit
There is not enough contiguous space on the volume (required when the catalog's secondary allocation is defined in tracks).
Programmer Response: Scratch unneeded data sets from the volume.
Delete all unnecessary entries from the catalog. The catalog may need
to be reallocated and rebuilt if these steps do not resolve the space
shortage.
I suggest contacting your DFSMS Administrator. I also suggest bookmarking the z/OS documentation for your version/release.