We have a application with a render engine developed in Direct3d/C++. Recently we have come across a crash( access violation) involving ID3DXConstantTable SetFloat/SetVector and shows inside D3dx9_42.dll when we attached a debugger in release binaries with PDBs. One of the ways this crash vanishes when we reduce the number of D3dPOOL Rendertarget textures which are used but from estimating the GPU memory load its no where close to even half of the total available as we are using 3GB NVIDIA cards.
Suspected it to be some heap corruptions due to memory overwrites we went about code checking and following that we used the Application Verifier along with a debugger to root out of memory overwrites which might crash at a later stage of running.. We came across few issues which we ironed out. But still that crash remains at the very first frame render ID3DXConstantTable SetFloat/SetVector . More info :This is 32 bit application running with LARGEADDRESSAWARE flag. Any pointers ?
Well a moment later only i found out the issue I executed the application with the registry switch MEM_TOP_DOWN(AllocationPreference=0x100000) and it instantly crashed at the first setfloat() location.Then goto to know the constant table had to be retrieved using D3DXGetShaderConstantTableEx() with the D3DXCONSTTABLE_LARGEADDRESSAWARE flag :) Thanks
Related
Context:
I have an Node.js application which memory seems to very high, I don't know if that is memory leak or not, because it is reduced after certain period time, but some times on heavy load it keeps on increasing takes much longer to get reduced.
So going through the articles and couple of videos, i figured that i have to heap snapshot and analyse what is causing the memory leak.
Steps:
I have taken 4 snap shots as of now in my local to reproduce the memory leak.
Snapshot 1: 800MB
Snapshot 2: 1400MB
Snapshot 3: 1600MB
Snapshot 4: 2000+MB
When i uploaded the heapdump files to chrome dev tools I see there a lot of information but i don't know how to proceed from there.
Please check below screenshot, it says there is constructor [array] which has 687206 as shallow Size & Retained Size is 721414 in the columns, so when expanded that constructor i can see there are 4097716 constructors created ( refer the second screenshot attached below ).
Question
What does internal array [] means ? Why is there 4097716 created ?
How can a filter out the constructor which created by my app and showing me that instead of some system/v8 engine constructor ?
In the same screenshot one of the constructor uses global variable called tenantRequire function, this is custom global function which is being used internally in some places instead of normal Node.js require, I see the variable across all the constructor like "Array", "Object". This is that global tenantRequire code for reference. It is just patched require function with trycatch. Is this causing the memory leak somehow ?
Refer screenshot 3, [string] constructor it has 270303848 as shallow size. When i expanded it shows modules loaded by Node.js. Question why is this taking that much size ? & Why is my lodash modules are repeated in that string constructor ?
Without knowing much about your app and the actions that cause the high memory usage, it's hard to tell what could be the issue. Which tool did you use to record the heap snapshot? What is the sequence of operations you did when you recorded the snapshot? Could you add this information to your question?
A couple of remarks
You tagged the question with node.js and showed Chrome DevTools. That's ok. You can totally take a heap snapshot of a Node.js application and analyze it in Chrome DevTools. But since both Node.js and Chrome use the same JS engine (V8) and the same garbage collector (Orinoco), it might be a bit confusing for someone who reads the question. Just to make sure I understand it correctly: the issue is in a Node.js app, not in a browser app. And you are using Chrome just to analyze the heap snapshot. Right?
Also, you wrote that you took the snapshots to reproduce the memory leak. That's not correct. You performed some action which you thought would cause a high memory usage, recorded a heap snapshot, and later loaded the snapshot in Chrome DevTools to observe the supposed memory leak.
Trace first, profile second
Every time you suspect a performance issue, you should first use tracing to understand which functions in your applications are problematic (i.e. slow, create a lot of objects that have to be garbage-collected, etc).
Then, when you know which functions to focus on, you can profile them.
Try these visual tools
There are a few tools that can help you with tracing/profiling your app. Have a look a FlameScope (a web app) and node-clinic (a suite of tools). There is also Perfetto, but I think it's for Chrome apps, not Node.js apps.
I also highly recommend the V8 blog.
I am using Lazarus 2.0.4 on Windows 10.
Trying to track memory leaks in a quite large project I tried to familiarize myself with the heaptrc and lineinfo units/compiler switches -gh and -gl.
While this gives me some information that there is a leak somewhere, I could not figure out how to pinpoint the exact location. I put together this quite basic leaking code which will leak 4 bytes every time the button is pressed. Upon exit of the program I get this:
While from the trace I see that there is a 4 byte memory leak, I cannot see any useful reference where to look for the leaking code. I see that the code was within the click handler of a TButton, but this isn't very useful, since there may be many Buttons.
Any way to get a reference to the exact location (line 42)?
I am looking after an app built with Node JS that's producing some interesting issues. It was originally running on Node JS v0.3.0 and I've since upgraded to v0.10.12. We're using Node JS to render charts on the server and we've noticed the memory usage keeps climbing chart after chart.
Q1: I've been monitoring the RES column in top for the Node JS process, is this correct or should I be monitoring something else?
I've been setting variables to null to try and reallocate memory back to the system resources (I read this somewhere as a solution) and it makes only a slight difference.
I've pushed the app all the way to 1.5gb and it then ceases to function and the process doesn't appear to die. No error messages which I found odd.
Q2: Is there anything else I can do?
Thanks
Steve
That is a massive jump in versions. You may want to share what code changes you may have made to get it working on latest stable. The api is not the same as back in v0.3, so that may be part of the problem.
If not then the issue you see it more likely from heap fragmentation than from an actual leak. In later v8 versions garbage collection is more liberal with cleanup to improve performance. (see http://code.google.com/p/chromium/issues/detail?id=112386 for some discussion on this)
You may try running the application with --max_old_space_size=32 which will limit the amount of memory v8 can use to around 32MB. Note the docs say "max size of the old generation", so it won't be exactly 32MB. Just around it, for lack of a better technical explanation.
Also you can track the amount of external memory usage with --trace_external_memory. This will allow you to know if external memory (i.e. Buffers) are being retained in your application.
You're note on the application hanging around 1.5GB would tell me you're probably on a 64-bit system. You only mentioned it ceases to function, but didn't note if the CPU is spinning during that time. Also since I don't have example code I'm not sure of what might be causing this to happen.
I'd try running on latest development (v0.11.3 at the time of this writing) and see if the issue is fixed. A lot of performance/memory enhancements are being worked on that may help your issue.
I guess you have somewhere a memory leak (in form of a closure?) that keeps the (not longer used?) diagrams(?) somewhere in memory.
The v8 sometimes needs a bit tweaking when it comes to > 1 GB of memory. Try out --noincremental_marking and/or --max_old_space_size=81920000 (if you have 8 GB available).
Check for more options with node --v8-options and go through the --trace*-parameters to find out what slows down/stops node.
Some days ago I set up my computer and installed a new copy of Windows 8 because of some hardware changes. Among others I changed the video card from Radeon HD 7870 to Nvidia GTX 660.
After setting up Visual Studio 11 again, I downloaded my last OpenGL project from Github and rebuilt the whole project. I ran the application out of Visual Studio and it crashed because of nvoglv32.dll.
Unhandled exception at 0x5D9F74E3 (nvoglv32.dll) in Application.exe: 0xC0000005: Access violation reading location 0x00000000.
In the old environment the application worked as expected. I didn't changed anything of the project or source code. The only difference was the language of the Visual Studio installtion which is English now and was German before. Therefore I created a new project and adopted all settings, but the error remains.
In order to locate the crash, I noticed that all initialization (window, shaders, ...) succeeded and the error is at the draw call glDrawElements() which referrs to the gemoetry pass of my deferred renderer.
After some reseach I found out that nvoglv32.dll is from Nvidia and is about a services called Compatible OpenGL ICD. Does that somehow mean that my application runs in a compatible mode? That sounds like a mode to support older applications and I want mine to run in a regular mode! By the way I installed the lastest stable drivers for my video card.
To be honest, I have no clue how to approach fixing this crash. What could cause it and how to fix it?
Update: I found a post on Geforce Forums about my issue. Although there was no reply, the autor could fix the problem by changing the order of two OpenGL calls.
Hi all,
After poking around with my application source code for a few hours, I found that calling the functions...
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, #)
glBindVertexArray(#)
...in that order causes the crash in nvoglv64.dll.
Reversing the order of these calls to...
glBindVertexArray(#)
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, #)
...prevents the crash and appears to be well-behaved.
Cheers,
Robert Graf
Since I do not use vertex arrays I cannot simple do this fix but there might be a similar issue. I will report my progress.
Update: I have absolutely no clue how to solve my problem. I tried different video driver versions but it makes no difference. I completely rewrote the renderer using minimal shaders and simple forward rendering. But the crash sill occurs at the first draw call.
In order to locate the crash, I noticed that all initialization (window, shaders, ...) succeeded and the error is at the draw call glDrawElements().
Most likely you had a out-of-bounds access in your code all the time, but the AMD Radeon Catalyst drivers did reserve more address space, or caught them beforehand. And now your NVidia GeForce driver's don't.
Either you're passing glDrawElements a too large number for count elements to draw, or your index buffer contains values that index beyond the range of your vertex arrays. If it's the later, then you're probably using client side vertex arrays, as VBOs usually catch out-of-bounds accesses; also those wouldn't crash your client side program, but just render garbage.
Finally I came up with a solution to fix the crash.
The SFML framework I use to create the window and more provides a function to reset the OpenGL state of the context. I called it right after window creation.
Even though I can't explain why, removing that function call solved the crash. Maybe it is because GLEW or something else isn't yet initialized that moment.
sf::RenderWindow window;
window.create(VideoMode(1024, 768), "Window Title");
window.resetGLStates(); // removing this line fixed the crash
window.setVerticalSyncEnabled(true);
I'm writing a MIDlet using LWUIT and images seem to eat up incredible amounts of memory. All the images I use are PNGs and are packed inside the JAR file. I load them using the standard Image.createImage(URL) method. The application has a number of forms and each has a couple of labels an buttons, however I am fairly certain that only the active form is kept in memory (I know it isn't very trustworthy, but Runtime.freeMemory() seems to confirm this).
The application has worked well in 240x320 resolution, but moving it to 480x640 and using appropriately larger images for UI started causing out of memory errors to show up. What the application does, among other things, is download remote images. The application seems to work fine until it gets to this point. After downloading a couple of PNGs and returning to the main menu, the out of memory error is encountered. Naturally, I looked into the amount of memory the main menu uses and it was pretty shocking. It's just two labels with images and four buttons. Each button has three images used for style.setIcon, setPressedIcon and setRolloverIcon. Images range in size from 15 to 25KB but removing two of the three images used for every button (so 8 images in total), Runtime.freeMemory() showed a stunning 1MB decrease in memory usage.
The way I see it, I either have a whole lot of memory leaks (which I don't think I do, but memory leaks aren't exactly known to be easily tracked down), I am doing something terribly wrong with image handling or there's really no problem involved and I just need to scale down.
If anyone has any insight to offer, I would greatly appreciate it.
Mobile devices are usually very low on memory. So you have to use some tricks to conserve and use memory.
We had the same problem at a project of ours and we solved it like this.
for downloaded images:
Make a cache where you put your images. If you need an image, check if it is in the cachemap, if it isn't download it and put it there, if it is, use it. if memory is full, remove the oldest image in the cachemap and try again.
for other resource images:
keep them in memory only for as long as you can see them, if you can't see them, break the reference and the gc will do the cleanup for you.
Hope this helps.
There are a few things that might be happening here:
You might have seen the memory used before garbage collection, which doesn't correspond to the actual memory used by your app.
Some third party code you are running might be pooling some internal datastructures to minimize allocation. While pooling is a viable strategy, sometimes it does look like a leak. In that case, look if there is API to 'close' or 'dispose' the objects you don't need.
Finally, you might really have a leak. In this case you need to get more details on what's going on in the emulator VM (though keep in mind that it is not necessarily the same as the phone VM).
Make sure that your emulator uses JRE 1.6 as backing JVM. If you need it to use the runtime libraries from erlyer JDK, use -Xbootclasspath:<path-to-rt.jar>.
Then, after your application gets in the state you want to see, do %JAVA_HOME%\bin\jmap -dump:format=b,file=heap.bin <pid> (if you don't know the id of your process, use jps)
Now you've got a dump of the JVM heap. You can analyze it with jhat (comes with the JDK, a bit difficult to use) or some third party profilers (my preference is YourKit - it's commercial, but they have time-limited eval licenses)
I had a similar problem with LWUIT at Java DTV. Did you try flushing the images when you don't need them anymore (getAWTImage().flush())?
Use EncodedImage and resource files when possible (resource files use EncodedImage by default. Read the javadoc for such. Other comments are also correct that you need to actually observe the amount of memory, even high RAM Android/iOS devices run out of memory pretty fast with multiple images.
Avoid scaling which effectively eliminates the EncodedImage.
Did you think of the fact, that maybe loading the same image from JAR, many times, is causing many separate image objects (with identical contents) to be created instead of reusing one instance per-individual-image? This is my first guess.