I’ve been trying to trace memory leaks in a hybrid-app built using ionic. I tried using the Google’s developer tools for the same. When i perform heap dump and try tracing any Constructor value, I get there is a Detached DOM tree is present but when i expand the same and view the origin in Objects in the Retainers tab, I’m not able to find which file the variable is located. With this I’m able to say that there is a memory leakage but not the file where it is located. Is there any way to find the file in which the variable or the array causing the leakage is present.
In below picture, I'm not finding a way as to how to trace the file of the memory leak present:
Related
Context:
I have an Node.js application which memory seems to very high, I don't know if that is memory leak or not, because it is reduced after certain period time, but some times on heavy load it keeps on increasing takes much longer to get reduced.
So going through the articles and couple of videos, i figured that i have to heap snapshot and analyse what is causing the memory leak.
Steps:
I have taken 4 snap shots as of now in my local to reproduce the memory leak.
Snapshot 1: 800MB
Snapshot 2: 1400MB
Snapshot 3: 1600MB
Snapshot 4: 2000+MB
When i uploaded the heapdump files to chrome dev tools I see there a lot of information but i don't know how to proceed from there.
Please check below screenshot, it says there is constructor [array] which has 687206 as shallow Size & Retained Size is 721414 in the columns, so when expanded that constructor i can see there are 4097716 constructors created ( refer the second screenshot attached below ).
Question
What does internal array [] means ? Why is there 4097716 created ?
How can a filter out the constructor which created by my app and showing me that instead of some system/v8 engine constructor ?
In the same screenshot one of the constructor uses global variable called tenantRequire function, this is custom global function which is being used internally in some places instead of normal Node.js require, I see the variable across all the constructor like "Array", "Object". This is that global tenantRequire code for reference. It is just patched require function with trycatch. Is this causing the memory leak somehow ?
Refer screenshot 3, [string] constructor it has 270303848 as shallow size. When i expanded it shows modules loaded by Node.js. Question why is this taking that much size ? & Why is my lodash modules are repeated in that string constructor ?
Without knowing much about your app and the actions that cause the high memory usage, it's hard to tell what could be the issue. Which tool did you use to record the heap snapshot? What is the sequence of operations you did when you recorded the snapshot? Could you add this information to your question?
A couple of remarks
You tagged the question with node.js and showed Chrome DevTools. That's ok. You can totally take a heap snapshot of a Node.js application and analyze it in Chrome DevTools. But since both Node.js and Chrome use the same JS engine (V8) and the same garbage collector (Orinoco), it might be a bit confusing for someone who reads the question. Just to make sure I understand it correctly: the issue is in a Node.js app, not in a browser app. And you are using Chrome just to analyze the heap snapshot. Right?
Also, you wrote that you took the snapshots to reproduce the memory leak. That's not correct. You performed some action which you thought would cause a high memory usage, recorded a heap snapshot, and later loaded the snapshot in Chrome DevTools to observe the supposed memory leak.
Trace first, profile second
Every time you suspect a performance issue, you should first use tracing to understand which functions in your applications are problematic (i.e. slow, create a lot of objects that have to be garbage-collected, etc).
Then, when you know which functions to focus on, you can profile them.
Try these visual tools
There are a few tools that can help you with tracing/profiling your app. Have a look a FlameScope (a web app) and node-clinic (a suite of tools). There is also Perfetto, but I think it's for Chrome apps, not Node.js apps.
I also highly recommend the V8 blog.
I am using Lazarus 2.0.4 on Windows 10.
Trying to track memory leaks in a quite large project I tried to familiarize myself with the heaptrc and lineinfo units/compiler switches -gh and -gl.
While this gives me some information that there is a leak somewhere, I could not figure out how to pinpoint the exact location. I put together this quite basic leaking code which will leak 4 bytes every time the button is pressed. Upon exit of the program I get this:
While from the trace I see that there is a 4 byte memory leak, I cannot see any useful reference where to look for the leaking code. I see that the code was within the click handler of a TButton, but this isn't very useful, since there may be many Buttons.
Any way to get a reference to the exact location (line 42)?
What are some examples of developer created memory leaks written in the 4D programming language?
By developer created memory leak, i am referring to a memory leak created by bad programming that could have been avoided with better programming.
32 bit
When ran in a 32 bit application it should eventually crash once it attempts to allocate more than 2^32 bytes (4 GB) of memory. If on the Mac OS X platform, the bottom of the crash report beneath the VM Region Summary should show a memory value around 3.7 GB:
TOTAL 3.7G
64 bit
When ran in a 64 bit application the code will continue to raise the amount of memory allocated and will not plateau, in that situation the OS will eventually complain that it has ran out of memory:
Overview
There are many ways that a developer can create there own memory leaks. Most of what you want to avoid is listed here.
use CLEAR VARIABLE when done using a variable
use CLEAR SET when done using a set
use CLEAR NAMED SELECTION when done using a named selection
use CLEAR LIST when done using a list
re-size your BLOBs to 0 with SET BLOB SIZE when done using the BLOB or use CLEAR VARIABLE
re-size your arrays to 0 when done using the array or use CLEAR VARIABLE
don't forget to close any open XML trees such as XML, DOM, SVG, etc (DOM CLOSE XML, SVG_CLEAR)
if using ODBC always remember to free the connection using ODBC_SQLFreeConnect
make sure to cleanup any offscreen areas used
Examples
Here are two specific examples of developer created memory leaks:
Forgetting to close XML
Bad code:
Repeat
$xmlRef:=DOM Create XML Ref("root")
Until (<>crashed_or_quit)
The code snippet above will leak memory because each call to DOM CREATE XML REF will create a new reference to a memory location, while the developer of this code has neglected to include a call to free the memory. Running this in a loop in a 32 bit host application will eventually cause a crash.
Fixed code:
This code can be easily fixed by calling DOM CLOSE XML when finished with the XML reference.
Repeat
$xmlRef:=DOM Create XML Ref("root")
DOM CLOSE XML($xmlRef)
Until (<>crashed_or_quit)
Forgetting to clear a list
Bad code:
Repeat
$listRef:=New list
Until (<>crashed_or_quit)
The code snippet above will leak memory because each time NEW LIST is called a reference to a new location in memory is returned. The developer is supposed to clear the the memory at the referenced location by using the CLEAR LIST($listRef) command. As a bonus, if the list has any sublists attached, the sublists can be cleared by passing the * parameter like CLEAR LIST($listRef;*).
Fixed code:
This can be easily fixed by calling CLEAR LIST($listRef;*) as seen in the following fixed example:
Repeat
$listRef:=New list
CLEAR LIST($listRef;*)
Until (<>crashed_or_quit)
There is a lot of tools designed to help analyzing portable executable files. For example PE Explorer. We can load .exe file into it and check things like number of sections, section alignment or virtual addresses of particular sections.
Is there any similar tool which allows me to do the same but for a portable executable already loaded into memory? Without access to it's .exe file?
EDIT:
Maybe I will try to clarify what I'm trying to achieve. Lets say (like #0x90 suggested) that I have two applications or maybe even three applications.
app1.exe - executed by user, creates new process basing on app3.exe and modifies its memory by putting app2.exe into it.
app2.exe - Injected into app3.exe memory by app1.exe.
app3.exe - Used to create the new process.
I have sources of all of three applications. I'm simply trying to learn about windows internals by practical exercises with injecting/Processes Hollowing. I have some bug in app1.exe which leads to:
The application was unable to start correctly (0xc0000018).
And I'm trying to find a way to debug this situation. My idea was to compare PE on disk with PE in memory and check if it looks correct. I was surprised that I can't find tool designed for such a propose.
For clarity, the application you wish to examine shall be called App1 while the application which loads the PE files contents into memory shall be called App2.
To my knowledge, all major disassemblers work on files.
This is due to the fact that App1 will be mapped into the address space of App2.
Your easiest solution is to dump the executable to disk from memory.
If you control the source of App2, this step is trivial.
If you don't, you will need to attack a debugger, identify the exact memory range where the PE file resides and use the debugger's functionality to dump that memory range to disk.
For an embedded ARM system running in-field there is a need to retrieve relevant debug information when a user-space application crash occurs. Such information will be stored in a non-volatile memory so it could be retreived at a later time. All such information must be stored during runtime, and cannot use third-party applications due to memory consumption concerns.
So far I have thought of following:
Signal ID and corresponding PC / memory addresses in case a kernel SIG occurs;
Process ID;
What other information do you think it's relevant in order to indentify the causing problem and be able to do a fast debug afterwards?
Thank you!
Usually, to be able to understand an issue, you'll need every register (from r0 to r15), the CPSR, and the top of the stack (to be able to determine what happened before the crash). Please also note that, when your program is interrupt for any invalid operation (jump to invalid address, ...), the processor goes to an exception mode, while you need to dump the registers and stack in the context of your process.
To be able to investigate, using those data, you also must keep the ELF files (with debug information, if possible) from your build, to be able to interpret the content of your registers and stack.
In the end, the more information you keep, the easier the debug is, but it may be expensive to keep every memory sections used by your program at the time of the failure (as a matter of fact, I've never done this).
In postmortem analysis, you will face some limits :
Dynamically linked libraries : if your crash occurs in a dynamically loaded and linked code, you will also need the lib binary you are using on your target.
Memory corruption : memory corruption usually results in the call of random data as code. On ARM with linux, this will probably lead to a segfault, as you can't go to an other process memory area, and as your data will probably be marked as "never execute", nevertheless, when the crash happens, you may have already corrupted the data that could have allow you to identify the source of the corruption. Postmortem analysis isn't always able to identify the failure cause.