This might be a big ask, but are there any tools that let me view the virtual memory usage of my process in Linux? I am talking detailed, probably graphical view of memory, including what is going into the reserved addresses, the BSS/text/etc segments, heap, stack growth, etc, while I am stepping over the program in a debugger?
I once used pmap to monitor memory consumption. It helped me a lot in discovering the source of a memory leak. It's far from being graphical but you can learn quite a lot from it.
Good question. For broad views, maybe just keep dumping /proc/pid/maps? I think pmap, mentioned above, shows info from here. But I guess you are talking about specific small allocs/deallocs.
Related
I have a query regarding Virtual Memory. First of all, I would like to mention that I am new to the field of programming. I have read up on Visual Memory.
Now I have a program which opens softwares which require large amounts of memory (like for example a picture viewer). The concerned computer, however, cannot spare that much of memory for this . And this is all done with Visual C++. The picture viewer is currently running on physical memory.
But once this software is distributed, it will be used on computers which cant spare that much of physical memory. So my task is to research and find out how to switch this program from using physical memory to virtual memory. In the end I will probably be implementing this myself.
So my question is, how do I alter the code in such a way that I can prevent the application from using physical memory and instead, switch over to virtual memory?
I'm not asking for someone to provide me with a copy paste code ofcourse, but just a method to do so. Also, if someone could explain the logic behind it, I would appreciate it.
Loads of thanks in advance.
The operating system is in charge of deciding what should be stored in RAM and what should be paged out to VM. Under certain abnormal circumstances, it can be useful to can provide advice to the OS from your app, but this is only recommended for experts. As a novice, your best bet is to trust that the OS will do the right thing.
Why do you think you need special behavior anyway? Pictures are generally small anyway. Unless your app is dealing with thousands and thousands, they will fit in RAM.
You can't use virtual memory without using some physical memory. There's a reason for the name swapfile. The processor cannot directly operate on data in secondary storage, such as hard disks. It must first copy it into RAM.
So is meminfo the best way to say there is X free memory? I ask this question because my company states that this is not an accurate way to represent free memory. In fact, it is the sum of memfree, cached, buffers, slab and if on a vmware box, the vmware ballon is added from /proc/vmmemctl. thoughts?
Regarding the vmware host case, I had a similar problem with one or our servers, see https://serverfault.com/questions/435307/redhat-linux-server-paging-sum-of-res-rss-buffers-cached-total-who-is-u
I think the problem with meminfo is that it doesn't clearly show the memory used by vmmemctl. Since vmmemctl is a driver, the memory it uses doesn't appear also in top/ps.
I believe that /proc/meminfo is accurate (but I can't prove that), and more generally the /proc pseudo-file system gives you a lot of information about the system (and each process).
Of course it gives information about the kernel providing it (so the kernel running inside your VMware virtual machine).
If you are concerned by a specific process, see also this answer
We have a a very large project which is basically an application which uses Linux Application programming and runs on PowerPC processor. This project was initially developed by another company. We acquired the project from the company and now we are maintaining the project.
The application is reported to have a lot of memory leak issue. Since this is a large project, it is not possible to go to each source code file and find out the memory leak. We have used Valgrid, mpatrol and other memory leak detection tools. These tools did not help much and the memory leak has not decreased by a significant percentage.
In this situation, how to go about to reduce the memory leak by a significant amount.Is there a general method which people use in these case to reduce the memory leak other than the memory leak detection tools like mentioned above.
Usually Valgrind belongs to the best tools for this tasks. If it does not work correctly, there might only be a couple of things you can still do.
First question: What language is the application in? Valgrind is very good for C and C++, but will not help you with garbage collected or scripting language. So check the language first. There might be something similar for java, but I have not used that much java, so you would have to ask someone else.
Play around a lot with the settings of valgrind. There are several plugins, that can help with this. One example could be using --leak-check=full or similar options. There are also plugins for valgrind, that can enhance it detection capabilities.
You say, that the application was reported to have a memory leak. How was this detected? Did the application detect this by itself. If it was detected by the application on it's own without any external tools, this probably means someone has added their own memory tracker inside the application. Custom memory tracker, memory pools etc. mess up valgrind and any other leak detection system very bad. So in case any custom memory handling is present in the application, your only choice is to either deactivate it (if possible) or to hook into this custom mechanism. How this could be done depends on your application only.
Add your own memory tracker. For example in C++ it is possible to hook into new/delete calls and get them to track the memory. There are a couple of libraries you can use for this. You can also write your own new/delete replacement in about 500 LOC. If you decide to use this method, be sure to read a lot of tutorials on replacing new/delete, since there are several things that are unusual in the C++ world when attempting this task.
What makes you so sure, there is an memory leak in the application (i.e. how was this detected)? If a tool just reported huge numbers of allocated memory, this might not even mean, there is an actual memory leak. A memory leak means that the handles to the memory are lost and hence it becomes impossible to every reach and free that memory again. In case your application just get's a lot of memory and keeps it accessible, you probably have a completely different problem. For example you simply might use an algorithm with a bad space complexity at one point or the other, leading to many allocations. In this case you will not need a leak detector, but rather a memory profiler, which gives you more detailed overview of the memory footprint of the code parts. However I have never used a profiler for this kind of task before, so I cannot give you any more hints on this.
You could replace all memory allocation calls with calls to your own allocation methods, which should call original methods and at the same time count memory usage and where it was allocated. This will allow you to find the leaks and eliminate them by hand.
There might also be automated tools that allow you to do this - not sure, haven't used any. But this method works.
Perhaps you might also consider using Boehm's garbage collector (that is using GC_malloc instead of malloc etc... and not bother about free-ing data).
My .NET application has memory leak. Few people seem to recommend using CLR Profiler for this pupose I am a bit lost on the idea. To me in order to find a memory leak, tool should compare two memory states that can give you statistics like growth in objects between two states. So in my mind, if a tool cannot compare two (or more) memory states, it cannot be used for detecting memroy leak. Obviously things like performance counters is bit different concept where you can trend the memory usage.
So my question is really if someone can explain how exactly CLR Profiler can be used to detect memory leaks?
Well it depends on what kind of memory leak you have.
We had a reproducible one, where we new that a certain chain of events should always leave a clean table after work was done - but it wasn't.
So we simple setup a test where we did it a couple of thousand times - then we looked at those objects (bigger in number) in the heap graph and at the "root"-object the cause of why the objects where still alive. It helped to solve our problem...
I couldn't find any documentation about memory management of iOS. Especially about memory fragmentation. If you know any document about this, please let me know.
Memory fragmentation correction is an implementation detail. You should never, in any way, directly deal with it. However if you are worried that your objects will be moved around without notifying your code, don't; apple never moves objects - end of story. Once memory is allocated, it's yours (even if a framework class's instance is occupying it) until it is deallocated. That being said, from an academic perspective I see no reason why the iPhone shouldn't use at least a similar strategy to MacOS, on which there is an excellent article here.