The machine on which I develop has more memory than the one on which the code will eventually run. I dont have access tothe machine on which it will actually run. This is a 64 bit application and I intend to use the address space but cap physical allocation. I dont want to lock down virtual memory, only physical memory. Is there a way to set limits on a linux machine such that it mimics a system with low RAM. I think ulimit does not differentiate between reserved address space vs actual allocation. If there is a way to do it without rebooting with different kernel parameters or, pulling out extra RAM that would be great. May be some /proc tricks.
See https://unix.stackexchange.com/questions/44985/limit-memory-usage-for-a-single-linux-process which suggests using "timeout" from here: https://github.com/pshved/timeout .
If You can change boot command line of the kernel and want to restrict available memory use
mem=
boot parameter.
For more information check:
https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html
Related
I am currently studying about virtual memory in operating system and I have few questions.
Is swap partition or swap file same as virtual memory in terms of Linux?
If yes, then in case I've no swapping enabled in my Linux system, does that mean my system has no virtual memory?
I have also read that virtual memory makes system more secure because with virtual memory, CPU generates virtual addresses which are then translated to actual physical addresses by MMU, therefore securing the system because no process can actually interact with the actual physical memory. So if I just enable swapping on my Linux system, will my CPU start generating virtual addresses and currently it's directly generating physical addresses as I have no swap partition?
How does CPU know if virtual memory is present or not?
Having no swap file/partition doesn't imply that you don't have virtual memory. Modern operating-systems always use paging/virtual memory no matter what.
Is swap partition or swap file same as virtual memory in terms of Linux?
No swap file and virtual memory is not the same in terms of any OS. Virtual memory just says that all memory accesses are going to be translated by the MMU using the page tables. Modern OSes always use paging.
If yes, then in case I've no swapping enabled in my Linux system, does that mean my system has no virtual memory?
Your system certainly has virtual memory. To use long mode (64bits mode), the OS must enable paging. I doubt that you have a system old enough to not use paging. Page swapping to the hard-disk is not virtual memory. It is more like a feature of virtual memory that can be used to extend physical memory because a page which isn't required immediatly can be swapped to the hard-disk momentarily.
I have also read that virtual memory makes system more secure because with virtual memory, CPU generates virtual addresses which are then translated to actual physical addresses by MMU, therefore securing the system because no process can actually interact with the actual physical memory. So if I just enable swapping on my Linux system, will my CPU start generating virtual addresses and currently it's directly generating physical addresses as I have no swap partition?
Your computer certainly has paging/virtual memory enabled. Having no swap partition doesn't mean that you don't have virtual memory. Paging can also be used to avoid fragmentation of RAM and for security. You are right that paging is securing your system because the page tables prevent a process from accessing the memory of another process. It also has ring privilege on a page to page basis which allows to differentiate between kernel mode and user mode code.
How does CPU know if virtual memory is present or not?
The OS just enables paging by setting a bit in a control register. Then the CPU starts blindly translating every memory accesses using the MMU.
No. Swap file is not the same as virtual memory.
Once the firmware/kernel sets up the necessary registers and/or in-memory data structures and switches the processor mode, virtual memory mappings are used for accessing the physical memory.
Yes, the inability of processes to refer to memory locations without a mapping allows the kernel to employ isolation and access control mechanisms.
Through active mappings, different virtual addresses can map to the same physical memory region at different times. The kernel can maintain the illusion that a larger amount of memory is available that the capacity of the actual physical memory, where only a subset of the virtual memory resides in the physical memory at any given time. The rest is stored in the swap file.
Accesses to virtual addresses where the corresponding data is currently in the swap file are trapped by the kernel (via a page fault) and might lead to the kernel swapping the data in, and swapping some other data from physical memory out.
If you disable the swap file, the kernel has no place store the swapped out data. This reduces the amount of virtual memory available.
Linux creates virtual memory pages for every program to use, and the OS handles mapping the virtual addresses to genuine hardware addresses, correct?
But how does Windows do this? Do Windows programs actually have memory that translates to real hardware addresses? I'm also aware that windows can use hard disk memory when RAM is over used, and this process is again called virtual memory, but I believe this is an entirely different concept?
Windows and Linux (at least on Intel 32/64 bit systems) both implement virtual memory using the same mechanism: hardware supported page tables. The OS and the hardware cooperate together to do the address mapping.
The entire concept of separating the logical addresses a program uses from the physical addresses is what is called virtual memory. The use of the hard disk as a backing store is an implementation of virtual memory that uses a swap file to increase the amount of virtual memory to an amount greater than the physical memory installed in the system.
Virtual memory is a pretty deep and wide subject. Maybe start with this Wiki article an Memory Management and then hit the googles for a deeper understanding.
I have ubuntu and installed on it several qemu-kvm guests, running also ubuntu.
I'm using libvirt to change the guests' memory allocation. But always encounter a constant difference between the requested memory allocation and the actual memory allocation I query from the Total field in the top command inside the guests.
The difference is the same in all the guests, and consistent.
In one machine I installed it is 134MB (allocated is less then requested), In another one it is 348MB.
I can live with it, I just don't know the reason. Does someone encounter this kind of problem? Maybe solved it?
Thanks
This constant difference is likely the space reserved by the kernel. Note that this amount of space will increase (at least in Linux) as you have more physical memory available in the system. The change you're seeing is probably due to kvm giving that particular guest more or less memory to work with than it was before.
If you're interested, here is a quick article on memory ballooning, as implemented by VMWare ESX Server.
I'm trying to get hold of CPU architecture information under Linux.
I understand the information is available via the sysfs filesystem.
I have CentOS 5 running in a Xen VM. The sysfs filesystem is mounted. However, the /sys/devices/system/cpu/cpu0/ directory is almost empty. The only entry is a single file, "online", with a value of "1".
What gives? where's all my CPU information?
The actual cpu information is still in /proc/cpuinfo.
The sysfs-files are used to control things like scheduling and frequency settings, not to get information on the cpus themselves.
Okay, I've just had a chat with a sysadmin at work.
Looking at some machines, it looks like this information simply is not pushed by VMs. The VMs think they have a virtual CPU - rather than a CPU of the type of the real underlying CPU - and the cache information simply is not published.
It is published (and it's nice to finally see it!) on real machines with reasonably modern kernels.
When writing embedded ARM code, it's easy to access to the built-in zero wait state memory to accelerate your application. Windows CE doesn't expose this to user-mode applications, but there is probably a way to do it. The internal SRAM is usually used for the video buffer, but there's usually some left over. Anyone know how to do it?
Thanks,
Larry B.
Unfortunately you can't access the high speed ram from usermode-processes.
The only way to get access to it on a WindowsCE-OS is to write a driver, map the fixed address of the TCM into the user-mode process address space and pass it to the user-mode process.