Google Compute Engine rents all size Linux VMs from 1 core to 64 cores at various prices. There are "preempt-able" instances for about 1/4 the price of guaranteed instances, but the the preempt-able instances can be terminated at any time (with an ACPI G2 soft off warning and ~ 30 secs until hard cutoff). Although you can provide a startup and shutdown script, the usual approach seems to lead to the unnecessary overhead of having to then create additional software to allow calculations to be interrupted, and to manage partial results of calculations whereas the suspend-to-disk/restore-from-disk scheme seen in laptops and desktops could be a much simpler approach to storing and resuming calculations and therefore preferable.
If I start a Linux preemptible VM on GCE, is it possible generally to suspend the state of the VM to the disk (aka hibernate) and restart a new preemptible VM from the disk afterwards? My idea is:
Start a new preemptible Linux VM.
When the OS receives the the preemption notice (ACPI G2 Soft Off signal), then trigger suspend the to disk - hibernate the Linux OS.
Start a new preemptible Linux VM from the suspended image, i.e. restore the former VM and continue in the computation.
How would I configure Linux to suspend/restore in this way?
Related
I have a script where I am using Throughput Shaping Timer to achieve 100 TPS/RPS.
When the script is executed on MAC System using GUI Mode, it is able to achieve ~99 TPS/RPS. But, when I execute it on Linux System it hardly goes beyond 60 RPS/TPS.
Following logs received on Linux OS (same script, so Thread Group settings remain as is):
No free threads available in current Thread Group Device Service
Some of the details given below:
JMeter version is 5.4.3 on both the systems (copied the same JMeter to Linux VM as well)
MAC OS version is: 11.6
Linux OS version is: Red Hat Enterprise Linux 8.6 (Ootpa)
Heap setting on both the systems are given below (even increased it to 13g on Linux VM):
: "${HEAP:="-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m"}"
Please let me know which settings I should do to achieve similar TPS/RPS as with my GUI on MAC.
Thread Group Setting shown in the attached image.
First of all GUI mode is for tests development and debugging, when it comes to test execution you should be running tests in command-line non-GUI mode in order to get accurate results
Make sure to use the same Java version
Make sure to use the same (or at least similar) hardware
Make sure to check resources consumption (CPU, RAM, Disk, Network, Swap usage, etc.) as it you're hitting the hardware or OS limits you might get false-negative results because JMeter cannot run requests as fast as it can and if this is the case you might need another Linux box and run JMeter in distributed mode
No free threads available in current Thread Group means that there is not enough threads in order to reach/maintain the desired throughput, you can try increasing the number of threads in the Thread Group or switch to Concurrency Thread Group and connect it to the Throughput Shaping Timer via Feedback function
Simple question: Is it dangerous to activate KSM on a running hypervisor (Debian 8 with 3.16 kernel)?
Or is it recommended to shut down all virtual machines (KVM/qemu) first, then activate KSM and then start the virtual machines again?
We expect a memory saving of approx. 50% (we have a similar system where KSM is already active and there we effectively save almost 50% due to the always very similar VM appliances).
As far as I understood, at the moment, Docker for Mac requires that I decide upfront how much memory and CPU cores to statically allocate to the virtualized linux it runs on.
So that means that even when Docker is idle, my other programs will run on (N-3) CPU cores and (M-3)GB of memory. Right?
This is very suboptimal!
In Linux, it's ideal because a container is just another process. So it uses and releases the system memory as containers starts and stop.
Is my mental model correct?
Will one day Docker for Mac or Windows dynamically allocate CPU and Memory resources?
The primary issue here is that, for the moment, Docker can only run Linux containers on Linux. That means on OS X or Windows, Docker is running in a Linux VM, and it's ability to allocate resources is limited by the facilities provided by the virtualization software in use.
Of course, Docker can natively on Windows, as long as you want to run Windows containers, and in this situation may more closely match the Linux "a container is just a process" model.
It is possible that this will change in the future, but that's how things stand right now.
So that means that even when Docker is idle, my other programs will run on (N-3) CPU cores and (M-3)GB of memory. Right?
I suspect that's true for memory. I believe that if the docker vm is idle it isn't actually using much in the way of CPU resources (that is, you are not dedicating CPUs to the VM; rather, you are setting maximum limits on how many resources the vm can consume).
While creating a KVM Virtual Machine in proxmox from the GUI, for the hard disk and CPU tab there are a couple of options that are confusing.
For example, this is the hard disk tab,
In that tab, what does "No backup", "Discard" and "Iothread" signify?
And similarly, this is the CPU tab,
In this tab, what does "Sockets", "Cores" and "Enable numa" mean?
I did not have any luck with google and the results that I got were conflicting.
No backup instructs proxmox to not perform any backups for that VM.
Discard allows the guest to use fstrim or the discard option to free up the unused space from the underlying storage system. This is only usable on a virtio_scsi driver.
Iothread sets the AIO mode to threads (instead of native). For more information check this presentation.
Sockets is the number of CPUs that the guest will see as installed/available.
Cores is the number of cores that the guest will be able to use for each CPU.
If your server has 2 sockets each with a CPU that has 6 cores you could put 2 in the Sockets and 6 in the Cores fields and the guest will be able to use 100% of both CPU's. You can also put 1 in Sockets and 3 in Cores fields and the guest will be able to use only 50% from one CPU, so only 25% of the available CPU power on the server.
Enable Numa will allow the guest to make use of the NUMA architecture (servers with more than 2 sockets) on specific servers. Check www.admin-magazine.com/Archive/2014/20/Best-practices-for-KVM-on-NUMA-servers
i am trying to measure code performance (basically speed-up when using threads). So far i was using cygwin via windows or linux on separate machine. Now i have the ability to set up a new system and i am not sure whether i should have dual boot (windows and ubuntu) or a virtual machine.
My concern is whether i can measure reliable speed up and possibly other stuff (performance monitors) via a linux virtual machine or if i have to go with with normal booting in linux.
anybody have an opinion?
If your "threading" relies heavily on scheduling, I won't recommend you to use VM. VM is just a normal process from the host OS's point of view, so the guest kernel and its scheduler will be affected by scheduling by the host kernel.
If your "threading" is more like parallel computation, I think it's OK to use VM.
For me, it is much safer to boot directly on the system and avoid using a VM in your case. Even when you don't use a VM, it is already hard to have twice the same results in multi-threading because the system being used for OS tasks, so having 2 OS running in the same time as for VM even increases the uncertainty on the results. For instance, running your tests 1000 times on a VM would lead to, let's say, 100 over-estimated time, while it would maybe be only 60 on a lonely OS. It is your call to know if this uncertainty is acceptable or not.