Weird EC2 CPU usage - linux

I'm really confused. Why does the load average and %CPU does not match the process CPU usage below. It seems like the process is eating up a lot of CPU while the AWS EC2 meters only says 25% CPU is used.

%CPU -- CPU Usage : The percentage of your CPU that is being used by the process. By default, top displays this as a percentage
of a single CPU. On multi-core systems, you can have percentages
that are greater than 100%. For example, if 3 cores are at 60% use,
top will show a CPU use of 180%.
You can toggle this behavior by hitting Shift+i while top is running to show the overall percentage of available
CPUs in use.
load average: 22.56, 24.99, 26.51
From left to right, these numbers show you the average load over the last 1 minute, the last 5 minutes, and the last 15 minutes.
us -- User CPU time
The time the CPU has spent running users' processes that are not niced.
sy -- System CPU time
The time the CPU has spent running the kernel and its processes.
ni -- Nice CPU time
The time the CPU has spent running users' proccess that have been niced.
wa -- iowait
Amount of time the CPU has been waiting for I/O to complete.
hi -- Hardware IRQ
The amount of time the CPU has been servicing hardware interrupts.
si -- Software Interrupts
The amount of time the CPU has been servicing software interrupts.
st -- Steal Time
The amount of CPU 'stolen' from this virtual machine by the hypervisor for other tasks (such as running another virtual machine).
See more details from In Linux “top” command what are us, sy, ni, id, wa, hi, si and st (for CPU usage).

after you run command "top" you can press "1" on your keyboard to see individual CPU utilization, more details when you run command "man top"
Note process "msqld" can use CPU from several resources and its utilization % could easily go beyond 100% in "top" display.

Hi maybe your app using single core and other cores are free. I think your instance has 4 CPU core and one is utilizing 100%. can you please check utilization by each core.

Related

htop shows that cpu usage of per core over 100%?

I'm using htop to monitor the CPU usage of my task. However, the CPU% value exceed 100% sometimes, which really confused me.
Some blogs explain that this is because I'm using a multi-core machine(this is true). If there are 8 (logic) cores, the max value of CPU% is gonna be 800%. CPU% over 100% means that my task is occupying more than one core.
But my question is: there is a column named CPU in htop window which shows the id of the core my task is running on. So how can the usage of this single core exceed 100%.
This is the screenshot. You can see the 84th core's usage is 375%!

CPU cores are underutilised while running n=-1 jobs

I have read that the Green bar represents the normal user process and the Red bar represents the time spent in kernel(kernel, iowait etc).
I'm running a jupyter-notebook that runs n=-1 jobs, and by definition all my cores should be used as normal user process and the htop should show high per core usage. And the normal behaviour while running this notebook was that the average per core usage used to hit 98-99%.
But weirdly now running the same notebook, the per core usage is limited to a max of 18% with an abrupt increase in kernel time.
I want to understand why this is happening.

Can CPU execute other processes if the iowait is 99% now

If there is only one cpu and the IOwait is 99%, is the cpu still able to execute other processes, if so, is only 1% cpu resource could be used? or the other process could also use 100% cpu resource although under the scene of 99% Iowait.
I read the following line in one book"
iowait is time spent receiving and handling hardware interrupts as a
percentage of processor ticks.
If so, the 99% iowait doesn't mean the cpu is idle or waiting, actually it's very busy to receive and handle the interrupts. If this is true, I guess the other processes only have 1% cpu resource can be used.
I got the answer: The 99% IOWait means cpu almost 99% idle. i.e IOWait is a subset of cpu Idle. The following url has an excellent explanation:
http://veithen.github.io/2013/11/18/iowait-linux.html

Meaning of values in CPU tab of resource monitor on windows 8.1

(Sorry for non english character in picture. Each column is thread/CPU/average CPU)
When I open CPU tab in resource monitor on Window 8.1, I see above values.
What's the difference between CPU and average CPU?
At first, I thought average CPU means avaerag usage per core but I have 4 cores so the value should be CPU=4*avg. CPU which is not.
Please let me know the meaning of CPU and average CPU values.
CPU. Current percent of CPU consumption by the process, or how much of the system's processing power is being devoted to this specific process.
Average CPU. This is average CPU consumption by the process over the past 60 seconds. This gives you a real-time look at what's happening on the system right now and for the past minute.
http://www.techrepublic.com/blog/the-enterprise-cloud/use-resource-monitor-to-monitor-cpu-performance/

Max CPU usage (max allowed CPU usage)

I have a quad core (with hyper threading Technology - HT)
I'm running an application which takes 270% CPU (according to TOP command)
What is the total available CPU usage? (is it 400% or 800%?)
I'm asking because according to Intel documentation, the HT can up the performance up to 30% cpu, so 800% seem to much, yeah?)
What is the relation between load averages and CPU usage?
1: 800. You have 8 cores visible to the OS - that they are not real cores (due to hyperthreading limitations) is not of concern.
2: Ever bothered reading documentation? Practically there is no relation between load average and CPU uage. Load average is "waiting processes" but that can mean they are waiting for IO, and the CPU may not be busy.

Resources