I'd like to use a linux pc purely to collect rss feeds, but would like to minimise it's power usage as much as possible.
Presumably I would create some kind of cron job to bring it in and out of sleep mode or are there better ways of tackling this?
Set wake up alarm in RTC, then put the computer in standby or suspend(-to-RAM) mode. Some motherboards support waking up from hibernation (suspend-to-disk), but I guess that'll be too slow.
echo `date '+%s' -d '+ 5 minutes'` > /sys/class/rtc/rtc0/wakealarm
echo -n "mem" > /sys/power/state
Replace "mem" with "standby" for stand-by instead of suspend.
BTW. MythTV's wiki has got some more in-depth info end examples.
http://www.mythtv.org/wiki/ACPI_Wakeup
Related
I'm working on a system on which ubuntu is running. I'm reading basic data like CPU frequency and temperature out of the thermal zones provided in /sys/class/thermal.
Unfortunately, I've got around 100 thermal_zones from which I need to read the data. I do it with:
for SENSOR_NODE in /sys/class/thermal/thermal_zone*; do printf "%s: %s\n" $(cat ${SENSOR_NODE}/type) $(cat ${SENSOR_NODE}/temp); done
To collect all data takes ~2.5-3 sec. which is way to long.
Since I want to collect the data every second my question is, if there is a way to "read" or "collect" the data faster?
Thank you in advance
There's only so much you can do while writing your code in shell, but let's start with the basics.
Command substitutions, $(...), are expensive: They require creating a FIFO, fork()ing a new subprocess, connecting the FIFO to that subprocess's stdout, reading from the FIFO and waiting for the commands running in that subshell to exit.
External commands, like cat, are expensive: They require linking and loading a separate executable; and when you run them without exec (in which case they inherit and consume the shell's process ID), they also require a new process to be fork()ed off.
All POSIX-compliant shells give you a read command:
for sensor_node in /sys/class/thermal/thermal_zone*; do
read -r sensor_type <"$sensor_node/type" || continue
read -r sensor_temp <"$sensor_node/temp" || continue
printf '%s: %s\n' "$sensor_type" "$sensor_temp"
done
...which lets you avoid the command substitution overhead and the overhead of cat. However, read reads content only one byte at a time; so while you're not paying that overhead, it's still relatively slow.
If you switch from /bin/sh to bash, you get a faster alternative:
for sensor_node in /sys/class/thermal/thermal_zone*; do
printf '%s: %s\n' "$(<"$sensor_node/type)" "$(<"$sensor_node/temp")"
done
...as $(<file) doesn't need to do the one-byte-at-a-time reads that read does. That's only faster for being bash, though; it doesn't mean it's actually fast. There's a reason modern production monitoring systems are typically written in Go or with a JavaScript runtime like Node.
My MacBook spontaneously wakes up from sleep mode with high fan activity.
I want to do a investigate this in RTC or power settings? Or by strace-ing of processes, etc (using some process/kernel magic!).
Hint: It is probably managed by "rtcwake".
I am not even sure if this is a scheduled task, or from a WiFi wakeup, or something else.
I don't want guesses about what usually causes this in Mojave, etc. Instead:
I need to do a systematic investigation on this on my MacOS (Mojave). Linux-related answers are also appreciated.
This is about system standby, sleep-mode, suspended mode. (Note that this is not about standup and wakeup of individual processes. The whole laptop turns on spontaneously.)
Reading the log file is the best way to debug the problem.
So, try this command in your Terminal to fetch the system logs,
this will tell you "wake up" history.
log show --style syslog | fgrep "Wake reason: EC.LidOpen"
To see the wake reason:
For macOS Sierra, Mojave, Catalina, and newer
log show |grep -i "Wake reason"
Or for MacOS El Capitan, Yosemite, Mavericks, and older
syslog |grep -i "Wake reason"
This will look like:
MacBookPro kernel[0] : Wake reason = OHC1
MacBookPro kernel[0] : Wake reason = PWRB
MacBookPro kernel[0] : Wake reason = EHC2
MacBookPro kernel[0] : Wake reason = OHC1
So what do these wake reason codes mean?
OHC: stands for Open Host Controller, is usually USB or Firewire. If you see OHC1 or OHC2 it is almost certainly an external USB keyboard or mouse that has woken up the machine.
EHC: standing for Enhanced Host Controller, is another USB interface, but can also be wireless devices and bluetooth since they are also on the USB bus of a Mac.
USB: a USB device woke the machine up
LID0: this is literally the lid of your MacBook or MacBook Pro when you open the lid the machine wakes up from sleep.
PWRB: PWRB stands for Power Button, which is the physical power button on your Mac
RTC: Real Time Clock Alarm, is generally from wake-on-demand services like when you schedule sleep and wake on a Mac via the Energy Saver control panel. It can also be from launchd setting, user applications, backups, and other scheduled events.
There may be some other codes (like PCI, GEGE, etc) but the above are the ones that most people will encounter in the system logs. Once you find out these codes, you can really narrow down what is causing your Mac to wake up from sleep seemingly at random.
Hope this will help :)
This answer is based on Linux, so it might not apply strictly to Mac.
To determine whether rtcwake is responsible for your MacOS wakeups, you could replace the executable (in my Ubutnu it is /usr/sbin/rtcwake) with a wrapper script that leaves a sign of rtcwake having run, e.g.
$ cd /usr/sbin/rtcwake
$ sudo mv rtcwake rtcwake_orig
and then write script /usr/sbin/rtcwake containing
#!/bin/bash
touch $HOME/rtcwake_ran
/usr/sbin/rtcwake_orig
Variants of the script would depend on your shell.
In particular, in the last line you would possibly run rtcwake in some alternative way, so as to not own the process (nohup / disown).
See https://unix.stackexchange.com/questions/152310/how-to-correctly-start-an-application-from-a-shell
To inspect possible causes of wakeup, you can check various relevant logs, at /var/log.
E.g., syslog*, acpi*.
See also https://unix.stackexchange.com/questions/83036/where-is-the-log-for-acpi-events
Do you have wakeonlan?
Here I am documenting my systematic approach. It is loosely based on, and initiated by, the answer by #vijay-rajpurohit, which is in turn based on comment by #Robert #1431720 . Note that the final result is particular to my MacOS machine, based on the logs shown below. It will be different in your MacOS.
In first attempt, I first checked the logs using: log show --style syslog | grep ... but it is taking too long. I accidentally checked /var/log/wifi.log after exploring the /var/log/ (I am also curious about /var/log/powermanagement/*.asl).
This turned out to be most useful:
cat /var/log/wifi.log|grep -i "Wake reason"
Then found this line: (note the EC. bit)
Thu Apr 23 22:41:32.359 Info: <airportd[219]> _systemWokenByWiFi: System wake reason: <EC.ARPT>, was woken by WiFi
Then googled for EC.ARPT, I found the following commands:
pmset -g log Useful stats about "Total Sleep/Wakes since boot".
pmset -g assertions This turned out to show the full answer to this question:
2020-04-24 02:23:38 +0100
Assertion status system-wide:
BackgroundTask 1
ApplePushServiceTask 0
UserIsActive 1
PreventUserIdleDisplaySleep 0
PreventSystemSleep 0
ExternalMedia 0
PreventUserIdleSystemSleep 0
NetworkClientActive 0
Listed by owning process:
pid 111(hidd): [0x0000200a000986a9] 00:00:00 UserIsActive named: "com.apple.iohideventsystem.queue.tickle.4295010950.3"
pid 85(apsd): [0x0003b830000b90bd] 00:00:10 ApplePushServiceTask named: "com.apple.apsd-waitingformessages-push.apple.com"
Kernel Assertions: 0x100=MAGICWAKE
id=504 level=255 0x100=MAGICWAKE mod=24/04/2020, 01:57 description=en0 owner=en0
Idle sleep preventers: IODisplayWrangler
In short, in a systematic approach, I explored the following keywords based on the logs, and googled each :
EC.ARPT (example link)
iohideventsystem (example link)
MAGICWAKE (example link)
ApplePushServiceTask (see below)
Most informative item emerged from the output of pmset -g assertions. For example ApplePushServiceTask in the following line:
pid 85(apsd): [0x0003b830000b90bd] 00:00:10 ApplePushServiceTask named: "com.apple.apsd-waitingformessages-push.apple.com"
The solution that seems to work in my particular case (not a general solution) was to disable :
/System/Library/LaunchDaemons/com.apple.apsd.plist using launchctl. But this cannot be done until you do a csrutil disable in the safe mode. I don't write instructions here because it need caution and you need to enable it later.
(to be updated)
I have a bash script that I plan to run every 5 or 15 mins using crontab based on the load it puts on server.
I can find time of running the script, but load, memory usage and CPU usage I am not sure how to find.
Can someone help me?
Also any suggestions of rough benchmark that will help me decide if the script puts too much load and should be run every 15 mins and not 5 mins.
Thanks in Advance!
You can use "top -b", top gives the CPU usage, memory usage etc,
Insert these lines in your script, this will process in background and will terminate the process as soon as your testing overs.
ssh server_name "nohup top -b -d 0.5 >> file_name &"
\top process will run in background because of &, -d 0.5 will give you the cpu status at every 0.5 secs, redirect the output in file_name for latter analysis.
for killing the process after your test, insert following in your script,
ssh server_name "kill \`ps -elf | grep 'top -b' | grep -v grep | sed 's/ */ /g' |cut -d ' ' -f4\`"
Your main testing script should be between top command and command for killing top.
I presumed you are running the script from client side, if not ignore "ssh server_name".
If you are running it from client side, because of "ssh", you will be asked for the password, for avoiding this follow these 3 simple steps
This will definitely solve the issue.
You can check following utilities
pidstat for CPU load, man page
pmap for memory load, man page
Although you might need to make measurements also for the child processes of your executable, in order to collect summarizing information
For memory, use free -m. Your actual memory available is the second number next to +/- buffers/cache (in megabytes with -m) (source).
For CPU, it's a bit more complicated. Start by looking at cat /proc/stat | grep 'cpu ' (note the space). You'll see something like this:
cpu 2255 34 2290 22625563 6290 127 456
The columns are from left to right, "user, nice, system, idle". CPU usage is usually calculated as (user+nice+system) / (user+nice+system+idle). However, these numbers show the number of "time units" that the CPU has spent doing that thing since boot, and thus are always increasing. If you were to do the aforementioned calculation, you'd get the CPU usage average since boot. To get a point-in-time usage, you have to take 2 samples, find their difference, and calculate the usage from that. To be clear, that will be the average CPU usage between your samples. (source)
I suppose this may be an odd question, but I have a small EC2-instance that costs quite a large sum of money every month. It's charged hourly though, so I only turn on this particular instance when I need it, and power it off when I'm done.
The purpose of this instance is for hosting a Counter-Strike: Global Offensive dedicated server which I only power on when I have a scrim to play.
Instead of forgetting to turn it off and being charged a lot, or having an unintelligent start-up script that asks the instance to power-off after 3 hours, I was thinking of a more intelligent design.
Here's my idea; that the instance intelligently powers itself off when it senses it is no longer in use, by determaining if a certain amount of network activity on UDP 27015 has not been recorded over the last 10 minutes, trying 3 times before powering off.
That way I can power-on, play the match, and not worry about powering off the server :-)
It sounds cool in my head. The question is how I go about solving the task. I imagine a bash-script executed every 10 minutes with the help of cron.
If I'm not being entirely crazy here, could a bash-script suggestion possibly be offered? Or maybe a better solution how I solve this quest I'm on, to save $$ by having the server power itself off when sensing it is no longer in use!
I'm not too familiar with EC2 instances, but if they are running some form of linux... Under Fedora I can use ifconfig to see how much data has been received/transmitted across the network interface. It's not just the single port but all ports on that interface... Would that number suffice for you? Ought to be pretty trivial to monitor it every few minutes and see when the load drops off...
Possibly a simple script to start with that is started when the EC2 instance is brought up and just logs the data. An hour after your game you can grab the log, manually shut down, and review it at your leisure to see if this will work. (It's amazing how many things use the network sometimes...)
Afterthought: Perhaps tcpdump would be better? Will it work with UDP port 27015? You might need some way to time it out, like running it as a background process, possibly with the -c option, sleeping for a while, and then killing the tcpdump process if it's still running. You may need to pipe through wc -l or just grep the final packets grabbed line. Caveat: tcpdump may need to be run as root.
E.g. /usr/sbin/tcpdump -n -nn -q -c 100 -i eth0 port 27015
Further afterthought:
#!/bin/bash --norc
/usr/sbin/tcpdump -n -nn -q -i eth0 port 27015 2>./logfile 1>/dev/null &
TCPDUMP_PID=$!
echo "sleeping... pid=$TCPDUMP_PID"
sleep 30
echo "wake up"
kill $TCPDUMP_PID
sleep 2
cat ./logfile
I'm running some JMeter tests against a Java process to determine how responsive a web application is under load (500+ users). JMeter will give the response time for each web request, and I've written a script to ping the Tomcat Manager every X seconds which will get me the current size of the JVM heap.
I'd like to collect stats on the server of the % of CPU being used by Tomcat. I tried to do it in a shell script using ps like this:
PS_RESULTS=`ps -o pcpu,pmem,nlwp -p $PID`
...running the command every X seconds and appending the results to a text file. (for anyone wondering, pmem = % mem usage and nlwp is number of threads)
However I've found that this gives a different definition of "% of CPU Utilization" than I'd like - according to the manpages for ps, pcpu is defined as:
cpu utilization of the process in "##.#" format. It is the CPU time used divided by the time the process has been running (cputime/realtime ratio), expressed as a percentage.
In other words, pcpu gives me the % CPU utilization for the process for the lifetime of the process.
Since I want to take a sample every X seconds, I'd like to be collecting the CPU utilization of the process at the current time only - similar to what top would give me
(CPU utilization of the process since the last update).
How can I collect this from within a shell script?
Use top -b (and other switches if you want different outputs). It will just dump to stdout instead of jumping into a curses window.
The most useful tool I've found for monitoring a server while performing a test such as JMeter on it is dstat. It not only gives you a range of stats from the server, it outputs to csv for easy import into a spreadsheet and lets you extend the tool with modules written in Python.
User load: top -b -n 2 |grep Cpu |tail -n 1 |awk '{print $2}' |sed 's/.[^.]*$//'
System load: top -b -n 2 |grep Cpu |tail -n 1 |awk '{print $3}' |sed 's/.[^.]*$//'
Idle load: top -b -n 1 |grep Cpu |tail -n 1 |awk '{print $5}' |sed 's/.[^.]*$//'
Every outcome is a round decimal.
Off the top of my head, I'd use the /proc filesystem view of the system state - Look at man 5 proc to see the format of the entry for /proc/PID/stat, which contains total CPU usage information, and use /proc/stat to get global system information. To obtain "current time" usage, you probably really mean "CPU used in the last N seconds"; take two samples a short distance apart to see the current rate of CPU consumption. You can then munge these values into something useful. Really though, this is probably more a Perl/Ruby/Python job than a pure shell script.
You might be able to get the rough data you're after with /proc/PID/status, which gives a Sleep average for the process. Pretty coarse data though.
also use 1 as iteration count, so you will get current snapshot without waiting to get another one in $delay time.
top -b -n 1
This will not give you a per-process metric, but the Stress Terminal UI is super useful to know how badly you're punishing your boxes. Add -c flag to make it dump the data to a CSV file.