My MacBook spontaneously wakes up from sleep mode with high fan activity.
I want to do a investigate this in RTC or power settings? Or by strace-ing of processes, etc (using some process/kernel magic!).
Hint: It is probably managed by "rtcwake".
I am not even sure if this is a scheduled task, or from a WiFi wakeup, or something else.
I don't want guesses about what usually causes this in Mojave, etc. Instead:
I need to do a systematic investigation on this on my MacOS (Mojave). Linux-related answers are also appreciated.
This is about system standby, sleep-mode, suspended mode. (Note that this is not about standup and wakeup of individual processes. The whole laptop turns on spontaneously.)
Reading the log file is the best way to debug the problem.
So, try this command in your Terminal to fetch the system logs,
this will tell you "wake up" history.
log show --style syslog | fgrep "Wake reason: EC.LidOpen"
To see the wake reason:
For macOS Sierra, Mojave, Catalina, and newer
log show |grep -i "Wake reason"
Or for MacOS El Capitan, Yosemite, Mavericks, and older
syslog |grep -i "Wake reason"
This will look like:
MacBookPro kernel[0] : Wake reason = OHC1
MacBookPro kernel[0] : Wake reason = PWRB
MacBookPro kernel[0] : Wake reason = EHC2
MacBookPro kernel[0] : Wake reason = OHC1
So what do these wake reason codes mean?
OHC: stands for Open Host Controller, is usually USB or Firewire. If you see OHC1 or OHC2 it is almost certainly an external USB keyboard or mouse that has woken up the machine.
EHC: standing for Enhanced Host Controller, is another USB interface, but can also be wireless devices and bluetooth since they are also on the USB bus of a Mac.
USB: a USB device woke the machine up
LID0: this is literally the lid of your MacBook or MacBook Pro when you open the lid the machine wakes up from sleep.
PWRB: PWRB stands for Power Button, which is the physical power button on your Mac
RTC: Real Time Clock Alarm, is generally from wake-on-demand services like when you schedule sleep and wake on a Mac via the Energy Saver control panel. It can also be from launchd setting, user applications, backups, and other scheduled events.
There may be some other codes (like PCI, GEGE, etc) but the above are the ones that most people will encounter in the system logs. Once you find out these codes, you can really narrow down what is causing your Mac to wake up from sleep seemingly at random.
Hope this will help :)
This answer is based on Linux, so it might not apply strictly to Mac.
To determine whether rtcwake is responsible for your MacOS wakeups, you could replace the executable (in my Ubutnu it is /usr/sbin/rtcwake) with a wrapper script that leaves a sign of rtcwake having run, e.g.
$ cd /usr/sbin/rtcwake
$ sudo mv rtcwake rtcwake_orig
and then write script /usr/sbin/rtcwake containing
#!/bin/bash
touch $HOME/rtcwake_ran
/usr/sbin/rtcwake_orig
Variants of the script would depend on your shell.
In particular, in the last line you would possibly run rtcwake in some alternative way, so as to not own the process (nohup / disown).
See https://unix.stackexchange.com/questions/152310/how-to-correctly-start-an-application-from-a-shell
To inspect possible causes of wakeup, you can check various relevant logs, at /var/log.
E.g., syslog*, acpi*.
See also https://unix.stackexchange.com/questions/83036/where-is-the-log-for-acpi-events
Do you have wakeonlan?
Here I am documenting my systematic approach. It is loosely based on, and initiated by, the answer by #vijay-rajpurohit, which is in turn based on comment by #Robert #1431720 . Note that the final result is particular to my MacOS machine, based on the logs shown below. It will be different in your MacOS.
In first attempt, I first checked the logs using: log show --style syslog | grep ... but it is taking too long. I accidentally checked /var/log/wifi.log after exploring the /var/log/ (I am also curious about /var/log/powermanagement/*.asl).
This turned out to be most useful:
cat /var/log/wifi.log|grep -i "Wake reason"
Then found this line: (note the EC. bit)
Thu Apr 23 22:41:32.359 Info: <airportd[219]> _systemWokenByWiFi: System wake reason: <EC.ARPT>, was woken by WiFi
Then googled for EC.ARPT, I found the following commands:
pmset -g log Useful stats about "Total Sleep/Wakes since boot".
pmset -g assertions This turned out to show the full answer to this question:
2020-04-24 02:23:38 +0100
Assertion status system-wide:
BackgroundTask 1
ApplePushServiceTask 0
UserIsActive 1
PreventUserIdleDisplaySleep 0
PreventSystemSleep 0
ExternalMedia 0
PreventUserIdleSystemSleep 0
NetworkClientActive 0
Listed by owning process:
pid 111(hidd): [0x0000200a000986a9] 00:00:00 UserIsActive named: "com.apple.iohideventsystem.queue.tickle.4295010950.3"
pid 85(apsd): [0x0003b830000b90bd] 00:00:10 ApplePushServiceTask named: "com.apple.apsd-waitingformessages-push.apple.com"
Kernel Assertions: 0x100=MAGICWAKE
id=504 level=255 0x100=MAGICWAKE mod=24/04/2020, 01:57 description=en0 owner=en0
Idle sleep preventers: IODisplayWrangler
In short, in a systematic approach, I explored the following keywords based on the logs, and googled each :
EC.ARPT (example link)
iohideventsystem (example link)
MAGICWAKE (example link)
ApplePushServiceTask (see below)
Most informative item emerged from the output of pmset -g assertions. For example ApplePushServiceTask in the following line:
pid 85(apsd): [0x0003b830000b90bd] 00:00:10 ApplePushServiceTask named: "com.apple.apsd-waitingformessages-push.apple.com"
The solution that seems to work in my particular case (not a general solution) was to disable :
/System/Library/LaunchDaemons/com.apple.apsd.plist using launchctl. But this cannot be done until you do a csrutil disable in the safe mode. I don't write instructions here because it need caution and you need to enable it later.
(to be updated)
I want to measure the time it takes to finish TCP three-way handshake. I want to measure this on my Linux server. What are best practices for this? Notice that I want to measure this latency on server side and for all connections that are being accepted.
Sorry, you're right I misunderstood the question.
I think you could achieve this using 'tcpdump' which is a really complete tool to see all the events in tcp traffic.
By your comment I see you want to measure the time between SYNC to the ACK packet.
With tcpdump you can filter the connections and specific packages:
tcpdump -r <interface> "tcp[tcpflags] & (tcp-syn|tcp-ack) != 0"
And by default the time will be displayed in the first column of tcpdump results.
Check this, I think it could help.
I don't know if it's the best practice. Also If you want to manipulate that data, you can pipe the results and use awk or something similar.
EDIT: By searching in google I also found this resource which is really interesting.
I'm working on sending large data files between two Linux computers via a 10 Gigabit Ethernet cable and netcat with a UDP transfer, but seem to be having issues.
After running several tests, I've come to the conclusion that netcat is the issue. I've tested the UDP transfer using [UDT][1], [Tsunami-UDP]2, and a Python UDT transfer as well, and all of which have not had any packet loss issues.
On the server side, we've been doing:
cat "bigfile.txt" | pv | nc -u IP PORT
then on the client side, we've been doing:
nc -u -l PORT > "outputFile.txt"
A few things that we've noticed:
On one of the computers, regardless of whether it's the client or server, it just "hangs". That is to say, even once the transfer is complete, Linux doesn't kill the process and move to the next line in the terminal.
If we run pipe view on the receiving side as well, the incoming data rate is significantly lower than what the sending side thinks it's sending.
Running Wireshark doesn't show any packet loss.
Running the system performance monitor in Linux shows that the incoming data rate (for the receiving side) is the same as the outgoing data rate from the sending side. This is in contrast to what pipe view thinks (see #2)
We're not sure where the issue is with netcat, and if there is a way around it. Any help/insights would be greatly appreciated.
Also, for what it's worth, using netcat with a TCP transfer works fine. And, I do understand that UDP isn't known for reliability, and that packet loss should be expected, but it's the protocol we must use.
Thanks
It could well be that the sending instance is sending the data too fast for the receiving instance. Note that this can occur even if you see no drops on the receiving NIC (as you seem to be saying), because the loss can occur at OS level instead. Your OS could have its UDP buffers overflowing. Run this command:
watch -d "cat /proc/net/snmp | grep -w Udp"
To see if your RcvbufErrors field is non-zero and/or growing while your file transfer is going on.
This answer (How to send only one UDP packet with netcat?) says that nc sends one packet per line. Assuming that's true, this could lead to a significantly higher number of packets than your other transfer mechanisms. Presumably, as #Smeeheey suggested, you're running out of receive buffers on the receiving end.
To cause your sending end to exit, you can add -q 1 to the command line (exit 1 second after seeing end of file).
But there's no way that the the receiving end nc can know when the transfer is complete. This is why these other mechanisms are "protocols" -- they have mechanisms built into them to communicate the bounds of a file. Raw UDP has no concept of end of file.
Tuning the Linux networking stack is a bit complicated, as there are many components to tune to figure out where data is being dropped.
If possible/feasible, I'd recommend that you start by monitoring packet drops throughout the entire network stack. Once you've done that, you can determine where exactly packets are being dropped and then adjust tuning parameters as needed. There are a lot of different files to measure with lots of different fields. I wrote a detailed blog post about monitoring and tuning each component of the Linux networking stack from top to bottom. It's a bit difficult to summarize all the information there, but take a look, I think it can help guide you.
I've written a Node.JS server which I would like to benchmark. It has the following components that I would like to benchmark separately:
- socket.io: how many continuous connections can it accept and process (where is the saturation point)
- redis: the same as above
- express: don't want to benchmark it
I know there is quite some (not a lot) documentation about that on the internet, but I don't like to reinvent the wheel, plus I don't want to actually spend countless hours of time trying some solution that turns out to be wrong for the job.
This is why I'm asking you guys here: what should I use to get a number/graph (whatever) on the number of simultaneous connections that server can process simultaneuosly without being bogged down? It would also be nice to monitor cpu, memory and swap of the process (yeah, yeah I know I can use countless of techniques or write my own script, but maybe something like that exists already).
I'm not looking for an answer where you'll paste a link to some solution that I already know it exists, I would like an answer in such a way, so that the person giving it has some actual experience and can really make a point or two and point me in the right direction.
Thank you
You can use ApacheBench ab to test the load that your server may take - man page
Some nice tutorials :
nixcraft/howto-performance-benchmarks-a-web-server
petefreitag/Using Apache Bench for Simple Load Testing
Usage :
$ ab -k -n 1000 -c 100 www.yourserver.com
-k - keep alive
-n N - will send N requests to the server
-c X - will send X packets concurrently
I need to test a serial port application on Linux, however, my test machine only has one serial port.
Is there a way to add a virtual serial port to Linux and test my application by emulating a device through a shell or script?
Note: I cannot remap the port, it hard coded on ttys2 and I need to test the application as it is written.
Complementing the #slonik's answer.
You can test socat to create Virtual Serial Port doing the following procedure (tested on Ubuntu 12.04):
Open a terminal (let's call it Terminal 0) and execute it:
socat -d -d pty,raw,echo=0 pty,raw,echo=0
The code above returns:
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/2
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/3
2013/11/01 13:47:27 socat[2506] N starting data transfer loop with FDs [3,3] and [5,5]
Open another terminal and write (Terminal 1):
cat < /dev/pts/2
this command's port name can be changed according to the pc. it's depends on the previous output.
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/**2**
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/**3**
2013/11/01 13:47:27 socat[2506] N starting data transfer loop with FDs
you should use the number available on highlighted area.
Open another terminal and write (Terminal 2):
echo "Test" > /dev/pts/3
Now back to Terminal 1 and you'll see the string "Test".
You can use a pty ("pseudo-teletype", where a serial port is a "real teletype") for this. From one end, open /dev/ptyp5, and then attach your program to /dev/ttyp5; ttyp5 will act just like a serial port, but will send/receive everything it does via /dev/ptyp5.
If you really need it to talk to a file called /dev/ttys2, then simply move your old /dev/ttys2 out of the way and make a symlink from ptyp5 to ttys2.
Of course you can use some number other than ptyp5. Perhaps pick one with a high number to avoid duplicates, since all your login terminals will also be using ptys.
Wikipedia has more about ptys: http://en.wikipedia.org/wiki/Pseudo_terminal
Use socat for this:
For example:
socat PTY,link=/dev/ttyS10 PTY,link=/dev/ttyS11
There is also tty0tty http://sourceforge.net/projects/tty0tty/ which is a real null modem emulator for linux.
It is a simple kernel module - a small source file. I don't know why it only got thumbs down on sourceforge, but it works well for me. The best thing about it is that is also emulates the hardware pins (RTC/CTS DSR/DTR). It even implements TIOCMGET/TIOCMSET and TIOCMIWAIT iotcl commands!
On a recent kernel you may get compilation errors. This is easy to fix. Just insert a few lines at the top of the module/tty0tty.c source (after the includes):
#ifndef init_MUTEX
#define init_MUTEX(x) sema_init((x),1)
#endif
When the module is loaded, it creates 4 pairs of serial ports. The devices are /dev/tnt0 to /dev/tnt7 where tnt0 is connected to tnt1, tnt2 is connected to tnt3, etc.
You may need to fix the file permissions to be able to use the devices.
edit:
I guess I was a little quick with my enthusiasm. While the driver looks promising, it seems unstable. I don't know for sure but I think it crashed a machine in the office I was working on from home. I can't check until I'm back in the office on monday.
The second thing is that TIOCMIWAIT does not work. The code seems to be copied from some "tiny tty" example code. The handling of TIOCMIWAIT seems in place, but it never wakes up because the corresponding call to wake_up_interruptible() is missing.
edit:
The crash in the office really was the driver's fault. There was an initialization missing, and the completely untested TIOCMIWAIT code caused a crash of the machine.
I spent yesterday and today rewriting the driver. There were a lot of issues, but now it works well for me. There's still code missing for hardware flow control managed by the driver, but I don't need it because I'll be managing the pins myself using TIOCMGET/TIOCMSET/TIOCMIWAIT from user mode code.
If anyone is interested in my version of the code, send me a message and I'll send it to you.
You may want to look at Tibbo VSPDL for creating a linux virtual serial port using a Kernel driver -- it seems pretty new, and is available for download right now (beta version). Not sure about the license at this point, or whether they want to make it available commercially only in the future.
There are other commercial alternatives, such as http://www.ttyredirector.com/.
In Open Source, Remserial (GPL) may also do what you want, using Unix PTY's. It transmits the serial data in "raw form" to a network socket; STTY-like setup of terminal parameters must be done when creating the port, changing them later like described in RFC 2217 does not seem to be supported. You should be able to run two remserial instances to create a virtual nullmodem like com0com, except that you'll need to set up port speed etc in advance.
Socat (also GPL) is like an extended variant of Remserial with many many more options, including a "PTY" method for redirecting the PTY to something else, which can be another instance of Socat. For Unit tets, socat is likely nicer than remserial because you can directly cat files into the PTY. See the PTY example on the manpage. A patch exists under "contrib" to provide RFC2217 support for negotiating serial line settings.
Using the links posted in the previous answers, I coded a little example in C++ using a Virtual Serial Port. I pushed the code into GitHub: https://github.com/cymait/virtual-serial-port-example .
The code is pretty self explanatory. First, you create the master process by running ./main master and it will print to stderr the device is using. After that, you invoke ./main slave device, where device is the device printed in the first command.
And that's it. You have a bidirectional link between the two process.
Using this example you can test you the application by sending all kind of data, and see if it works correctly.
Also, you can always symlink the device, so you don't need to re-compile the application you are testing.
Would you be able to use a USB->RS232 adapter? I have a few, and they just use the FTDI driver. Then, you should be able to rename /dev/ttyUSB0 (or whatever gets created) as /dev/ttyS2 .
I can think of three options:
Implement RFC 2217
RFC 2217 covers a com port to TCP/IP standard that allows a client on one system to emulate a serial port to the local programs, while transparently sending and receiving data and control signals to a server on another system which actually has the serial port. Here's a high-level overview.
What you would do is find or implement a client com port driver that would implement the client side of the system on your PC - appearing to be a real serial port but in reality shuttling everything to a server. You might be able to get this driver for free from Digi, Lantronix, etc in support of their real standalone serial port servers.
You would then implement the server side of the connection locally in another program - allowing the client to connect and issuing the data and control commands as needed.
It's probably non trivial, but the RFC is out there, and you might be able to find an open source project that implements one or both sides of the connection.
Modify the linux serial port driver
Alternately, the serial port driver source for Linux is readily available. Take that, gut the hardware control pieces, and have that one driver run two /dev/ttySx ports, as a simple loopback. Then connect your real program to the ttyS2 and your simulator to the other ttySx.
Use two USB<-->Serial cables in a loopback
But the easiest thing to do right now? Spend $40 on two serial port USB devices, wire them together (null modem) and actually have two real serial ports - one for the program you're testing, one for your simulator.
-Adam
$ socat -d -d pty,link=/tmp/vserial1,raw,echo=0 pty,link=/tmp/vserial2,raw,echo=0
Will generate symlinks of /tmp/vserial1 and /tmp/vserial2 for generated virtual serial ports in /dev/pts/*
Resource
Combining all other amazingly useful answers, I found the below command to be VERY useful for testing on different types of Linux distros where there's no guarantee you're going to get the same /dev/pts/#'s every time and/or you need to test multiple psuedo serial devices and connections at once.
parallel 'i="{1}"; socat -d -d pty,raw,echo=0,link=$HOME/pty{1} pty,raw,echo=0,link=$HOME/pty$(($i+1))' ::: $(seq 0 2 3;)
Breaking this down:
parallel runs the same command for each argument supplied to it.
So for example if we run it with the --dryrun flag it gives us:
i="0"; socat -d -d pty,raw,echo=0,link=$HOME/pty0 pty,raw,echo=0,link=$HOME/pty$(($i+1))
i="2"; socat -d -d pty,raw,echo=0,link=$HOME/pty2 pty,raw,echo=0,link=$HOME/pty$(($i+1))
This is due to the $(seq x y z;) at the end, where
x = start #, y = increment by, and z = end # (or # of devices you need to spawn)
parallel 'i="{1}"; echo "make psuedo_devices {1} $(($i+1))"' ::: $(seq 0 2 3;)
Outputs:
make psuedo_devices 0 1
make psuedo_devices 2 3
Gathering all this together the final above command symlinks the proper psuedo devices together regardless of whats in /dev/pts/ to whatever directory supplied to socat via the link flag.
pstree -c -a $PROC_ID gives:
perl /usr/bin/parallel i="{1}"; socat -d -d pty,raw,echo=0,link=$HOME/pty{1} pty,raw,echo=0,link=$HOME/pty$(($i+1)) ::: 0 2
├─bash -c i="0"; socat -d -d pty,raw,echo=0,link=$HOME/pty0 pty,raw,echo=0,link=$HOME/pty$(($i+1))
│ └─socat -d -d pty,raw,echo=0,link=/home/user/pty0 pty,raw,echo=0,link=/home/user/pty1
└─bash -c i="2"; socat -d -d pty,raw,echo=0,link=$HOME/pty2 pty,raw,echo=0,link=$HOME/pty$(($i+1))
└─socat -d -d pty,raw,echo=0,link=/home/user/pty2 pty,raw,echo=0,link=/home/user/pty3
ls -l $HOME/pty* yield:
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty0 -> /dev/pts/4
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty1 -> /dev/pts/6
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty2 -> /dev/pts/7
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty3 -> /dev/pts/8
This was all because I was trying running tests against a platform where I needed to generated a lot of mach-serial connections and to test their input/output via containerization (Docker). Hopefully someone finds it useful.