I have been trying various ways of getting an RFCOMM server up and running using python's subprocess package for some time now but have hit a wall. What I want to do is start a process in the background with this RFCOMM server running and getting the usual return from the command which is something like "Connected /dev/rfcomm0 to xx:xx:xx:xx:xx:xx on channel n" and "Press CTRL-C for hangup". The thing is, the process starts with two different ways I have tried it but getting this return and letting it sit in the background is an issue for me.
Alternative 1:
ref = Popen("sudo rfcomm connect 0 xx:xx:xx:xx:xx:xx 1 -i hci0 &", stdout=PIPE, stderr=PIPE, shell=True)
The above starts the RFCOMM server but either if I use communicate() or try to read from ref.stdout/stderr the program freezes.
Alternative 2:
res = run("sudo rfcomm connect 0 xx:xx:xx:xx:xx:xx 1 -i hci0 &", stdout=PIPE, stderr=PIPE, shell=True)
Same thing here, if I try to access the CompletedProcess objects stdout/stderr fields, the program freezes and refuses to continue. The RFCOMM server starts but the script does not terminate and I cannot read the stdout/stderr fields.
Alternative 3:
res = run(["sudo", "rfcomm", "connect", "0", "xx:xx:xx:xx:xx:xx", "1", "-i", "hci0"], stdout=PIPE, stderr=PIPE)
Same thing here, even though shell is left at its default and the command is not to be run in the background, the run function does not return and the script does not terminate. The RFCOMM server starts but the script does not terminate and I cannot read the stdout/stderr fields.
So, what I want is to start this RFCOMM server in the background, read the two lines it should show, and continue with my program so that I can poll its availability through other commands.
Quite strange answer in my opinion for this one, but the answer lied in that I assigned stdout and stderr to the PIPE constant. This for some reason "binds" the script to the execution and will not let it finish. When removing these two assignments the script finishes happily and I have to find another way to get the output that I in my question listed I wanted, the "Connected ..." message that is. Seems like there is no way for me to retrieve it for now.
Script that works and starts the server in the background:
mac_addr = "xx:xx:xx:xx:xx:xx"
run("sudo rfcomm connect 0 " + mac_addr + " 1 -i hci0 &", shell=True)
So, as you can see, I just removed stdout and stderr... If someone in turn can answer why that would essentially "lock" the script I would be grateful.
Related
I want to run a game server inside my Ubuntu machine. I want to run it in the background and write the live output of that process inside a log file. I tried using nohup and running the game server using "&" at the end but I couldn't make it work the way I wanted.
Then I started reading about named pipes and actually gave it a go. I made a simple script that in theory should work. But, of course I am missing something.
First, I made a pipe using the mkfifo command.
mkfifo testpipe
Then I created a small script:
#!/bin/bash
./mta-server64 > pipe &
pid=$!
echo $pid // so I know the pid of the process
cat < pipe > log.txt &
(Note: I wrote this code from memory.)
The code works only when there is an error and the process stops. It actually records the game console error. But when the game server is running I get no output in the log file.
I want to read the output (stdout and stderr if I am not mistaken) of a process running in background and record it those inside a log file.
I also thought about using screen as it logs everything inside a file but I would prefer not using it if there is a better solution.
EDIT:
First of all: thank you for the interest you had in helping me. In the same way, I have to apologize for only giving scarce details about what I intend to do with this small project and for my limited understanding of stdout and stderr.
Let's go to the first base.
I want to run a game server named Multi Theft Auto (https://multitheftauto.com/). This is GTA San Andreas but multiplayer.
I can easily run this game server in my Ubuntu server by calling the executable ./mta-server-64. After calling it the game server console appears:
[|] MTA: San Andreas :: 0/32 players :: 196 resources :: 125 fps (25)
MTA:BLUE Server for MTA:SA
==================================================================
= Multi Theft Auto: San Andreas v1.5.6 [64 bit]
==================================================================
= Server name : Default MTA Server
= Server IP address: auto
= Server port : 22884
=
= Log file : /root/mta/mods/deathmatch/logs/server.log
= Maximum players : 32
= HTTP port : 22564
= Voice Chat : Disabled
= Bandwidth saving : Medium
==================================================================
[09:49:07] Resource 'mapmanager' requests some acl rights. Use the command 'aclrequest list mapmanager'
[09:49:07] Resources: 196 loaded, 0 failed
[09:49:07] Starting resources...
[09:49:07] Server minclientversion is now 1.5.6-9.16588.0
[09:49:07] INFO: MAPMANAGER: Some important ACL permissions are missing. To ensure the correct functioning of Mapmanager, please write: aclrequest allow mapmanager all
[09:49:07] Gamemode 'play' started.
[09:49:07] Authorized serial account protection is enabled for the ACL group(s): `Admin` See http://mtasa.com/authserial
[09:49:07] WARNING: <owner_email_address> not set
[09:49:07] Server started and is ready to accept connections!
[09:49:07] To stop the server, type 'shutdown' or press Ctrl-C
[09:49:07] Type 'help' for a list of commands.
[09:49:07] Querying MTA master server... success! (Auto detected IP:xxx.xxx.xxx.xxx)
I am using the following script to run the process in the background and (try to) get the live output from:
#!/bin/bash
newport=$(shuf -i 22003-22900 -n 1)
newip=$(shuf -i 22003-22900 -n 1)
rm -rf ~/server/*
cp -r /home/user*/ftp/server/mtaserver/serverfiles/* ~/server
sed -i "s/<httpport>[0-9][0-9][0-9][0-9][0-9]<\/httpport>/<httpport>$newport<\/httpport>/g" ~/server/mods/deathmatch/mtaserver.conf
sed -i "s/<serverport>[0-9][0-9][0-9][0-9][0-9]<\/serverport>/<serverport>$newip<\/serverport>/g" ~/server/mods/deathmatch/mtaserver.conf
~/server/mta-server64 2>&1 | tee -a outfile &
mta_pid=$!
echo $mta_pid
sleep 6
pkill $mta_pid
(Note: Because of some technical problems I had to add the first few lines of script which automatically replace the game files with new ones and also replace the existing ports with random ones.)
This script starts the server and tries to log the output of the process. The process is automatically killed after few seconds so there is only one instance of the game server at any given time.
THE ISSUE:
This script only logs the output if there is an error. I still cannot get the live output of the process when it is still running. Maybe this is an issue with the game server but truly believe there should be a way to make it work the way I intend.
I believe you want to use tee command to split the pipe output to log file.
I suggest you read this article and these answers 1 2.
Usually this is enough nohup somecommand > somecommand.log 2>&1 & then, tail -F somecommand.log to follow the logs.
After 2 days I finally figured out a way to make it work (the way I intended to work, without taking in consideration any major security/performance risks).
Reading the comments made me realize I was attacking the wrong point. The stdout of the game server is buffered, thus making it impossible to log it into a log file using the methods I tried when I posted my question At least this is what I came to understand).
I did some research on how to run the application without having the stdout buffered: https://serverfault.com/questions/294218/is-there-a-way-to-redirect-output-to-a-file-without-buffering-on-unix-linux
My code now:
stdbuf -o0 ~/server/mta-server64 >> pipe &
cat < pipe | tee -a outfile &
After creating the named pipe it executes the game server inside that pipe and then appends the stdout into the log file.
The stdbug -o0 command disables the stdout buffering (as noted in the link above).
This works for me and I cannot guarantee it will work for anybody else. I am still not aware if disabling the buffering is a safe approach to my issue but for now it is what I need.
I have a host machine that gets rebooted or reconnected quite a few times.
I want to have a script running on my dev machine that continuously tries to log into that machine and if successful runs a specific command (tailing the log data).
Edit: To clarify, the connection needs to stay open. The log command keeps tailing until I stop it manually.
What I have so far
#!/bin/bash
IP=192.168.178.1
if (("$#" >= 1))
then
IP=$1
fi
LOOP=1
trap 'echo "stopping"; LOOP=0' INT
while (( $LOOP==1 ))
do
if ping -c1 $IP
then
echo "Host $IP reached"
sshpass -p 'password' ssh -o ConnectTimeout=10 -q user#$IP '<command would go here>'
else
echo "Host $IP unreachable"
fi
sleep 1
done
The LOOP flag is not really used. The script is ended via CTRL-C.
Now this works if I do NOT add a command to be executed after the ssh and instead start the log output manually. On a disconnect the script keeps probing the connection and logs back in once the host is available again.
Also when I disconnect from the host (CTRL-D) the script will log right back into the host if CTRL-C is not pressed fast enough.
When I add a command to be executed after ssh the loop is broken. So pressing (CTRL-C) does not only stop the log but also disconnects and ends the script on the dev machine.
I guess I have to spawn another shell somewhere or something like that?
1) I want the script to keep probing, log in and run a command completely automatically and fall back to probing when the connection breaks.
2) I want to be able to stop the log on the host (CTRL-C) and thereby fall back to a logged in ssh connection to use it manually.
How do I fix this?
Maybe best approach on "fixing" would be fixing requirements.
The problematic part is number "2)".
The problem is from how SIGINT works.
When triggered, it is sent to the current control group related to your terminal. Mostly this is the shell and any process started from there. With more modern shells (you seem to use bash), the shell manages control groups such that programs started in the background are disconnected (by having been assigned a different control group).
In your case the ssh is started in the foreground (from a script executed in the foreground), so it will receive the interrupt, forward it to the remote and terminate as soon as the remote end terminated. As by that time the script shell has processed its signal handler (specified by trap) it is going to exit the loop and terminate itself.
So, as you can see, you have overloaded CTRL-C to mean two things:
terminate the monitoring script
terminate the remote command and continue with whatever is specified for the remote side.
You might get closer to what you want if you drop the first effect (or at least make it more explicit). Then, calling a script on the remote side that does not terminate itself but just the tail command, will be step. In that case you will likely need to use -t switch on ssh to get a terminal allocated for allowing normal shell operation later.
This, will not allow for terminating the remote side with just CTRL-C. You always will need to exit the remote shell that is going to be run.
The essence of such a remote script might look like:
tail command
shell
of course you would need to add whatever parts will be necessary for your shell or coding style.
An alternate approach would be to keep the current remote command being terminated and add another ssh call for the case of being interrupted that is spanning the shell for interactive use. But in that case, also `CTRL-C will not be available for terminating the minoring altogether.
To achieve this you might try changing active interrupt handler with your monitoring script to trigger termination as soon as the remote side returns. However, this will cause a race condition between the user being able to recognize remote command terminated (and control has been returned to local script) and the proper interrupt handler being in place. You might be able to sufficiently lower that risk be first activating the new trap handler and then echoing the fact and maybe add a sleep to allow the user to react.
Not really sure what you are saying.
Also, you should disable PasswordAuthentication in /etc/ssh/sshd_config and log by adding the public key of your home computer to `~/.ssh/authorized_keys
! /bin/sh
while [ true ];
do
RESPONSE=`ssh -i /home/user/.ssh/id_host user#$IP 'tail /home/user/log.txt'`
echo $RESPONSE
sleep 10
done
In Linux I used to use "hidd --connect mmac" to connect with BT devices but that is now gone since Bluez5.
I can use bluetoothctl to make the connection manually but I need to use these commands from my app and using bluetoothctl would be difficult.
What are the hcitool equivalent commands to do what bluetoothctl does?
For example, I would type in bluetoothctl:
select <cmac>
scan on
trust <mmac>
pairable on
pair <mmac>
connect <mmac>
I can use "hcitool scan" for the scanning but I haven't figured out connecting.
I've tried using "hcitool cc mmac" followed by "hcitool auth mmac" but nothing works.
Or can hcitool do what bluetoothctl does?
I am using bluetoothctl from scripts like this:
#!/bin/bash
bluetoothctl << EOF
power on
EOF
And it is possible to specify multiple commands as one command per line.
Strangely enough, it does not work like this for me:
echo "power on" | bluetoothctl
(I am using bluez-5.21-r1 - not sure whether this is version dependent)
You can pass commands to bluetoothctl like this:
echo -e 'power on\nquit' | bluetoothctl
You can even use tab to autocomplete:
echo -e 'power on\nconnect \t \nquit' | bluetoothctl
I am not adding this as a comment on Jiri's answer so it is more visible.
Another solution (the best in my opinion) would be to use expect TCL scripting with bluetoothctl.
I use it to automatically connect to bluetooth devices using bluetoothctl without having to interact with it.
For example to connect to a device identified by its MAC address
#!/usr/bin/expect -f
set address [lindex $argv 0]
set prompt "#"
log_user 0
spawn bluetoothctl
expect $prompt
send -- "remove $address\r"
expect $prompt
send -- "scan on\r"
expect "Discovery started"
sleep 10
send -- "scan off\r"
expect "Discovery stopped"
expect $prompt
send -- "trust $address\r"
expect "trust succeeded"
expect $prompt
send -- "pair $address\r"
expect "Pairing successful"
expect "Device $address Connected: no"
expect $prompt
send -- "connect $address\r"
expect "Connection successful"
expect $prompt
send "quit\r"
expect "eof"
You can launch this script as it ./myExpectScript <MAC_addr>
If you want to see the output just set the log_user value to 1
I solved this using tmux, i.e.:
Install tmux:
apt install tmux
Create Session:
tmux new-session -d -s ServerFault 'sudo bluetoothctl -a |& tee /run/shm/BLUETOOTH_OUTPUT'
Then you can issue commands like:
tmux send-keys -t ServerFault "pair AC:22:0B:9F:0C:D6" Enter
I wrote a python3 script to auto-connect my gamepads on my game cabinet. You have to run it for each device you want to connect, but no user interaction is needed. It uses the expect python module, similar to the above answers, to communicate with bluetoothctl. I found it a little easier to use than the expect/tcl scripts. If python can't find pexpect, you would need to install python3-pexpect.
sudo apt install python3-pexpect
You'll want to change the mylist list variable to search for the MACs that match the first 3 bytes (the vendor part) of your bluetooth devices. So, for example, if the first 3 bytes of the MACs on your devices start with AA:BB:CC:, then change the EF\:17\:D8\: part to AA\:BB\:CC\:
You can add as many devices you want to scan for in the mylist variable. My example searches for two different vendors, one starting with EF\:17\:D8\:, and one starting with 16\:04\:18\: The script will reject all other bluetooth devices that may be transmitting, and only connect the gamepad MACs you've configured in the mylist variable.
mylist = ['E4\:17\:D8\:[0-9A-F].[:][0-9A-F].[:][0-9A-F].', '16\:04\:18\:[0-9A-F].[:][0-9A-F].[:][0-9A-F].',pexpect.EOF]
Here is the python3 script:
#!/usr/bin/python3
import os,sys,time,pexpect
def findaddress():
address=''
p = pexpect.spawn('hcitool scan', encoding='utf-8')
p.logfile_read = sys.stdout
mylist = ['E4\:17\:D8\:[0-9A-F].[:][0-9A-F].[:][0-9A-F].', '16\:04\:18\:[0-9A-F].[:][0-9A-F].[:][0-9A-F].',pexpect.EOF]
p.expect(mylist)
address=p.after
if address==pexpect.EOF:
return ''
else:
return address
def setbt(address):
response=''
p = pexpect.spawn('bluetoothctl', encoding='utf-8')
p.logfile_read = sys.stdout
p.expect('#')
p.sendline("remove "+address)
p.expect("#")
p.sendline("scan on")
mylist = ["Discovery started","Failed to start discovery","Device "+address+" not available","Failed to connect","Connection successful"]
while response != "Connection successful":
p.expect(mylist)
response=p.after
p.sendline("connect "+address)
time.sleep(1)
p.sendline("quit")
p.close()
#time.sleep(1)
return
address=''
while address=='':
address=findaddress()
time.sleep(1)
print (address," found")
setbt(address)
I wrote another python3 script that wraps the entire process in a Vte and shows the process as it is happening, and lets you to exit it, if needed. If you want to see that, just let me know.
You can give commands as arguments directly to bluetoothctl from the shell, without needing expect scripts.
I use this in a Bash script in Ubuntu 20.04 :
mac="90:03:B7:17:00:08"
# turn on bluetooth in case it's off
rfkill unblock bluetooth
bluetoothctl power on
bluetoothctl connect $mac
To disconnect, use
bluetoothctl disconnect
This assumes the destination $mac is already paired of course. If it isn't, you can first do
bluetoothctl pair $mac
To list all available commands:
bluetoothctl help
I'm running ffmpeg on another machine for screen capture. I'd like to be able to stop it recording remotely. FFMPEG requires that q is pressed to stop encoding as it has to do some finalization to finish the file cleanly. I know I could kill it with kill/killall however this can lead to corrupt videos.
Press [q] to stop encoding
I can't find anything on google specifically for this, but some there is suggestion that echoing into /proc//fd/0 will work.
I've tried this but it does not stop ffmpeg. The q is however shown in the terminal in which ffmpeg is running.
echo -n q > /proc/16837/fd/0
So how can I send a character to another existing process in such a way it is as if it were typed locally? Or is there another way of remotely stopping ffmpeg cleanly.
Here's a neat trick I discovered when I was faced with this problem: Make an empty file (it doesn't have to be a named pipe or anything), then write 'q' to it when it's time to stop recording.
$ touch stop
$ <./stop ffmpeg -i ... output.ext >/dev/null 2>>Capture.log &
$ wait for stopping time
$ echo 'q' > stop
FFmpeg stops as though it got 'q' from the terminal STDIN.
Newer versions of ffmpeg don't use 'q' anymore, at least on Ubuntu Oneiric, instead they say to press Ctrl+C to stop them. So with a newer version you can simply use 'killall -INT' to send them SIGINT instead of SIGTERM, and they should exit cleanly.
Elaborating on the answer from sashoalm, i have tested both scenarios, and here are the results:
My experiments shows that doing
killall --user $USER --ignore-case --signal INT ffmpeg
Produces the following on the console where ffmpeg was running
Exiting normally, received signal 2.
While doing
killall --user $USER --ignore-case --signal SIGTERM ffmpeg
Produces
Exiting normally, received signal 15.
So it looks that ffmpeg is fine with both signals.
System: Debian GNU/Linux 9 (stretch), 2020-02-28
You can also try to use "expect" to automate the execution and stop of the program. You would have to start it using some virtual shell like screen, tmux or byobu and then start the ffmpeg inside of it. This way you would be able to get again the virtual shell screen and give the "q" option.
Locally or remotely start a virtual shell session, lets say with "screen". Name the session with -S option, like screen -S recvideo Then you can start the ffmpeg as you like. You can, optionally, detach from this session with a Ctrl+a + d.
Connect to the machine where the ffmpeg is running inside the screen (or tmux or whatever) and reconnect to it: screen -d -RR recvideo and then send the "q"
To do that from inside a script you can then use expect, like:
prompt="> "
expect << EOF
set timeout 20
spawn screen -S recvideo
expect "$prompt"
send -- "ffmpeg xxxxx\r"
set timeout 1
expect eof
EOF
Then, in another moment or script point or in another script you recover it:
expect << EOF
set timeout 30
spawn screen -d -RR recvideo
expect "$prompt"
send -- "q"
expect "$prompt"
send -- "exit\r"
expect eof
EOF
You can also automate the whole ssh session with expect, passing a sequence of commands and "expects" to do what you want.
The question has already been answered for Linux, but it came up when I was looking for the windows equivalent, so I'm gonna add that to the answers:
On powershell, you start the process like this:
$((Start-Process ffmpeg -passthru -argument "FFMPEG_ARGS").ID)
This sends back the PID of the FFMPEG process that you can store in a variable, or echo, and then you send the windows equivalent of sigint (Ctrl + C) using taskkill
taskkill /pid FFMPEG_PID
I tried with Stop-Process (which is what comes up when looking how to do this on Google) but it actually kills the process. (And yes, taskkill doesn't kill it, it gently asks the process to stop... good naming :D)
I have a bash script that acts as the default shell for a user loging in trough ssh.
It provides a menu with several options one of wich is sending a file using netcat.
The netcat of the embedded linux I'm using lacks the -w option, so if the user closes the ssh connection without ever sending the file, the netcat command waits forever.
I need to know if the user abruptly closes the connection so the script can kill the netcat command and exit gracefully.
Things I've tried so far:
Trapping the SIGHUP: it is not issued. The only signal issued i could find is SIGCONT, but I don't think it's reliable and portable.
Playing with the -t option of the read command to detect a closed stdin: this would work if not for a silly bug in the embedded read command (only times out on the first invocation)
Edit:
I'll try to answer the questions in the comments and explain the situation further.
The code I have is:
nc -l -p 7576 > /dev/null 2>> $LOGFILE < $TMP_DIR/$BACKUP_FILE &
wait
I'm ignoring SIGINT and SIGTSTP, but I've tried to trap all the signals and the only one received is SIGCONT.
Reading the bash man page I've found out that the SIGHUP should be sent to both script and netcat and that the SIGCONT is sent to stopped jobs to ensure they receive the SIGHUP.
I guess the wait makes the script count as stopped and so it receives the SIGCONT but at the same time the wait somehow eats up the SIGHUP.
So I've tried changing the wait for a sleep and then both SIGHUP and SIGCONT are received.
The question is: why is the wait blocking the SIGHUP?
Edit 2: Solved
I solved it polling for a closed stdin with the read builtin using the -t option. To work around the bug in the read builtin I spawn it in a new bash (bash -c "read -t 3 dummy").
Does the Parent PiD change? If so you could look up the parent in the process list and make sure the process name is correct.
I have written similar applications. It would be helpful to have more of the code in your shell. I think there may be a way of writing your overall program differently which would address this issue.