Testing the simple example that should execute the external application and catch both inputs there.
# external_application.py
print('First_line:')
x = input()
print('Second_line:')
y = input()
print(f'Done: {x} and {y}')
And the runner:
# runner.py
import subprocess
import pexpect.fdpexpect
p = subprocess.Popen(("python", "external_application.py"), stdin=subprocess.PIPE, stdout=subprocess.PIPE)
session = pexpect.fdpexpect.fdspawn(p.stdout.fileno())
session.expect("First_line:")
p.stdin.write(b'1')
print('first_done')
session.expect("Second_line:")
p.stdin.write(b'2')
print('second_done')
p.stdin.close()
print(f'Result: {p.stdout.read()}')
I can see that output of the print('first_done') and then it locks on the second expect. If we remove the second input and a second expect then everything works correctly till the end.
Running on the windows, python 3.7.9, pexpect 4.8.0
Am I missing some timeout or a flush?
When I check the dump, session matches b'First_line:\r\n', but p.stdin.write doesn't move the pexpect cursor forward to look for "Second_line:", as what happens with send() or sendline().
May I suggest a simpler way, using PopenSpawn? The docs state that it "provides an interface like pexpect.spawn interface (sic) using subprocess.Popen":
NOTE - Tested in Windows 11, using Python 3.9.
import pexpect
from pexpect import popen_spawn
session = pexpect.popen_spawn.PopenSpawn("python external_application.py")
session.expect("First_line:")
session.sendline("1")
print("first_done")
session.expect("Second_line:")
session.sendline("2")
print("second_done")
print(session.read().decode("utf-8"))
Output:
first_done
second_done
Done: 1 and 2
Related
After upgrading from python 3.8.0 to python 3.9.1, the tremc front-end of transmission bitTorrent client is throwing decodestrings is not an attribute of base64 error whenever i click on a torrent entry to check the details.
My system specs:
OS: Arch linux
kernel: 5.6.11-clear-linux
base64.encodestring() and base64.decodestring(), aliases deprecated since Python 3.1, have been removed.
use base64.encodebytes() and base64.decodebytes()
So i went to the site-packages directory and with ripgrep tried searching for the decodestring string.
rg decodestring
paramiko/py3compat.py
39: decodebytes = base64.decodestring
Upon examining the py3compat.py file,i found this block:
PY2 = sys.version_info[0] < 3
if PY2:
string_types = basestring # NOQA
text_type = unicode # NOQA
bytes_types = str
bytes = str
integer_types = (int, long) # NOQA
long = long # NOQA
input = raw_input # NOQA
decodebytes = base64.decodestring
encodebytes = base64.encodestring
So decodebytes have replaced(aliased) decodestring attribute of base64 for python version >= 3
This must be a new addendum because tremc was working fine in uptil version 3.8.*.
Opened tremc script, found the erring line (line 441), just replaced the attribute decodestring with decodebytes.A quick fix till the next update.
PS: Checked the github repository, and there's a pull request for it in waiting.
If you don't want to wait for the next release and also don't want to hack the way i did, you can get it freshly build from the repository, though that would be not much of a difference than my method
I have a script that opens a subprocess, this process awaits multiple inputs.
I tried with subprocess.Popen() but when combining the 3 standards streams it gets stuck in deadlocks...
So I tired to use wexpect (Apparently Windows version of pexecpt) it also didn't worked.
Now I'm using Sarge and my code right now looks like this:
import os
import subprocess
import sarge
import sys
def get_env():
env = os.environ.copy()
return env
def interactive(list_command: list, instruction_dict: dict):
env = get_env()
std_out = ""
for a_number in instruction_dict:
capture_stdout = sarge.Capture(buffer_size=-1)
capture_stderr = sarge.Capture(buffer_size=-1)
child_proc = sarge.Command(list_command, stdout=capture_stdout, stderr=capture_stderr,
shell=True, env=env)
child_proc.run(input=subprocess.PIPE,
async_=True)
print(capture_stdout.readlines(timeout=1.0), capture_stderr.readlines(timeout=1.0))
if instruction_dict[a_number][0] is not None:
#if capture_stdout.expect(instruction_dict[a_number][0]) is not None or capture_stderr.expect(instruction_dict[a_number][0]) is not None:
# self.test_manager.log_event(msg="found line : " + instruction_dict[a_number][0] + "in PIPE", current_event_type=LogLevel.DEBUG, screen_only=False)
print("in exec line : ", child_proc.stdout.readline(timeout=1.0), child_proc.stderr.readlines(timeout=1.0))
else:
print("in exec line : ", child_proc.stdout.readlines(timeout=1.0), child_proc.stderr.readlines(timeout=1.0))
if instruction_dict[a_number][1] is not None:
print(f"sending \"{instruction_dict[a_number][1]}\" to process in interactive")
temp_bytes = str.encode((instruction_dict[a_number][1]+"\n"))
child_proc.stdin.write(temp_bytes)
try:
child_proc.stdin.flush() # blind flush
except:
pass
print("Last line : ", child_proc.stdout.readlines(timeout=1.0), child_proc.stderr.readlines(timeout=1.0))
child_proc.wait()
std_out += child_proc.stdout.read().decode('utf-8')
child_proc.kill() # kill the subprocess
print("output: ", std_out)
interactive(list_command=["process.exe", "-delete_datamodel", "MODELNAME"],instruction_dict={1 : (None, "y")})
Output looks like this:
[] []
in exec line : [] []
send "y" to process in interactive
Last line : [] [b'User root connected to DB MODELNAME\r\n']
output: Are you sure to delete DB MODELNAME? (y/n)\n\nDelete Data Model Ok\n
In console it looks like this:
C:\>process.exe -delete_data_model -db_name MODELNAME
Are you sure to delete DB MODELNAME ? (y/n)
y
User root connected to DB MODELNAME
Delete Data Model Ok
There are multiple problems:
I can't catch the first stdout "Are you sure to delete DB MODELNAME ? (y/n)" during the loop I only get [].
capture_stdout.expect crash but in fact it might be the same problem as the first one, if it can't read the first stdout.
lines are going in stderr instead of stdout (it might be because of the process, I don't know how to test this)
for another command I ll need to answer more than one interactive by looking of the question before answering them that way my instruction_dict will look like this
instruction_dict = {1 : ("Are you sure to recover the DB", "y"),
2 : ("Do you want to stop the service", "y"),
3 : ("Do you want to stop the DB", "y"),
4 : ("Do you want to drop and create the DB", "y")}
Is this even possible? I have search for a working exemple and didn't found one, I know I might be bad at searching but...
Sorry for the terrible syntax(and bad english), I want to know if there is a pure python solution (I have one solution working in tcl but I'm trying to get rid of it/ I didn't wrote the tcl code).
I'm trying to make a python script wich reports me the port on 0.tcp.ngrok.io is started when I run the code on terminal (after moving ngrok executable file to /usr/local/bin)
ngrok tcp 22
I get ths kind of output
ngrok by #inconshreveable (Ctrl+C to quit)
Session Status connecting
Version 2.2.4
Region United States (us)
Web Interface http://127.0.0.1:4041
Forwarding tcp://0.tcp.ngrok.io:13014 -> localhost:22
Connections ttl opn rt1 rt5 p50 p90
0 0 0.00 0.00 0.00 0.00
My first attempt is to log the subprocess stdout to a variable , but as the stdout is cyclic the stdout.read() never ends this is the code
import subprocess
ngrok = subprocess.Popen(['ngrok','tcp','22'],stdout = subprocess.PIPE)
output_text = ngrok.stdout.read() # script stops here forever
[**code for getting domain:port from output_text**]
how can I get a "snapshot" of stdout to a variable , without stoping ngrok?
Is there another way of doing this (next try would be a webscraper on localhost , but it would be nice to have this knowledge for other commands , such as "top")
thanks in advance
I had the same issue when I was working with ngrok http and all the alternatives weren't working, resulting in deadlocks and I even could print the child process response created with ngrok. Thus, reading the ngrok docs I notice that there is a way to get the ngrok public url with requests.
Adding the code below:
localhost_url = "http://localhost:4041/api/tunnels" #Url with tunnel details
tunnel_url = requests.get(localhost_url).text #Get the tunnel information
j = json.loads(tunnel_url)
tunnel_url = j['Tunnels'][0]['PublicUrl'] #Do the parsing of the get
So, tunnel_url will return what you need. Adding the imports the full code would be like this:
import subprocess
import requests
import json
ngrok = subprocess.Popen(['ngrok','tcp','22'],stdout = subprocess.PIPE)
localhost_url = "http://localhost:4041/api/tunnels" #Url with tunnel details
tunnel_url = requests.get(localhost_url).text #Get the tunnel information
j = json.loads(tunnel_url)
tunnel_url = j['Tunnels'][0]['PublicUrl'] #Do the parsing of the get
not enough reputation to comment, feel free to update or comment on #Rodolfo great answer and then delete this
Perhaps they've changed the api slightly, this is what worked for me:
(ngrok executable next to the script, serving http on port 5000 and picking the https tunneling url)
import subprocess
import requests
import json
import time
if __name__ == '__main__':
ngrok = subprocess.Popen(['./ngrok','http','5000'],
stdout=subprocess.PIPE)
time.sleep(3) # to allow the ngrok to fetch the url from the server
localhost_url = "http://localhost:4040/api/tunnels" #Url with tunnel details
tunnel_url = requests.get(localhost_url).text #Get the tunnel information
j = json.loads(tunnel_url)
tunnel_url = j['tunnels'][1]['public_url'] #Do the parsing of the get
print(tunnel_url)
This is working for me:
def open_tunnel():
process = subprocess.Popen(f'/snap/bin/ngrok tcp {PORT} --log "stdout"', shell=True, stdout=PIPE, stderr=PIPE) # You can also use a list and put shell = False
while True:
output = process.stdout.readline()
if not output and process.poll() is not None:
break
elif b'url=' in output:
output = output.decode()
output = output[output.index('url=tcp://') + 10 : -1]
return output.split(':')
I use /snap/bin/ngrok because my pycharm does not recognize path, sorry for that. You can replace it by saying only ngrok
.
I'm trying to build a scapy program that scans for Beacon Frames. Every router should send beacon frames to the air in an interval of X milliseconds so the possible hosts know the router(AP) is alive.
I'm getting nothing, the only kind of Dot11 frames I've been able to get so far is Prob Request, very rarely some data or control frames as well. I setup my wireless card to monitor mode before running the script and it supports it as well. I don't what I might be doing wrong... Here's the code :
from scapy.all import *
global list_prob
list_prob = []
def search_prob(packet1):
if (packet1.haslayer(Dot11)) and (packet1[Dot11].type == 0) and\
(packet1[Dot11].subtype == 8) : #type 4 == ProbRequest
if packet1[Dot11].addr2 not in list_prob:
if packet1[Dot11].info not in list_prob:
print('[>]AP',packet1[Dot11].addr2,'SSID',packet1[Dot11].info)
list_prob.append(packet1[Dot11].addr2)
list_prob.append(packet1[Dot11].info)
sniff(iface='wlan0mon',prn=search_prob)
Ive also tried it with Dot11Beacon instead of subtype 8 and nothing changed . I'm programming with python3.5 on Linux.
Any ideas ?
Code to constantly change channel of network interface using python :
from threading import Thread
import subprocess,shlex,time
import threading
locky = threading.Lock()
def Change_Freq_channel(channel_c):
print('Channel:',str(channel_c))
command = 'iwconfig wlan1mon channel '+str(channel_c)
command = shlex.split(command)
subprocess.Popen(command,shell=False) # To prevent shell injection attacks !
while True:
for channel_c in range(1,15):
t = Thread(target=Change_Freq_channel,args=(channel_c,))
t.daemon = True
locky.acquire()
t.start()
time.sleep(0.1)
locky.release()
Firstly, I'd like to say I just begin to learn python, And I want to execute maven command inside my python script (see the partial code below)
os.system("mvn surefire:test")
But unfortunately, sometimes this command will time out, So I wanna to know how to set a timeout threshold to control this command.
That is to say, if the executing time is beyond X seconds, the program will skip the command.
What's more, can other useful solution deal with my problem? Thanks in advance!
use the subprocess module instead. By using a list and sticking with the default shell=False, we can just kill the process when the timeout hits.
p = subprocess.Popen(['mvn', 'surfire:test'])
try:
p.wait(my_timeout)
except subprocess.TimeoutExpired:
p.kill()
Also, you can use in terminal timeout:
Do like that:
import os
os.system('timeout 5s [Type Command Here]')
Also, you can use s, m, h, d for second, min, hours, day.
You can send different signal to command. If you want to learn more, see at:
https://linuxize.com/post/timeout-command-in-linux/
Simple answer
os.system not support timeout.
you can use Python 3's subprocess instead, which support timeout parameter
such as:
yourCommand = "mvn surefire:test"
timeoutSeconds = 5
subprocess.check_output(yourCommand, shell=True, timeout=timeoutSeconds)
Detailed Explanation
in further, I have encapsulate to a function getCommandOutput for you:
def getCommandOutput(consoleCommand, consoleOutputEncoding="utf-8", timeout=2):
"""get command output from terminal
Args:
consoleCommand (str): console/terminal command string
consoleOutputEncoding (str): console output encoding, default is utf-8
timeout (int): wait max timeout for run console command
Returns:
console output (str)
Raises:
"""
# print("getCommandOutput: consoleCommand=%s" % consoleCommand)
isRunCmdOk = False
consoleOutput = ""
try:
# consoleOutputByte = subprocess.check_output(consoleCommand)
consoleOutputByte = subprocess.check_output(consoleCommand, shell=True, timeout=timeout)
# commandPartList = consoleCommand.split(" ")
# print("commandPartList=%s" % commandPartList)
# consoleOutputByte = subprocess.check_output(commandPartList)
# print("type(consoleOutputByte)=%s" % type(consoleOutputByte)) # <class 'bytes'>
# print("consoleOutputByte=%s" % consoleOutputByte) # b'640x360\n'
consoleOutput = consoleOutputByte.decode(consoleOutputEncoding) # '640x360\n'
consoleOutput = consoleOutput.strip() # '640x360'
isRunCmdOk = True
except subprocess.CalledProcessError as callProcessErr:
cmdErrStr = str(callProcessErr)
print("Error %s for run command %s" % (cmdErrStr, consoleCommand))
# print("isRunCmdOk=%s, consoleOutput=%s" % (isRunCmdOk, consoleOutput))
return isRunCmdOk, consoleOutput
demo :
isRunOk, cmdOutputStr = getCommandOutput("mvn surefire:test", timeout=5)