Related
I have a function in my code that asks the user for input:
def function_1():
...
x = input('Please provide input')
...
return something
I want to be able to run my code, and when the program eventually reaches function_1 and asks the user for input, automatically provide it with some specified input. When unittesting, I can use the mock library to simulate keyboard input as below
#mock.patch('builtins.input', side_effects=[1,2,3])
function_1()
function_1()
function_1()
This calls the function three times and provides the inputs {1, 2, 3}. I'm wondering if there is a way to do the same thing outside of unittesting.
I'm aware that I can rewrite the code, or use pipe in terminal. But I'm more curious about whether this can be solved in the manner described above.
One way is to overwrite sys.stdin:
import sys
from io import StringIO
oldstdin = sys.stdin
sys.stdin = StringIO("1\n2\n3\n")
assert input() == "1"
assert input() == "2"
assert input() == "3"
sys.stdin = oldstdin
The great thing about Python is that you can override just about any function, even built-ins.
def override():
from itertools import count
counter = count()
return lambda *args, **kwargs: next(counter)
input = override()
def x():
return input("Testing123")
print(x()) # 1
print(x()) # 2
print(x()) # 3
Though, this has to be done before your functions are called.
I'm trying to use Click to create a CLI for my Python 3 app. Basically I need the app to run continuously, waiting for user commands and executing them, and quitting if a specific command (say, "q") is entered. Couldn't find an example in Click docs or elsewhere.
An example of interactive shell would be like this:
myapp.py
> PLEASE ENTER LOGIN:
mylogin
> PLEASE ENTER PASSWORD:
mypwd
> ENTER COMMAND:
a
> Wrong command!
> USAGE: COMMAND [q|s|t|w|f] OPTIONS ARGUMENTS
> ENTER COMMAND:
f
> (output of "f" command...)
> ENTER COMMAND:
q
> QUITTING APP...
I've tried like so:
import click
quitapp = False # global flag
#click.group()
def cli():
pass
#cli.command(name='c')
#click.argument('username')
def command1(uname):
pass # do smth
# other commands...
#cli.command(name='q')
def quitapp():
global quitapp
quitapp = True
def main():
while not quitapp:
cli()
if __name__ == '__main__':
main()
But the console just runs the app once all the same.
I've actually switched to fire and managed to make a shell-like continuous function like so:
COMMAND_PROMPT = '\nCOMMAND? [w to quit] >'
CAPTCHA_PROMPT = '\tEnter captcha text (see your browser) >'
BYE_MSG = 'QUITTING APP...'
WRONG_CMD_MSG = 'Wrong command! Type "h" for help.'
EMPTY_CMD_MSG = 'Empty command!'
class MyClass:
def __init__(self):
# dict associating one-letter commands to methods of this class
self.commands = {'r': self.reset, 'q': self.query, 'l': self.limits_next, 'L': self.limits_all,
'y': self.yandex_logo, 'v': self.view_params, 'h': self.showhelp, 'c': self.sample_captcha, 'w': None}
# help (usage) strings
self.usage = '\nUSAGE:\t[{}] [value1] [value2] [--param3=value3] [--param4=value4]'.format('|'.join(sorted(self.commands.keys())))
self.usage2 = '\t' + '\n\t'.join(['{}:{}'.format(fn, self.commands[fn].__doc__) for fn in self.commands if fn != 'w'])
def run(self):
"""
Provides a continuously running commandline shell.
The one-letter commands used are listed in the commands dict.
"""
entered = ''
while True:
try:
print(COMMAND_PROMPT, end='\t')
entered = str(input())
if not entered:
print(EMPTY_CMD_MSG)
continue
e = entered[0]
if e in self.commands:
if self.commands[e] is None:
print(BYE_MSG)
break
cmds = entered.split(' ')
# invoke Fire to process command & args
fire.Fire(self.commands[e], ' '.join(cmds[1:]) if len(cmds) > 1 else '-')
else:
print(WRONG_CMD_MSG)
self.showhelp()
continue
except KeyboardInterrupt:
print(BYE_MSG)
break
except Exception:
continue
# OTHER METHODS...
if __name__ == '__main__':
fire.Fire(MyClass)
Still, I'd appreciate if someone showed how to do that with click (which appears to me to be more feature-rich than fire).
I've finally found out other libraries for interactive shells in Python: cmd2 and prompt, which are way more advanced for REPL-like shells out of the box...
There's a quick example of how to do a continuous CLI application with Click here: python click module input for each function
It only has a way of running click commands on a loop, but you can put in any custom logic you want, either in commands or the main body of the loop. Hope it helps!
Here I found click in the loop but it is error prone when we try to use different commands with different options
!Caution: This is not a perfect solution
import click
import cmd
import sys
from click import BaseCommand, UsageError
class REPL(cmd.Cmd):
def __init__(self, ctx):
cmd.Cmd.__init__(self)
self.ctx = ctx
def default(self, line):
subcommand = line.split()[0]
args = line.split()[1:]
subcommand = cli.commands.get(subcommand)
if subcommand:
try:
subcommand.parse_args(self.ctx, args)
self.ctx.forward(subcommand)
except UsageError as e:
print(e.format_message())
else:
return cmd.Cmd.default(self, line)
#click.group(invoke_without_command=True)
#click.pass_context
def cli(ctx):
if ctx.invoked_subcommand is None:
repl = REPL(ctx)
repl.cmdloop()
# Both commands has --foo but if it had different options,
# it throws error after using other command
#cli.command()
#click.option('--foo', required=True)
def a(foo):
print("a")
print(foo)
return 'banana'
#cli.command()
#click.option('--foo', required=True)
def b(foo):
print("b")
print(foo)
# Throws c() got an unexpected keyword argument 'foo' after executing above commands
#cli.command()
#click.option('--bar', required=True)
def c(bar):
print("b")
print(bar)
if __name__ == "__main__":
cli()
I'm parsing the last line of a continuously updating log file. If it matches, I want to return the match to a list and start another function using that data. I need to keep watching for new entries and parse them even while the new function continues.
I've been working this from a few different angles for about a week with varying success. I tried threading, but ran into issues getting the return value, I tried using a global var but couldn't get it working. I'm now trying asyncio, but having even more issues getting that to work.
def tail():
global match_list
f.seek(0, os.SEEK_END)
while True:
line = f.readline()
if not line:
time.sleep(0.1)
continue
yield line
def thread():
while True:
tail()
def somefun(list):
global match_list
#do things here
pass
def main():
match_list = []
f = open(r'file.txt')
thread=threading.Thread(target=thread, args=(f,))
thread.start()
while True:
if len(match_list) >= 1:
somefun(match_list)
if __name__ == '__main__':
main()
Wrote the above from memory..
I want tail() to return the line to a list that somefun() can use.
I'm having issues getting it to work, I will use threading or asyncio.. anything to get it running at this point.
In asyncio you might use two coroutines, one that reads from file, and the other that processes the file. Since they communicate using queue, they don't need the global variable. For example:
import os, asyncio
async def tail(f, queue):
f.seek(0, os.SEEK_END)
while True:
line = f.readline()
if not line:
await asyncio.sleep(0.1)
continue
await queue.put(line)
async def consume(queue):
lines = []
while True:
next_line = await queue.get()
lines.append(next_line)
# it is not clear if you want somefun to receive the next
# line or *all* lines, but it's easy to do either
somefun(next_line)
def somefun(line):
# do something with line
print(f'line: {line!r}')
async def main():
queue = asyncio.Queue()
with open('file.txt') as f:
await asyncio.gather(tail(f, queue), consume(queue))
if __name__ == '__main__':
asyncio.run(main())
# or, on Python older than 3.7:
#asyncio.get_event_loop().run_until_complete(main())
The beauty of an asyncio-based solution is that you can easily start an arbitrary number of such coroutines in parallel (e.g. you could start gather(main1(), main2()) in an outer coroutine, and run that), and have them all share the same thread.
with a few small fixes you almost run this :) (comments inside)
match_list # should be at the module scope
def tail():
# f = open(...) ???
f.seek(0, os.SEEK_END)
while True:
line = f.readline()
if not line:
time.sleep(0.1)
continue
yield line
def thread():
for line in tail():
match_list.append(line) # append line
print("thread DONE!")
def somefun(list):
#do things here
while match_list:
line = match_list.pop(0)
print(line)
def main():
match_list = []
f = open(r'file.txt')
thread=threading.Thread(target=thread, args=(f,))
thread.start()
while True:
if match_list:
somefun(match_list)
time.sleep(0.1) # <-- don't burn the CPU :)
I'm working on a bot for a competition that receives its input through sys.stdin and uses Python's print() for output. I have the following:
import sys
def main():
while True:
line = sys.stdin.readline()
parts = line.split()
if len(parts) > 0:
# do stuff
The problem is that the input comes in through a stream and using the above, blocks me from printing anything back until the stream is closed. What can I do to make this work?
By turning blocking off you can only read a character at a time. So, there is no way to get readline() to work in a non-blocking context. I assume you just want to read key presses to control the robot.
I have had no luck using select.select() on Linux and created a way with tweaking termios settings. So, this is Linux specific but works for me:
import atexit, termios
import sys, os
import time
old_settings=None
def init_any_key():
global old_settings
old_settings = termios.tcgetattr(sys.stdin)
new_settings = termios.tcgetattr(sys.stdin)
new_settings[3] = new_settings[3] & ~(termios.ECHO | termios.ICANON) # lflags
new_settings[6][termios.VMIN] = 0 # cc
new_settings[6][termios.VTIME] = 0 # cc
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, new_settings)
#atexit.register
def term_any_key():
global old_settings
if old_settings:
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_settings)
def any_key():
ch_set = []
ch = os.read(sys.stdin.fileno(), 1)
while ch is not None and len(ch) > 0:
ch_set.append( ord(ch[0]) )
ch = os.read(sys.stdin.fileno(), 1)
return ch_set
init_any_key()
while True:
key = any_key()
if key is not None:
print(key)
else:
time.sleep(0.1)
A better Windows or cross-platform answer is here: Non-blocking console input?
You can use selectors for handle I/O multiplexing:
https://docs.python.org/3/library/selectors.html
Try this out:
#! /usr/bin/python3
import sys
import fcntl
import os
import selectors
# set sys.stdin non-blocking
orig_fl = fcntl.fcntl(sys.stdin, fcntl.F_GETFL)
fcntl.fcntl(sys.stdin, fcntl.F_SETFL, orig_fl | os.O_NONBLOCK)
# function to be called when enter is pressed
def got_keyboard_data(stdin):
print('Keyboard input: {}'.format(stdin.read()))
# register event
m_selector = selectors.DefaultSelector()
m_selector.register(sys.stdin, selectors.EVENT_READ, got_keyboard_data)
while True:
sys.stdout.write('Type something and hit enter: ')
sys.stdout.flush()
for k, mask in m_selector.select():
callback = k.data
callback(k.fileobj)
The above code will hold on the line
for k, mask in m_selector.select():
until a registered event occurs, returning a selector_key instance (k) and a mask of monitored events.
In the above example we registered only one event (Enter key press):
m_selector.register(sys.stdin, selectors.EVENT_READ, got_keyboard_data)
The selector key instance is defined as follows:
abstractmethod register(fileobj, events, data=None)
Therefore, the register method sets k.data as our callback function got_keyboard_data, and calls it when the Enter key is pressed:
callback = k.data
callback(k.fileobj)
A more complete example (and hopefully more useful) would be to multiplex stdin data from user with incoming connections from network:
import selectors
import socket
import sys
import os
import fcntl
m_selector = selectors.DefaultSelector()
# set sys.stdin non-blocking
def set_input_nonblocking():
orig_fl = fcntl.fcntl(sys.stdin, fcntl.F_GETFL)
fcntl.fcntl(sys.stdin, fcntl.F_SETFL, orig_fl | os.O_NONBLOCK)
def create_socket(port, max_conn):
server_addr = ('localhost', port)
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.setblocking(False)
server.bind(server_addr)
server.listen(max_conn)
return server
def read(conn, mask):
global GO_ON
client_address = conn.getpeername()
data = conn.recv(1024)
print('Got {} from {}'.format(data, client_address))
if not data:
GO_ON = False
def accept(sock, mask):
new_conn, addr = sock.accept()
new_conn.setblocking(False)
print('Accepting connection from {}'.format(addr))
m_selector.register(new_conn, selectors.EVENT_READ, read)
def quit():
global GO_ON
print('Exiting...')
GO_ON = False
def from_keyboard(arg1, arg2):
line = arg1.read()
if line == 'quit\n':
quit()
else:
print('User input: {}'.format(line))
GO_ON = True
set_input_nonblocking()
# listen to port 10000, at most 10 connections
server = create_socket(10000, 10)
m_selector.register(server, selectors.EVENT_READ, accept)
m_selector.register(sys.stdin, selectors.EVENT_READ, from_keyboard)
while GO_ON:
sys.stdout.write('>>> ')
sys.stdout.flush()
for k, mask in m_selector.select():
callback = k.data
callback(k.fileobj, mask)
# unregister events
m_selector.unregister(sys.stdin)
# close connection
server.shutdown()
server.close()
# close select
m_selector.close()
You can test using two terminals.
first terminal:
$ python3 test.py
>>> bla
open another terminal and run:
$ nc localhost 10000
hey!
back to the first
>>> qwerqwer
Result (seen on the main terminal):
$ python3 test.py
>>> bla
User input: bla
>>> Accepting connection from ('127.0.0.1', 39598)
>>> Got b'hey!\n' from ('127.0.0.1', 39598)
>>> qwerqwer
User input: qwerqwer
>>>
#-----------------------------------------------------------------------
# Get a character from the keyboard. If Block is True wait for input,
# else return any available character or throw an exception if none is
# available. Ctrl+C isn't handled and continues to generate the usual
# SIGINT signal, but special keys like the arrows return the expected
# escape sequences.
#
# This requires:
#
# import sys, select
#
# This was tested using python 2.7 on Mac OS X. It will work on any
# Linux system, but will likely fail on Windows due to select/stdin
# limitations.
#-----------------------------------------------------------------------
def get_char(block = True):
if block or select.select([sys.stdin], [], [], 0) == ([sys.stdin], [], []):
return sys.stdin.read(1)
raise error('NoChar')
This is a posix solution, similar to the answer by swdev.
As they stated, you have to play with termios.VMIN and termios.VTIME to catch more than one char without requiring user to press Enter. Trying to only use raw mode will be a problem as special keys like arrows can mess next keypress.
Here we use tty.setcbreak() or tty.setraw() as a shortcut, but they have short internals.
import termios
import tty
import sys
import select
def get_enter_key():
fd = sys.stdin.fileno()
orig_fl = termios.tcgetattr(fd)
try:
tty.setcbreak(fd) # use tty.setraw() instead to catch ^C also
mode = termios.tcgetattr(fd)
CC = 6
mode[CC][termios.VMIN] = 0
mode[CC][termios.VTIME] = 0
termios.tcsetattr(fd, termios.TCSAFLUSH, mode)
keypress, _, _ = select.select([fd], [], [])
if keypress:
return sys.stdin.read(4095)
finally:
termios.tcsetattr(fd, termios.TCSANOW, orig_fl)
try:
while True:
print(get_enter_key())
except KeyboardInterrupt:
print('exiting')
sys.exit()
note that there are two potential timeouts you could add here:
one is adding last parameter to select.select()
another is playing with VMIN and VTIME
Might I suggest nobreak? If'n you are willing to use curses.
https://docs.python.org/3/library/curses.html#curses.window.nodelay
You should be able to get read of a stream with either
sys.stdin.read(1)
to read utf-8 decoded chars or:
sys.stdin.buffer.read(1)
to read raw chars.
I would do this if I wanted to get raw data from the stdin and do something with it in a timely manner, without reading a newline or filling up the internal buffer first. This is suitable for running programs remotely via ssh where tty is not available, see:
ssh me#host '/usr/bin/python -c "import sys; print(sys.stdin.isatty())"'
There are some other things to think about to make programs work as expected in this scenario. You need to flush the output when you're done to avoid buffering delays, and it could be easy to assume a program hasn't read the input, when you've simply not flushed the output.
stdout.write("my data")
stdout.flush()
But usually it's not the input reading that's the problem but that the terminal (or program) supplying the input stream is not handing it over when you expect, or perhaps it's not reading your output when you expect. If you have a tty to start with (see ssh check above) you can put it into raw mode with the tty module.
import sys
import termios
import tty
old = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin)
c = None
try:
c = sys.stdin.read(1)[0]
finally:
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old)
print(c)
... if using Mac/Linux. If using Windows you could use msvcrt.getch().
Use a generator - thankfully sys.stdin is already a generator!
A generator enables you to work on an infinite stream. Always when you call it, it returns the next element. In order to build a generator you need the yield keyword.
for line in sys.stdin:
print line
if a_certain_situation_happens:
break
Do not forget to place a break statement into the loop if a certain, wished situation happens.
You can find more information about generators on:
http://www.dabeaz.com/generators/index.html
http://linuxgazette.net/100/pramode.html
How could you set a variable equal to 'O' or '-' and then put that in an if statement like the one below:
if variable == 'O':
print 'hi'
how could you do that for:
import threading
from array import array
from Queue import Queue, Full
import pyaudio
CHUNK_SIZE = 1024
MIN_VOLUME = 500
BUF_MAX_SIZE = CHUNK_SIZE * 10
def main():
stopped = threading.Event()
q = Queue(maxsize=int(round(BUF_MAX_SIZE / CHUNK_SIZE)))
listen_t = threading.Thread(target=listen, args=(stopped, q))
listen_t.start()
record_t = threading.Thread(target=record, args=(stopped, q))
record_t.start()
try:
while True:
listen_t.join(0.1)
record_t.join(0.1)
except KeyboardInterrupt:
stopped.set()
listen_t.join()
record_t.join()
def record(stopped, q):
while True:
if stopped.wait(timeout=0):
break
chunk = q.get()
vol = max(chunk)
if vol >= MIN_VOLUME:
# TODO: write to file
print "O",
else:
print "-",
def listen(stopped, q):
stream = pyaudio.PyAudio().open(
format=pyaudio.paInt16,
channels=2,
rate=44100,
input=True,
frames_per_buffer=1024,
)
while True:
if stopped.wait(timeout=0):
break
try:
q.put(array('h', stream.read(CHUNK_SIZE)))
except Full:
pass # discard
if __name__ == '__main__':
main()
Could you use that so if the output is 'O' then print hi? Will somebody write the code for me because I have been trying for a little bit to write this code and I have still not been able to make the code work for me. Thank You.
In order to use if then statement in Python, first we need to declare variable value with proper syntax according to requirement and type.
s = "O"
if s == 'O':
print 'hi'