Working on python script that uses stdin and stdout. The default
behaviour is the script will write to a file, so I have this function
that is called to handle writing to a file:
def run_loop(data,name):
with open(name, 'w') as fo:
fo.write("webstat output "+t) # t is a date str created in global scope
for x in data:
fo.write(processer(x))
fo.write('\n')
I have a few places in the script where I use this function, so it
would be less convenient to have to rewrite it or write a substituent
of it to work with stdout. When I try using stdout with open,
sys.stdout says its a type _io.TextIOWrapper. I get an error when
tying to call run_loop(same_data,sys.stdout) I get this error:
TypeError: expected str, bytes or os.PathLike object, not _io.TextIOWrapper
I'm wondering if there is away to use open() with stdout? I would
guess that since stdout is file io, I should be able to write to it as
a file. Ideally there would be a way to change _io.TextIOWrapper into
an object that open() can use, but I've been looking at types of the
methods inside of sys.stdout and they don't seem to provide this
facility. I'm not sure why the fo object from open() couldn't be
set to stdout?
The behaviour I'm trying to achieve is
cat input-data.txt | myscript.py > out.txt
I've been able to get the stdin portion to work, but I'm still at a
loss for how to write to stdout with open().
Supposedly, I could forego open(), if I could keep compatibility with
the other places I call this function: data being a list and name
usually being a string of the file name where the output should be saved.
Python 3.7.5rc1
Related
import sys
data = []
for line in sys.stdin:
data.append(line)
I did not use .read() or .readline()
but this code works reading the data. It read the data line by line. Input data is separated by '\n'. data is typed by user. like 'input()' .
My Question:
Is .read() or .readline() not necessary?
I wonder why for loop works on sys.stdin and how it reads data line by line?
If you look at the documentation for sys, you'll see that sys.stdin (and sys.stdout / sys.stderr) are just file objects.
These streams are regular text files like those returned by the open() function.
The documentation about open() says:
The type of file object returned by the open() function depends on the mode. When open() is used to open a file in a text mode ('w', 'r', 'wt', 'rt', etc.), it returns a subclass of io.TextIOBase (specifically io.TextIOWrapper).
TextIOWrapper inherits from IOBase, which has an __iter__ method. This __iter__ method allows looping over lines in the file. I can't find where this is noted in the Python documentation, but it is given in the source code for IOBase.
IOBase (and its subclasses) support the iterator protocol, meaning that an IOBase object can be iterated over yielding the lines in a stream.
Consider the following function
import time
def foo():
for i in range(5):
print(f"{i}. Hello world!")
time.sleep(1)
I would like to save all these print calls in a variable without preventing them from reaching the terminal in real time. Essentially, print would output to stdout and a variable.
I have tried:
from contextlib import redirect_stdout
import io
stdout = io.StringIO()
with redirect_stdout(stdout):
foo()
stdout_content = stdout.getvalue()
print(stdout_content)
However, this blocks printing to the terminal until foo returns.
I would like foo to keep printing to the terminal in real time while an object stores the calls.
How can this be achieved?
One approach is to provide your own file like object to redirect_stdout. Your object class will implement the write by writing both to a file and the original sys.stdout.
You can read about sys.stdout here.
You can see the various file like classes here for solid examples.
I made 2 classes:
Input which implements read method, and
Output which implements write method
I am trying to call a shell command and capture the input and output. This is my code:
import subprocess
command = "date"
output = Output()
input = Input()
process = subprocess.Popen(command,
stdout=output, stdin=input, shell=False)
but when I check the output instance it says that it contains no data.
Technically, those class instances are probably invalid parameters, unless they inherit from a file object.
According to the documentation (emphasis mine):
stdin, stdout and stderr specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values are PIPE, DEVNULL, an existing file descriptor (a positive integer), an existing file object, and None. PIPE indicates that a new pipe to the child should be created. DEVNULL indicates that the special file os.devnull will be used. With the default settings of None, no redirection will occur; the child’s file handles will be inherited from the parent. Additionally, stderr can be STDOUT, which indicates that the stderr data from the applications should be captured into the same file handle as for stdout.
I am new to python.
Can anybody explain what's the difference between a string variable and io.StringIO . In both we can save character.
e.g
String variable
k= 'RAVI'
io.stringIO
string_out = io.StringIO()
string_out.write('A sample string which we have to send to server as string data.')
string_out.getvalue()
If we print k or string_out.getvalue() both will print the text
print(k)
print(string_out.getvalue())
They are similar because both str and StringIO represent strings, they just do it in different ways:
str: Immutable
StringIO: Mutable, file-like interface, which stores strs
A text-mode file handle (as produced by open("somefile.txt")) is also very similar to StringIO (both are "Text I/O"), with the latter allowing you to avoid using an actual file for file-like operations.
you can use io.StringIO() to simulate files, since python is dynamic with variable types usually if you have something that accepts a file object you can also use io.StringIO() with it, meaning you can have a "file" in memory that you can control the contents of without actually writing any temporary files to disk
I am new to python and programming and I am trying to understand this code. I have spent the past few hours reading documentation and watching videos on subprocessing but I am still confused (I added snidbits of information of what I found online to comment the code as best I could).
Here are some questions I have pertaining to the code below:
when is subprocess used?
when should I use Popen verses the more convenient handles with subprocess?
what does PIPE do?
what does close_fds do?
basically I need this line of code explained
my_process=Popen(['player',my_video_File_path], stdin=PIPE, close_fds=True)
full code here:
#run UNIX commands we need to create a subprocess
from subprocess import Popen, PIPE
import os
import time
import RPi.GPIO as GPIO
my_video_file_path='/home/pi/green1.mp4'
#stdin listens for information
# PIPE connnects the stdin with stdout
#pipe, (like a pipe sending info through a tunnel from one place to another )
#STDIN (channel 0):
#Where your command draws the input from. If you don’t specify anything special this will be your keyboard input.
#STDOUT (channel 1):
#Where your command’s output is sent to. If you don’t specify anything special the output is displayed in your shell.
#to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE.
#Popen interface can be used directly with subprocess
# with pipe The return value is an open file object connected to the pipe, which can be read or written depending on whether mode is 'r' (default) or 'w'.
#If we pass everything as a string, then our command is passed to the shell;
#"On Unix, if args is a string, the string is interpreted as the name or path of the program to execute. "
my_process=Popen(['player',my_video_File_path], stdin=PIPE, close_fds=True)
GPIO.setmode(GPIO.BCM)
GPIO.setup(17,GPIO.IN,pull_up_down=GPIO.PUD_UP)
GPIO.setup(22,GPIO.IN,pull_up_down=GPIO.PUD_UP)
while True:
button_state=GPIO.input(17)
button_state1=GPIO.input(22)
if button_state==False:
print("quite video")
my_process.stdin.write("q")
time.sleep(.09)
if button_state1==False:
print("full video")
my_process.stdin.write("fs")
time.sleep(5)
Regarding the difference between subprocess and Popen, here is a line from the python 3.5 docs:
For more advanced use cases, the underlying Popen interface can be used directly. (compared to using subprocess)
So, this means that subprocess is a wrapper on Popen.
PIPE is used so that your python script communicate with the subprocess via standard IO operations (you can think of print in python as a type of stdout).
So, when you do my_process.stdin.write("fs") you are sending this text (or piping it) to the standard input of your subprocess. Then your subprocess reads this text and does whatever processing it needs to do.
To further understand subprocessing, try to read standard input into a python program to see how it works. You can follow this How do you read from stdin in Python? question to work on this exercise.
Also, it might be worthwhile to learn about piping in the more general linux style. Try to read through this article.