I've got a python script where I run a cmd using subprocess.getoutput(), and store the resulting output in a list. Now, I need to be able to have the script change to a target dir and run the command there. It should be simple, but passing the cwd arg to getoutput() is not working.
Any ideas?
Example:
out = subprocess.getoutput(" ".join(cmd), cwd='/my/target/path').splitlines()
From the doc it looks like I can easily do this with subprocess.Popen, but then it's difficult to get the output into a list of strings. I've only been able to get the results into a list of binary strings.
subprocess.getoutput is a Legacy Shell Invocation Function. It doesn't take cwd argument and returns a tuple of (status, output). You've got several problems before you even get to the list of bytes.
When python runs a program, it doesn't know what encoding its output is going to have you need to supply that somehow. Assuming the encoding is `utf-8', the basic operation is
mylist = []
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, cwd='/my/target/path')
for line in mylist:
mylist.append(line.decode('utf-8'))
proc.wait()
In this implementation, anything written to stderr just goes to your programs stderr. Notice also that I kept the command as a list and didn't do shell=True. There are several helper functions that do some of the work for you, but that's pretty simple already.
Related
I'm writing a module with custom logging utilities to be imported in other scripts.
It's based on the standard-library logging module.
One of these utilities looks like this:
import argparse as ap
def parse_log_args() -> dict:
log_arg_parser = ap.ArgumentParser(description='Parses arguments that affect logging')
log_arg_parser.add_argument(
'--level',
dest='level',
help='Sets logging level',
choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']
)
log_args, _ = log_arg_parser.parse_known_args()
return vars(log_args)
This function looks for arguments that have to do with logging (even though only --level is defined for the time being) and parses those independently of (and before) all others so that the logging can be configured early on and used in the rest of the script.
The goal here is to remain flexible and be able to quickly plug-in support for these arguments, both in scripts that expect no other arguments and in those that do.
From the point of view of simply parsing arguments this works: this function runs first, parses --level and then the script-specific parser comes and handles the rest.
The problem, however, is the help text. When I run a script that calls this function with --help it only displays the help text from this first parser and not from the script-specific one. So something like this:
Parses arguments that affect logging
optional arguments:
-h, --help show this help message and exit
--level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
Sets logging level
Is there a way to combine the help-texts from all the ArgumentParser instances in a script?
Alternatively: Is there a different way to achieve this in a flexible/plug-in kind of way, that is, without modifying existing ArgumentParsers or having to add them to scripts that don't yet use them?
PS: A similar question has been asked before here: Argparse combine --help directives but the proposed ideas don't really solve the problem:
Define the first parser with add_help=False: This would hide the option from the user which I would prefer not to do.
Use subcommands somehow: doesn't seem to be applicable here.
I think this might fit the bill:
import argparse
part1 = argparse.ArgumentParser(add_help=False)
#... some parsing takes place ...
part2 = argparse.ArgumentParser(add_help=True, parents=[part1])
part1 parser must be fully initialized for parents to work.
More on the topic:
https://docs.python.org/3/library/argparse.html#parents
I'm building a command-line interface using the argparse library to parse user input. At one stage, I'd like to take user input such as "'test', x=False" and use it in a function such as func('test', x=False).
I've tried using ast.literal_eval to do this but it encounters a syntax error at the equals sign. (I did ast.literal_eval("("+args+")") where args was above example)
Does anyone know of a safe way to parse the user input like that? Preferably without eval although worst-case scenario I could use eval as, well, it's a CLI tool.
Edit (to people that have said to use input manually(): I need the tool to parse input from when the command is run (it's a python module that I want to be able to be called like python3 -m hmcli.raw --args "'test', x=False" where the args can be flexible as the function used can differ.
I'm very new to Python and programming overall, so if I seem to struggle to understand you, please bear with me.
I'm reading "Learn Python 3 the Hard Way", and I'm having trouble with exercise 23.
I copied the code to my text editor and ended up with this:
import sys
script, input_encoding, error = sys.argv
def main(language_file, encoding, errors):
line = language_file.readline()
if line:
print_line(line, encoding, errors)
return main(language_file, encoding, errors)
def print_line(line, encoding, errors):
next_lang = line.strip()
raw_bytes = next_lang.encode(encoding, errors=errors)
cooked_string = raw_bytes.decode(encoding, errors=errors)
print(raw_bytes, "<====>", cooked_string)
languages = open("languages.txt", encoding = "utf-8")
main(languages, input_encoding, error)
When I tried to run it I got the following error message:
Traceback (most recent call last):
File "pag78.py", line 3, in <module>
script, input_encoding, error = sys.argv
ValueError: not enough values to unpack (expected 3, got 1)
which I am having difficulties understanding in this context.
I googled the exercise, to compare it something other than the book page and, if I'm not missing something, I copied it correctly. For example, see this code here for the same exercise.
Obviously something is wrong with this code, and I'm not capable to identify what it is.
Any help would be greatly appreciated.
When you run the program, you have to enter your arguments into the command line. So run the program like this:
python ex23.py utf-8 strict
Copy and paste all of that into your terminal to run the code. This exercise uses argv like others do. It says this in the chapter, just a little bit later. I think you jumped the gun on running the code before you got to the explanation.
Let's record this in an answer for sake of posterity. In short, the immediate problem described lies not as much in the script itself, but rather in how it's being called. No positional argument was given, but two were expected to be assigned to input_encoding and error.
This line:
script, input_encoding, error = sys.argv
Takes (list) of arguments passed to the script. (sys.argv) and unpacks it, that is asigns its items' values to the variables on the left. This assumes number of variables to unpack to corresponds to items count in the list on the right.
sys.argv contains name of the script called and additional arguments passed to it one item each.
This construct is actually very simple way to ensure correct number of expected arguments is provided, even though as such the resulting error is perhaps not the most obvious.
Later on, you certainly should check out argparse for handling of passed arguments. It is comfortable and quite powerful.
I started reading LPTHW a couple of weeks ago. I got the same error as 'micaldras'. The error arises because you have probably clicked the file-link and opened an IEExplorer window. From there, (I guess), you have copied the text into a notepad file and saved. it.
I did that as well and got the same errors. I then downloaded the file directly from the indicated link (right click on the file and choose Save Target As). The saves the file literally as Zed intended and the program now runs.
I am new to programming, and new to Python. I am running Python 3 on Windows 10, and I am having a strange problem. I built the following script, and it doesn't work:
def count_item(sequence, item):
return sequence.count(item)
count_item([1,2,1,1], 1)
When I run it, it comes up empty. Not so much as an error, or the "none" that Python likes to give.
However, when I run it from the interactive shell:
>>> item = 1
>>> sequence = [1,2,1,1]
>>> sequence.count(item)
3
I am guessing that this has something to do with how functions work on a deep level in Python, but I am just not sure.
Any help would be appreciated.
The REPL, or interactive shell, is built to be interactive. One way that is manifested is that you don't need to print variables and other objects - you can just type their name, hit Enter, and the relevant info shows up.
However, when actually running a program, you need to explicitly call the print() function in order to make anything show up on the screen. return just returns a value to the calling function or statement. So, for your code, modify the last line to:
print(count_item([1,2,1,1], 1))
this is my first time asking a question so let me know if I am doing something wrong (post wise)
I am trying to create a function that writes into a .txt but i seem to get two very different results between calling it from within a module, and writing the same loop in the shell directly. The code is as follows:
def function(para1, para2): #para1 is a string that i am searching for within para2. para2 is a list of strings
with open("str" + para1 +".txt", 'a'. encoding = 'utf-8') as file:
#opens a file with certain naming convention
n = 0
for word in para2:
if word == para1:
file.write(para2[n-1]+'\n')
print(para2[n-1]) #intentionally included as part of debugging
n+=1
function("targetstr". targettext)
#target str is the phrase I am looking for, targettext is the tokenized text I am
#looking through. this is in the form of a list of strings, that is the output of
#another function, and has already been 'declared' as a variable
when I define this function in the shell, I get the correct words appearing. However, when i call this same function through a module(in the shell), nothing appears in the shell, and the text file shows a bunch of numbers (eg: 's93161), and no new lines.
I have even gone to the extent of including a print statement right after declaration of the function in the module, and commented everything but the print statement, and yet nothing appears in the shell when I call it. However, the numbers still appear in the text file.
I am guessing that there is a problem with how I have defined the parameters or how i cam inputting the parameters when I call the function.
As a reference, here is the desired output:
‘She
Ashley
there
Kitty
Coates
‘Let
let
that
PS: Sorry if this is not very clear as I have very limited knowledge on speaking python
I have found the solution to issue. Turns out that I need to close the shell and restart everything before the compiler recognizes the changes made to the function in the module. Thanks to those who took a look at the issue, and those who tried to help.