Use CMD inside Python - python-3.x

Good evening everyone, this is the first time I post here.
I have many codes I need to run in CMD but I'm not good at coding with CMD, so I want to use python to modify the code that I have to repeat often
for example
temp1 = ["C:/dir1", "C:/dir2", "C:/dir3"]
temp2 = ["1", "2", "3"]
for i, p in zip(temp1, temp2):
os.system('7z -a "{}" C:/target/output"{}" -m5'.format(i, p))
With my code, I also need to see the Shell prompt, because some of the codes I do show a very long terminal data output that I want to monitor actively, I prefer to see it in the python terminal as an output, or print or something similar.
Thanks in advance, it is my first post hope it's clear.

os.system() implicitly uses cmd.exe on Windows (by passing the specified command line to cmd /c) and implicitly passes the command's output through to stdout.
Therefore, your command should work in principle, though I recommend double-quoting the target path in full ("C:/target/output{}"), not just the substituted part (C:/target/output"{}")[1]:
import os
temp1 = ["C:/dir1", "C:/dir2", "C:/dir3"]
temp2 = ["1", "2", "3"]
for i, p in zip(temp1, temp2):
os.system('7z -a "{}" "C:/target/output{}" -m5'.format(i, p))
If you're running this code from a GUI script (a script invoked via pythonw.exe / pyw.exe), the console window created for each os.system() call automatically closes when the command finishes, which doesn't allow you to inspect the output after the fact. To address this, you have two options:
Append & pause to the command, which waits for keypress before closing the console window:
import os
temp1 = ["C:/dir1", "C:/dir2", "C:/dir3"]
temp2 = ["1", "2", "3"]
for i, p in zip(temp1, temp2):
os.system('7z -a "{}" "C:/target/output{}" -m5 & pause'.format(i, p))
Invoke your command via cmd /k, which starts an interactive cmd.exe session that stays open after the command finishes; note that each such session then blocks further execution of your Python script until you manually close the console window in which the persistent cmd.exe session runs.
import os
temp1 = ["C:/dir1", "C:/dir2", "C:/dir3"]
temp2 = ["1", "2", "3"]
for i, p in zip(temp1, temp2):
os.system('cmd /k " 7z -a "{}" "C:/target/output{}" -m5 "'.format(i, p))
Note the unusual quoting (an outer "..." string in which embedded " aren't escaped), which is what cmd.exe supports, however.
[1] Typically, this shouldn't make a difference, but you never know how programs parse their command line on Windows, and double-quoting an argument in full is more likely to work.

Related

How to call a forward the value of a variable created in the script in Nextflow to a value output channel?

i have process that generates a value. I want to forward this value into an value output channel. but i can not seem to get it working in one "go" - i'll always have to generate a file to the output and then define a new channel from the first:
process calculate{
input:
file div from json_ch.collect()
path "metadata.csv" from meta_ch
output:
file "dir/file.txt" into inter_ch
script:
"""
echo ${div} > alljsons.txt
mkdir dir
python3 $baseDir/scripts/calculate.py alljsons.txt metadata.csv dir/
"""
}
ch = inter_ch.map{file(it).text}
ch.view()
how do I fix this?
thanks!
best, t.
If your script performs a non-trivial calculation, writing the result to a file like you've done is absolutely fine - there's nothing really wrong with this approach. However, since the 'inter_ch' channel already emits files (or paths), you could simple use:
ch = inter_ch.map { it.text }
It's not entirely clear what the objective is here. If the desire is to reduce the number of channels created, consider instead switching to the new DSL 2. This won't let you avoid writing your calculated result to a file, but it might mean you can avoid an intermediary channel, potentially.
On the other hand, if your Python script actually does something rather trivial and can be refactored away, it might be possible to assign a (global) variable (below the script: keyword) such that it can be referenced in your output declaration, like the line x = ... in the example below:
Valid output
values
are value literals, input value identifiers, variables accessible in
the process scope and value expressions. For example:
process foo {
input:
file fasta from 'dummy'
output:
val x into var_channel
val 'BB11' into str_channel
val "${fasta.baseName}.out" into exp_channel
script:
x = fasta.name
"""
cat $x > file
"""
}
Other than that, your options are limited. You might have considered using the env output qualifier, but this just adds some syntactic-sugar to your shell script at runtime, such that an output file is still created:
Contents of test.nf:
process test {
output:
env myval into out_ch
script:
'''
myval=$(calc.py)
'''
}
out_ch.view()
Contents of bin/calc.py (chmod +x):
#!/usr/bin/env python
print('foobarbaz')
Run with:
$ nextflow run test.nf
N E X T F L O W ~ version 21.04.3
Launching `test.nf` [magical_bassi] - revision: ba61633d9d
executor > local (1)
[bf/48815a] process > test [100%] 1 of 1 ✔
foobarbaz
$ cat work/bf/48815aeefecdac110ef464928f0471/.command.sh
#!/bin/bash -ue
myval=$(calc.py)
# capture process environment
set +u
echo myval=$myval > .command.env

Is it possible to interact with the subprocess 3 standards streams on windows?

I have a script that opens a subprocess, this process awaits multiple inputs.
I tried with subprocess.Popen() but when combining the 3 standards streams it gets stuck in deadlocks...
So I tired to use wexpect (Apparently Windows version of pexecpt) it also didn't worked.
Now I'm using Sarge and my code right now looks like this:
import os
import subprocess
import sarge
import sys
def get_env():
env = os.environ.copy()
return env
def interactive(list_command: list, instruction_dict: dict):
env = get_env()
std_out = ""
for a_number in instruction_dict:
capture_stdout = sarge.Capture(buffer_size=-1)
capture_stderr = sarge.Capture(buffer_size=-1)
child_proc = sarge.Command(list_command, stdout=capture_stdout, stderr=capture_stderr,
shell=True, env=env)
child_proc.run(input=subprocess.PIPE,
async_=True)
print(capture_stdout.readlines(timeout=1.0), capture_stderr.readlines(timeout=1.0))
if instruction_dict[a_number][0] is not None:
#if capture_stdout.expect(instruction_dict[a_number][0]) is not None or capture_stderr.expect(instruction_dict[a_number][0]) is not None:
# self.test_manager.log_event(msg="found line : " + instruction_dict[a_number][0] + "in PIPE", current_event_type=LogLevel.DEBUG, screen_only=False)
print("in exec line : ", child_proc.stdout.readline(timeout=1.0), child_proc.stderr.readlines(timeout=1.0))
else:
print("in exec line : ", child_proc.stdout.readlines(timeout=1.0), child_proc.stderr.readlines(timeout=1.0))
if instruction_dict[a_number][1] is not None:
print(f"sending \"{instruction_dict[a_number][1]}\" to process in interactive")
temp_bytes = str.encode((instruction_dict[a_number][1]+"\n"))
child_proc.stdin.write(temp_bytes)
try:
child_proc.stdin.flush() # blind flush
except:
pass
print("Last line : ", child_proc.stdout.readlines(timeout=1.0), child_proc.stderr.readlines(timeout=1.0))
child_proc.wait()
std_out += child_proc.stdout.read().decode('utf-8')
child_proc.kill() # kill the subprocess
print("output: ", std_out)
interactive(list_command=["process.exe", "-delete_datamodel", "MODELNAME"],instruction_dict={1 : (None, "y")})
Output looks like this:
[] []
in exec line : [] []
send "y" to process in interactive
Last line : [] [b'User root connected to DB MODELNAME\r\n']
output: Are you sure to delete DB MODELNAME? (y/n)\n\nDelete Data Model Ok\n
In console it looks like this:
C:\>process.exe -delete_data_model -db_name MODELNAME
Are you sure to delete DB MODELNAME ? (y/n)
y
User root connected to DB MODELNAME
Delete Data Model Ok
There are multiple problems:
I can't catch the first stdout "Are you sure to delete DB MODELNAME ? (y/n)" during the loop I only get [].
capture_stdout.expect crash but in fact it might be the same problem as the first one, if it can't read the first stdout.
lines are going in stderr instead of stdout (it might be because of the process, I don't know how to test this)
for another command I ll need to answer more than one interactive by looking of the question before answering them that way my instruction_dict will look like this
instruction_dict = {1 : ("Are you sure to recover the DB", "y"),
2 : ("Do you want to stop the service", "y"),
3 : ("Do you want to stop the DB", "y"),
4 : ("Do you want to drop and create the DB", "y")}
Is this even possible? I have search for a working exemple and didn't found one, I know I might be bad at searching but...
Sorry for the terrible syntax(and bad english), I want to know if there is a pure python solution (I have one solution working in tcl but I'm trying to get rid of it/ I didn't wrote the tcl code).

How to run a process that have an argument containing new-lines?

I have a command that have the structure :
xrdcp "root://server/file?authz=ENVELOPE&Param1=Val1" local_file_path
The problem is that ENVELOPE in text that should be unquoted in command line
and it contains a lot of new-lines
I cannot use repr as it will replace new-line with \n
Moreover subprocess seems to automatically use repr on the items from the list arguments
In bash this command is usually run with
xrdcp "root://server/file?authz=$(<ENVELOPE)&Param1=Val1" local_file
So, is there a way to run a command while keeping the new lines in the arguments?
Thank you!
Later Edit:
my actual code is :
envelope = server['envelope']
complete_url = "\"" + server['url'] + "?" + "authz=" + "{}".format(server['envelope']) + xrdcp_args + "\""
xrd_copy_list = []
xrd_copy_list.extend(xrdcp_cmd_list)
xrd_copy_list.append(complete_url)
xrd_copy_list.append(dst_final_path_str)
xrd_job = subprocess.Popen(xrd_copy_list, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = xrd_job.communicate()
print(stdout)
print(stderr)

How to execute an exe file supply the input text file and get the output in a variable in Python 3 on Windows?

I have an exe file mycode.exe at "D:\projFolder\mycode.exe" and an input text file in.txt at "D:\projFolder\in.txt".
I am writing a Python3 script which will execute this exe file with the supplied input text and compare the output.
To achieve it, I am just trying to execute Windows command as :
cmd> "D:\projFolder\mycode.exe" < "D:\projFolder\in.txt"
and want to save the result of the above command in a variable say, resultstdout, and later use it to compare with an expected output file out.txt.
My problem, how to execute the Windows command "D:\projFolder\mycode.exe" < "D:\projFolder\in.txt" in Python3 script?
I was previously working on Python2 and I was achieving it as follows:
baseDirectory = "D:/projFolder"
( stat, consoleoutput ) = subprocess.getstatusoutput(baseDirectory + "/mycode.exe"+ " < " + contestDirectory+ "/" +in.txt)
if(stat == 0):
# Perform result comparision
else:
# Some execption while executing the command.
However, I am not sure on how to refactor the above code in Python 3.

Is it possible to use pexpect to run another python script on switch?

Connecting to switch using expect spawn
child = pexpect.spawn('telnet ' + self.device_ip, timeout = 30)
When trying to run the below line, Im getting nothing as output
child.sendline('python script.py')
Can someone help me to run the python file over here?
I used
print("%s"%(child.before))
to print the output but it prints nothing
Make sure when you are using child.before that you have a child.expect statement beforehand. To put simply expect defines the text buffer that before references. Ex.
child = pexpect.spawn('telnet ' + self.device_ip, timeout = 30)
child.sendline('python script.py')
child.expect('user#abcd1234>')
remotehostoutput = (child.before)

Resources