I have a script, lets say "foo.py", which depends on some file, let's say "dep.par", and I would call these as
python foo.py --parameters=dep.par
If foo.py writes only one output file, "bar.dat", I would say:
env=Environment()
env.Command("bar.dat", "dep.par", "python foo.py --parameters=dep.par")
However, I need the case where foo.py outputs more than one file, let's say "bar1.dat, bar2.dat, ..., barN.dat".
I'm at a loss, any help would be appreciated, thanks.
The target (and/or source) can be a list of targets as follows:
env=Environment()
env.Command(["bar1.dat", "bar2.dat"], "dep.par", "python foo.py --parameters=dep.par")
Related
Im currently writing a python3 script that checks out a C source file by running the C code with various of input files. the compilation is done by GCC if it matters.
in some case, the C code enters into an infinite loop (I figured it out because I ran out of memory).
is there a way that I can "protect" my code like a watchdog or something that
tells me a after X minutes that I ran into an infinite loop?
I cant assume anything about the input so i cant have answers like change it or something...
#runfile is an exe file, code file is .c file, inputlist/outputlist are directories
import subprocess as sbp
import os
sbp.call('gcc -o {0} {1}'.format(runfile,codefile), shell = True)
for i in range(numoFiles):
#run the file with input i and save it to output i in outdir
os.system("./{0} <{1} >{2}".format(ID,inputList[i],outputList[i]))
Look up the "Halting Problem". It is not possible to determine whether an arbitrary program will eventually finish or if it will be stuck in a loop forever.
I figured out a way to avoid enter infinite loop by this method:
import subprocess
for i in range(numofiles):
cmd="gcc -o {0} {1}".format(runfile,codefile)
subprocess.call(cmd, shell = True)
try:
cmd="./{0} <{1} >'/dev/null'".format(Cfile,inputfile) #check if runtime>100 sec
subprocess.run(cmd,shell=True,timeout=100)
except subprocess.TimeoutExpired:
print("infinite loop")
continue
cmd="./{0} <{1} >{2}".format(Cfile,inputfile,outputfile)
subprocess.run(cmd,shell=True) #print output to txt file
Does any one know why i got an error of "FileNotFoundError: [Errno 2] No such file or directory: 'bcs.xlsx'" when i'm loading this file of size 2mb it has around 60,000 rows and 4 columns.
i tried using csv instead of xlsx but i get the same error and i've checked hundreds times that the script and the file are at he same directory.
This is because Python does not find your file, errors are not lying.
But there's a misunderstanding in your question, you checked that the file is in the same directory as your script, but that's not the check you have to do. You have to check the file is in the current working directory of your python script.
To see your current working directory, use:
import os
print(os.getcwd())
And as we're at it you can list this directory:
print(os.listdir())
I don't know how you execute your script, but if you're using a terminal emulator, a typical way to give a file name to a program is by argument, not hardcoding its name, like by using argparse. And if you do this way, your shell completion may help you naming your file properly, like:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('file', type=argparse.FileType('r'))
args = parser.parse_args()
print(args.file.read())
Now on a shell if you type:
python3 ./thescript.py ./th[TAB]
your shell will autocomplete "./th" to "./thescript.py" (if and only if it exists), highly reducing the probablity of having a typo. Typically if there's a space in the filename like "the script.py", your shell should properly autocomplete the\ script.py.
Also if you use argparse with the argparse.FileType as I did, you'll have a verbose error in case the file does not exist:
thescript.py: error: argument file: can't open 'foo': [Errno 2] No such file or directory: 'foo'
But… you already have a verbose error.
I'm fairly new to Python, so please bear with me.
Currently, I'm using Python 3.5 in an Anaconda environment on Pycharm, and I am trying to understand how to set/define/use sys.argv so that I can automate several processes before uploading my changes onto github.
For example:
python function/function.py input_folder/input.txt output_folder/output.txt
This means that function.py will take input.txt from input_folder, apply whatever script written in function.py, and store the results into output.txt in folder output_folder.
However, when I type this into terminal, I get the following error:
python: can't open file 'function/function.py': [Errno 2] No such file or directory
Then, typing sys.argv into Python console, I receive the following:
['C:\\Program Files (x86)\\JetBrains\\PyCharm 2016.2\\helpers\\pydev\\pydevconsole.py',
'53465',
'53466']
My guess is that if I were to set sys.argv[0:1] correctly, then I should be able to apply function.py to input.txt and store the results into output.txt.
I've already tried to define these directories, but they wouldn't work. Any help would be awesome!
your issue is that python does not know where the function directory exists. If you are trying to run a script from a sub directory like so
function
|_function.py
|
input_folder
|_input.txt
|
|output_folder
|_output.txt
you must tell python that the function folder is local, so
python ./function/function.py ./input_folder/input.txt ./output_folder/output.txt
or
python $PWD/function/function.py $PWD/input_folder/input.txt $PWD/output_folder/output.txt
$PWD is a bash variable that gives the current directory
I'm going nuts trying to control when files are built in scons. I have a very simple example build tree (see below), with a Poem builder that just takes a .txt file and converts it to lower case in a corresponding .eectxt file.
In my SConstruct and SConscript files, I declare dependencies of 3 .txt files.
But I can't figure out what's putting these into the default build!
sconstest/
SConstruct
tiger.txt
src/
SConscript
hope.txt
jabberwocky.txt
where the *.txt files are poems and my SConstruct and SConscript look like this:
SConstruct:
env = Environment();
def eecummings(target, source, env):
if (len(target) == 1 and len(source) == 1):
with open(str(source[0]), 'r') as fin:
with open(str(target[0]), 'w') as fout:
for line in fin:
fout.write(line.lower());
return None
env['BUILDERS']['Poem'] = Builder(action=eecummings, suffix='.eectxt', src_suffix='.txt');
Export('env');
poems = SConscript('src/SConscript');
tigerPoem = env.Poem('tiger.txt');
src/SConscript:
Import('env');
input = ['jabberwocky.txt', 'hope.txt'];
output = [env.Poem(x) for x in input];
Return('output');
What I want to do is to declare the dependency of the .eectxt files from the corresponding .txt files, but not cause them to be built unless I explicitly put them into the Default() build in the SConstruct file, or I request them explicitly at the command line.
How can I do this?
By default, a directory depends on all files and/or targets which reside in it.
So running:
scons
Will then build all targets under the current directory.
I figured out how to do what I want, but I still don't understand why I need to do it this way. Acceptance to the first decent answer that explains it.
Here's what works, if I add the following to the root SConstruct file:
env.Ignore('.', tigerPoem);
env.Ignore('src', poems);
env.Alias('poems', [tigerPoem]+poems);
This ignores the 3 poems from the default target, and then adds them as targets aliased to "poems", so if I run scons it builds nothing, but if I run scons poems it builds the files.
Why does this work? Why does calling env.Poem(...) add something to the default targets?
I want to inject a "Cleanup" target which depends on a number of other targets finishing before it goes off and gzip's some log files. It's important that I not gzip early as this can cause some of the tools to fail.
How can I inject a cleanup target for Scons to execute?
e.g. I have targets foo and bar. I want to inject a new custom target called 'cleanup' that depends on foo and bar and runs after they're both done, without the user having to specify
% scons foo cleanup
I want them to type:
% scons foo
but have scons execute as though the user had typed
% scons foo cleanup
I've tried creating the cleanup target and appending to sys.argv, but it seems that scons has already processed sys.argv by the time it gets to my code so it doesn't process the 'cleanup' target that I manually append to sys.argv.
you shouldn't use _Add_Targets or undocumented features, you can just add your cleanup target to BUILD_TARGETS:
from SCons.Script import BUILD_TARGETS
BUILD_TARGETS.append('cleanup')
if you use this documented list of targets instead of undocumented functions, scons won't be confused when doing its bookkeeping. This comment block can be found in SCons/Script/__init__.py:
# BUILD_TARGETS can be modified in the SConscript files. If so, we
# want to treat the modified BUILD_TARGETS list as if they specified
# targets on the command line. To do that, though, we need to know if
# BUILD_TARGETS was modified through "official" APIs or by hand. We do
# this by updating two lists in parallel, the documented BUILD_TARGETS
# list, above, and this internal _build_plus_default targets list which
# should only have "official" API changes. Then Script/Main.py can
# compare these two afterwards to figure out if the user added their
# own targets to BUILD_TARGETS.
so I guess it is intended to change BUILD_TARGETS instead of calling internal helper functions
One way is to have the gzip tool depend on the output of the log files. For example, if we have this C file, 'hello.c':
#include <stdio.h>
int main()
{
printf("hello world\n");
return 0;
}
And this SConstruct file:
#!/usr/bin/python
env = Environment()
hello = env.Program('hello', 'hello.c')
env.Default(hello)
env.Append(BUILDERS={'CreateLog':
Builder(action='$SOURCE.abspath > $TARGET', suffix='.log')})
log = env.CreateLog('hello', hello)
zipped_log = env.Zip('logs.zip', log)
env.Alias('cleanup', zipped_log)
Then running "scons cleanup" will run the needed steps in the correct order:
gcc -o hello.o -c hello.c
gcc -o hello hello.o
./hello > hello.log
zip(["logs.zip"], ["hello.log"])
This is not quite what you specified, but the only difference between this example and your requirement is that "cleanup" is the step that actually creates the zip file, so that is the step that you have to run. Its dependencies (running the program that generates the log, creating that program) are automatically calculated. You can now add the alias "foo" as follows to get the desired output:
env.Alias('foo', zipped_log)
In version 1.1.0.d20081104 of SCons, you can use the private internal SCons method:
SCons.Script._Add_Targets( [ 'MY_INJECTED_TARGET' ] )
If the user types:
% scons foo bar
The above code snippet will cause SCons to behave as though the user had typed:
% scons foo bar MY_INJECTED_TARGET