I created a special builder in SCons for creating a virtualenv in Python from a requirements file. If given a flag --virtualenv, I would like to set an envrionment variable called HOSTPYTHON which changes the Python that scons use to build and test the rest of my code.
This means that I need the virtualenv builder to always run first and before all other builders. How can I do that?
Instead of defining the virtualenv as a builder, you could instead consider making it a simple Python function and run it using the SCons Execute() function. Execute will always be executed before any of the builders.
I dont know how to change the Python version being used by SCons in the middle of a build, so you may have to run SCons twice. The first time, check for the --virtualenv command line argument, call Execute() if present (or always call Execute() and check the cmd line internally), and then Exit().
Related
I need to run a script inside a Popen inside a packaged project (with pyinstaller). The command NEEDS to run in python3 and I (obviously) need it to be portable, which means that I can't rely on the config (python/python3 notation) used on the machine I've used to generate the package neither the one where I am using it (I can't also guarantee that there will be any python in the deployment machine...).
I have computers where python is already the correct version, and others where the correct one is python3. I tried to insert #!/usr/bin python3 in the beginning of the file and then run as python but it didn't work.
subprocess.Popen(['python', 'internal.py', arg1, arg2], universal_newlines=True)
In the non-frozen environment I am able to run using sys.executable instead of python. I tried to "pyinstall" the internal.py then, "pyinstall" the rest copying the internal folder to the new folder and then call:
subprocess.Popen(['./internal/internal', arg1, arg2], universal_newlines=True)
But this doesn't work... In the console (moving to the bigger project folder) it does, but when running the package, it doesn't...
Any idea?
Ps. I can't just import the script as a class or whatever. I can't modify system environment vars (except the ones inside the pyinstaller thing...)
I saw something about the PyInstaller.compat that has a exec_python and a __wrap_python, but I couldn't use it I get errors when trying to run the package due to pkg_resources.DistributionNotFound: The 'PyInstaller' distribution was not found and is required by the application, I tried to create a hook, to add as hidden import... None worked (but the hook part maybe I got it wrong...)
It seems this question has been asked very often before but none of the solutions seem to apply in my case.
I'm in a CMake/Linux environment and have to run an executable binary during the build step (protoc in particular).
This binary needs a library but it's not installed (and cannot be) in the in the standard directories like /usr, so the library cannot be found.
Unfortunately I cannot manipulate the protoc call because it's embedded in a 3rd party script.
I can now set LD_LIBRARY_PATH before every make or set it system wide but this is very inconvenient especially when it comes to IDEs in which the build takes place or distributed build scenarios with continuous build environments.
I tried to set LD_LIBRARY_PATH via
set(ENV{LD_LIBRARY_PATH} "/path/to/library/dir")
but this seems to have no effect during the build step.
So my question is: can I set a library search path in CMake which is used during the build?
Try this
SET(ENV{LD_LIBRARY_PATH} "/path/to/library/dir:$ENV{LD_LIBRARY_PATH}")
I also used this dirty trick to temporary change some environment variables:
LD_LIBRARY_PATH="/path/to/library/dir:$LD_LIBRARY_PATH" cmake ...
After execution of this line LD_LIBRARY_PATH is not changed in the current shell.
Also, I do not find it bad to change LD_LIBRARY_PATH before invoking cmake:
export LD_LIBRARY_PATH=...
It won't change anything system-wide, but it would be used for your current shell, current build process. The same holds for CI builds. You can save the variable and restore it after cmake invocation:
MY_LD=$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=...
cmake...
export LD_LIBRARY_PATH=$MY_LD
I have recently run into a somewhat similar problem.
My solution was to incorporate sourcing a file that set the appropriate environment into every command.
For example, this custom command:
add_custom_command(
OUTPUT some_output
COMMAND some_command
ARGS some_args
DEPENDS some_dependencies
COMMENT "Running some_command some_args to produce some_output"
)
Would become:
set(my_some_command_with_environment "source my_environment_script.sh && some_command")
add_custom_command(
OUTPUT some_output
COMMAND bash
ARGS -c "${my_some_command_with_environment} some_args"
DEPENDS some_dependencies
COMMENT "Running some_command some_args to produce some_output"
VERBATIM
)
Obviously, this has some disadvantages:
It relies on a bash shell being available.
It sources the environment script for every command invocation (performance issue) and you will have to change all invocations of commands that rely on that environment variables.
It changes the normal syntax of having the command follow COMMAND and the arguments follow ARGS, as now the 'real' command is part of the ARGS.
My CMake-Fu has proven insufficient to find a syntactically nicer way of doing this, but maybe somebody can comment a nicer way.
I had a similar issue for an executable provided by a third party library. The binary was linked against a library not provided by the distribution but the required library was included in the libs directory of the third party library.
So running LD_LIBRARY_PATH=/path/to/thirdparty/lib /path/to/thirdparty/bin/executable worked. But the package config script didn't set up the executable to search /path/to/thirdparty/lib for the runtime dependent so CMake would complain when CMake tried to run the executable.
I got around this by configuring a bootstrap script and replacing the IMPORTED_LOCATION property with the configured bootstrapping script.
_thirdpartyExe.in
#!/bin/bash
LD_LIBRARY_PATH=#_thirdpartyLibs# #_thirdpartyExe_LOCATION# "$#"
CMakeLists.txt
find_package(ThirdPartyLib)
get_target_property(_component ThirdPartyLib::component LOCATION)
get_filename_component(_thirdpartyLibs ${_component} DIRECTORY)
get_target_property(_thirdpartyExe_LOCATION ThirdPartyLib::exe IMPORTED_LOCATION)
configure_file(
${CMAKE_CURRENT_LIST_DIR} _thirdpartyExe.in
${CMAKE_BINARY_DIR}/thirdpartyExeWrapper #ONLY
)
set_target_properties(ThirdPartyLib::exe PROPERTIES IMPORTED_LOCATION ${CMAKE_BINARY_DIR}/thirdpartyExeWrapper)
Honestly I view this as a hack and temporary stop gap until I fix the third party library itself. But as far as I've tried this seems to work on all the IDE's I've thrown at it, Eclipse, VSCode, Ninja, QtCreator, ... etc
I made a Python script that runs whenever the main binary is changed (using Requires), so at the moment it runs with every build. However, the script can be time consuming in certain cases, and I'm trying to figure out how to run it only if a flag is given.
For example, if I type "scons -script" the script will run after the compilation of the main binary, but typing "scons" will just build the file and do nothing beyond that.
I ended up solving it by using AddOption() to define the flag:
AddOption("--script", action="store_true", help="Run the script")
And writing a method that checks the flag using GetOption() and adds the script to the requirements:
if GetOption("script"):
script = Command(target=...,
source=...,
action=[...])
Requires(script, binary_node)
So I've got an executable with conflicting dependencies with the build system it is running in. (AKA Xilinx doesn't play well with others). I'd love to run a cmake script as part of the build process, but it is depended on different dlls. I can try to figure out a version of cmake that matches the xylinx dlls. Xylinx redirects the dependencies to the different C++ runtimes by setting a bunch of environment variables.
Sanitize environment with command or bash script?
Will completely clean out all environments, but what i really want to do is run the script after the .bash_rc or .bash_profile is called. (cmake seems to require some of that to be set, as env -i cmake returns an error) I'd like to do it without having to require the login credentials. Is that possible?
You can pass environment variables directly in a command line, right before the executable name:
ENV_VAR=foo command -options ...
If you want to pass multiple variables just add them one after one:
ENV_VAR_FOO=foo ENV_VAR_BAR=bar command -options ...
In your case, you'll need to find out which environment variable cmake needs to access the libraries. Then pass this to the call, like:
LIB_PATH=/path/to/libs cmake -options ...
I've been looking, and looking, and I can't find an answer to my question.
I've just started learning scons tonight, and it looks awesome! I'm running into a little confusion though.
For ease of development, I often like to have my make file build my target, and then run it so that I can test a change with one keystroke. This is very simple in a make file:
run: $(exe)
chmod a+x $(exe)
$(exe)
I have figured out that I can do it using subprocess like so:
import subprocess import os.path
env = Environment();
result = env.Program(target = "FOO", source = "BAR");
location = os.path.abspath(result[0].name)
subprocess.call([location])
But there's a problem with this solution. As far as I have experimented, scons won't wait until your program is finished building before it starts the subprocess call, so you end up running the old executable, or having an error if it's a build after a clean.
What you do in your scons file is a typical beginner error in scons. Your assume that you are writing a script for building your project.
Scons doesn't work like that. The scons files is a script that add targets to the project. This is done through python, and the various objects allows you to create and manipulate targets until the script is done. First then will the project start building.
What you do in your code is to describe the Environment to use, the Program to create, and after that you call a subprocess that runs some program. After this the project will start building - no wonder the old executable is run, the new one haven't started to be built yet.
What you should do is to use a custom builder for executing the program.
env = Environment() #construct your environment
files = "test.cpp" #env.Glob or list some files
#now we create some targets
program = env.Program("test",files) #create the target *program*
execution = env.Command(None,None,"./test") #create the execution target (No input & output
Depends(execution,program) #tell scons that execution depends on program
#there might be a way to get SCons to figure out this dependency itself
#now the script is completed, so the targets are built
Here the dependencies are clear, the program must be built before the execution is done, and it will
I may be a little bit late for you, but I have this solution using Alias.
By using the following command, it will build and run the program:
$ scons run
# Define the different target output
program = env.Program('build/output', Glob('build/test/*.cpp'))
env.Default(program)
env.Alias('run', program, program[0].abspath)
note that we use the abspath, so it can be cross platform win/linux (for linux you need to add the "./" before the program name if your PATH is not correctly set.
Ok, I'm a little nervous to answer my own question, but I found a more or less acceptable solution.
I have just set up a simple chain.
I set up a Makefile with something like this in it:
run:
scons -s
./name_of_executable
This calls scons in silent mode, and runs your program automatically afterwards. It's not a scons-only solution, but it works. I'd still be interested to see if anyone has another answer.
Thanks!
Murphy