I would like to run my py_test with python 3 in Bazel.
py_library(
name = "foo",
srcs = ["foo.py"]
)
py_test(
name = "foo_test",
srcs = glob(["foo_test.py",]),
deps = [":foo"]
)
py_runtime(
name = "python-3.6.3",
files = [],
interpreter_path = "/usr/local/bin/python3",
)
I was able to achieve this using command
bazel test --python_top=//path/to/foo:python-3.6.3 foo_test
However, I would like to import python3 to bazel sandbox with new_http_archive and provide the interpreter_path for the py_runtime rule that points to that http_archive within bazel sandbox. So far I am not able to find what is the interpreter_path... Do I have to reference the http_archive label somewhere from the py_runtime or somewhere else?
new_http_archive(
name = "python_version",
urls = ["https://www.python.org/ftp/python/3.6.3/Python-3.6.3.tgz"],
strip_prefix = "Python-3.6.3",
build_file_content = """
py_library(
name = "python_srcs",
srcs = glob(["Lib/*.py"]),
visibility = ["//visibility:public"]
)"""
)
The tgz that you're downloading doesn't contain an interpreter. It contains the source code for the interpreter. If you want to build the interpreter as part of your build, you could do something like this
new_http_archive(
name = "python_version",
urls = ["https://www.python.org/ftp/python/3.6.3/Python-3.6.3.tgz"],
strip_prefix = "Python-3.6.3",
build_file_content = """
genrule(
name = "build_python",
srcs = glob(["**"]),
outs = ["python"],
cmd = "./external/python_version/configure && make && cp python $#",
visibility = ["//visibility:public"],
)""",
)
And then your py_runtime rule would set the interpreter attribute (not interpreter_path):
py_runtime(
name = "python-3.6.3",
files = [],
interpreter = "#python_version//:python",
)
Related
I am trying to create a Bazel project which includes cucumber-cpp. I could not figure out how its BUILD file would look like.
As Google Test now includes it's own BUILD file it is as easy as it get's. Something similar would be nice.
My WORKSPACE file looks like this
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "googletest",
sha256 = "927827c183d01734cc5cfef85e0ff3f5a92ffe6188e0d18e909c5efebf28a0c7",
strip_prefix = "googletest-release-1.8.1",
url = "https://github.com/google/googletest/archive/release-1.8.1.zip",
)
http_archive(
name = "cucumber-cpp",
sha256 = "73fddda099e39cc51ebee99051047067f6dcd437fbde60601ac48cb82a903dac",
url = "https://github.com/cucumber/cucumber-cpp/archive/v0.5.zip",
)
My specification BUILD file
cc_test(
name = "app-spec",
srcs = glob(["**/*.cpp"]),
deps = [
"//src:app-lib",
"#cucumber-cpp//:main", //do not know if this is correct
],
)
cc_test(
name = "app-spec",
srcs = glob(["**/*.cpp"]),
deps = [
"//src:app-lib",
"#cucumber-cpp//:main", //do not know if this is correct
],
)
Test BUILD file
cc_test(
name = "app-test",
srcs = glob(["**/*.cpp"]),
deps = [
"//src:app-lib",
"#googletest//:gtest_main",
],
)
But obviously the cucumber-cpp is not built so I wonder how it's Bazel BUILD file would look like?
I also wanted to do this, but couldn't find anything where anyone had attempted it. In the end I wrote a dedicated bazel extension for using cucumber and gherkin feature specs. Currently this only supports (linux|osx)+cpp+cucumber, but I may add support for windows and other languages further down the line. To use this add this to your WORKSPACE file;
load("#bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(
name = "rules_gherkin",
commit = "ef361f40f9716ad8a3c6a8a21111bb80d4cbd927", # Update this to match latest commit
remote = "https://github.com/silvergasp/rules_gherkin.git"
)
load("#rules_gherkin//:gherkin_deps.bzl","gherkin_deps")
gherkin_deps()
load("#rules_gherkin//:gherkin_workspace.bzl","gherkin_workspace")
gherkin_workspace()
An example BUILD file would look like this;
load("//gherkin:defs.bzl", "gherkin_library", "gherkin_test")
gherkin_library(
name = "feature_specs",
srcs = glob(["**/*.feature"]),
)
gherkin_test(
name = "calc_test",
steps = ":calculator_steps",
deps = [":feature_specs"],
)
load("//gherkin:defs.bzl", "cc_gherkin_steps")
cc_gherkin_steps(
name = "calculator_steps",
srcs = [
"CalculatorSteps.cpp",
],
visibility = ["//visibility:public"],
deps = [
"//examples/Calc/src:calculator",
"#cucumber_cpp//src:cucumber_main",
"#gtest",
],
)
A complete example can be found here.
I am new to bazel and have some question:
I have defined a library xxx like this:
cc_library(
name = "xxx",
srcs = glob(["lib/*.c"]),
hdrs = glob(["include/*.h"]),
copts = ["-Iinclude -Werror"],
)
Using pkg_tar, I saw that it produces a target files :xxx which have the .so and the .a :
load("#bazel_tools//tools/build_defs/pkg:pkg.bzl", "pkg_tar")
pkg_tar(
name = "libxxx",
package_dir = "/usr/lib/",
srcs = [":xxx"],
mode = "0644",
)
I want to get only the static library .a, how do I do, for the moment, I found only this solution:
pkg_tar(
name = "libxxx-static",
package_dir = "/usr/lib/",
srcs = [":xxx"],
# FIXME
strip_prefix = "libxxx.so",
mode = "0644",
)
How do I get only one file in target files ?
You can use the linkstatic field of cc_library(),
The linkstatic attribute has a different meaning if used on a cc_library() rule. For a C++ library, linkstatic=True indicates that only static linking is allowed, so no .so will be produced. linkstatic=False does not prevent static libraries from being created. The attribute is meant to control the creation of dynamic libraries.
Like so:
cc_library(
name = "xxx",
srcs = glob(["lib/*.c"]),
hdrs = glob(["include/*.h"]),
copts = ["-Iinclude -Werror"],
linkstatic = True,
)
Let's say I want to strip all the debug symbols in the shared libraries that I build whiling keeping the original file name.
I tried to add an command in the method:
def mySharedLibrary(self, *args, **kwargs):
# do some common work for every shared library like add a soname or append some lib files to LIBS parameter
target = SharedLibary(*args, **kwargs)
target = env.Command(target,target, "objcopy --strip-debug ${SOURCE}")
return target
I get this error: two different method was given to the same target,
I guess it's because the two targets returned by env.Command and SharedLibrary are exactly the same name.
Any ideas to do this?
Thanks in advance!
I had the same problem and got the same error. What I had to do was to create an intermediate target/library. The intermediate and final targets each had their own library name, so SCons doesnt get confused.
You could probably do something like the following:
env.SharedLibrary(target = 'namePreStrip', source = 'yourSource')
env.Command(target = 'name', source = 'namePreStrip', 'objcopy...')
I used objcopy to build a library out of several libraries. Here's the actual source code I implemented:
#
# Build an object file out of several other source files, objects, and libraries
# Optionally execute objcopy on the resulting library, depending if objcopyFlags
# has been populated
#
# env - SCons Environment used to build, Mandatory arg
# target - resulting library name, without LIBPREFIX and LIBSUFFIX, ej 'nsp2p',
# Mandatory arg
# sourceFiles - list of '.cc' files that will be compiled and included in the
# resulting lib, Optional arg
# objects - list of already compiled object files to be included in resulting lib,
# Optional arg
# libraries - list of libraries to be included in resulting lib, Optional arg
# objcopyFlags - list of flags to pass to objcopy command. objcopy will only
# be executed if this list is populated, Optional arg
#
# One of [sourceFiles, objects, or libraries] must be specified, else nothing
# will be performed
#
# Not using a custom builder because I dont like the way SCons prints the
# entire command each time its called, even if its not going to actually
# build anything AND I need more method args than provided by custom builders
#
def buildWholeArchive(self, env, target, sourceFiles, objects, libraries, objcopyFlags):
if len(sourceFiles) == 0 and len(objects) == 0 and len(libraries) == 0:
print "Incorrect use of buildWholeArchive, at least one of [sourceFiles | objects | librarires] must be specified, no build action will be performed"
return None
# Compile each source file
objNodes = []
if len(sourceFiles) > 0:
objNodes = env.Object(source = sourceFiles)
cmdList = []
cmdList.append(env['CXX'])
cmdList.append('-nostdlib -r -o $TARGET -Wl,--whole-archive')
for obj in objNodes:
cmdList.append(env.File(obj).abspath)
for obj in objects:
cmdList.append(env.File(obj).abspath)
for lib in libraries:
cmdList.append(lib)
cmdList.append('-Wl,--no-whole-archive')
cmd = ' '.join(cmdList)
libTarget = '%s%s%s' % (env['LIBPREFIX'], target, env['LIBSUFFIX'])
if len(objcopyFlags) > 0:
# First create the library, then run objcopy on it
objTarget = '%s%s_preObjcopy%s' % (env['LIBPREFIX'], target, env['LIBSUFFIX'])
preObjcopyTarget = env.Command(target = objTarget, source = [], action = cmd)
env.Depends(preObjcopyTarget, [objNodes, sourceFiles, objects, libraries])
objCmdList = [env['OBJCOPY']]
objCmdList.extend(objcopyFlags)
objCmdList.append('$SOURCE $TARGET')
objcopyCmd = ' '.join(objCmdList)
archiveTarget = env.Command(target = libTarget, source = preObjcopyTarget, action = objcopyCmd)
else:
# Just create the library
archiveTarget = env.Command(target = libTarget, source = [], action = cmd)
env.Depends(archiveTarget, [objNodes, sourceFiles, objects, libraries])
return archiveTarget
And here is how I called it:
sourceFiles = ['file1.cc', 'file2.cc']
libSource = []
if 'OcteonArchitecture' in env:
libSource.append(lib1)
libSource.append(lib2)
libSource.append(lib3)
objcopy = []
if 'OcteonArchitecture' in env:
objcopy.extend([
'--redefine-sym calloc=ns_calloc',
'--redefine-sym free=ns_free',
'--redefine-sym malloc=ns_malloc',
'--redefine-sym realloc=ns_realloc'])
archiveTarget = clonedEnv.buildWholeArchive(target = libName,
sourceFiles = sourceFiles,
objects = [],
libraries = libSource,
objcopyFlags = objcopy)
env.Alias('libMyLib', archiveTarget)
Im new to scons and am having problems with scons dependancies
in a hierarchichal build with a variant directory.
Im able to reproduce the problem in a reduced environment that consists of
2 subdirs under the SConscript directory (moduleA and moduleB) as follows:
.
|-- SConstruct
|-- file.conf
|-- moduleA
| |-- SConscript
| `-- conf2cc
`-- moduleB
|-- SConscript
`-- fileB.cc
Here is the flow of what needs to be done:
moduleA executes a shell script: conf2cc, input: $projRootDir/file.conf, output: moduleA/$variantDir/source.cc
moduleA compiles source.cc and creates moduleA/$variantDir/libmoduleA.a
moduleB needs to copy moduleA/$variantDir/source.cc to moduleB/source.cc
moduleB needs to compile moduleB/source.cc and moduleB/fileB.cc into its
library libmoduleB.a
Its entirely possible that Im doing several things wrong here. For example, I know
Im not using $TARGET/$SOURCE in moduleA Command(), but thats on purpose, since the
script needs the absolute path names, and scons doesnt remove the leading '#'
The problem I have is the Command() builder in moduleB (step 3 above) never executes.
Here are the SConstruct and SConscript files:
Sconstruct
import os
env = Environment()
env['variantDir'] = 'linux' # for this example, just make variantDir = linux
modules = ['moduleA', 'moduleB']
for dir in modules:
SConscript(
os.path.join(dir, 'SConscript'),
variant_dir = os.path.join(dir, env['variantDir']),
exports = ['env'],
duplicate = 0)
moduleA/Sconscript
import os
Import('env')
scriptInput = '#file.conf'
sourceFile = os.path.join('#moduleA', env['variantDir'], 'source.cc')
conf2ccScript = File('#moduleA/conf2cc').abspath
# The script needs abspaths for input and output, not the scons '#' prepended
# the script syntax is: script <inputFile> <outputFile>
cmd = '%s %s %s' % (conf2ccScript, File(scriptInput).abspath, File(sourceFile).abspath)
# Generate source.cc file based on file.conf
conf2ccNode = env.Command(target = sourceFile,
source = scriptInput,
action = cmd)
libNode = env.Library(target = 'moduleA', source = sourceFile)
env.Depends(libNode, conf2ccNode)
moduleB/Sconscript
import os
Import('env')
sourceFiles = ['fileB.cc', 'source.cc']
# Get the source.cc file
externalSourceFile = os.path.join('#moduleA', env['variantDir'], 'source.cc')
sourceTarget = os.path.join('#moduleB', 'source.cc')
cmdNode = env.Command(target = sourceTarget,
source = externalSourceFile,
action = Copy('$TARGET', '$SOURCE'))
libNode = env.Library(target = 'moduleB', source = sourceFiles)
env.Depends(libNode, cmdNode)
Here is the output when I execute scons:
Any help would be greatly appreciated!
Brady
notroot#ubuntu:~/projects/sconsTest/sconsTestHierDeps$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
/home/notroot/projects/sconsTest/sconsTestHierDeps/moduleA/conf2cc /home/notroot/projects/sconsTest/sconsTestHierDeps/file.conf /home/notroot/projects/sconsTest/sconsTestHierDeps/moduleA/linux/source.cc
g++ -o moduleA/linux/source.o -c moduleA/linux/source.cc
ar rc moduleA/linux/libmoduleA.a moduleA/linux/source.o
ranlib moduleA/linux/libmoduleA.a
g++ -o moduleB/linux/fileB.o -c moduleB/fileB.cc
scons: *** [moduleB/linux/source.o] Source `moduleB/source.cc' not found, needed by target `moduleB/linux/source.o'.
scons: building terminated because of errors.
I suggest problem in incorrect dependency what you use and filenames.
May be problem in variant_dir and source files for moduleB, you use Command to generate #/moduleB/source.cc, but in sourceFiles you have 'source.cc'.
So, one of ways to help you may be correct moduleB SConscript :
# Get the source.cc file
externalSourceFile = os.path.join('#moduleA', env['variantDir'], 'source.cc')
sourceTarget = os.path.join('#moduleB', 'source.cc')
sourceFiles = ['fileB.cc', sourceTarget]
cmdNode = env.Command(target = sourceTarget,
source = externalSourceFile,
action = Copy('$TARGET', '$SOURCE'))
libNode = env.Library(target = 'moduleB', source = sourceFiles)
And try to use Command like source file. instead filename. It's looks more correct.
moduleA :
conf2ccNode = env.Command(target = sourceFile,
source = scriptInput,
action = cmd)
libNode = env.Library(target = 'moduleA', source = conf2ccNode)
moduleB:
cmdNode = env.Command(target = sourceTarget,
source = externalSourceFile,
action = Copy('$TARGET', '$SOURCE'))
libNode = env.Library(target = 'moduleB', source = ['fileB.cc', cmdNode])
I found a solution to the problem, but I dont really understand why it works.
If I add a call to env.Default() with the targets I need built, then it works. So the SConscript files would then look like this:
moduleA/Sconscript
import os
Import('env')
scriptInput = '#file.conf'
sourceFile = os.path.join('#moduleA', env['variantDir'], 'source.cc')
conf2ccScript = File('#moduleA/conf2cc').abspath
# The script needs abspaths for input and output, not the scons '#' prepended
# the script syntax is: script <inputFile> <outputFile>
cmd = '%s %s %s' % (conf2ccScript, File(scriptInput).abspath, File(sourceFile).abspath)
# Generate source.cc file based on file.conf
conf2ccNode = env.Command(target = sourceFile,
source = scriptInput,
action = cmd)
libNode = env.Library(target = 'moduleA', source = sourceFile)
env.Depends(libNode, conf2ccNode)
env.Default([conf2ccNode, libNode])
moduleB/Sconscript
import os
Import('env')
sourceFiles = ['fileB.cc', 'source.cc']
# Get the source.cc file
externalSourceFile = os.path.join('#moduleA', env['variantDir'], 'source.cc')
sourceTarget = os.path.join('#moduleB', 'source.cc')
cmdNode = env.Command(target = sourceTarget,
source = externalSourceFile,
action = Copy('$TARGET', '$SOURCE'))
libNode = env.Library(target = 'moduleB', source = sourceFiles)
env.Depends(libNode, cmdNode)
env.Default(cmdNode, libNode)
So that leads to the question: If I dont specify Default() targets and there is more than one target, how does scons know which one to build?
Also, I still dont understand why scons doesnt resolve the dependancy in moduleB/SConscript that libNode has on the cmdNode.
My solution:
def CreateLibrary(env, name, sources, shared=True):
def GetObjectFile(sourceFileName):
def GetFileNameWithoutExtension(path):
return os.path.splitext(os.path.basename(path))[0]
def IsFileNameExist(newFileName):
return fileNames.count(newFileName) > 0
sourceAbsPath = os.path.abspath(sourceFileName)
fileNameWithoutExtension = GetFileNameWithoutExtension(sourceAbsPath)
destFileName = fileNameWithoutExtension
attemptNumber = 0
while IsFileNameExist(destFileName):
attemptNumber += 1
destFileName = fileNameWithoutExtension + str(attemptNumber)
fileNames.append(destFileName)
destFilePath = os.path.join(compilationDirRoot, destFileName)
if shared:
return env.SharedObject(destFilePath, sourceAbsPath)
else:
return env.StaticObject(destFilePath, sourceAbsPath)
objFiles = []
fileNames = []
compilationDirRoot = Dir('.').abspath
for src in sources:
if isinstance(src,str):
objFiles.append(GetObjectFile(src))
elif isinstance(src, SCons.Node.FS.File):
objFiles.append(GetObjectFile(SCons.Node.FS.File.rstr(src)))
else:
for f in src:
objFiles.append(GetObjectFile(str(f)))
if shared:
return env.SharedLibrary(name, objFiles, no_import_lib=True)
else:
return env.StaticLibrary(name, objFiles)
Example of use:
theora = CreateLibrary(env, 'theora', sources)
Easy question but I don't know the answer.
Let's say I have a scons build where my CCFLAGS includes -O1. I have one file needsOptimization.cpp where I would like to override the -O1 with -O2 instead. How could I do this in scons?
update: this is what I ended up doing based on bialix's answer:
in my SConscript file:
Import('env');
env2 = env.Clone();
env2.Append(CCFLAGS=Split('-O2 --asm_listing'));
sourceFiles = ['main.cpp','pwm3phase.cpp'];
sourceFiles2 = ['serialencoder.cpp','uartTestObject.cpp'];
objectFiles = [];
objectFiles.append(env.Object(sourceFiles));
objectFiles.append(env2.Object(sourceFiles2));
...
previously this file was:
Import('env');
sourceFiles = ['main.cpp','pwm3phase.cpp','serialencoder.cpp','uartTestObject.cpp'];
objectFiles = env.Object(sourceFiles);
...
Use Object() builder for fine-grained control over compilation, and then pass these objects to Program() builder.
E.g. instead of:
env = Environment()
env.Program(target='foo', source=['foo.cpp', 'bar.cpp', 'needsOptimisation.cpp'])
You need to use following:
env = Environment()
env_o1 = env.Clone()
env_o1.Append(CCFLAGS = '-O1')
env_o2 = env.Clone()
env_o2.Append(CCFLAGS = '-O2')
# extend these lists if needed
SRC_O1 = ['foo.cpp', 'bar.cpp']
SRC_O2 = ['needsOptimisation.cpp']
obj_o1 = [env_o1.Object(i) for i in SRC_O1]
obj_o2 = [env_o2.Object(i) for i in SRC_O2]
env.Program(target='foo', source=obj_o1+obj_o2)
You can avoid creation of separate clone of env variable if you provide CCFLAGS='-O2' right in the Object() call:
obj_o2 = [env.Object(i, CCFLAGS=env['CCFLAGS'] + ['-O2']) for i in SRC_O2]
Avoiding creating of a separate env variable requires (ref: bialix's answer) needs something like this.
obj_o2 = env.Object(SRC_O2, CCFLAGS=env['CCFLAGS'] + ['-O2']);
If you just do this (or in the for loop like bialix does)
obj_o2 = env.Object(SRC_O2, CCFLAGS='-O2');
then you lose all the builtin flags.