My team is creating a build system based on SCons. We have created a bunch of helper classes in our own site_scons/site_tools folder.
My task is to create and run tests on our code, using pyunit. The test code would probably live in a subfolder, with the directory layout looking something like:
SConstruct
our_source_code/
Sconscript
site_scons/
site_tools/
a.py
b.py
c.py
tests/
test_a.py
test_b.py
test_c.py
Now my question is: What is the best way to invoke our tests, given they will probably require the correct SCons environment set up? (that is a.py uses SCons.Environment)
Do I add a Builder or a Command? Or something else?
I think the best approach would be to use the test setup code from SCons itself. This requires a SVN checkout of SCons, as the test files are not shipped with the regular SCons tarballs. This is probably workable, as not everyone in your team would be writing tools and running tests on them.
For example, this is the test for javac. Basically you write out files that you want, run a SConstruct, then check the results are what you expected. You can mock tools with Python scripts, to ensure they are really called with the flags and files that you expect. For example:
import TestSCons
test = TestSCons.TestSCons()
test.write('SConstruct', '''env = Environment(tools = ["yourtool"])
env.RunYourTool()''')
test.write('sourcefile.x', 'Content goes here')
test.run(arguments = '.', stderr = None)
test.must_match('outputfile', 'desired contents')
test.pass_test()
There are also more instructions on writing SCons tools tests on the swtoolkit wiki, which is a seemingly-defunct SCons extension from Google. The info on the wiki is still useful, and there are some good examples on how to write tests for custom SCons tools.
Related
I have a few test files residing in test/ directory which has an __init__.py file. The test files are basically Python files but have the extension as .thpy (Python test files). Each of these tests internally use unittest.
For code coverage purposes, I'm using the trace module. coverage.py, unfortunately would have been ideal but at this time, doesn't give information on the number of line hits per line.
unittest and trace are not really compatible. unittest.py doesn't play well with trace.py - why?
For example: I have a file name cluster_ha.thpy. From the solution given for the above question, I need to mention module='cluster_ha' inside cluster_ha.thpy but because of the extension, Python doesn't look at it as a Python module.
Is there any way to get around this? Is there a hack that can make other extensions to be considered as a Python module? Or perhaps, there's another module out there which can help me get code coverage?
I have a code snippet similar to this:
# Compile protobuf headers
env.Protoc(...)
# Move headers to 'include' (compiled via protobuf)
env.Command([include headers...], [headers...], move_func)
# Compile program (depends on 'include' files)
out2 = SConscript('src/SConscript')
Depends(out2, [include headers...])
Basically, I have Protoc() compiling protobuf files, then the headers are moved to the 'include' directory by env.Command() and finally the program is compiled through a SConscript file in the 'src'.
Since these are header files that are being moved (that the src compilation depends on), they are not explicitly defined as a dependency by scons (as far as I understand). Thus, the compilation runs, but the header files haven't been moved so it fails. I have tried exposing the dependency via Depends() and Requires() without success.
I understand that in the usual case, scons should "figure-out" dependencies, but I don't know how it could do that here.
Thanks!
You seem to be thinking in "make" ways about your build process, which is the wrong approach when using SCons. You can't order single build steps by putting them in different SConscripts, and then including those in a special order. You have to define proper dependencies between your actual sources (C/CPP files for example) and a target like a program or PDF file. Then SCons is able to figure out the correct build order, and will traverse through the folder structure of your project automatically. If required, it will enter subfolders more than once when the dependency graph (DAG) dictates this. Defining this kind of dependencies between inputs and outputs is usually done, using a Builder...and in your case the Install() builder would be a good fit. Please also regard the hints for #2 in the list of "most frequently-asked FAQs" ( https://bitbucket.org/scons/scons/wiki/FrequentlyAskedQuestions).
Further, I can only recommend to read a little more in the UserGuide ( http://www.scons.org/doc/production/HTML/scons-user.html ) to get a better feeling for how to do things in a more "SConsy" way. If you get stuck, feel free to ask further questions on our mailing list at scons-users#scons.org (see http://www.scons.org/lists.php ).
Finally, if you have a lot of steps that you want to execute in serial, and that don't require any special input/output files, SCons is probably not the right tool for your current task. It's designed as a file-oriented build system with automatic parallelization in mind, a simple (Python?) script might be better at the mere serial stuff...
I have a multimodule maven project with modules such as: moduleA, moduleB, moduleC. Then I have an entirely separate moduleTest that has integration tests in it run by the failsafe plugin.
I want to have a report generated by cobertura(or any other maven plugin) that can tell me which lines in all of moduleA, B and C are covered by my integration tests.
I don't think http://jira.codehaus.org/browse/MCOBERTURA-65 helps me. Is there an easy way to achieve this?
One of possible solutions is to use Emma. You shall setup code instrumentation in your source code modules by using instrument goal:
http://mojo.codehaus.org/emma-maven-plugin/instrument-mojo.html
After successful compilation and instrumentation, tests execution will generate coverage data. Then You can execute emma standalone tool to generate report based on it:
java emma report -r txt,xml,html -in coverageA.em,coverageB.em,coverageC.em,coverage.ec -sp srcA/,srcB,srcC
coverage*.em shall be replaced with proper paths to generated by Emma metadata in source code modules, coverage.ec is path to coverage file generated in test module, src* directories shall be replaced with paths to source code directories. Here is detailed documentation:
http://emma.sourceforge.net/reference/ch02s04s03.html
You can do it also with jacoco (also in a quite tricky way) but due too low reputation I cannot put more than 2 links. So like it like it! :)
I have a project build by scons.
In the project there are multiple components, including client, shell, engine and etc...
Each component uses different compile options so they are split into different env.
And both shell and engine are going to require client libraries built first.
In the environment settings, both shell and engine has something like "-lclient -L[installpath]/lib", and SConscriptClient is going to build the libclient.a in [installpath]/lib.
So I expect SConscriptClient is run before everything else.
so in the code I have something like:
clientbuild = clientEnv.SConscript ( 'SConscriptClient', variant_dir=clientDir )
if hasShell:
shellbuild = shellEnv.SConscript ( 'SConscriptShell', variant_dir=shellDir )
Depends ( shellbuild, clientbuild )
if hasEngine:
enginebuild = engineEnv.SConscript ( 'SConscriptEngine', variant_dir=engineDir )
Depends ( enginebuild, clientbuild )
However it seems scons is not smart enough to understand the dependencies between client/shell and engine ( that means the Depends call doesn't take effect ). It still try to run SConscriptShell before SConscriptClient
Is there anything i can do to set the dependeices between sconscript?
You shouldn't have to explicitly set these dependencies with Depends(). If the client library is a target built by SCons and both the shell and engine link that library, then SCons should be able to implicitly determine the dependencies and build the client first.
Basically, I see 2 issues here:
Why doesn't SCons implicitly figure out the dependencies?
Why isn't it working as is with the explicit calls to Depends()?
If we figure out number 1, then we wont have to figure out number 2. But just to be complete, I think number 2 isnt working because of what the call to SConscript() is returning. In the subsidiary SConscript scripts (SConscriptClient, SConscriptShell, and SConscriptEngine) are you returning the target? If not, I would imagine the clientbuild variable would be None. To return the target use the Return() SCons function and pass it the return value of the Library() builder.
As for why SCons cant figure out the dependencies implicitly, we would need to see the subsidiary SConscript build scripts. But I can imagine its because you are probably specifying the client library "by hand", so SCons doesnt see the dependency.
The way to build a program with a library so that SCons can see the dependency, you need to use the LIBS construction variable, as follows:
env.Append(LIBS='client')
env.Append(LIBPATH='path/to/client/lib')
env.Program(target='shell', source='shell.cc')
Notice I don't use the -l nor the -L flags above, SCons will add those in a platform independent manner. If you're specifying the library "by hand" by specifying it like this: '-lclient' then SCons wont see the dependency. This is by design, and is more interesting with include paths: if you have a lot of include paths with header files that will almost never change, then (for performance reasons) you don't want SCons to scan them for changes, and thus specify them "by hand".
One additional comment, normally the different environments are passed to the subsidiary SConscript build scripts differently, as follows:
clientEnv = Environment()
# set the clientEnv accordingly
SConscript ('SConscriptClient', variant_dir=clientDir, exports=['clientEnv'] )
SConscriptClient:
Import('clientEnv')
# You may want to clone the clientEnv here, if you want to make
# changes that you don't want seen in the rest of the build
I'm working on implementing a build system using scons for a somewhat large software project. There is a directory structure which separates the code for individual libraries and programs into their own directories. With our existing make system, I can do a "make clean" in a single program directory and it will only clean the files associated with the source in that directory. If I do an "scons -c" though, it recognizes that the program depends on a slew of libraries that are in sibling (or cousin) directories and cleans all of the files for those as well. This is not what I want since I then have to rebuild all of these libraries which can take several minutes.
I have tried playing with the "NoClean()" command, but have not gotten it to work in the way I need. Given the size of the code base and complexity of the directory structure, I can't realistically have a NoClean() line for every file in every library.
Is there any way to tell scons to ignore any dependencies above the current directory when doing a clean (i.e. scons -c) ?
I'd love to have a good answer to this myself.
The only solution that I can offer for now is that you get Noclean working.
So in your library, you should have something like this
lib_objs = SharedObject(source_list)
mylib = SharedLibrary('libname', lib_objs)
So for this we want to protect the library and the sources from being cleaned.
NoClean([mylib, lib_objs])
Notice that I had to split the building of the object files from the library because I want to be able to pass them to NoClean as well.
Try using the target name when cleaning.
scons -c aTargetName
You can use the SCons Alias() function to simplify the target name and to also group several target names into one alias.
With this approach you'll have to add an alias in each appropriate subdir, which isn't necessarily a bad thing :)