Should I use an environment to invoke SConscript()? - scons

What difference in effect is there, if any, between these?
SConscript('subdir/SConscript')
env.SConscript('subdir/SConscript')
I initially wondered if this affected the called SConscript's DefaultEnvironment, so I tried this experiment:
SConstruct:
env = Environment(CC='my_cc')
SConscript('SConscript', exports={'program':'p1.c'})
env.SConscript('SConscript', exports={'program':'p2.c'})
SConscript:
Import('program')
Program(program)
Result:
$ scons -n -Q
gcc -o p1.o -c p1.c
gcc -o p1 p1.o
gcc -o p2.o -c p2.c
gcc -o p2 p2.o
But, as you can see, the construction environment used in SConscript seems unaffected.

If you simply write:
Program('main','main.cpp')
SCons will use what is called the "DefaultEnvironment" internally. It was added for people, like starting developers in Bioinformatics, when they don't need to work with special environment settings...or don't have several different environments in their builds.
It makes typing a little less verbose, but you're stuck with one Environment. However, one can always freely mix the DefaultEnvironment with self-defined ones...it's just a matter of taste.
But what's important to note is, that the DefaultEnvironment and additional Environments are not automatically related or connected, regarding their settings.
Finally, the DefaultEnvironment can also be accessed directly by using:
import SCons.DefaultEnvironment
SCons.DefaultEnvironment(tools=[])
env = Environment(...)
which here has the effect of stopping SCons from searching for tools/apps in the system twice...once for the DefaultEnv, and once for your special environment.
Internally, a command like "Program()" is simply a wrapper for "SCons.DefaultEnvironment.Program()"...that's all there is to it.
Based on all of the above, the SConscript() method is available in any Environment...and it should always be safe to include other build spec files with a simple
SConscript('...')
, which is also the method I prefer personally.

Related

Run `rustc` to check a program without generating any files

As cargo check shows, it's often useful to check if your program is well-formed without actually generating code (an often pretty time-consuming task). I want to check a single (library) Rust file with rustc directly (I cannot use Cargo!).
cargo check apparently works by calling this:
rustc --emit=metadata -Z no-codegen
This only emits metadata, a .rmeta file. Cargo actually needs that to check crates dependent on the checked crate. In my case I really don't need the metadata file.
I tried the following:
rustc --crate-type=lib --emit=
rustc --crate-type=lib --emit=nothing
But both didn't work. I use --crate-type=lib because my file doesn't have a main function. I need a platform-independent solution (I don't just want to use it on my machine, but use it in a public script).
How do I make rustc not write a single file?
You can just skip the --emit flag.
The final command would then be: rustc -Z no-codegen rust.rs
To quote my own GitHub comment about this very question, there are a few options for stable Rust:
rustc --emit=mir -o /dev/null seems to work in 1.18 and newer, writing nothing. (--emit=mir is the only helpful --emit option—the others try to create silly files like /dev/null.foo0.rcgu.o, except --emit=dep-info, which does no checking.)
rustc -C extra-filename=-tmp -C linker=true (i.e. use /bin/true as a “linker”) seems to work in all versions, writing some intermediate files but cleaning them up.
rustc --out-dir=<new empty temporary directory> is less clever and therefore perhaps less likely to break?
Note that linker errors, if any, will not be found by the first two options (nor by the nightly-only -Zno-codegen option).

How to properly create autoconf setup of netcdf 4.x?

I am not sure exactly what my question is as I get seriously turned around by autoconf/automake/libtoolize etc. Several of us are trying to autoconferize mbsystem. I've thrown a repo up of the work to date here:
https://bitbucket.org/schwehr/mbsystem
I'm trying to improve the netcdf setup to use nc-config, but am uncertain how to do this correctly. I am working on configure.in. It seems unable to find a header with AC_CHECK_HEADER("netcdfcpp.h") after INCLUDES="$INCLUDES ``$nc_config --cflags``" (pardon the incorrect back ticks) as taken from the gdl netcdf check. What is the correct way to update the path from nc-config --cflags?
http://gnudatalanguage.cvs.sourceforge.net/viewvc/gnudatalanguage/gdl/configure.in?revision=1.121
I then tried to use AX_PATH_GENERIC and get stuck on this error with m4_include([m4/ax_path_generic.m4])
Running autoconf ...
configure.in:29: error: possibly undefined macro: AC_SUBST
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure:12992: error: possibly undefined macro: AC_MSG_RESULT
Any help in getting better at creating a netcdf check that actually will work with funky non-standard install locations via nc-config and figuring out how to properly put a macro in the m4 directory would be a huge help.
A pointer to a package doing this really cleanly would be a super help. I've been looking at the netcdf, gdal, geos and gdl sources for examples. And things like the octopus netcdf check do not use nc-config... http://www.tddft.org/trac/octopus/browser/trunk/m4/netcdf.m4
The current setup with fink for netcdf 4.x:
nc-config --cflags --libs
-I/sw/opt/netcdf7/include -I/sw/include
-L/sw/opt/netcdf7/lib -lnetcdf
Thanks!
See Makefile.am: How to use curl-config and xml2-config in configure.ac? and substitute xml2/curl by netcdf.
Just use
PKG_CHECK_MODULES([libnetcdf], [netcdf])
in configure.ac, and then, in Makefile.am:
AM_CPPFLAGS = ${libnetcdf_CFLAGS}
bin_PROGRAMS = foo
foo_SOURCES = ...
foo_LDADD = ${libnetcdf_LIBS}
The "correct" way to use a third party m4 macro is to use aclocal (usually via automake) to generate aclocal.m4. If you are using automake, just add
ACLOCAL_AMFLAGS = -I m4
to Makefile.am and put
AC_CONFIG_MACRO_DIR([m4])
in configure.ac (after renaming configure.in).
If you are not using automake, add '-I m4' when you invoke aclocal. If you are not using aclocal, then you'll have to append the definition of the macro to the end of aclocal.m4 (and be careful to never run aclocal, as that will overwrite the file.)
There is no good example of a clean way to use conf scripts to do a build because using such scripts is an inherently flawed approach. A slightly cleaner approach is to stop using custom scripts and use pkg-config via PKG_CHECK_MODULES, but the cleanest way to do this is to educate your users. If the user wants to install the library in funky non-standard locations then they need to be educated enough to set LDFLAGS and CPPFLAGS appropriately.

Reusing custom makefile for static library with cmake

I guess this would be a generic question on including libraries with existing makefiles within cmake; but here's my context -
I'm trying to include scintilla in another CMake project, and I have the following problem:
On Linux, scintilla has a makefile in (say) the ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk directory; if you run make in that directory (as usual), you get a ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/bin/scintilla.a file - which (I guess) is the static library.
Now, if I'd try to use cmake's ADD_LIBRARY, I'd have to manually specify the sources of scintilla within cmake - and I'd rather not mess with that, given I already have a makefile. So, I'd rather call the usual scintilla make - and then instruct CMAKE to somehow refer to the resulting scintilla.a. (I guess that this then would not ensure cross-platform compatibility - but note that currently cross-platform is not an issue for me; I'd just like to build scintilla as part of this project that already uses cmake, only within Linux)
So, I've tried a bit with this:
ADD_CUSTOM_COMMAND(
OUTPUT scintilla.a
COMMAND ${CMAKE_MAKE_PROGRAM}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk
COMMENT "Original scintilla makefile target" )
... but then, add_custom_command adds a "target with no output"; so I'm trying several approach to build upon that, all of which fail (errors given as comment):
ADD_CUSTOM_TARGET(scintilla STATIC DEPENDS scintilla.a) # Target "scintilla" of type UTILITY may not be linked into another target.
ADD_LIBRARY(scintilla STATIC DEPENDS scintilla.a) # Cannot find source file "DEPENDS".
ADD_LIBRARY(scintilla STATIC) # You have called ADD_LIBRARY for library scintilla without any source files.
ADD_DEPENDENCIES(scintilla scintilla.a)
I'm obviously quote a noob with cmake - so, is it possible at all to have cmake run a pre-existing makefile, and "capture" its output library file, such that other components of the cmake project can link against it?
Many thanks for any answers,
Cheers!
EDIT: possible duplicate: CMake: how do i depend on output from a custom target? - Stack Overflow - however, here the breakage seems to be due to the need to specifically have a library that the rest of the cmake project would recognize...
Another related: cmake - adding a custom command with the file name as a target - Stack Overflow; however, it specifically builds an executable from source files (which I wanted to avoid)..
You could also use imported targets and a custom target like this:
# set the output destination
set(SCINTILLA_LIBRARY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk/scintilla.a)
# create a custom target called build_scintilla that is part of ALL
# and will run each time you type make
add_custom_target(build_scintilla ALL
COMMAND ${CMAKE_MAKE_PROGRAM}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk
COMMENT "Original scintilla makefile target")
# now create an imported static target
add_library(scintilla STATIC IMPORTED)
# Import target "scintilla" for configuration ""
set_property(TARGET scintilla APPEND PROPERTY IMPORTED_CONFIGURATIONS NOCONFIG)
set_target_properties(scintilla PROPERTIES
IMPORTED_LOCATION_NOCONFIG "${SCINTILLA_LIBRARY}")
# now you can use scintilla as if it were a regular cmake built target in your project
add_dependencies(scintilla build_scintilla)
add_executable(foo foo.c)
target_link_libraries(foo scintilla)
# note, this will only work on linux/unix platforms, also it does building
# in the source tree which is also sort of bad style and keeps out of source
# builds from working.
OK, I think I have it somewhat; basically, in the CMakeLists.txt that build scintilla, I used this only:
ADD_CUSTOM_TARGET(
scintilla.a ALL
COMMAND ${CMAKE_MAKE_PROGRAM}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scintilla/gtk
COMMENT "Original scintilla makefile target" )
... and then, the slightly more complicated part, was to find the correct cmake file elsewhere in the project, where the ${PROJECT_NAME} was defined - so as to add a dependency:
ADD_DEPENDENCIES(${PROJECT_NAME} scintilla.a)
... and finally, the library needs to be linked.
Note that in the commands heretofore, the scintilla.a is merely a name/label/identifier/string (so it could be anything else, like scintilla--a or something); but for linking - the full path to the actual `scintilla.a file is needed (which in this project ends up in a variable ${SCINTILLA_LIBRARY}). In this project, the linking basically occurs through a form of a
list(APPEND PROJ_LIBRARIES ${SCINTILLA_LIBRARY} )
... and I don't really know how cmake handles the actual linking afterwards (but it seems to work)
For consistency, I tried to use ${SCINTILLA_LIBRARY} instead of scintilla.a as identifier in the ADD_CUSTOM_TARGET, but got error: "Target names may not contain a slash. Use ADD_CUSTOM_COMMAND to generate files". So probably this could be solved smarter/more correct with ADD_CUSTOM_COMMAND - however, I read that it "defines a new command that can be executed during the build process. The outputs named should be listed as source files in the target for which they are to be generated."... And by now I'm totally confused so as to what is a file, what is a label, and what is a target - so I think I'll leave at this (and not fix it if it ain't broken :) )
Well, it'd still be nice to know a more correct way to do this eventually,
Cheers!

How to work around "scons: warning: Two different environments were specified for target"

Suppose I have an SConstruct file that looks like this:
env = Environment()
env.Program("a", ["a.c", "util.c"])
env.Program("b", ["b.c", "util.c"])
This build works properly with no SCons warning messages. However, if I modify this to specify different libraries for each Program build (the actual libraries are not relevant):
env.Program("a", ["a.c", "util.c"], LIBS="m")
env.Program("b", ["b.c", "util.c"], LIBS="c")
then I get the warning:
scons: warning: Two different environments were specified for target util.o,
but they appear to have the same action: $CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
This appears to be caused by the Program builder automatically creating a new environment for building the sources, even though it is just the LIBS variable that is different (and so only the link step needs to have a different environment). I can work around this by doing something like:
util = env.Object("util.c")
env.Program("a", ["a.c"] + util, LIBS="m")
env.Program("b", ["b.c"] + util, LIBS="c")
This uses a single Object builder for building util.c, then using the precompiled object file in each Program build, thus avoiding the warning. However, this should not really be necessary. Is there a more elegant way to work around this problem? Or is this actually a bug in SCons that should be fixed?
Context: I have nearly 2000 C source files compiled into about 20 libraries and 120 executables with lots of shared sources. I created the SConstruct file from the previous proprietary build system using a conversion script I wrote. There are about 450 "Two different environments" warning messages produced by SCons for a full build using my current SConstruct.
I found a workaround that doesn't involve creating extra variables to hold the object file nodes:
env.Program("a", ["a.c", env.Object("util.c")], LIBS="m")
env.Program("b", ["b.c", env.Object("util.c")], LIBS="c")
This isolates the build of util.c within a single environment. Although it is specified twice, once for each Program, SCons doesn't warn about this because it's the same source built with the same env object. Of course SCons only compiles the source once in this case.
You may use the Split function and a custom helper to simplify the build process for large projects:
def create_objs(SRCS, path=""):
return [env.Object(path+src+".cpp") for src in SRCS]
prg1 = Split("file_1 file_2 file_N")
prg2 = Split("file_2 file_5 file_8")
env.Program("a", create_objs(prg1), LIBS="x")
env.Program("b", create_objs(prg2), LIBS="y")
The object files are created only once, and they can be used in multiple builds. Hope this helps...
One issue I found in my code was that I was not using the target object path correctly. Or in otherwords I had a variant dir directive, but instead of using BUILDPATH i ended up using my original source code path. This way Scons was finding the object generated in target BUILDPATH and source path.
Creating a static library out of the first set of files and linking the library to the next set of files (which have some files in common with the first set) to create a target works as well.
env.StaticLibrary ("a", ["a.c","util.c"], LIBS = "m")
env.Program ("b", ["b.c","util.c"], LIBS = ["c","a"])

Installing and Linking PhysX Libraries in Debian Linux

I am trying to get PhysX working using Ubuntu.
First, I downloaded the SDK here:
http://developer.download.nvidia.com/PhysX/2.8.1/PhysX_2.8.1_SDK_CoreLinux_deb.tar.gz
Next, I extracted the files and installed each package with:
dpkg -i filename.deb
This gives me the following files located in /usr/lib/PhysX/v2.8.1:
libNxCharacter.so
libNxCooking.so
libPhysXCore.so
libNxCharacter.so.1
libNxCooking.so.1
libPhysXCore.so.1
Next, I created symbolic links to /usr/lib:
sudo ln -s /usr/lib/PhysX/v2.8.1/libNxCharacter.so.1 /usr/lib/libNxCharacter.so.1
sudo ln -s /usr/lib/PhysX/v2.8.1/libNxCooking.so.1 /usr/lib/libNxCooking.so.1
sudo ln -s /usr/lib/PhysX/v2.8.1/libPhysXCore.so.1 /usr/lib/libPhysXCore.so.1
Now, using Eclipse, I have specified the following libraries (-l):
libNxCharacter.so.1
libNxCooking.so.1
libPhysXCore.so.1
And the following search paths just in case (-L):
/usr/lib/PhysX/v2.8.1
/usr/lib
Also, as Gerald Kaszuba suggested, I added the following include paths (-I):
/usr/lib/PhysX/v2.8.1
/usr/lib
Then, I attempted to compile the following code:
#include "NxPhysics.h"
NxPhysicsSDK* gPhysicsSDK = NULL;
NxScene* gScene = NULL;
NxVec3 gDefaultGravity(0,-9.8,0);
void InitNx()
{
gPhysicsSDK = NxCreatePhysicsSDK(NX_PHYSICS_SDK_VERSION);
if (!gPhysicsSDK)
{
std::cout<<"Error"<<std::endl;
return;
}
NxSceneDesc sceneDesc;
sceneDesc.gravity = gDefaultGravity;
gScene = gPhysicsSDK->createScene(sceneDesc);
}
int main(int arc, char** argv)
{
InitNx();
return 0;
}
The first error I get is:
NxPhysics.h: No such file or directory
Which tells me that the project is obviously not linking properly. Can anyone tell me what I have done wrong, or what else I need to do to get my project to compile? I am using the GCC C++ Compiler. Thanks in advance!
It looks like you're confusing header files with library files. NxPhysics.h is a source code header file. Header files are needed when compiling source code (not when linking). It's probably located in a place like /usr/include or /usr/include/PhysX/v2.8.1, or similar. Find the real location of this file and make sure you use the -I option to tell the compiler where it is, as Gerald Kaszuba suggests.
The libraries are needed when linking the compiled object files (and not when compiling). You'll need to deal with this later with the -L and -l options.
Note: depending on how you invoke gcc, you can have it do compiling and then linking with a single invocation, but behind the scenes it still does a compile step then a link step.
EDIT: Extra explanation added...
When building a binary using a C/C++ compiler, the compiler reads the source code (.c or .cpp files). While reading it, there are frequently #include statements that are used to read .h files. The #include statements give the names of files that must be loaded. Those exact files must exist in the include path. In your case, a file with the exact name "NxPhysics.h" must be found somewhere in the include path. Typically, /usr/include is in the path by default, and so is the current directory. If the headers are somewhere else such as a subdirectory of /usr/include, then you always need to explicitly tell the compiler where to look using the -I command-line switches (or sometimes with environment variables or other system configuration methods).
A .h header file typically includes data structure declarations, inline function definitions, function and class declarations, and #define macros. When the compilation is done, a .o object file is created. The compiler does not know about .so or .a libraries and cannot use them in any way, other than to embed a little bit of helper information for the linker. Note that the compiler also embeds some "header" information in the object files. I put "header" in quotes because the information only roughly corresponds to what may or may not be found in the .h files. It includes a binary representation of all exported declarations. No macros are found there. I believe that inline functions are omitted as well (though I could be wrong there).
Once all of the .o files exist, it is time for another program to take over: the linker. The linker knows nothing of source code files or .h header files. It only cares about binary libraries and object files. You give it a collection of libraries and object files. In their "headers" they list what things (data types, functions, etc.) they define and what things they need someone else to define. The linker then matches up requests for definitions from one module with actual definitions for other modules. It checks to make sure there aren't multiple conflicting definitions, and if building an executable, it makes sure that all requests for definitions are fulfilled.
There are some notable caveats to the above description. First, it is possible to call gcc once and get it to do both compiling and linking, e.g.
gcc hello.c -o hello
will first compile hello.c to memory or to a temporary file, then it will link against the standard libraries and write out the hello executable. Even though it's only one call to gcc, both steps are still being performed sequentially, as a convenience to you. I'll skip describing some of the details of dynamic libraries for now.
If you're a Java programmer, then some of the above might be a little confusing. I believe that .net works like Java, so the following discussion should apply to C# and the other .net languages. Java is syntactically a much simpler language than C and C++. It lacks macros and it lacks true templates (generics are a very weak form of templates). Because of this, Java skips the need for separate declaration (.h) and definition (.c) files. It is also able to embed all the relevant information in the object file (.class for Java). This makes it so that both the compiler and the linker can use the .class files directly.
The problem was indeed with my include paths. Here is the relevant command:
g++ -I/usr/include/PhysX/v2.8.1/SDKs/PhysXLoader/include -I/usr/include -I/usr/include/PhysX/v2.8.1/LowLevel/API/include -I/usr/include/PhysX/v2.8.1/LowLevel/hlcommon/include -I/usr/include/PhysX/v2.8.1/SDKs/Foundation/include -I/usr/include/PhysX/v2.8.1/SDKs/Cooking/include -I/usr/include/PhysX/v2.8.1/SDKs/NxCharacter/include -I/usr/include/PhysX/v2.8.1/SDKs/Physics/include -O0 -g3 -DNX_DISABLE_FLUIDS -DLINUX -Wall -c -fmessage-length=0 -MMD -MP -MF"main.d" -MT"main.d" -o"main.o" "../main.cpp"
Also, for the linker, only "PhysXLoader" was needed (same as Windows). Thus, I have:
g++ -o"PhysXSetupTest" ./main.o -lglut -lPhysXLoader
While installing I got the following error
*
dpkg: dependency problems prevent configuration of libphysx-dev-2.8.1:
libphysx-dev-2.8.1 depends on libphysx-2.8.1 (= 2.8.1-4); however:
Package libphysx-2.8.1 is not configured yet.
dpkg: error processing libphysx-dev-2.8.1 (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
*
So I reinstalled *libphysx-2.8.1_4_i386.deb*
sudo dpkg -i libphysx-2.8.1_4_i386.deb

Resources