My project contains third-party library sources that located in separate directory:
/prj
/src
/app
/lib1
/lib2
/third-party-lib
SConscript
...
SConstruct
Compiling of the third-party-lib is quite long because of large library size. I'm never change the sources of the library and the only case when the library needs to be rebuild is changes of the build options (compiler flags, for example).
To rebuild the project I issue commands:
scons -c && scons
In this case SCons removes all build products including the third-party-lib and subsequent build consumes a significant time due to third-party-lib compiling which, as said above, never changed. Method:
lib = env.StaticLibrary(Target, obj)
env.NoClean(lib)
does not give desired result - this preserves only final library file (lib.a) from clean. I've tried to preserve object files:
obj = env.Object(Sources)
env.NoClean(obj)
but this solves the problem only partially, because some object files compiled implicitly - the library code contains Qt code which processed by Qt meta-object compiler (MOC), therefore these object files do not included in 'obj' list.
Is there a way to prevent such third-part library rebuilding every time when the project rebuild carried out?
The SCons way of doing things would be to not call "scons -c", but only
scons
if you want to rebuild your project.
It is SCons main strength to get all the dependencies (implicit and explicit) right, even for an iterative rebuild where only a handful of files have changed. By using the "-c" option, like you may be used from other build tools like "make", you're shortcutting this feature and creating problems where there would be none usually.
I assume that you're using your "make clean; make all" approach because you haven't properly defined all the dependencies in your project yet. Please do that first, it will help your build in the long run.
And no, there is no method that will prevent "cleaning" for a whole folder and its subdirs.
Use env.Glob(), it will see files which SCons knows about but are not yet created when that logic is run.
Related
I work on a Rust project that has a lot of packages as explicit or implicit dependencies (~420). When I want to rebuild the target after changing the .env file (that configures things like IP to download files from), I would like to rebuild only the packages that I authored, not all the dependencies.
How can I tell cargo build to use the previously compiled dependencies, but not use the previously compiled package that uses the .env file as input?
Ideally, cargo build would realize that the .env file has changed and automatically decide to rebuild only the parts that use the .env file, but unfortunately this doesn't seem to be the case.
So the second best solution is to manually tell cargo build at which point in the build graph to start off again.
We're using the dotenv crate https://crates.io/crates/dotenv crate to read the .env file.
I tried cargo clean -p nextclade to tell it to clean only the package in question that I'm working on - but that still cleans up all the dependencies which cause my build to take 5 minutes rather than 2 minutes (if using compiled dependencies).
There's a question that seems to ask a similar question, but that question is actually a different use case/set up, so it's not a duplicate: How does cargo decide whether to rebuild the deps or not?
I am currently building some docker images.
I found that the Linux distribution I was using was hard to adapt to Docker multi-stage builds until I found Nix.
With Nix, I can copy files among images (COPY --from=source/image /nix/store /nix/store) without worrying about conflicts and breaking things.
But I found that it installed too many things after running nix-env -i curl command.
warning: there are multiple derivations named 'curl-7.60.0'; using the first one
installing 'curl-7.60.0'
these paths will be fetched (49.44 MiB download, 203.64 MiB unpacked):
/nix/store/0yaiablzxhd8ki5qan156ydz78grlav7-nghttp2-1.32.0-bin
/nix/store/0zvcf4dnlcd4bk84qmxcxm1pbc534chv-openssl-1.0.2o-bin
/nix/store/3xvnr0y2mx7g8b796rb9p77bjfbaw03h-linux-headers-4.15
/nix/store/4bikvz91b83sycavf35lmby65m6zxgch-libssh2-1.8.0-dev
/nix/store/504vcw350rp1yh31razv0mq2vsgp0izh-libkrb5-1.15.2-dev
/nix/store/5gzy6cacylfb0lha2yd0i0as0k1d0d5v-libev-4.24
/nix/store/5xnniwzazzlg6qinhrwammxxwsq5c1di-nghttp2-1.32.0-dev
/nix/store/7l1smzwil1kxyyfayzl6lg1hw9m4iwmw-nghttp2-1.32.0
/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23
/nix/store/93ljbaqhsipwamcn1acrv94jm6rjpcnd-acl-2.2.52
/nix/store/dgp8mnf40pmwh8ghpcfda1vcwcy34w6z-curl-7.60.0-devdoc
/nix/store/gbddfvxzjjqpgkr17whn8ynh9z8afz8l-curl-7.60.0-debug
/nix/store/imfm3gk3qchmyv7684pjpm8irvkdrrkk-gcc-7.3.0
/nix/store/jg9yh6cm4iwcpl4l18g7mr9y7sdwav5q-curl-7.60.0-dev
/nix/store/jsmnk16iwb9xrm3c6jv2fyxkh7xr7q3j-curl-7.60.0-man
/nix/store/lyd89mv72m8a0aw1a4idfimyi0rb2b13-glibc-2.27-dev
/nix/store/n7qp8pffvcb5ff52l2nrc3g2wvxfrk75-coreutils-8.29
/nix/store/pa4q0szxz23bd6srry91gmw08fmwgfw2-libkrb5-1.15.2
/nix/store/q239yikz665n4a5rff7rg2vc7jpay6xb-openssl-1.0.2o-dev
/nix/store/rmq6gnybmxxzpssj3s63sfjivlq4inrm-attr-2.4.47
/nix/store/szdi35clpzj13c8dhfzh55fj6hk0z8j6-glibc-2.27-bin
/nix/store/v5xh3glylamhfg586hcykn6hlk4n41dh-nghttp2-1.32.0-lib
/nix/store/vawc9a89l53mf05yq0k1910q7dakd99w-perl-5.24.3
/nix/store/vl5k9m1pjkd6cm9125afic1kj06y4i6b-curl-7.60.0-bin
/nix/store/y8cfvcvya61l260jil989lcmkia5b5gh-zlib-1.2.11-dev
/nix/store/z4k2pbdd8pz9mjc0p5394j0zp435fcc5-curl-7.60.0
It is important to keep docker images slim and I do not think curl need dependencies like gcc or linux-headers at runtime.
Is there a way for Nix to exclude the dependencies of these source or dev libraries?
Build dependencies become runtime dependencies whenever a path name to the build dependency is included in the package. This is necessary because there is no general way to tell whether such a reference is actually used by a program.
The best way to avoid having build dependencies in your closures is by not referencing them in the first place. The next best thing is to understand why the reference is there and, if safe, modify the package build script to remove the reference.
In order to figure out where these references come from, you can make use of the Nix 2.0 nix why-depends command. It will tell you the shortest path, or all paths that lead from the first argument package to the second argument package. You can also use store paths instead of the attribute paths in the examples of nix why-depends --help.
The method for removing the dependency depends on the referencing package, so there's no general formula for that. General hacks to remove the reference in unsafe ways exist, but they are probably not worth the risk.
I have a stack multi-package project where the "multi-package" is zip-conduit-0.2.2.2 (I needed to tweak some dependency versions to get it to compile with nightly-2015-10-12).
I use zip-conduit in the main package and I also want to in Setup.hs in order to be able to package up zip files.
I have build-type: Custom in my cabal file and stack build attempts to build Setup.hs. The problem is that building Setup.hs cannot find the dependency zip-conduit-0.2.2.2.
Can I use a multi-package dependency in Setup.hs? If so how? and if not, what is my alternative?
Note: I have seen this reference https://github.com/commercialhaskell/stack/pull/899, but I don't understand it.
It looks like you need to set the explicit-setup-deps setting to true for that package. It's unfortunate that Stack can't handle this situation automatically, but without dependency information in .cabal files, there seems to be no alternative.
See the linked issues from the documentation if you're interested in the history here.
I don't think it is possible to declare package dependencies for Setup.hs itself.
Some ideas:
Require that the dependent packages have already been installed.
Have Setup.hs shell out to an external command to perform the zipping, and just require that the external command has to exist.
Make the installer program an additional exe target in the cabal file. Before proceeding with the build of the library, Setup.hs can build the installer program target, then build the library, then use the built installer exe to install the library.
The source tree for happy contains AttrGrammarParser.ly and Parser.ly and the source tree for alex contains Scan.x. Yet, as far as I can tell in order to compile happy, we need to transform the .ly files into .lhs files using... happy, and in order to compile alex we need to transform the .x files into .hs files using... alex.
So it seems like there must be some bootstrapping going on here in order to compile either tool.
The Setup.lhs files for each project contain some template expansion, but, as far as I can tell, don't do anything in particular to do the bootstrapping.
How and where is the bootstrapping done?
I see that you are looking at the source tree of the darcs repositories for these packages on darcs.haskell.org. If you look at the actual tarballs on Hackage, you'll see something a bit different:
https://hackage.haskell.org/package/alex-3.1.4/src/dist/build/alex/alex-tmp/
https://hackage.haskell.org/package/happy-1.19.5/src/dist/build/happy/happy-tmp/
So basically the build artifacts necessary are shipped with the Hackage tarball. Cabal then just uses the build artifacts during the build process, thus avoiding the need to bootstrap locally. Cabal also knows how to preserve such build artifacts when you run cabal sdist for your own packages which you want not to depend on happy or alex, but last I checked this doesn't interoperate well with sandboxes, fwiw.
By the way, alex and happy development has moved to github:
https://github.com/simonmar/alex/
https://github.com/simonmar/happy/
I try to build my multiproject application in Visuas C++ 10 in release mode.
I get the next link error:
2> All outputs are up-to-date.
2>PidAppLib.lib(ThreadWin32.obj) : error LNK2011: precompiled object not linked in; image may not run
2>C:\infinite_memories_svn\projects\sw\NewPidTools\PidWriter\Release\PidLabler.exe : fatal error LNK1120: 1 unresolved externals
2>
2>Build FAILED.
How can I know which object unresolved?
ThreadWin32.obj is external lib which was downloaded from internet.
Ok.
I've solved the problem.
I have 2 libraries, let say a.lib and b.lib, and project, c.
a.lib uses b.lib so in librarian dependencies of a.lib I added b.lib, c uses a.lib so I added
it to dependencies.
It works in debug but in release I should also add b.lib to dependencies of c.
Strange but it solved the problem
I had this error in a large project I was refactoring. During the refactoring process I would occasionally compile individual source files to check changes I'd made without having to build the entire project. Depending on your project configuration this can result in .obj files being output in a different directory to where they would normally be output when you build the project. In my case the .obj files for source files compiled manually were output in the source code directory instead of the intermediate build directory.
A side-effect of this appeared to be that when building the whole project, source files that had been manually compiled might not be recompiled even if other changes elsewhere meant they should be (like a change that required recompiling the precompiled header file).
The solution for me was to manually delete all .obj files generated by manually compiling individual source files. Depending on project configuration executing Clean on the project might also work.