I have a main Haskell executable program with a cabal file. There, I specify the ghc-options.
This executable links against other libraries out in the wilderness. Will the ghc-options of the cabal files of these libraries be ignored?
I'm basically wondering whether the executable's ghc-options will be used for the entire enchilada (main executable + libraries).
Additional bounty notes: Please also expand on chi's comment below, namely, what exactly is the difference between ghc-options for compiling vs. linking. Which are which, and which are never needed in libraries? Maybe you can talk about some of the most important ones, such as the -threaded mentioned below.
Under the normal cabal-install workflow (and the stack workflow built atop it), flags specified in your Cabal file are local to your package, and should not trigger rebuilds. Similarly, options specified with --ghc-options on the command line are local to your package.
To your specific questions about -threaded, this flag has no effect on library code (as cabal-install will tell you), only on executables.
A brief listing of GHC flags is available here. In particular, note that -threaded is listed under Linking options, with a further link to Options affecting linking. From this information, we conclude that -threaded is only meaningful for executables because it signals to GHC that we wish to use the threaded runtime. If your package doesn't provide an executable, it has no need for any runtime, threaded or otherwise.
For a high-level explanation of compiling vs. linking: they are two of the steps between source code and executable. Compilation is the process of producing an object file from source code. Linking is the process of connecting the numerous object files which compose your executable. When you compile an executable, it has no idea that a function, say map exists unless you defined it, so it just compiles under the assumption that it does. The linking step is where we make all those names available and meaningful. In the case of -threaded, we are making the linking process aware of the threaded runtime, which all code calling on the runtime will use.
Since I don't know if you're using the standard cabal workflow, stack, or the new cabal.project workflow, here's a digression to discuss this behavior in the cabal.project case.
This is actually an open bug, right now.
The bug is tracked as issue 3883 on the Cabal GitHub (and somewhat in the related issue 4247).
Relevant to your question, according to current behavior, specifying flags in a ghc-options stanza in a cabal.project file causes those dependencies to be compiled (or recompiled, as the case may be) with those flags.
Related
When I run cabal build it uses some Haskell compiler to build the executables and/or test-suites in my .cabal file.
Can I control which compiler is used for the different targets? Ideally, I would like to have separate build targets that use ghc and ghcjs in the same .cabal file. It seems to me that someone might want to use ghc and hugs or two version of ghc in the same project. Is this currently possible?
Also, how does cabal decide what compiler to use when running cabal build? I saw there is a compiler option in my ~/.cabal/config file but changing it from ghc to ghcjs and uncommenting it, did not seem to change what cabal build does.
The compiler to use is determined during the configure step (or during an install step's implicit configure step, which does not share configuration options with a previous configure step). It is also determined by the entity building the package and cannot be influenced by the person writing the package. Probably what happened to you is that a previous cabal build implicitly invoked the configure step and chose a compiler; future builds will keep a previous choice of compiler over one stuck in your global configuration file. You can countermand that by simply manually running cabal configure again.
It is possible to cause a build to fail with the wrong implementation, e.g.
library
if impl(ghc)
buildable: False
will prevent cabal from trying to build the package using GHC. However, this isn't really useful for building separate parts of a package with separate compilers, as cabal will refuse to install a package unless it can build the whole thing with a single compiler.
Probably the best way forward is to make separate packages for things that should be built by separate compilers.
One modern Linux security hardening tactic is to compile & link code with the option -Wl,-z-noexecstack, this marks the DLL or binary as not needing an executable stack. This condition can be checked using readelf or other means.
I have been working with uClibc and noticed that it produces objects (.so files) that do not have this flag set. Yet uClibc has a configuration option UCLIBC_BUILD_NOEXECSTACK which according to the help means:
Mark all assembler files as noexecstack, which will mark uClibc
as not requiring an executable stack. (This doesn't prevent other
files you link against from claiming to need an executable stack, it
just won't cause uClibc to request it unnecessarily.)
This is a security thing to make buffer overflows harder to exploit.
...etc...
On some digging into the Makefiles this is correct - the flag is only applied to the assembler.
Because the flag is only passed to the assembler does this mean that the uClibc devs have missed an important hardening flag? There are other options, for example UCLIBC_BUILD_RELRO which do result in the equivalent flag being added to the linker (as -Wl,-z,relro)
However a casual observer could easily misread this and assume, as I originally did, that UCLIBC_BUILD_NOEXECSTACK is actually marking the .so file when it is in fact not. OpenWRT for example ensures that that flag is set when it builds uClibc.
Why would uClibc not do things the 'usual' way? What am I missing here? Are the libraries (e.g. librt.so, libpthread.so, etc) actually not NX?
EDIT
I was able to play with the Makefiles and get the noexecstack bit by using the -Wl,-z,noexecstack argument. So why would they not use that as well?
OK, it turns out after list conversation and further research that:
the GNU linker sets the DLL / executable stack state based on the 'lowest common denominator' i.e. if any linked or referenced part has an exec stack then the whole object is set this way
the 'correct' way to resolve this problem is actually to find and fix assembly / object files that use an exec stack when they dont need to.
Using the linker to 'fix' things is a workaround if you can't otherwise fix the root cause.
So for uClibc solution is to submit a bug so that the underlying objects get fixed. Otherwise anything linked with static libraries wont get a non-exec stack.
For my own question, if building a custom firmware not using any static libraries it is possibly sufficient to use the linker flag.
References:
Ubuntu Security Team - Executable Stacks
I wish to profile my program written in Haskell.
On compilation, I am told that I do not have profiling libraries for certain dependencies (e.g., criterion) installed and cabal aborts.
I have no interest in profiling parts of those dependencies; code called from main doesn't even use them.
How can I profile my application without installing profiling libraries I don't need and without removing all those dependencies?
A good way to circumvent having to compile everything with profiling is to use cabal sandbox. It allows you to set up a sandbox for one application only, and thereby you won't have to re-install your entire ~/.cabal prefix. You'll need a recent version of Cabal, so run cabal update && cabal install cabal-install first.
Once you initialise a sandbox, create a file cabal.config to include the necessary directives (in your case library-profiling: True; executable-profiling: True may also be handy.)
A side-effect of this is that you can test your code with dependencies that need not be installed globally, for example, experimental versions, or outdated versions.
EDIT: btw, I don't think you need to have profiling enabled for criterion to work. In any case, it works for me without profiling being enabled. Just write a Main module that contains main = defaultMain benchmarks where benchmarks has type [Benchmark], i.e. a list of benchmarks that you've written.
You then compile that file (say, we call it benchmarks.hs with ghc --make -o bench benchmarks.hs, and run the program, ./bench with the appropriate arguments (consult the criterion documentation for details. A good default argument is, say ./bench -o benchmarks.html which will generate a nifty report similar to this one)
I had the same problem this week, and although I had recompiled everything by hand, I was instructed in the IRC channel to do the following:
Go to your cabal config file (in case you don't know where)
Edit the line for enable library profiling (and while you are at it, enable documentation)
Run Cabal Install World
As mentioned in the question you refer to in your comment, a good way to solve this problem in the future is to enable profiling in the cabal configuration. This way all libraries are installed with profiling support. This might not be a satisfying solution but I guess many are opting for it.
If you are only interested in getting an impression of the memory usage of your program you can generate a heap profile of your program using -hT. More precisely, you have to compile the program with -rtsopts to enable RTS options then execute it using +RTS -hT. The compiler generates a file with the extension hp. You can convert the hp file into a postscript file with a heap profile using hp2ps. This should work without any profiling support, but note that I am too lazy to verify it as I have installed all libraries with profiling support ; )
In JVM based programs, you can compile a file to a .class file and be able to run the binary again, without compiling necessarily all the files.
Is it possible to do it in haskell? Is it imperative to compile and link all the files in the project? If yes, why?
What if there is no binary, you are only installing a library?
For GHC, you can change and recompile a single module without having to recompile modules depending on that, provided the exposed interface doesn't change. GHC's --make mode (default as of ghc-7.*) checks whether recompilation is necessary and recompiles only those modules where it can't determine that it's not necessary.
If you have a cabal package and you cabal build after changing one module, you can see from the compiler output that it doesn't recompile all modules in the package in general, only the changed module and [maybe] the ones depending on it.
If you build an executable, that of course has to be relinked, but many of the old object files can be reused.
If you build a library, the library archive of course has to be rebuilt, but many of the old object files can be reused.
Only a minimum amount of work is done
at compile time by the linker; it only
records what library routines the
program needs and the index names or
numbers of the routines in the
library. (source)
So it means ld.so won't check all libraries in its database,only those recorded by the application programe itself, that is to say, only those specified by gcc -lxxx.
This contradicts with my previous knowledge that ld.so will check all libraries in its database one by one until found.
Which is the exact case?
I will make a stab at answering this question...
At link time the linker (not ld.so) will make sure that all the symbols the .o files that are being linked together are satisfied by the libraries the program is being linked against. If any of those libraries are dynamic libraries, it will also check the libraries they depend on (no need to include them in the -l list) to make sure that all of the symbols in those libraries are satisfied. And it will do this recursively.
Only the libraries the executable directly depends on via supplied -l parameters at link time will be recorded in the executable. If the libraries themselves declared dependencies, those dependencies will not be recorded in the executable unless those libraries were also specified with -l flags at link time.
Linking happens when you run the linker. For gcc, this usually looks something like gcc a.o b.o c.o -lm -o myprogram. This generally happens at the end of the compilation process. Underneath the covers it generally runs a program called ld. But ld is not at all the same thing as ld.so (which is the runtime loader). Even though they are different entities they are named similarly because they do similar jobs, just at different times.
Loading is the step that happens when you run the program. For dynamic libraries, the loader does a lot of jobs that the linker would do if you were using static libraries.
When the program runs, ld.so (the runtime loader) actually hooks the symbols up on the executable to the definitions in the shared library. If that shared library depends on other shared libraries (a fact that's recorded in the library) it will also load those libraries and hook things up to them as well. If, after all this is done, there are still unresolved symbols, the loader will abort the program.
So, the executable says which dynamic libraries it directly depends upon. Each of those libraries say which dynamic libraries they directly depend upon, and so forth. The loader (ld.so) uses that to decide which libraries to look in for symbols. It will not go searching through random other libraries in a 'database' to find the appropriate symbols. They must be in libraries that are in the dependency chain.