I'm a bit confused about how a dependency on a package affects including it in Stackage LTS; specifically, if
package A requires package B, and
package A works when package B is installed as an extra-dep on top of LTS-X.Y, but
package B itself is not in LTS-X.Y,
does package A have to be excluded from LTS-X.Y, particularly if
the only reason B is excluded is because of a test suite dependency, not a dependency in the library itself?
I'll copy/paste my answer on github
does package A have to be excluded?
No, it doesn't have to be excluded. Here's why:
even if the only reason B is excluded is because of a test suite dependency
In this case, we can add B to the build plan and mark it under the skipped-tests section in order to avoid pulling in its test suite dependencies. This is true of both LTS and nightly snapshots.
(However, a preferable course of action would be to remedy B's dependency issue so that the test suite can be run.)
To further clarify, in response to #bergey's answer:
packages are only included if the package's maintainer agrees to keep it up to date with respect to its dependencies
This is only true of packages explicitly included. Some packages are transitive dependencies, which are included implicitly, and are not necessarily held to such strict standards. (However, in the future we may eliminate the concept of implicit inclusion and instead include all packages explicitly.)
Exceptions can also be made so that a package may be included, even though its test suite or its benchmarks have incompatible dependency constraints with the snapsnot.
Of course the preferred way to go is to not need to make such exceptions, and we encourage all maintainers to keep all of their build targets up to date.
Finally, allow me to note that this question would probably be more well suited for the stackage mailing list, which is admittedly not well publicied or utilized.
Yes, for every package in a given Stackage snapshot, all of its transitive dependencies are also in the snapshot. Also, packages are only included if the package's maintainer agrees to keep it up to date with respect to its dependencies. There are more details about this in the README on github. An excerpt:
All packages are buildable and testable from Hackage. We recommend the Stack Travis script, which ensures a package is not accidentally incomplete.
All packages are compatible with the newest versions of all dependencies (You can find restrictive upper bounds by visiting http://packdeps.haskellers.com/feed?needle=PACKAGENAME).
All packages in a snapshot are compatible with the versions of libraries that ship with the GHC used in the snapshot (more information on lenient lower bounds).
Related
It is my understanding that when using stack to compile a project, no version constraints for dependencies should go in the .cabal (or package.yaml) files, because the resolver picks specific versions for you. This includes the GHC version and its base library. However, when creating a new project with stack new, it automatically adds a version constraint to the dependency to base.
Excerpt of auto-generated package.yaml
dependencies:
- base >= 4.7 && < 5
Why is that?
I don't know the canonical answer. But here's one reason why it might be nice.
For what is currently the only realistic implementation of Haskell, namely, GHC, the base version and the compiler version are inextricably linked. This means that suitable base constraints also communicate which version of the compiler is intended to be used.
That latter piece is interesting information to know about a package at a glance.
Now, it's also true that stack resolvers and GHC versions are inextricably linked. So you might think that information is already available. BUT currently, Hackage (the place that most Haskell packages get hosted for use by others) displays a bunch of information taken from cabal files, but no information taken from stack files. So if you want the information about GHC version to be conveniently available from the autogenerated Hackage summary of the package, this is one way to easily and automatically do that.
My project has a dependency on another project, and I'm using git dependency as follows in the setup.py file:
setup(
name="cake",
version="0.1",
install_requires=[
"flan # git+ssh://git#github.com/terrymcguire/flan.git#egg=flan"
]
)
Suppose they both depend on pyyaml. Is it best practice to include a "pyyaml==5.1.2" inside both projects' setup.py, install_requires: ... (or requirements.txt as you prefer), and make sure the versions are the same, or is it recommended to only have pyyaml listed as a dependency in the flan project, and then inherit the version in the parent project, even though it's then less clear that pyyaml is a dependency of the parent project, and if one day I no longer depend on flan, I might not notice I may have broken other code?
1.
Is it best practice to include a "pyyaml==5.1.2" inside both projects' setup.py, install_requires: ... (or requirements.txt as you prefer) [...]?
Only applications should (possibly) pin requirements to a specific version. Libraries should restrict to a range of known compatible versions (as accurate as possible).
In general I believe pinning the versions of dependencies in setup.py (or pyproject.toml) is a bad idea, since those can not be (easily) overruled by the end user, the one ultimately installing the projects (doesn't matter if applications or libraries) and the one who should have the last word on what gets installed. On the other hand it is good practice to give a recommendation of a combination of pinned versions of dependencies that is known to work well (because it has been tested) in the form of a requirements.txt file that the end user might opt to use or not (for applications, this doesn't make much sense for libraries).
Read for example Donald Stufft's article "setup.py vs requirements.txt".
2.
is it recommended to only have pyyaml listed as a dependency in the flan project, and then inherit the version in the parent project, even though it's then less clear that pyyaml is a dependency of the parent project [...]?
The general (obvious) rule is that all projects should list all of their own dependencies and only their own dependencies. Anything else doesn't make any sense (of course there might be exceptions as always).
Common practice for version numbers of npm dependencies in package.json has been to enter exact version numbers (like 1.2.4) instead of inexact version numbers (like ^1.2.4 which allows installing bug fix releases like 1.2.5) to make sure a future installation will not break due to changes in dependencies (see for example this article).
Using exact version numbers has a drawback in that you can't automatically update bug fix versions of dependencies. This is an issue when it's nested dependencies having security fixes or bug fixes. For example, at this moment the package karma-browserstack-launcher uses browserstack, which is using an outdated version of https-proxy-agent containing a security vulnerability. This becomes very visible right now thanks to npm audit which looks for security issues in dependencies.
Since some time we have package-lock.json, which is used to lock down the version numbers of all dependencies. This may change the way we deal exact or inexact version numbers in package.json.
My question is: given package.json and package-lock.json, what is the best strategy nowadays to deal with version numbers of dependencies? Use exact versions or not? How can I deal with security issues in nested dependencies if they don't get upgraded?
My feeling is that
packages that are libraries and meant to be used to others should have inexact version numbers and should specify the minimum they require in order to work; and
top-level projects that aren't going to be included elsewhere should specify the full version numbers of their requirements, so they can have the most control over when things are updated.
I am a little bit confused while reading about Cabal Hell, as the term is overloaded. I guess originally Cabal Hell referred to the diamond dependency problem, which was solved by restricting the build plan to have only a single version of any package in each build plan (two different versions of a package can't exist in a single build plan) as explained in this answer.
However, the term is also used in various other contexts. Such as destructive re-installations, incorrect package dependency boundaries (lower/upper version bounds), inconsistent environments ... (or any other error reported by Cabal).
Particular among these, I am confused about 1) destructive re-installations and 2) inconsistent environments? What do they mean, and how cabal new-build solves these problems (is it just sandboxing like cabal sandbox)? And what role ghc-pkg has to play here?
Any references or a simple example where these problems could be reproduced would be very appreciated.
Regarding "destructive re-installations": If I am not wrong, GHC has a package manager of itself (ghc-pkg), and the packages are installed as dynamically linkable libraries i.e: base depends on ghc-prim, so if ghc-prim is removed it will break base, am I right? And since GHC only allows one instance of a package with the same version, cabal install might register a newer build of the same (package, version) such that it breaks the dependents of the unregistered package. If the above understanding regarding "destructive re-installations" are correct; how does cabal new-build help here?
The only meaningful use of the term is the one given in the linked answer. Related are the follow-on problems from having lots of different packages in the global database, which can make encountering diamond dependencies more common, requiring destructive reinstalls to resolve, etc.
The other usages of the term are not helpful and just mean "problems somehow involving cabal."
That said, let me answer your other questions.
1) ghc-pkg is not a package manager, but rather a tool for managing ghc package databases. It is used by cabal to register packages into databases, and can be used by end-users to inspect the contents of the databases. Think of it as part of the underlying substrate provided by ghc, not a competing tool.
2) new-build eliminates and replaces the standard notion of a packagedb entirely. Instead of saying that a db consists of packages and versions, with at most one of each pair, instead a db consists of potentially many copies of packages at any given version, each with potentially different versions of its dependencies, all of which are managed in part by hash-addressing, so marked by a unique "fingerprint". This is called the store. When you new-build, cabal calculates a build plan irrespective of any previously installed dependencies, from scratch. If a particular fingerprint (consisting of a package, version, and the versions of all its dependencies, certain flags, etc) already exists in the store, then it makes use of it. If it does not, it calculates it.
As such, the only "diamond dependencies" that can occur are the truly insoluble ones, and not the ones occasioned by having fixed too-early (due to already-installed deps) some portion of the dependency tree.
tldr; you write "since GHC only allows one instance of a package with the same version" but new-build partially lifts this restriction in the store which allows the solver to produce better, more reproducible plans more often.
I am relatively new to haskell, stack, ghc, etc.
Have been trying a few projects with ghcjs and haven't been able to build any of them, including reflex-dom-stack-demo. I am getting the following error:
In the dependencies for semigroupoids-5.0.0.4:
tagged-0.8.1 from stack configuration does not match >=0.8.5 && <1 (latest matching version is 0.8.5)
needed due to ghcjs-0.2.0 -> semigroupoids-5.0.0.4
Now I cannot understand whether I misconfigured something or there is truly a broken dependency. Deleted ~/.stack multiple times throughout my experiments.
I found this bug in stackage but am unsure whether this is what affects me, and whether it would be fixed once the fix moves through.
Using Ubuntu 17.10..
Any insight is welcome.
The recomended way to create a development environnement for reflex-dom is to use try-reflex.
It is tricky to build reflex-dom with stack, because some needed changes have not yet been added to the upstream libraries.
If you really want to build a reflex-dom environnement with stack, please consider these hints:
Do not use a GHC compiler with a version higher than 8.0.2
Do not use the reflex /reflex-dom versions from Hackage, they are outdated.
Use versions of reflex / reflex-dom from Github.
This repo contains a stack.yaml file, that used to work.
You may also try the stack.yaml file from the answer to this SO question.