Some open-source projects make combined releases where the version number of each package(library) is increased to the same version.
Examples in Java are:
org.springframework
com.fasterxml.jackson
org.hamcrest
This implies that some packages may get a new version even though they have not changed (nor their dependencies). I don't think this violates semantic versioning.
Benefits I see is that:
Users can use a single version to monitor and upgrade
All users likely to use the same combination of libraries
Drawbacks:
Users using just one out of many libraries might be notified about an "update" though the package to download has not changed
If many users use just a sub-package, then all bug reports for one version are equally for a range of versions, which is difficult to track. Reverting to the previous "different" version to avoid a bug becomes more complex.
One alternative to single-versioning is to use a BOM (Bill-of-materials).
Different concepts of BOMs exist:
A BOM can list several dependencies to include in their versions (e.g. Linux apt Meta-packages)
A BOM can define versions (and other restrictions) for dependencies to be used if the dependency is included (e.g. Java Maven dependencyManagement section of BOM)
The BOM allows to declare which configuration(combination) of library-versions have been tested together, and allows separate groups of users to all use the same configuration, helping with bug reports and reproducibility.
Not all software distribution and buildsystems support the BOM concept equally well, though.
Related
Suppose a project that consists of multiple parts
A file format definition
A library to interact with the file format
A CLI utility application that uses the library to perform tasks
I can do non-backwards compatible changes to (3) without breaking (1) or (2).
I can do non-backwards compatible changes to (2) without breaking (1), but (3) would break.
I can do non-backwards compatible changes to (1) without breaking (3), but (2) would break.
Does this mean that these parts must be different project, so they can have different version numbers?
A version string can refer to a collection of independently versioned API's/packages, a single API, an API and implementation, etc. Basically you have to be cognizant of exactly what it is you are applying a version number to, and what your customers expect. Semantic versioning is all about telling your customers about the risks you think they will take, if they accept an update from you.
From your description, I think you have three things that need to be versioned:
The file format.
The library.
the CLI utility application.
Whether you develop them in one, two or three repos is up to you. It's not uncommon for a file format specification to be developed and versioned independently of any implementations that depend on it. Whether your library and CLI utility belong together or in separate repo's is a matter of your development process logistics and customer expectations. Do you have consumers of the library that don't require your utility?
Whether they reside in one repo, two or three, you should package and version them separately. This implies independent feeds and packaging scripts.
I'm quite new to the Haskell stack, so I might be misunderstanding how things are intended to work here, but I have a problem that I've seen a few times and am wondering if I'm going about the wrong way.
In short, I sometimes want to use a package with a version spec that is capped at a lower version of base than what my Stack resolver includes.
For example, I can't use the lts-12.14 resolver with PSQueue-1.1, because the former includes base-4.11.1 and the latter requires base >=4 && <4.11.
The two ways I have found to resolve that has been to either
add a specific version of base to my extra-deps, making sure that the pinned version is within the range my package allows; or
choose a different resolver (using e.g. https://www.stackage.org/diff/ to figure out which one is the latest resolver with an early enough version of base)
Both of these feel suboptimal, especially since I might want to use packages which have non-overlapping ranges (e.g. one >=3 && <4.11 and one >=4.11). I realize that using such a combination toghether might fail, especially if they are locked on different major-versions of base (assuming base uses semver), but so far I'm only writing pretty small programs, so if they seem to work I'm quite happy even if there are other, non-exercised code paths that will fail on the specific combination of packages and versions. In other words: I know the risk - but I can't chance it, because I don't know how to.
Is there a way to force Stack to allow a newer version of base than the one specified in the requirements of a dependency?
I think you are looking for allow-newer - https://docs.haskellstack.org/en/stable/yaml_configuration/#allow-newer
Ignore version bounds in .cabal files. Default is false.
I need to install the same C++/Fortran library compiled with different compilers on the system with CMake. Is there a standard location where to install the different compiler-specific versions of the same library on the system? For example, assuming that lib.so and lib.a have already been installed using the system package manager under /usr/, is it good practice to install each of the additional compiler-specific versions in a different folder under let's say usr/local. Or is there a better way of doing this that you can advise?
It depends on how many compilers/libraries/versions you have. If you have just a few of them, I think that (almost) any choice on the location is right but I personally prefer /opt/ paths for manually installed code. But if you start having several combinations of them you easily get in trouble. Besides, I think that the question on the "best" location is related to the question on the "best" way to switch from the usage of one library to another one, possibly avoiding to manually set LD_LIBRARY_PATH, libraies to link or similar things.
I give some personal recommendations according to my experience for systems where you want to support many libraries/applications with many compilers/versions and also provide them for many users:
Do not use root user to install compiled software: just use an "installer account" and give read and execute permissions when needed to other users
Select a path for compiled software, e.g. /opt and define two subfolders /opt/build and /opt/install, the first one for your sources and where you compile them, the second as compilation target
Create some subfolders based on categories, e.g. /compilers, /libraries, /applications, ... from both /opt/build and /opt/install
Start preparing compilers under /compilers, e.g. /compilers/gnu/6.3 or /compilers/intel/2017. When possible compile them, e.g. from /opt/build/compilers/gnu/6.3 to /opt/install/compilers/gnu/6.3 or just put them into /install folder, e.g. /opt/install/compilers/intel/2017
Prepare the tree for libraries (or applications) adding subfolders which specify the version, the compiler and compiler version, e.g. compile from /opt/build/libraries/boost/1.64.0/gnu/6.3 and install to /opt/install/libraries/boost/1.64.0/gnu/6.3
At this stage, you have well organized things. But:
It is difficult to decide which library you want to use, you have to specify LD_LIBRARY_PATH or manually link the right one and the situation is worse when you deal also with applications
You are not considering dependencies between libraries: how can I force using g++ 6.3 when linking against boost/1.64.0/gnu/6.3?
To address these and many other issues, a good way of doing is using a tool which can help you, e.g. http://modules.sourceforge.net/ so that you can easily switch to one library to another one, force dependency, get help, and in general have something less error prone in the daily usage.
There is a 3rd-party library my project uses that has split its functionality into multiple imported packages so that a project can install just what it needs. In package.json, several entries are present for the different sub-packages, like...
"dependencies": {
"#lib/dogs": "^1.0.3",
"#lib/cats": "^1.0.3",
"#lib/iguanas": "^1.0.3"
...lots more of the same...
}
I don't want to spend time thinking about compatibility issues if one of the sub-packages installs a different version# than the others through semver-range-picking or another developer fixing a problem by incrementing the version on just one sub-package. I suspect there is some risk of bugs if the sub-package versions get out of sync, even if the intent of the package maintainers is to respect the meaning of breaking changes in their versioning. It seems simpler to just have all the sub-packages on the same version by default.
Should I try to enforce (or at least promote) that the sub-packages have the same version?
Promote, but don't enforce.
Your current set-up, which uses Caret Ranges is the default used when installing with the --save flag for a reason: it's the most flexible and robust range to use for dependencies that correctly follow the semver conventions. This means that whenever someone update's your module as a dependency to theirs, it will automatically bump their sub-dependencies to the latest version that is backwards-compatible with the one explicitly specified after the ^.
Because of this, and the fact that scoped packages don't have interdependencies since they behave identically to normal dependencies, leaving identical caret ranges for each of them should already be sufficient enough to avoid compatibility issues by default.
Don't protect developers from themselves
A good methodology to follow when considering how to deal with compatibility issues is to avoid the antipattern of "protecting developers from themselves." In this situation, you propose to put a lock in place that prevents 3rd parties from editing the relative versions of your dependencies, to avoid compatibility issues. This is a very vague goal since you haven't actually run into any problems yet, as you've pointed out.
Sometimes, yes, developers might not know what they're doing, in which case they'll probably avoid tampering with your default dependency versions, but sometimes they do know, and it can be frustrating when a developer knows they can resolve a bug and are unnecessarily prevented from doing so. So hold their hand, don't cuff them.
npm already chose to avoid this antipattern, you should too.
If a 3rd-party developer chooses to use your module as a dependency, they should have the default amount of freedom available to manage their sub-dependencies through npm by using features like package-lock.json, which unlocks a very clean pattern for precisely managing sub-dependency versions without editing the source code of their dependencies.
In conclusion, what you have now is a very clean and flexible approach, following common conventions and not going out of the way to constrain 3rd-party developers.
I am using alternative version numbering approach for my projects. I have encountered strange behavior by cabal and stack that does not allow me to fully enjoy benefits of this approach. Both cabal and stack enforce version to be of format Int.Int.Int, which does not cover the case of another version format I use for branches (0.x.x, 1.x.x, 1.0.x, etc).
If I have line version: 0.x.x in my .cabal file, I am getting Parse of field 'version' failed. error when running cabal build or Unable to parse cabal file {PROJECT_NAME}.cabal: NoParse "version" 5 when running stack init.
Is there a way to disable version parsing on cabal and stack commands? Is there a flag for it? Or do I have to request this kind of change (adding flags, disabling version parsing) from the developers of cabal and stack?
Why is there any parsing at all? How does it help with building a package? Does cabal or stack automatically increment build numbers on some event? If yes, where could I read more about this? How could I influence the way version numbering incrementation gets implemented in cabal and stack? I want developers of haskell packages take into account the possibility of alternative version numbering approaches.
PS. For all interested folks, I want to quickly summarize the idea behind "weird" version numbers, such as 0.x.x, 1.x.x, 1.0.x. I use the version numbers with x's to describe streamlines of development that allow code changes while such version numbers as 1.0.0, 1.1.0, 2.35.46 are used to describe frozen states of development (to be precise, they are used for released versions of software). Note that such version numbers as 0.x.0, 1.x.15, 2.x.23 are also possible (used for snapshots/builds of software) and they mean that codebase has been inherited from branches with version numbers 0.x.x, 1.x.x and 2.x.x correspondingly.
Why do I need such version numbers as 0.x.x, 1.x.x and 2.x.x at all? In brief, different number of x's mean branches of different types. For example, version number pattern N.x.x is used for support branches, while pattern N.M.x is used for release branches. Idea behind support branches is that they get created due to incompatibility of the corresponding codebases. Release branches get created due to feature freeze in corresponding codebase. For example, branches 1.0.x, 1.1.x, 1.2.x, ... get created as a result of feature freezes (or releases) in branch 1.x.x.
I know this is all confusing, but I worked hard to establish this version numbering approach and I continue working on awareness about the inconsistencies of version numbering through my presentations and other projects. This all makes sense once you think more about the pitfalls of semver approach (you can find detailed slideshare presentation on the matter following the link). But I do not want to defend it for now. For the time being, I just want cabal and stack to stop enforcing their, as I perceive them, unjustified rules to my project. Hope you can help me with that.
You can't. The version will be parsed to Version, which is:
data Version = PV0 {-# UNPACK #-} !Word64
| PV1 !Int [Int]
Stack uses Cabal as a library but has its own Version type:
newtype Version =
Version {unVersion :: Vector Word}
deriving (Eq,Ord,Typeable,Data,Generic,Store,NFData)
Neither cabal nor stack have a way to customize the parsing. You have to write your own variant of those programs if you want to use another version type. But then again, you're not winning anything at that point: neither Hackage nor Stackage will recognize your package's version.
So the 1.x.x isn't possible at the moment. You could exchange x with 99999999 or something similar to mitigate the problem. That being said, it's not clear what cabal install should then install. The 99999999 version? Or the latest stable variant?
If you can express the semantics, a discussion on the mailing list as well as a feature request might change the behaviour in the (far away) future, but for now, you either have to patch the programs yourself or use another numbering scheme.
Is there a way to disable version parsing on cabal and stack commands? Is there a flag for it?
No.
Or do I have to request this kind of change (adding flags, disabling version parsing) from the developers of cabal and stack?
You can of course ask, but there are so many outstanding issues that you are unlikely to get any traction. You will have to be very convincing -- convincing enough to overturn more than 20 years of experience that says the current versioning scheme is basically workable. Realistically, if you want this to happen you'll probably have to maintain a fork of these tools yourself, and provide an alternative place to host packages using this scheme.
Why is there any parsing at all? How does it help with building a package?
Packages specify dependencies, and for each dependency, specify what version ranges they work with. The build tools then use a constraint solver to choose a coherent set of package/version pairs to satisfy all the (transitive) dependencies. To do this, they must at a minimum be able to check whether a given version is in a given range -- which requires parsing the version number at least a little bit.
Does cabal or stack automatically increment build numbers on some event? If yes, where could I read more about this?
There is nothing automatic. But you should take a look at the Package Version Policy, which serves as a social contract between package maintainers. It lets one package maintainer say, "I am using bytestring version 0.10.0.1 and it seems to work. I'm being careful about qualifying all my bytestring imports; therefore I can specify a range like >=0.10 && <0.11 and be sure that things will just work, while giving the bytestring maintainer the ability to push security and efficiency updates to my users." without having to pore through the full documentation of bytestring and hope its maintainer had written about what his version numbers mean.
How could I influence the way version numbering incrementation gets implemented in cabal and stack?
As with your previous question about changing the way the community does things, I think modifications to the Package Versioning Policy are going to be quite difficult, especially changes as radical as you seem to be proposing here. The more radical the change, the more carefully motivated it will have to be to gain traction.
I honestly don't know what a reasonable place to take such motivation and discussion would be; perhaps the haskell-cafe mailing list or similar.