I am pretty new to Haskell as well as stack.
import Data.Set
import Data.Stack
The statements above trigger compilation error: Could not find module 'Data.Set'. Perhaps you meant 'Data.Int'. I tried to google and found nothing similar.
Hence, my question is: do I need to specify external dependencies manually or just my stack build command somewhy fails to grab appropriate modules from somewhat cache or repository?
In case I have to specify my dependencies manually, should I prefer .cabal or .yaml? What's the correct way to deal with versioning?
[Do] I need to specify external dependencies manually [...]?
Yes.
Since you are using Stack, it is easy to specify the dependent packages you import in your code. Depend on your Stack version, the default configuration might be a little bit different:
If you created your project with the latest version of Stack, you will see package.yaml in the root of your project (hpack is used in this case to specify the configurations). You need to add package dependencies there, e.g., containers for Data.Set. Here's an example of a dependencies section in one of my projects:
dependencies:
- base >= 4.7 && < 5
- containers
- time
- network
- bytestring
If you are using an older version of stack and do not see package.yaml, you need to edit your-project-name.cabal to add the dependencies. Here's the complete document telling you how to do it: https://docs.haskellstack.org/en/stable/GUIDE/#adding-dependencies
Personally, I prefer the new system with hpack (package.yaml). Basically, it is simpler, and it saves you time declaring all modules you have (not about dependency). If you have package.yaml, do not edit .cabal, which is automatically generated by hpack.
Related
I have a project with a a library in a few directories. The cabal file is produced with hpack and looks ok. The project builds with cabal build and the main can be run with cabal run xx.
Using repl in vscode, I get occasionally
Could not load module ‘GIS.Subdivisions’
It is a member of the hidden package ‘CatCoreConcepts-0.2’.
Perhaps you need to add ‘CatCoreConcepts’ to the build-depends in your .cabal file.
The package is, of courese, listed in the dependencies. The error is not always occuring and I assume it is a probem with some data cached in the vscode Haskell HLS plugin. Is there a simple way to clean the cache of the plugin? Restart HLS and Developer: reload window in vscode is not having any effect.
I have had the problem where VSCode thinks modules are hidden. Many answers on SO, such as this one. Possible solution, which hopefully helps,
ghc-pkg expose CatCoreConcepts
This is not an answer to the main question, but to some OP comments.
I'd say cabal documentation is good (but lengthy).
A brief summary as I understand (not official documentation)
A Package is a set of components
A component is a set of modules (haskell files)
Module-name and file-name must be equal except for entrypoints (see below)
Components can be classified in two groups:
runnables: tests, benchmarks and executables
runnables must have a unique main function called the entrypoint
In the cabal file, you can use keywords executable, test-suite and benchmark for defining runnables
Within each runnable section in the cabal file there must be a field main-is pointing to the file with the entrypoint
The file-name of the entrypoint can be whatever but the module name must be Main.
Notice that runnable components can have multiple modules (haskell files). For example you can have a test suit consisting in a file with auxiliar functions and other file with the main function. Both files conform a component (a test-suite in this case)
non-runnables: libraries
In the cabal file, you can use keywords library for defining non-runnable component
For libraries you must specify the modules which are exposed with the exposed-modules keyword.
Runnable components can import non-runnable ones even if they are defined in the same package. But viceversa is not true.
As a good practice, each component should be separated in a different folder. Some libraries expect naming convention. For example: tests to be in a folder with the same name and with test_XXXX.hs format (this is regular across programing languages, not only Haskell)
When libraries are separated in folders the field hs-source-dirs must be specified to point to the folder with the haskell files
(the name runnable is not part of the official docs, but is the way I understand)
stack works pretty much the same as cabal since the former is just a different front-end for the later. So aside from different keywords and yaml format, all of the above can be applied to stack (notice that cabal updates more often than stack, therefore some features supported by cabal can be missing in stack.)
If you find this useful, I think I am opening an issue to the cabal docs to include it.
I have a library coming up deep in the node module hierarchy which is causing security issues. I am not directly referencing that module in my package.json. One of the module which I reference is loading up another module and that module is loading this module. So it's the third layer in the dependency tree. I can find out the library dependence tree using npm ls.
I tried updating package.json, but that's not correct I think.
How can I update the version of this particular module without touching the top modules? Should I have to use shrinkwrap?
One (horrible way) (to answer your question directly) you could carefully manage all of those dependencies on your own and build that structure outside of NPM. I hate it. There is a ton of dependency management overhead and no guarantee any of the hand assembled components would work together - so testing overhead too. but in "theory" it could work. FWIW I don't think shrinkwrap helps with sub dependencies at all.
I recommend this course (I understand this isn't what you asked for - but it is the best approach IMO):
Fork/Branch the library and make the change there.
Then issue a pull request (Back to the main branch)
Until it is is merged back in, you cab reference it via the GIT url in your package.json
from: https://docs.npmjs.com/files/package.json
git+ssh://git#github.com:npm/npm.git#v1.0.27
git+ssh://git#github.com:npm/npm#semver:^5.0
git+https://isaacs#github.com/npm/npm.git
git://github.com/npm/npm.git#v1.0.27
I am building a custom binary of NodeJS from the latest code base for an embedded system. I have a couple modules that I would like to ship as standard with the binary - or even run a custom script the is compiled into the binary and can be invoked through a command line option.
So two questions:
1) I vaguely remember that node allowed to include custom modules during build time but I went through the latest 5.9.0 configure script and I can't see anything related - or maybe I am missing it.
2) Did someone already do something similar? If yes, what were the best practices you came up with?
I am not looking for something like Electron or other binary bundlers but actually building into the node binary.
Thanks,
Andy
So I guess I figure it out much faster that I thought.
For anyone else, you can add any NPM module to it and just add the actual source files to the node.gyp configuration file.
Compile it and run the custom binary. It's all in there now.
> var cmu = require("cmu");
undefined
> cmu
{ version: [Function] }
> cmu.version()
'It worked!'
> `
After studying this for quite a while, I have to say that the flyandi's answer is not quite true. You cannot add any NPM module just by adding it to the node.gyp.
You can only add pure JavaScript modules this way. To be able to embed a C++ module (I deliberately don't use the word "native", because that one is quite ambiguous in nodeJS terminology - just look at the sources).
To summarize this:
To embed a JS module to your custom nodejs, just add it in the library_files section of the node.gyp file. Also note that it should be placed within the lib folder, otherwise you'll have troubles requiring the module. That's because the name/path listed in node.gyp / library_files is used to encode the id of the module in the node_javascript.cc intermediate file which is then used when searching for the built-in modules.
To embed a native module is much more difficult. The best way I have found so far is to build the module as a static library instead of dynamic, which for cmake(-js) based module you can achieve by changing the SHARED parameter to STATIC like this:
add_library(${PROJECT_NAME} STATIC ${SRC})
instead of:
add_library(${PROJECT_NAME} SHARED ${SRC})
And also changing the suffix:
set_target_properties(
${PROJECT_NAME}
PROPERTIES
PREFIX ""
SUFFIX ".lib") /* instead of .node */
Then you can link it from node.gyp by adding this section:
'link_settings': {
'libraries' : [
"path/to/my/library.lib",
#...add other static dependencies
],
},
(how to do this with node-gyp based project should be quite ease to google)
This allows you to build the module, but you won't be able to require it, because require() function in node can only be used to load built-in JS modules, external JS modules or external dynamic node modules. But now we have a built-in C++ module. Well, lot of node integrated modules are C++, but they always have a JS wrapper in /lib, and those wrappers they use process.binding() to load the C++ module. That is, process.binding() is sort of a require() function for integrated C++ modules.
That said, we also need to call require.binding() instead of require to load our integrated module. To be able to do that, we have to make our module "built-in" first.
We can do that by replacing
NODE_MODULE(mymodule, InitAll)
int the module definition with
NODE_BUILTIN_MODULE_CONTEXT_AWARE(mymodule, InitAll)
which will register it as internal module and from now on we can process.binding() it.
Note that NODE_BUILTIN_MODULE_CONTEXT_AWARE is not defined in node.h as NODE_MODULE but in node_internals.h so you either have to include that one, or copy the macro definition over to your cpp file (the first one is of course better because the nodejs API tends to change quite often...).
The last thing we need to do is to list our newly integrated module among the others so that the node knows to initialize them (that is include them within the list of modules used when searching for the modules loaded with process.binding()). In node_internals.h there is this macro:
#define NODE_BUILTIN_STANDARD_MODULES(V) \
V(async_wrap) \
V(buffer) \
V(cares_wrap) \
...
So just add the your module to the list the same way as the others V(mymodule).
I might have forgotten some step, so ask in the comments if you think I have missed something.
If you wonder why would anyone even want to do this... You can come up with several reasons, but here's one most important to me: Those package managers used to pack your project within one executable (like pkg or nexe) work only with node-gyp based modules. If you, like me, need to use cmake based module, the final executable won't work...
I have developed a module (M.hs) which depends upon 3 other modules (A.hs, B.hs and C.hs). Now I want to use the module M across multiple other projects. So I have to install this module. But for learning purpose I don't want to use cabal, I want to do it manually. I want to install it in my home dir.
What is a proper course of action? Which files to be created, copied? where? How to use this module in other project?
Additional info:
I am using Debian 6
I am using GHC 6.12
You say you don’t want to use cabal, but would you use Cabal?
cabal is the name of the command line tool provided by cabal-install which can download packages from Hackage and resolve dependencies.
Cabal is the library that Haskell code uses to drive the compilation (e.g. pre-process files, build in the right order, build variants, generate documentation) and install into the right location.
I would not recommend not using Cabal, even for learning purposes, until you want to write a replacement for it. But if you really want to do it, here is the rough outline, with enough details to figure out for a good learning experience:
Build your files with -package-name yourpkgname-version.
Link the generated files to form a libyourpkgname-version.a file.
Create a package configuration file like /var/lib/ghc/package.conf.d/mtl-2.1.2.conf, and pay attention to name, `exposed-modules, import-dirs, library-dirs and hs-libraries
Register package by passing it to ghc-pkg register
Is there a way to specify OS specific dependencies in a npm package.json file?
For example, I would only want to install 'dbus' (https://npmjs.org/package/dbus) as a dependency for my module if the user is running Linux. I would have a different dependency for Mac and Windows.
There's a possible good way of doing this, depending on your setup.
npm package.json supports an os key,
and also optionalDependencies
os can be used to specify which OS a module can be installed on.
optionalDependencies are module dependencies that if they cannot be installed, npm skips them and continues installing.
In this way you can have your module have an optional dependency for each OS, and only the one which works will be loaded/installed ^.^
EDIT: As #Sebastien mentions below, this approach is dangerous.
For any given OS, at least one of your dependencies is "required" and the rest "optional". Making all versions of the dependency optional means that if your installation fails for a legitimate reason, it will silently skip installation and you will be missing a dependency you really need.
I think the short answer is no. I can think of a couple of workarounds though - the simplest is to just add everything to package.json regardless of OS, and then require() the correct one at runtime.
If that doesn't work for you, you might be able to use an install script to get the result you're going for - https://docs.npmjs.com/misc/scripts
I haven't tested this but I think it would work:
Add something like this to your package.json:
,"scripts": {
"install": "node install_dependencies.js"
}
And then add a install_dependencies.js file that checks the OS and runs the appropriate npm install ... commands.
There's also the bindings-shyp module:
https://www.npmjs.com/package/bindings-shyp
Helper module for loading your native module's .node file
This is a helper module for authors of Node.js native addon modules. It is basically the "swiss army knife" of require()ing your native module's .node file.
Throughout the course of Node's native addon history, addons have ended up being compiled in a variety of different places, depending on which build tool and which version of node was used. To make matters worse, now the gyp build tool can produce either a Release or Debug build, each being built into different locations.
This module checks all the possible locations that a native addon would be built at, and returns the first one that loads successfully.
Quoting #npm_support at:
https://twitter.com/npm_support/status/968195526989512705
2/2 If you'd like to avoid installation problems related to dependencies, one route is for you to write a wrapper that's required as a regular dependency, and to make sure that it has optionalDeps (and also ensure that the wrapper verifies you have everything needed to work).
But IMHO it looks more like a workaround than solving the problem for real.
I can understand that npm wants to preserve portability and avoid to deal with platform specifics, but it has to be done anyway and IMHO doing this at runtime is not optimal (specialty if one wants do optimize code size).
So today I have no optimal solution to share but an open discussion for proposal.
Can't "conditional dependencies" be supported in npm ?
The 1st thing that came to my mind was to to add a "override" section that will change (+add, -remove, =replace) current parsed sections.
For example:
dependencies: { "common-stuff": "*" }
overrides: {
"os: { linux: { dependencies: { "+best-linux-module" } } }
}
And other option suggested by a developer I know, would be to introduce a provides keyword, then several modules could provide a same semantic than would be satisfied by resolver (a la debian), but it's generating similar overhead.
I am looking for a generic approach not only focused on OS support but also on other flavors of package (depending on engines for instance).
Do you know any related issue in NPM tracker ? if not I am considering to file a bug to be tracked at:
https://github.com/npm/npm/issues?q=dependencies+conditional
Feedback welcome on this idea.