Is it okay to install `npm` as a dependency? - node.js

We want to use AsyncAPI to document our RabbitMQ messaging. Therefore, we installed asyncapi/generator as a npm dependency.
If you have a look at the package.json you can see that it references npmi as a dependency which in turn is referencing to global-npm. If we want to run it, a globally installed node and npm is necessary.
Now if we run the generator ($ ag ./docs/asyncapi.yaml #asyncapi/html-template --output ./docs/asyncapi/ --force-write) on a machine which has no globally installed npm following error message appears:
/path/to/project/node_modules/global-npm/index.js:13
throw err
^
Error: Cannot find module 'npm'
at throwNotFoundError (/path/to/project/node_modules/global-npm/index.js:11:13)
at /path/to/project/node_modules/global-npm/index.js:39:5
...
As a workaround we declared npm itself as a dependency:
"dependencies": {
"#asyncapi/generator": "^1.1.4",
"#asyncapi/html-template": "^0.15.4",
"#asyncapi/markdown-template": "^0.11.1",
"npm": "^6.14.9",
...
I've never seen such a thing. Is this acceptable or do we need to install our npm on our machines separatly?

if you run ag you must have installed it with npm initially right? so npm is most probably on this machine already.
The error you have, I saw it on windows only, when you have the generator as dependency, and most likely you use nvm.
Solution was this, so manual bump of global-npm to have this fixed in npmi. This is a workaround,long term I think we need to get rid of npmi dependency from the generator I think

We only had this issue in our CI/CD pipeline working with maven-frontend-plugin which is installing node/npm. npm is located in node/node_modules. This is no location where AsyncAPI looks for npm. In order to fix this issue we link the npm-cli.js (which is npm) from
the maven-frontend-plugin to a well known place where AsyncApi looks up for npm node_modules/.bin.
- ln -sf "$(pwd)/node/node_modules/npm/bin/npm-cli.js" "$(pwd)/node_modules/.bin/npm"

Related

How to build node.js projects against local versions of the dependencies?

I've been trying to build a relatively complex node.js project (https://github.com/edrlab/thorium-reader/) against local versions of some dependencies.
I can build and run the project with the non-local dependencies without problems.
I've tried different "routes", such as adding the dependencies using npm install --save <path-to-dependency> or just adding a file:<path-to-dependency> reference to the package.json file. I've checked out the exact versions of each dependency.
npm install doesn't show any errors.
Now when I run npm run start I get type errors that don't quite understand, such as:
Argument of type 'import("~/repositories/thirdparty/thorium/r2-opds-js/dist/es6-es2015/src/opds/opds2/opds2-facet").OPDSFacet'
is not assignable to parameter of type
'import("~/repositories/thirdparty/thorium/thorium-reader/node_modules/r2-opds-js/dist/es6-es2015/src/opds/opds2/opds2-facet").OPDSFacet'.
Types of property 'Links' are incompatible.
Does anyone have a hint for me what I am doing wrong here ?
I'm using node.js version 17.2.0 and npm version 8.2.0.
Best,
N
Ok, after fiddling around for hours I found that using npm link does the trick. Not sure what the differences are, npm install didn't work at all, neither did putting the references in package.json.
That is, go to the dependency's repository, run npm link, build the dependency as a module, THEN go to the main repository and run npm link <dependency> --save.
Now the build process works.
Not sure why there are the two different method, one of which doesn't work a all in this case.

Cannot find module dtrace-provider

I have a simple nodejs application that is throwing "Cannot find module './build/Release/DTraceProviderBindings'". I look it up online and it looks like that a lot of people are having the same problem when using restify on windows (which is my case, I'm using restify on windows 10). Apparently, dtrace-provider is a optional module for restify and there is no version of it for windows. So, what I tried so far:
Update node to v6.2.0;
Uninstall all modules and run npm install --no-optional;
Uninstall only restify and run npm install restify --no-optional;
And my most desperate move npm install dtrace-provider.
Everything I tried where found on github issues, I've seen same error on OSX users with other modules. Not sure what else to try.
Note: This exception does not stop my application, not even prints the error on the console, I just notice that this was happening using the debugger, in other words, my application runs fine, but this keeps happening on the background.
List of other modules I'm using:
"dependencies": {
"restify": "latest",
"request": ">=2.11.1",
"cheerio": ">=0.10.0",
"xml2js": ">=0.2.0",
"botbuilder": "^0.11.1",
"applicationinsights": "latest"
}
This worked for me after switching to Node 6.1 (and when re-installing node modules didn't work):
Install and save dtrace-provider
$ npm install dtrace-provider --save
Delete 'node_modules' folder
Re-install node modules
$ npm install
I found this thread before combining your attempts with another solution on the Github project issues for restify (https://github.com/restify/node-restify/issues/1093) and simplified best as possible.
I recently ran into this error as well on node 6.11.1.
I ran npm rebuild dtrace-provider and that resolved the problem.
The restify team followed an approach of trying to load the module by requiring it on a try/catch block. You should just ignore the exception.
I had success with the following (elaborate) sequence:
Adjust my path to not have spaces
rm -rf node_modules
rm -rf ~/Library/Caches/node-gyp/
npm cache clean --force
V=1 npm install -S dtrace-provider#0.8.8 --python=python2.7 (repeat this step, resolving as you go, until the install is completely successful … if it fails, check the version - I had rogue dtrace-provider#0.6.0 building at one point)
npm install
At this point everything should have installed cleanly, and I was congratulating myself on a job well done. Then I executed my code and still got the DTraceProviderBindings error. The cause was nested dependencies with the wrong version of dtrace-provider (especially bunyan).
To confirm, do npm list | grep dtrace -B6.
If there's anything lower than 0.8.8, edit package-lock.json, following the method in How do I override nested NPM dependency versions?. Replace requires with dependencies for dtrace-provider and update the version.
Back round to get everything clean: rm -rf node_modules
Then, again, npm install --python=python2.7
I had to iterate round npm list a few times because I thought I'd caught everything and I hadn't.
The key points were to use the required version of python, have a unix-friendly path, and hunt down all nested dependencies. The python version issues gave a big messy error, the space issue gave a much more missable error.
I know this is an old issue but I wanted to comment on it in case anyone else has the same issue I had.
My issue was caused by having parentheses in my path.
/users/karlgroves/Dropbox (Personal)/foo/bar/bat/project...
Moving the project to a path without the parens worked for me.
You'll need to wipe out node_modules and reinstall again.
I recently ran into this error as well on node v8.8.1
as #Derek mentioned, I ran npm rebuild dtrace-provider and that resolved the problem.
tl;dr; dtrace-provider utilized node-gyp which required python version >= 2.5 and NOT 3.5
I had this issue on OSX and found a post that showed using environment variable
V=/Users/your_user/your_project npm i dtrace-provider
This let me know that there was a dependency on node-gyp that was failing to build...Once I knew the issue was with this module was able to focus my attention at troubleshooting node-gyp.
This led to some log output indicating that my python version 3.5 was unsupported and it required version >= 2.5.
Went and downloaded python 2.7.x and checked /usr/bin/python 2.7.x to ensure it was there. Uninstalled the node module that was ultimately requiring this module, then used npm cache clean then reinstalled the module and this time it appeared to pick up the right python version to be able to build.
Hope this helps someone =)
I have tried many suggestions but get the same error again.
Finally, I found the correct way to solve this question.
Go the node.js website and download the latest version of node.js pkg.
After installed, reinstall your software, everything will be ok.
i managed to get this working by running this command
npm install --python=python2.7

Update local file dependency with npm

I have a project with a local file dependency in my package.json like this:
"dependencies": {
"dep_1": "file:../../dep_1"
}
}
When I do npm install it is installed into node_modules. But if I make changes to dep_1 how do I update the module version in node_modules?
I tried doing npm update but nothing happens.
If you are using a relatively new version of npm (I used version 2.14.2) you can bump the version number in package.json and npm update dep_1 should work. Otherwise how can npm know that something needs to be updated?
Note: This will only work if the version is higher than what has previously been installed. You will have to clean the cache to reset this behaviour.
However, you can forceably (and lazily) update local modules by simply running npm install again. e.g.
npm install dep_1
It should be fast since its on your local computer and you don't have to play around with version numbers.
For more detail see the discussion about this issue on the official npm repository page: https://github.com/npm/npm/issues/7426

npm install fails because package is missing in registry

I have an issue with a project where we are using node and brunch. The issue is current specific to brunch, but could occur for any module would be my guess.
The easiest way to currently reproduce this, is to do the following in a new folder:
npm init
npm install --save-dev brunch
The issue here is that brunch depends on loggy, which in turn depends on ansi-color, which no longer has an entry in the npmregistry:
https://registry.npmjs.org/ansi-color
I think this might be the github project: https://github.com/loopj/commonjs-ansi-color
In any case, I am unable to proceed, and all our builds fail because they are not able to fetch the given dependency.
I could perhaps use npm shrinkwrap in some way, but that depends on the modules already existing in node_modules, which I am currently missing.
So how can I force npm to use ansi-color from a different location, or ignore the dependency?
Not sure about npm 2 but you can fix this with beta npm 3. npm 3 has flat node_modules directory. So sub modules can sit in the top level. Read the Changelog.
The missing modules can be installed directly from their Github repo as a toplevel dependency in your project. If npm finds the module with the same version in node_modules directory, it won't look for it anymore in the registry.
Install npm 3:
npm install -g npm#3-latest
Then install depencies:
//install missing module from other location
npm install https://github.com/loopj/commonjs-ansi-color.git --save-dev
npm install --save-dev brunch
It looks like ansi-color is back on the npm registry ("https://registry.npmjs.org/ansi-color" is back online)

What is the difference between --save and --save-dev?

What is the difference between:
npm install [package_name]
and:
npm install [package_name] --save
and:
npm install [package_name] --save-dev
What does this mean? And what is really the effect of --save and -dev keywords?
The difference between --save and --save-dev may not be immediately noticeable if you have tried them both on your own projects. So here are a few examples...
Let's say you were building an app that used the moment package to parse and display dates. Your app is a scheduler so it really needs this package to run, as in: cannot run without it. In this case you would use
npm install moment --save
This would create a new value in your package.json
"dependencies": {
...
"moment": "^2.17.1"
}
When you are developing, it really helps to use tools such as test suites and may need jasmine-core and karma. In this case you would use
npm install jasmine-core --save-dev
npm install karma --save-dev
This would also create a new value in your package.json
"devDependencies": {
...
"jasmine-core": "^2.5.2",
"karma": "^1.4.1",
}
You do not need the test suite to run the app in its normal state, so it is a --save-dev type dependency, nothing more. You can see how if you do not understand what is really happening, it is a bit hard to imagine.
Taken directly from NPM docs docs#dependencies
Dependencies
Dependencies are specified in a simple object that maps a package name
to a version range. The version range is a string that has one or
more space-separated descriptors. Dependencies can also be identified
with a tarball or git URL.
Please do not put test harnesses or transpilers in your dependencies
object. See devDependencies, below.
Even in the docs, it asks you to use --save-dev for modules such as test harnesses.
--save-dev is used to save the package for development purpose.
Example: unit tests, minification..
--save is used to save the
package required for the application to run.
By default, NPM simply installs a package under node_modules. When you're trying to install dependencies for your app/module, you would need to first install them, and then add them to the dependencies section of your package.json.
--save-dev adds the third-party package to the package's development dependencies. It won't be installed when someone runs npm install directly to install your package. It's typically only installed if someone clones your source repository first and then runs npm install in it.
--save adds the third-party package to the package's dependencies. It will be installed together with the package whenever someone runs npm install package.
Dev dependencies are those dependencies that are only needed for developing the package. That can include test runners, compilers, packagers, etc.
Both types of dependencies are stored in the package's package.json file. --save adds to dependencies, --save-dev adds to devDependencies
npm install documentation can be referred here.
--
Please note that --save is now the default option, since NPM 5. Therefore, it is not explicitly needed anymore. It is possible to run npm install without the --save to achieve the same result.
Let me give you an example,
You are a developer of a very SERIOUS npm library which uses different testing libraries to test the package.
Users download your library and want to use it in their code. Do they need to download your testing libraries as well? Maybe you use jest for testing and they use mocha. Do you want them to install jest as well? Just To run your library?
No. right? That's why they are in devDependencies.
When someone does, npm i yourPackage only the libraries required to RUN your library will be installed. Other libraries you used to bundle your code with or testing and mocking will not be installed because you put them in devDependencies. Pretty neat right?
So, Why do the developers need to expose the devDependancies?
Let's say your package is an open-source package and 100s of people are sending pull requests to your package. Then how they will test the package? They will git clone your repo and when they would do an npm i the dependencies as well as devDependencies.
Because they are not using your package. They are developing the package further, thus, in order to test your package they need to pass the existing test cases as well write new. So, they need to use your devDependencies which contain all the testing/building/mocking libraries that YOU used.
A perfect example of this is:
$ npm install typescript --save-dev
In this case, you'd want to have Typescript (a javascript-parseable coding language) available for development, but once the app is deployed, it is no longer necessary, as all of the code has been transpiled to javascript. As such, it would make no sense to include it in the published app. Indeed, it would only take up space and increase download times.
As suggested by #andreas-hultgren in this answer and according to the npm docs:
If someone is planning on downloading and using your module in their program, then they probably don't want or need to download and build the external test or documentation framework that you use.
However, for webapp development, Yeoman (a scaffolding tool that installs a peer-reviewed, pre-written package.json file amongst other things) places all packages in devDependencies and nothing in dependencies, so it appears that the use of --save-dev is a safe bet in webapp development, at least.
--save-dev saves semver spec into "devDependencies" array in your package descriptor file, --save saves it into "dependencies" instead.
--save-dev is used for modules used in development of the application,not require while running it in production environment
--save is used to add it in package.json and it is required for running of the application.
Example: express,body-parser,lodash,helmet,mysql all these are used while running the application use --save to put in dependencies while mocha,istanbul,chai,sonarqube-scanner all are used during development ,so put those in dev-dependencies .
npm link or npm install will also install the dev-dependency modules along with dependency modules in your project folder
Read Complete And Forget --save-dev Headache
Simplest answer is that --save-dev is useful when you are creating packages for other developers and want to host your package at NPM Registry like lodash, mongoose, express etc. When you are building or writing a Node Server there is no difference between --save and --save-dev because your Node Server implementation is private to you and you will never publish it on NPM.
How NPM Install Works
Whenever we install a new package using npm like npm install express then NPM installs that package to our system and put it into node_modules folder, now NPM will analyze the package.json file of newly installed package i.e express in this case, after analyzing NPM will install all those packages which were mentioned in dependencies section of package.json file of express package. After installing those packages on which express was dependent NPM again analyze the package.json file of all newly installed packages and again install the packages for them, this cycle goes on until all packages are available into node_modules folder to function properly. You can check package dependencies by running npm list in terminal where terminal should point location of your project directory.
How --save-dev Is Related To Above Explained Stuff
Suppose you want to create a new package like express, now while development of this new package you probably want to write some unit testing code and test the package with any other available testing package let's assume mocha in this case. Now you know mocha is only required to test the package not required to use the package. In this case you should install mocha using --save-dev flag, otherwise NPM will install it whenever a developer install your package using NPM. So if we want a dependency not installed when someone install our package from NPM we must install that package using --save-dev in development phase.
Last Thing
Do not mix --save-dev with collaboration development, if someone cloned your package code from a source version control system like github then NPM will surely install all devDependencies i.e package installed using --save-dev also.
Clear answers are already provided. But it's worth mentioning how devDependencies affects installing packages:
By default, npm install will install all modules listed as dependencies in package.json . With the --production flag (or when the NODE_ENV environment variable is set to production ), npm will not install modules listed in devDependencies .
See: https://docs.npmjs.com/cli/install
When you install an npm package using npm install <package-name>, you are installing it as a dependency.
The package is automatically listed in the package.json file, under the dependencies list (as of npm 5: before you had to manually specify --save).
ex. npm install lodash
After pressing enter check your package.json file.
"dependencies": {
"lodash": "4.x",
},
When you add the -D flag, or --save-dev, you are installing it as a development dependency, which adds it to the devDependencies list.
ex. npm install --save-dev lite-server
After pressing enter check your package.json file
"devDependencies": {
"lite-server": "^2.6.1"
},
Development dependencies are intended as development-only packages, that are unneeded in production. For example testing packages, webpack, or Babel.
When you go in production, if you type npm install and the folder contains a package.json file, they are installed, as npm assumes this is a development deploy.
You need to set the --production flag (npm install --production) to avoid installing those development dependencies.
All explanations here are great, but lacking a very important thing: How do you install production dependencies only? (without the development dependencies).
We separate dependencies from devDependencies by using --save or --save-dev.
To install all we use:
npm i
To install only production packages we should use:
npm i --only=production
You generally don't want to bloat production package with things that you only intend to use for Development purposes.
Use --save-dev (or -D) option to separate packages such as Unit Test frameworks (jest, jasmine, mocha, chai, etc.)
Any other packages that your app needs for Production, should be installed using --save (or -S).
npm install --save lodash //prod dependency
npm install -S moment // " "
npm install -S opentracing // " "
npm install -D jest //dev only dependency
npm install --save-dev typescript //dev only dependency
If you open the package.json file then you will see these entries listed under two different sections:
"dependencies": {
"lodash": "4.x",
"moment": "2.x",
"opentracing": "^0.14.1"
},
"devDependencies": {
"jest": "22.x",
"typescript": "^2.8.3"
},
--save-dev (only used in the development, not in production)
--save (production dependencies)
--global or -g (used globally i.e can be used anywhere in our local system)
People use npm on production to do wicked cool stuff, Node.js is an example of this, so you don't want all your dev tools being run.
If you are using gulp (or similar) to create build files to put on your server then it doesn't really matter.
Basically We Write
npm install package_name
But specially for Testing Purpose we don't need to run some package while Application is Running in Normal State so that Node introduce good way to solve this problem. Whenever we write
npm install package_name --save-dev
at that time this package is only installed for development purpose.
I want to add some of my ideas as
I think all differences will appear when someone uses your codes instead of using by yourself
For example, you write an HTTP library called node's request
In your library,
you used lodash to handle string and object, without lodash, your codes cannot run
If someone uses your HTTP library as a part of his code. Your codes will be compiled with his.
your codes need lodash, So you need to put in dependencies to compile
If you write a project like monaco-editor, which is a web editor,
you have bundled all your codes and your product env library using webpack, when build completed, only have a monaco-min.js
So someone doesn't care whether --save or --save-dependencies, only he needs is monaco-min.js
Summary:
If someone wants to compile your codes (use as a library),
put lodash which used by your codes into dependencies
If someone want to add more feature to your codes, he needs unit test and compiler, put these into dev-dependencies
as --save is default option for npm, so I use
npm i package
and for --save-dev, I use
npm i package -D
default option will install package as project dependency where as -D is for development dependencies like testing, lint etc. and install package for development process
you can find all the flags here https://docs.npmjs.com/cli/v8/commands/npm-install

Resources