I have a node projection which has a dependency let's say depA. After I run yarn install on my project, it downloads all dependencies for depA in node_modules/depA/node_modules which makes the node_modules directory very big. It doesn't download this folder for other dependencies. Is there anything I should look at why it happens on depA?
Its how dependencies get installed in node.js, a folder with name node_modules is created and then all dependency mentioned in your package.json is fetched from npm server and downloaded.
Now comes the twist, say in your package.json has dependency depA only. but library depA internally is dependent on depSubA, depSubB then these 2 will also get downloaded so that depA can work.
In the previous version of npm (before 5 I guess), there used to be subfolders inside node_modules which had their independent dependencies creating chances of duplicities and huge folder, the latest version now shares these common dependencies.
check for more details https://docs.npmjs.com/configuring-npm/folders.html
Related
As per title, I have been developing my react app in a very bad manner.
Since I worked in an offline environment, whenever I needed to install a new package top be used in my application, I would manually copy it into my node_modules folder. The problem is that I normally do not update my package.json file with the newly installed dependency.
Because of this, after a long time, my node_modules folder grew quite large (300 ish).
When I send the application to my colleauge to develop and he ran npm install "some package", npm deletes more than half of my manually installed packages (ouch). (Also this explains npm's behavior on this https://github.com/npm/npm/issues/17929#issuecomment-322881421 )
Is there a way for me to update my package.json file with all of my dependencies manually installed in my node_modules folder? Besides having to manually type all 300 plus modules (+ modules that are downloaded as it is a dependancy of another node_module)?
Im pretty desperate so any advice will really be appreciated.
You could try deleting your package-lock.json, run npm shrinkwrap, and then paste the dependencies in the generated npm-shrinkwrap.json into your package.json. This will be way more verbose than your package.json would normally be, because I think it will explicitly list the dependencies of all your dependencies (like package-lock.json), but it should give you a file that your application can be installed from by a colleague.
I am learning Angular and each time it takes like 5 minutes to create a new project because of 100MB folder "node_modules" which CLI creates. And the files in this folder are always the same (unless you add some dependencies which I never do). Is there a way to use one node_modules folder for every project?
Have a look in to https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/
Yarn Workspaces is a feature that allows users to install dependencies
from multiple package.json files in subfolders of a single root
package.json file, all in one go.
Making Workspaces native to Yarn enables faster, lighter installation
by preventing package duplication across Workspaces. Yarn can also
create symlinks between Workspaces that depend on each other, and will
ensure the consistency and correctness of all directories.
npm install -g yarn
You can install all dependencies globally or create a symlink from one place to every project.
BUT it is bad practice, correct way is to use separate node_modules for each project, even if you are using same packages. Once you will need use different versions of same package in different projects and common node_modules will cause a lot headache.
Try to use npm cache and npm install --prefer-offline if you just want to install package faster and don't care about version very match. I didn't use it but believe it should work.
Only packages installed by node (npm install) can be in the node_modules folder. This is because, if someone wants to install your project, instead of downloading the whole project with the node_modules included. They type npm install.
Based on the packag.json the node_modules will now be downloaded to the node_modules folder.
So you can put angular in the node_modules folder if it is an npm package. No you cannot put your own files in this folder.
So you can just copy your package.json to every project and run npm install. Then all the node_modules will be the same.
I am new to nodejs. And I found that there are many unknown modules in node_modules folder after I installed three modules (express, jade, gulp) in my local project.
Unknown module examples in the node_modules:
vary
statuses
send
promise
From the tutorials I see from others, after they installed gulp, there will be only one "gulp" folder in their node_modules folder, but this is not my case. Why? Thank you.
The node package manager (npm) updated recently, as part of that update, all modules are installed in the top level node_modules folder. This includes modules that your dependencies need to install. In the past these modules would be nested inside another node_modules folder in express (for example).
This is why the tutorials you read say different, likely they were written before this update.
It is mentioned in the npm changelog here
Your dependencies will now be installed maximally flat. Insofar as is
possible, all of your dependencies, and their dependencies, and THEIR
dependencies will be installed in your project's node_modules folder
with no nesting. You'll only see modules nested underneath one another
when two (or more) modules have conflicting dependencies.
#3697 This will hopefully eliminate most cases where windows users ended up with paths that were too long for Explorer and other standard
tools to deal with.
#6912 (#4761 #4037) This also means that your installs will be deduped from the start.
#5827 This deduping even extends to git deps.
#6936 (#5698) Various commands are dedupe aware now. This has some implications for the behavior of other commands:
npm uninstall removes any dependencies of the module that you
specified that aren't required by any other module. Previously, it
would only remove those that happened to be installed under it,
resulting in left over cruft if you'd ever deduped. npm ls now shows
you your dependency tree organized around what requires what, rather
than where those modules are on disk.
#6937 npm dedupe now flattens the tree in addition to deduping. And bundling of dependencies when packing or publishing changes too:
#2442 bundledDependencies no longer requires that you specify deduped sub deps. npm can now see that a dependency is required by something
bundled and automatically include it. To put that another way,
bundledDependencies should ONLY include things that you included in
dependencies, optionalDependencies or devDependencies.
#5437 When bundling a dependency that's both a devDependency and the child of a regular dependency, npm bundles the child dependency. As a
demonstration of our confidence in our own work, npm's own
dependencies are now flattened, deduped, and bundled in the npm#3
style. This means that npm#3 can't be packed or published by npm#2,
which is something to be aware of if you're hacking on npm.
I just run a simple npm install morgan in a folder and for some reason it adds all the sub dependencies to the parent folder. See image attached
Yes, this is a new feature in npm 3.x, you can read about it here:
https://github.com/npm/npm/releases/tag/v3.0.0
Your dependencies will now be installed maximally flat. Insofar as is possible, all of your dependencies, and their dependencies, and THEIR dependencies will be installed in your project's node_modules folder with no nesting. You'll only see modules nested underneath one another when two (or more) modules have conflicting dependencies.
Basically, it now handles dependencies with as little nesting as possible.
Npm has changed the way they organize dependencies. So instead of 2 separate modules requiring the same dependency and installing them in their own node_modules folder. The dependency is only installed once at the same folder level the node module is installed at.
I am trying to publish my first module to NPM, it renders markdown with EJS templating, it uses two other npm modules marked and ejs, I have these as dependencies in my package.json file. I have a .gitignore file that contains my node_modules directory, and also a .npmignore file that is empty.
I have successfully published to npm.
However when I try to install my module by putting it into the package.json of a test app and doing npm install -d it installs, but it does not install its dependencies, if I go into the test app root node_modules directory and then into my newly published module's installed directory, It has not installed any of its dependencies, it has not have a nested node_modules directory of its own.
There should be a way to get my module's dependencies to install with it correct, when I include express as a dependency, it installs its own node_modules folder with connect and other modules installed, I want to do the same with two other npm modules.
I know it would work if it would install its nested node_modules dependencies, when I do this it works.
$ npm install -d
$ cd node_modules/my_module
$ npm install -d
$ cd ../..
$ node app
EDIT: Here is a link to the GitHub repo for my module, and here is the package.json.
EDIT: Note, this has only happens when I had my dependencies marked and ejs already installed in my test app. When I then installed my module, it did not install marked and ejs in its own node_modules directory. However, if I remove all modules from the test app and install only my module, it will install them. Is there anyway to get it to work regardless of whether my dependencies have been installed beforehand.
It should all "just work" as is.
npm does not install any new modules, because it sees there are already appropriate modules in a node_modules directory at a higher level. Because node's require will look up into the tree, trying node_modules subdir for each directory, your module's require statements will work, without having a node_modules directory of its own.
If you'd install the module in any place which does not have the right dependencies already installed, they would be installed under the module's own node_modules directory. Alberto confirmes this.
You may want to specify a more specific version of the dependencies in package.json though. This will ensure your module gets to use the version of the dependencies you have tested it with.