As per title, I have been developing my react app in a very bad manner.
Since I worked in an offline environment, whenever I needed to install a new package top be used in my application, I would manually copy it into my node_modules folder. The problem is that I normally do not update my package.json file with the newly installed dependency.
Because of this, after a long time, my node_modules folder grew quite large (300 ish).
When I send the application to my colleauge to develop and he ran npm install "some package", npm deletes more than half of my manually installed packages (ouch). (Also this explains npm's behavior on this https://github.com/npm/npm/issues/17929#issuecomment-322881421 )
Is there a way for me to update my package.json file with all of my dependencies manually installed in my node_modules folder? Besides having to manually type all 300 plus modules (+ modules that are downloaded as it is a dependancy of another node_module)?
Im pretty desperate so any advice will really be appreciated.
You could try deleting your package-lock.json, run npm shrinkwrap, and then paste the dependencies in the generated npm-shrinkwrap.json into your package.json. This will be way more verbose than your package.json would normally be, because I think it will explicitly list the dependencies of all your dependencies (like package-lock.json), but it should give you a file that your application can be installed from by a colleague.
Related
I am a total Javascript newbie aiming to configure my Mac nicely for development.
NPM is installed.
I notice that folder node_modules exists in my Users/MyName directory.
I think this is a result of having either installed Node/NPM or specifically run npm install airtable the other day, which I did at the time in Users/MyName.
When I npm uninstall airtable, it removes airtable and its dependency folders from nodule_modules, leaving the following: #types and package-lock.json (hidden).
If I cd to new project-specific directory Users/MyName/Projects/Code/myusername/airtable-test and run npm install airtable from there, I expected the packages may get installed in that folder. However, again, they get installed up at Users/MyName/node_modules.
In both cases, .package-lock.json (non-hidden) and package.json are in Users/MyName, which seems messy to me. (I haven't done anything non-standard in install).
Is this the way things should be?
Attempts to solve:
I seem to read, including from questions on Stackoverflow, that storing modules at Users/MyName/node_modules is effectively storing them globally, accessible to any app, and such that projects don't have to get committed to server with all dependencies in tow - the idea being that, after you deploy your app, you then run npm install whilst in its folder, prompting it to install all dependencies.
Is this right? Should I be looking at storing all dependency modules in a project folder, or above and outside of it?
(If the answer to this question is opinion-based, I wasn't aware of that).
Here is what I believe is happening. You have your package.json in folder Users/MyName and you are running npm install in Users/MyName/Projects/Code/myusername/airtable-test. But the problem is you do not have package.json file in the folder Users/MyName/Projects/Code/myusername/airtable-test. So npm goes up in the directory to find the package.json and it found it in Users/MyName so it is installing the package there.
This is happening because the way npm identifies a project is by looking for package.json. If it does not find it in current directory than it assumes that you must be inside some sub directory of the project and start searching upwards in the folder hierarchy to find the package.json.
solution
Do npm init in the folder Users/MyName/Projects/Code/myusername/airtable-test. This will initialize the folder as a npm package (by creating package.json).
I am learning Angular and each time it takes like 5 minutes to create a new project because of 100MB folder "node_modules" which CLI creates. And the files in this folder are always the same (unless you add some dependencies which I never do). Is there a way to use one node_modules folder for every project?
Have a look in to https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/
Yarn Workspaces is a feature that allows users to install dependencies
from multiple package.json files in subfolders of a single root
package.json file, all in one go.
Making Workspaces native to Yarn enables faster, lighter installation
by preventing package duplication across Workspaces. Yarn can also
create symlinks between Workspaces that depend on each other, and will
ensure the consistency and correctness of all directories.
npm install -g yarn
You can install all dependencies globally or create a symlink from one place to every project.
BUT it is bad practice, correct way is to use separate node_modules for each project, even if you are using same packages. Once you will need use different versions of same package in different projects and common node_modules will cause a lot headache.
Try to use npm cache and npm install --prefer-offline if you just want to install package faster and don't care about version very match. I didn't use it but believe it should work.
Only packages installed by node (npm install) can be in the node_modules folder. This is because, if someone wants to install your project, instead of downloading the whole project with the node_modules included. They type npm install.
Based on the packag.json the node_modules will now be downloaded to the node_modules folder.
So you can put angular in the node_modules folder if it is an npm package. No you cannot put your own files in this folder.
So you can just copy your package.json to every project and run npm install. Then all the node_modules will be the same.
I'm quite new to ReactNative so sorry if it's obvious, but..
Each RN project init-ed via CLI has a large number of node modules stored in project_root/node_modules. Not that I would mind, but if you have several projects it seems redundant and takes up time/space to move it to the source versioning system.
Wouldn't it be possible to retrieve all these same modules from the general node_modules on the machine instead ?
You never want to store dependencies nested in node_modules in your source control... it defeats the whole purpose of versioning and dependencies in general. Your package.json file will specify the versions so when you run npm install it knows exactly which dependencies to grab.
As an alternative, Yarn is an up and rising package client that Facebook developed that does a much better job of caching your packages locally so that way if multiple projects reuse the same depencencies, it will still satisfy the need to keep them in node_modules but doesn't need to perform http requests for each one.
Yarn doesn't replace NPM as a package registry, just a better client to download, maintain, and cache those packages.
Yarn also adds a yarn.lock file (similar to Ruby's Gemfile.lock) that allows you to lock in the specific versions used in your app, regardles of the package.json. This file can be stored in version control, which is probably what you were wanting to achieve by saving the node_modules in version control.
Some good reads...
Yarn vs NPM
Scotch.io Yarn Tutorial
Why I'm working on Yarn (Yehuda Katz)
I would echo Brad's answer: Don't put node_modules in version control. npm install will install the correct versions from the package.json. Just put package.json in version control, not node_modules.
However, if you still want to save disk space, you can install some of your dependencies in a general node_modules folder by using the link option:
npm config set link true -g
You can read more about link here: https://docs.npmjs.com/misc/config#link.
Note that you must not include node_modules in your version control when using this option since npm will put symlinks to the globally installed packages in node_modules. The global install location varies from machine to machine, so if node_modules is in version control, it may link to non-existent locations.
I read from here that the dependencies in the package.json file allow people to install the dependencies if they install your project through npm-
Finally, the dependencies field is used to list all the dependencies
of your project that are available on npm. When someone installs your
project through npm, all the dependencies listed will be installed as
well. Additionally, if someone runs npm install in the root directory
of your project, it will install all the dependencies to
./node_modules.
Where will all the dependencies be installed to if someone doesn't run npm install in the root directory of your project?
Also, what if they choose to clone this project through Github instead? It would be ready to go anyway, right? Then at that point what is the purpose of the package.json file besides giving the user meta data about the project?
Where will all the dependencies be installed to if someone doesn't run npm install in the root directory of your project?
If by that you mean 'where will they be installed if you run the command in a different directory', NPM will search upwards through the parent directories until it finds package.json, and then install the dependencies in a node_modules folder next to that file. I.E. they'll always end up in the project root.
Also, what if they choose to clone this project through Github instead? It would be ready to go anyway, right? Then at that point what is the purpose of the package.json file besides giving the user meta data about the project?
This isn't the case! Node projects just about always have a .gitignore file which explicitly excludes node_modules from being committed to version control, and expect you to run npm install after downloading the source.
There's very few good reasons to have your dependencies in your GitHub repository - as long as a project adheres to Semantic Versioning (the vast majority of packages do), npm install will never cause incompatible versions to be downloaded, and if you absolutely need to lock down the versions of your dependencies, you can just use npm shrinkwrap.
EDIT: As Matt's comment very helpfully pointed out, there's several features of NPM that go beyond simple metadata - the one I probably get the most use out of is Scripts, which allow you to create project-specific aliases for command-line operations.
An example of where this has come in handy for me is running the Webpack development server - it's installed locally to my project in the devDependencies (which you can do using the --save-dev option when installing a package), so if I was doing it manually, I would have to type something along the lines of:
"./node_modules/.bin/webpack-dev-server" --inline --hot
Which quite frankly, would be a bit of a pain. Instead, I can just add this to my package.json (note that node_modules/.bin is automatically added to the system path when using an NPM script, so you don't need to type that every time):
"scripts": {
"dev": "webpack-dev-server --inline --hot"
}
And then all I have to run is:
npm run dev
Beyond this simple use-case, there's also several 'special' script names which are automatically called upon certain events - for example, prepublish is run before publishing a package to the registry.
Each Node.js project must contain at least one package.json file, usually located in the root directory of your project. This file identifies the project and lists the packages your project depends on, making your build reproducible.
You can create a package.json file by using a text editor, but the quickest way is to run the npm init command and pass it the -y flag
npm init -y
I'm working in a web application (JavaScript/C#, version controlled by TFS) and our team wants to start using Visual Studio 2015. Microsoft is moving developers to use existing popular tools like Gulp for automated tasks, so I've written a few Gulp tasks that will run on the server.
My problem is that our automated builds generate new project folders on the build server, so I can't run gulp myBuildTask without first running npm install. The npm install adds over 2 minutes to the build process, and it seems very inefficient to download the same dependencies for every build (since they will change rarely).
Is there anyway I can run a Gulp task on a new project folder without first running npm install?
Options I've considered:
Include node_modules in TFS. I couldn't add the node_modules folder to TFS (which would cause it to exist in each new build folder) because bower's nested dependencies have file paths that are too long for Windows. I could go this route without bower, but I'm not certain I want all those files in my solution (much of which is not needed, like readme's and test files).
Run npm install after each automated build.
As already mentioned, I don't want to do this because it adds several minutes to the build process.
Install NPM modules globally.
I'm not sure if this is even possible, but I'm wondering if I can install all project dependencies globally on the build server (avoiding having to install at the project level). My concern with an approach like this is that I don't want to have to manually update the build server's globally installed NPM modules every time we add a gulp plugin.
Ideally, the solution would be something like #3. The modules would install globally, but every build could run an npm install which would verify every module is installed. If a new npm module was added to the package.json, it would be downloaded. This npm install would be pretty fast since in most cases, all modules would already exist (globally installed on the build server).
There are a few things you might do:
Make npm install run faster. For this purpose, use newest npm (if possible) or use npm dedupe. Running dedupe may result in having less dependencies than with plain npm install. Then run npm shrinkwrap which creates npm-shrinkwrap.json file which contain 'freezed' info about what exactly gets installed (and in which version) during npm install.
Remember, node_modules is just a directory, if you can copy / rsync it to your installation, you can skip the npm install phase altogether
Node package resolution approach is to first try local node_modules directory and if not successful, (node_modules not there or dependency missing in node_modules) check out node_modules of the parent directory, then grandparent directory and so on. This means, you don't have to install packages globally, semi-global installation is quite sufficient
:
my_project
node_modules/
dependency1
dependency2
build_001/
build_002/
build_00x/
no node_modules here,
no deps here
Note however, that this, naturally, works only if your dependencies are really not changing. Since in real life you install something new from time to time, slightly enhanced approach might be helpful: organize your directories as follows:
my_project
ver_af729b
node_modules
build_001
build_002
ver_82b5f3
node_modules
build_003
build_004
af729b and 82b5f3 being (prefixes of) sha hashes of your npm-shrinkwrap.json file. If you then add new dependency, shrinkwrap file gets updated, build script creates new ver_something directory and executes npm install in it. Doing all this would naturally require extra work, but it should work great.
------------------ EDIT -------------------
If you are not trying to avoid npm install completely (you just want it to be quick) you can stick to the typical scenario: you checkout the sources always to the same directory, and let npm install re-use the old node_modules as much as possible.
If you want always to create a new directory for your build, you may still create a node_modules symlink to the older version of node_modules - also in this scenario, npm will reuse as much as possible from symlinked folder.