I am in the process of migrating a machine from an older Linux version to a newer one. We have some node projects that seem to be more difficult to transplant to the newly desired setup.
On the old machine the build script from package.json could be run without problems because node_modules is tracked by the source control.
We don't want to use that method anymore since it is not very scalable as we want to make sure we can easily upgrade all the packages for all the users at once without the hassle of putting hundreds of subfolders (from node_modules) together with various resources.
Did you encounter such issues and, if so, how did you manage it?
Can we use a global node_modules somehow to avoid multiple node_modules for each project?
Thank you!
Can we use a global node_modules somehow to avoid multiple node_modules for each project?
There are (at least) three possibilities. Here they are in the order that I would recommend using them:
If Node.js does not find a package in the application's/module's own node_modules directory, it will check in the parent directory (../node_modules) and then that directory's parent directory (../../node_modules) etc. So if all your projects have a shared parent directory, you can put the dependencies in a node_modules directory. See "Loading from node_modules folders" in the Node.js documentation.
A second option is to set the NODE_PATH environment variable to the directory where you would like your projects to search for modules not found elsewhere. See "Loading from the global folders" in the Node.js documentation.
Lastly, it's legacy behavior, but Node.js will also search in three other locations before giving up, so you can use one of those as well: $HOME/.node_modules, $HOME/.node_libraries, and $PREFIX/lib/node. See (again) "Loading from the global folders" for more information.
All that said, please note that the docs also correctly say "It is strongly encouraged to place dependencies in the local node_modules folder. These will be loaded faster, and more reliably."
If you must do this, I would use the first option above and place the node_modules folder as far down in the directory hierarchy as you can.
Related
I am starting a JavaScript project and its parent directory contains 2 folders: client and server. Both of these will require Javascript packages but I'm confused as to whether I should make a node_modules folder in each of them for their respective dependencies or whether I should create only one and install all the packages there. Basically what I'm asking is whether I should do this:
or this:
I should make a node_modules folder in each of them for their
respective dependencies or whether I should create only one and
install all the packages there.
As you have client- Frontend and server - Backend which are two different side of things and they will have their own dependencies with package.json. You might be installing some specific packages which only works on browser and same for server-side environment.
It is good to segregate both the environment to keep dependencies separate. You should check this link to check out things in details.
I'm developing various Angular 2 projects and I want to share node_modules folder between multiple projects. I would like to create a structure like this:
MainFolder
- Project1
- Project2
- package.json
so I would have just 1 package.json for all the projects. My answer: is it possible to do this?
If possible, I have to lunch npm install with -g?
I can't understand how -g works.
Can someone give me instructions how to proceed?
Very thanks
I forgot to say that I build the projects with angular-cli.
The way I go around this for small/learning/test projects is with (I call it) "git projects". Basically I manage the various projects via git, and just "load" the project I want to work on. Of course this doesn't work if you want to have access to multiple projects at the same time.
I like to use a git client for this purpose because it's easier to visualize my existing "projects".
So my workflow is this...
Create my main/base folder. This will contain the git repo, the single node_modules folder, and whatever else that should be common to all projects.
I create the basic package.json file (using npm init). No description, no nothing, just the basic skeleton package.json file. (However, if you know you will use certain packages in ALL of your projects, you can npm install them first, so they will be added to package.json as your "base" modules.)
Now I check the bare package.json into the repo (and anything else that you may want to have in all of your projects, but usually it's just the package.json file). This will be the bare-bones starting branch for all projects.
Once this is checked in, I create a branch off of this in the git repo. This will be "Project 1" - or whatever you want to call it. Then build up your project however you want, installing modules, checking in changes, etc, etc.
When I want to start a new project, I simply check out the first bare-bones project (which is just the empty, or almost empty, package.json file) and do another branch off of it. This will be my 2nd project.
And so forth...
So the main thing is that every new "project" will be a new branch in the git repo, and to create a new project, just switch back to the original bare-bones one and do a new branch off of that.
Of course it is possible to create branches within a project, too. It's all about naming conventions. You could, for example, prefix a new project branch with "P_" or "PROJECT_", etc, so you can quickly tell in your git client which branches are projects. And of course use a different naming scheme if you just need a new branch within an existing project. That's basically how I go about it.
You may not like this workflow, but this way I don't need to install packages globally. When I do a backup, I can simply delete the single (possibly huge) node_modules folder. All project related modules can be reinstalled by simply checking out a branch for a particular project and run "npm install" on its package.json. Hope it makes sense.
Here is documentation on the various npm install arguments
In global mode (ie, with -g or --global appended to the command), it
installs the current package context (ie, the current working
directory) as a global package.
The -g install locations based on environment can be found here
One way you can achieve what you want is to have one solution for both projects and each project route uses it's own lazy loaded module.
Unless you have a specific business need to share resources, it's better to keep each project separate with it own resources and configuration.
-g Stands for global Installation, i.e. the packages you install will be available for all applications.
And why do you want to share node_modules and package.json file?
Keep them seperate for each seperate project. And if you need to share your project, you may share your package.json instead of sharing the node_modules folder.
Also to point out, if you manually install packages by listing their names, then you can use -g (global) flag, but if you do use only npm install then your packages won't be installed as global packages.
If it really is just for testing simple applications, could rename tha app folder in some way provide a solution. It assumes that all the dependencies are the same or at least a subset of the dependencies provided.
Recently started working with Gulp and I can't figure out is it really necessary to have a copy of node_modules directly in folder with current project?
E.g. I have this structure:
mysite
└─builder
└──node_modules
└─work
└─work2
How can I access node_modules in folder 'builder' from folder 'work' or 'work2' without copying it? It is quite large, about 100mb, and seems to me it has no sense to have a copy of it for every new project.
I tried this line export NODE_PATH='D:\OpenServer\domains\mysite\build' in file package.json and then tried command gulp but it replied[10:24:27] Local gulp not found in d:\OpenServer\domains\mysite\work
[10:24:27] Try running: npm install gulp
Short answer
Don't do it. Let NPM work the way it's designed to. However, to save space, you can delete the node_modules folder on projects that are currently dormant, and recreate it with a single shot of npm install when you switch back to them.
Justification
Even if you share your node_modules, you'll probably have redundancies in it anyway. What will you do about them next ?
It is the essence of NPM to replicate modules per project. If you dig into the node_modules folder tree, you may notice that it can even contain several replications of a same library under one given dependencies tree. Say you requested two modules explicitely, and both these modules themselves pulled a dependency that takes care of a lot of things, and is therefore called lib_DADDYMUMMY :
node_modules
+ a_module_you_use v0.5
+ lib_DADDYMUMMY v0.1 (pulled as a dependency of this module)
+ another_module_that_you_requested v0.3
+ lib_DADDYMUMMY v0.1 (again ! pulled as a dependency of this other module)
This comes in handy when your two module start needing different versions of lib_DADDYMUMMY. This comes in handy when you maintain long-lived projects ! And hell knows that in the JavaScript world, with fast changing APIs, you can consider most any decent project as long-lived. :)
One could imagine having all dependencies being shared by everyone, living in a flat structure, with several versions of a library living next to each other and every one finding what he needs there. That repository could be called, say, .m2. But that's just not the way NPM works unfortunately.
NPM considers that storage space is cheap. That's its price for helping you manage versions in dependencies, dependencies of dependencies, and dependencies of dependencies of dependencies. I consider that it's an affordable price for taking care of the dirty jobs the day when work and work2, as their lives go on, take diverging maintenance paths. I wouldn't try getting in its way by forcing a half-Maven-like folder model.
Maybe you should put your package.json into your root directory(mysite/package.json),
then try to install node_modules on the root.
In addition, you write gulpfile on the same dir.
eg.
mysite
|- package.json
|- node_modules
|- gulpfile.js
└─builder
└─work
└─work2
However, I recommend that you write one single gulpfile for each project.
One problem why you shouldn't do this is because of versioning. If your modules require different versions of the same package, you're going to run into problems. One package is going to win, and it might break another package.
Further, you get into the problem of having to merge the dependency lists in some way - meaning, you'll have to get the dependencies from work/package.json, work2/package.json, etc. and then install all of them at once.
Merging node_modules/ won't solve your problem, either - believe me, don't try.
Paste the node_modules folder inside your mySite directory.
All npm packages such as gulp will work in your work or work2 directory.
But, now(your folder structure) work folders can't find node_modules in their parent directory.
I want to use gulp on my windows machine and it actually works pretty fine, unless I try to use the created files (like pushing to github or deleting). Then it breaks, because the filepaths are too long and it seems to be a fairly common problem. https://github.com/joyent/node/issues/6960#issuecomment-45569604
I understand that the problem arises through npm's nested directories, which extend the maximal char count for Windows directories, but in my understanding there is not any solution yet.
As I see it right now I have three options:
Try to reduce the chars of npm's directories, by changing the default from 'node_modules' to 'n_m' and hope that problem ist postponed. Like suggested here:
https://github.com/joyent/node/issues/6960#issuecomment-45569604
Then it is my question, how exactly do I change the default 'node_modules' directory name?
Change my development environment to Ubuntu, which is frankly a solution I dislike, because I've never used Ubuntu.
Stop using gulp overall.
So, how do I change the default 'node_modules' directory created through npm or what solution do you actually suggest?
There is one more tricky option.
Main problem is that gulp has a lot of nested dependencies and it creates very long nested file pathes.
But if you install some of npm modules that gulp requires in your main node_modules directory gulp will not download them as nested.
Currently you have something similar to (this may be not real path you have but idea is the same):
\node_modules\gulp\node_modules\lodash.bind\node_modules\lodash._createwrapper...
If you will add "lodash.bind" module to your project's package.json as dependency it will be installed in one level with gulp and not as gulp's dependency
\node_modules\gulp
\node_modules\lodash.bind\node_modules\lodash._createwrapper
And this will shorter all urls. You will need to fix only one or two with the longest pathes and it will work.
In my project it was enough to add this dependencies: “lodash.createcallback” and “lodash.bind” to package.json to fix everything.
Take in mind that befor doing this you probably would need to clear current node_modules folder. If you are not able to do that because off too long url you can create symbolic link to temporary short file path and delete it.
What would be the preferred way to handle internal modules within a node.js application?
I mean, currently we have a fairly big application and there are several dependencies, installed via npm. We have node_modules in .gitignore. Therefore it's not in the repository. We use npm shrinkwrap to freeze the modules versions, so, upon deployment, an npm install puts together eveything we need into node_modules.
The problem is, since out app is getting bigger, we want to split it to smaller modules. Now, if I create a foo module and put it in node_modues, I need to allow it in the repo, which does not seem so neat to have both ignored and checked out modules in node_modules.
We can not publish these on npm registry because they are not really "public".
Is there any obvious solution that I'm not aware of?
I agree with your aesthetic of not mixing 3rd-party non-repo stuff with your in-house revision-controlled contents.
The NODE_PATH search path variable can help you here, you can use a different directory with contents under revision control.
http://nodejs.org/api/modules.html
Depending on your appetite for flexibility vs complexity, you can consider a build step that actually copies your various node modules from wherever you keep them, into an output directory (probably "named node_modules" but not necessarily).
A common solution is to store the modules in a private Git repository somewhere, and then use a git:// URL as a dependency; npm knows how to clone repositories as dependencies.