I have a very special requirement from my client. We have been using npm to install karma and phantomjs for quite a while. Everything works fine until we have to move everything off the cloud to internal infrastructure. Now things get complicated. The internal infrastructure doesn't have internet access so we cannot use npm to resolve dependencies anymore. We tried to move node_modules folder dev machine to the internal infrastructure machine. It didn't work because dev machine is OSX and Windows and the server is Centos and phantomjs is OS specific but npm is able to workout the versioning. What options do we have to resolve dependencies? I just learn that node_modules name cannot be changed. I was thinking of checking in OS specific node_modules but that wouldn't work since npm only looks for node_modules folder.
I got the same error as this thread PhantomJS Crash - Exit Code 126 when I was trying to use node_modules from OSX in Centos.
Install all dependencies on first OS (i.e. OSX), assuming that you have package.json with all dependencies.
npm install
Rename created npm_modules to npm_modules_mac
Repeat steps above for different OS (i.e. Windows), rename node_modules to something like node_modules_windows.
On target OS, move folders created above to your app folder, create symbolic link (node_modules), which will point to appropriate folder (npm_modules -> npm_modules_mac in OSX)
Why don't you just host your private registry? You can store the registry in the internal infrastructure.
The defacto registry is #isaacs own npmjs.org. This can be found here:
https://github.com/isaacs/npmjs.org
It does require using CouchDB as the database, however, and that can be daunting. There are alternatives that allow you to do this. For example, reggie:
https://github.com/mbrevoort/node-reggie
Related
I'm planning to setup private npm registry for our internal Node.js/web projects, and seems that Verdaccio is the best open-source choice for it.
Before starting to publish my private packages there I want to be sure that an easy way to move Verdaccio installation with all published packages to another server exists. Here is a similar question for Sinopia Verdaccio project is forked from. Folks there say that we should simply move entire sinopia directory to another server. But what is the exact directory and what is the directory (or, probably, few directories) for Verdaccio?
I have Node.js installed by nvm script and Verdaccio installed globally the following way:
npm install -g verdaccio
Here Verdaccio core maintainer. Not long time ago I've written down the required steps to move from latest Sinopia to any Verdaccio (v3,v4-alpha).
In a nutshell (for UNIX):
The folder ~/.local/share/sinopia must be renamed to ~/.local/share/verdaccio
The folder ~/.config/sinopia must be renamed to ~/.config/verdaccio
There is an additional step, not required, but recommended:
The file ~/.config/sinopia/storage/.sinopia-db.json must be renamed to ~/.local/share/verdaccio/storage/.verdaccio-db.json
To find the Windows location, check the following link.
I hope this helps. Original source in the link below.
https://verdaccio.org/blog/2019/02/24/migrating-verdaccio#migrating-from-sinopia-140-to-verdaccio-2x-3x
What is the best practice for deploying a nodejs application?
1) Directly moving the node_modules folders from the development server to production server, so that our same local environment can be created in the production also. Whatever changes made to any of the node modules remotely will not affect our code.
2) Run npm install command in the production server with the help of package.json. Here the problem is, any changes in the node modules will affect our code. I have faced some issues with the loopback module (issue link).
Can anyone help me?
Running npm install in production server cannot be done in certain scenario (lack of compiling tools, restricted internet access, etc...) and also if you have to deploy the same project on multiple machines, can be a waste of cpu, memory and bandwidth.
You should run npm install --production on a machine with the same libraries and node version of the production server, compress node_modules and deploy on production server. You should also keep the package-lock.json file to pinpoint versions.
This approach allows you also to build/test your code using development packages and then pruning the node_modules before the actual deploy.
Moving node_modules folder is overkilled.
Running npm install might break the version dependencies.
The best approach is npm ci. It uses the package_lock file and installs the required dependencies without modify the versions.
npm ci meant for continuous integration projects. LINK
I am an ASP.NET Core developer but I recently started working with Node.js apps. For me this was one of the challenges you mentioned to move the node_modules folder to production. Instead of moving the whole folder to production or only running the npm install command on production server, I figured out and tried a way of bundling my Node.js app using Webpack into a single/multiple bundles, and I just got rid of the mess of managing node_modules folder. It only picks up the required node_modules packages that are being used/referred in my app and bundles up in a single file along with my app code and I deploy that single file to production without moving the entire node_modules folder.
I found this approach useful in my case but please suggest me if this is not the correct way regarding the performance of the app or if any cons of this approach.
Definitely npm install. But you shouldn't do this by your own hand when it comes to deploying your app.
Use the tool for this like PM2.
As for your concern about changes in packages, the short answer is package-lock.json.
My guess is that by asking this question you don't really understand the point of the package.json file.
The package.json file is explicitly intended for this purpose (that, and uploading to the npm registry), the transfer of a node package without having to transfer the sizeable number of dependencies along with it.
I would go as far as to say that one should never manually move the node_modules directory at all.
Definitely use the npm install command on your production server, this is the proper way of doing it. To avoid any changes to the node_modules directory as compared to your local environment, use the package lock file. That should help with minimising changes to the source code in node_modules.
I mean no bad intent by saying this
I want to build my Node.JS application in a Azure Website.
There will be an usage of different NPM packages via my packages.json file.
My problem is that I often receive error messages which are related to missing NPM files.
Normally I put my files via FTP or edit them per VS Studio 15 Azure plugin directly on the server. This may be the reason why NPM isn't triggering as Microsoft intended it.
I would prefer a way in which I can just run commands with elevated privileges to have full control over NPM by myself.
Which ways are possible to avaid these problems?
If you're publishing your nodeJS application 'manually' via FTP there are little concerns about that.
First of All, 'manually' means manually.
Git
If you use continuous deployment via Git the final deployment step is to call npm install in your current application folder, this will install all the packages listed in package.json file.
The node_modules folder is excluded by default in .gitignore file, so all packages are downloaded by the server
Web deployment
If you're using web deployment from visual studio or command line, all the files contained by your solution are copied to Hosting environment including node_modules folder , because of this the deployment would take a long time to finish due the huge amount of dependencies and files that the folder contains.
Even worst: this scenario could take you to the same scenario you're facing right now.
FTP deployment
You're copying everything yourself. So the same thing occurs in Web Deployment is happen in FTP deployment method.
--
The thing is that when you copy all those node_modules folder contents you're assuming that those dependencies remains the same in the target enviroment, most of the cases that's true, but not always.
Some dependencies are platform dependent so maybe in you're dev environment a dependency works ok in x86 architectures but what if your target machine or website (or some mix between them) is x64 (real case I already suffer it).
Other related issues could happen. May be your direct dependencies doesn't have the problem but the linked dependencies to them could have it.
So always is strongly recommended to run npm install in your target environment and avoid to copy the dependencies directly from your dev environment.
In that way you need to copy on your target environment the folder structure excluding node_modules folder. And then when files are copied you need to run npm install on the server.
To achieve that you could go to
yoursitename.scm.azurewebsites.net
There you can goto "Debug Console" Tab, then goto this directory D:\home\site\wwwroot> and run
npm install
After that the packages and dependencies are downloaded for the server/website architecture.
Hope this helps.
Azure tweak the Kudu output settings, in local Kudu implementations looks the output is normalized.
A workaround -non perfect- could be this
npm install --dd
Or even more detailed
npm install --ddd
The most related answer from Microsoft itself is this
Using Node.js Modules with Azure applications
Regarding control via a console with elevated privileges there is the way of using the Kudu console. But the error output is quite weird. It's kind of putting blindly commands in the console without much feedback.
Maybe this is a way to go. But I didn't tried this yet.
Regarding deployment it looks like that Azure wants you to prefer Continuous Deployment.
The suggested way is this here.
I have a project setup that is as follows:
workspace
└cache
└node_modules
└gulp (and gulp-plugins, express etc.)
└nodejs
└node.exe
└project1
└gulpfile.js
└project2
└gulpfile.js
Now I want to execute the gulpfile in the project directories:
set NODE_PATH='C:\workspace\cache\node_modules\'
cd C:\workspace\project1\
C:\workspace\nodejs\node.exe C:\workspace\cache\node_modules\gulp\bin\gulp.js watch
and I get the following output:
[12:06:04] Local gulp not found in C:\workspace\project1
[12:06:04] Try running: npm install gulp
In both project folders the gulpfile is similar and uses a similar set of plugins. I'd really like to have the dependencies only once (because potentially I have up to 25 projects sharing the same node_modules).
Is this setup possible, or does the seperate project directories need to have their own node_modules folders?
Gulp requires you to have both a global installation as well as a local one. So you need to have your Gulp relatively to your Gulpfile. If your package.json would be located in workspace and your node_modules would be in workspace/node_modules everything would work fine because of Node's search tree, but if you can't move them, the only way to make it work is to "fake" the node_modules folder.
You can do this by creating a symbolic link.
Here's on Unix/Linux/Mac:
ln -s ../cache/node_modules node_modules
Here's on Windows
mklink /D node_modules ../cache/node_modules
(the latter one might work different, I'm not on a Win machine)
You could also try pkglink
From description:
Space saving Node.js package hard linker. pkglink locates common JavaScript/Node.js packages from your node_modules directories and hard links the package files so they share disk space.
Edit: ddprt
On Windows
mklink /D node_modules "C:/fullPATH/cache/node_modules"
you could always use the '-g' parameter with npm install 'package-name', so as to make the module available globally to access across different projects.
See the following links
what does the "-g" flag do in the command "npm install -g <something>"?
How do I install a module globally using npm?
https://docs.npmjs.com/files/folders
Packages are dropped into the node_modules folder under the prefix. When installing locally, this means that you can require("packagename") to load its main module, or require("packagename/lib/path/to/sub/module") to load other modules.
Global installs on Unix systems go to {prefix}/lib/node_modules.
Global installs on Windows go to {prefix}/node_modules (that is, no
lib folder.)
Scoped packages are installed the same way, except they are grouped
together in a sub-folder of the relevant node_modules folder with the
name of that scope prefix by the # symbol, e.g. npm install
#myorg/package would place the package in
{prefix}/node_modules/#myorg/package.
I've been using Node.js and npm for a few weeks with great success and have started to question the best practice for installing local modules. I understand the Global vs Local argument, however, my question has more to do with where to place a local install. Let's say that I have a project located at ~/ProjectA/ which is version controlled and worked on by multiple developers. When initially playing with Node.js and npm I wasn't aware of the default local installation paths and just simply installed the necessary modules in a default terminal which resulted in a installation path of ~/node_modules. What this ended up doing is requiring all the other developers working on the project to install the modules on their own machines in order to run the application. Having seen where some of the developers ran npm install I'm still really surprised that it worked on their machines at all (I guess it relates to how Node.js and require() looks for modules), but needless to say, it worked.
Now that the project is getting past the "toying around" stage, I would like to setup the project folder correctly. So, my question is, should the modules be installed at ~/ProjectA/node_modules and therefore be part of the version controlled project files, or should it continue to be located at a developer-machine specific location...or does it not really matter at all?
I'm just looking for a little "best-practice" guidance on this one and what others do when setting up your projects.
I think that the "best practice" here is to keep the dependencies within the project folder.
Almostly all Node projects I've seen so far (I'm a Node developer has about 8 months now) do that.
You don't need to version control the dependencies. That's how I manage my Node projects:
Keep the versions locked in the package.json file, so everyone gets the same working version, or use the npm shrinkwrap command in your project root.
Add the node_modules folder to your VCS ignore file (I use git, so mine is .gitignore)
Be happy, you're done!