Setup:
A VM with an Internet connection where npm install will be executed to install all the app dependencies. The result will be a folder with the app and its dependencies in node_modules.
Between the app modules is fi: mongoose, which on installation time uses node-gyp to compile a native BSON extension.
The app folder is copied to another VM without an Internet conection and it is fully functional, but then the compiled extensions don't work but its .js fallbacks does.
Question:
How can I reinstall/recompile/regenerate all the app modules on the new VM without an Internet conection?
This is precisely what the npm rebuild command does. Just run npm rebuild inside your app directory after it is copied over to the new VM and any binary add-ons will be recompiled to match the current CPU architecture and node version. If the initial npm install before the copy was completely successful, the npm rebuild on the second VM will not need to download anything. Just make sure the second VM has a reasonably-close version of node and the appropriate lower level compilers, libraries, etc (build-essential and python on debian, for example).
Related
Is there anyway we can setup SharePoint framework development environment offline without internet connection?
Following this instruction,
https://learn.microsoft.com/en-us/sharepoint/dev/spfx/set-up-your-development-environment
It failed on the second step,
npm install -g yo gulp
If your node modules were already installed, then yes, you can develop offline. But the npm install command very specifically downloads and installs the package you specify.
If you can find a colleague who already has those modules installed and you can copy from their machine to yours, you could potentially get it setup without an internet connection, but you are going to have to get those SPFx packages (Yeoman Gulp and Microsoft Generator) from somewhere.
My node app will deploy in a bank that can't access the internet. Can I download all the node modules in a local directory, and install them from it, or just copy the node_modules to the bank machines? Any safe and practical advise will be highly appreciated, thank you.
OR
How to add node_modules dependencies without using internet .
Add offline-npm to your project to serve a npm compatible tgz file wich contains all dependencies for offline installation with npm install.
Additionally you can use offline-npm -n to install packages from your local npm cache directory
For installation
npm install -g offline-npm
Usage
1) Open terminal and go to your project you want to prepare for offline use. This folder needs to contain a package.json file.
2) Prepare your project for offline use
offline-npm --add
3) This changes the package.json file and adds a offline folder which will contain all your dependencies.
Pack your project
npm pack
For more details , visit the Link
Hope this answer gets useful to you .
Typically you will want to bundle your node_modules into your deployable artifact.
There are some gotchas here around any native dependencies as you need to match the nodejs version of the build environment with the target system.
The easiest way to achieve this in my experience is by using docker to build and package your deployment. Though it is possible to do when running directly on host machines, you may find it safest to just avoid the usage of native dependencies to remove any risk of things breaking from a nodejs or os update.
I've also successfully achieved this packaging the nodejs binary into my deployment artifact that was deployed directly on centos hosts, however we had a mixture of centos 6 and centos 7 hosts at the time, which brought additional complexity related to different glibc versions causing nodejs to fail to start with the system provided library.
In short if you can I would use docker to package your application into a completely self contained image.
you can just copy the node_modules into the deployment machine but it is bad practice.
there are other solutions like using local-npm package... your npm installs are fetched from the registry and then modules and their dependencies get stored in a local PouchDB database. This caches them so subsequent npm installs use the local cache rather than calling to the network. local-npm also takes care of keeping modules updated when they change. It does this by listening for changes to the remote registry so you don't have to worry about staleness.
or bundling your packages to use them offline you could visit this link for more details for offline installation of npm packages:
https://addyosmani.com/blog/using-npm-offline/
How to install node modules in local network?
1. Solution: yarn offline and 'Offline Mirro'
One of the main advantages of Yarn is that it can install node_modules from files located in file system. We call it “Offline Mirror” because it mirrors the files downloaded from registry during the first build and stores them locally for future builds.
2. New problem: How to use yarn in local network?
Download yarn.tar.gz into the local repository, and install it in local node_modules directory.
npm install yarn.tar.gz --no-save
3. Usage
# run yarn install, and download the node modules (.tar.gz) into the offline mirror directory '$REPOSITORY/yarn/yarn-offline-mirror'.
npm run online-install
# with yarn.lock file, install node_modules from offline mirror directory '$REPOSITORY/yarn/yarn-offline-mirror'
npm run offline-install
4. Problems
4.1. problem 1
4.1.1. description
error can`t make a request in offline mode("http://....")
4.1.2. reason
the indirect dependencies could not be downloaded into offline mirror directory
4.1.3. resolution
yarn config set yarn-offline-mirror-pruning false
5. Github demo
yarn-offline-deploy-demo
6. Reference
1. Running Yarn offline
I am new to node, i have made few small application using node, but everytime i have to use npm install for every application which download the required dependencies in node_modules folder. There are many libraries which are common.
I tried installing using npm install express -g but i was not sure how to use this dependency in other application which is in some other folder.
Is there any way i can have only one folder like in D:\Users\User\AppData\Roaming\npm\node_modules from where my all applications can have the module which they need ?
Can anyone let me know how to do the settings for the same ?
Any help would be highly appreciated !!
Every node application that has a package.json has a specific set of rules for using specific versions of it's modules. You can install globally only one version of a specific module, but if you happen to have an application that needs an older / newer version that is not installed globally on your dev environment, then it will fail to work.
The recommended way of using node modules ( packages ) is to have a local directory inside your project, which contains all libraries that the project needs. This practice is everywhere and so you should follow it.
There are some ways to mitigate the slow npm install, though.
There is a new npm-replacement, created and maintained by Facebook, called yarn.
What yarn does is it creates a local cache of all installed packages and then symlinks them to your project folder from your local computer cache. This way the npm install procedure becomes very fast.
Is it possible to copy a set of NPM installed files and associated files from a Mac computer to a Windows computer, and for all those files to work?
For example, transfering Node.js files with some other NPM files from Mac to Windows, then running node app.js in that directory (on the Windows Command Prompt).
Thanks! :)
The binary, npm, that you install is platform dependent, as is node.js. That's why there are different releases for each platform available on the download site.
For the most part, your project files are platform independent. For example, most of your JavaScript files will all be used by node.js and work just fine without having to worry about what platform you are on because the system details will be dealt with by node.js itself.
However, some modules are platform dependent. For example, anything that uses node-gyp will try to compile on your platform whenever the module is installed by npm. You do not have to worry about that though because it is handled by npm, that's why you're using a package manager.
Copying node_modules can be done; but it's more often than not better and easier to just run npm i on whatever machine is going to be running your application. You can avoid having to worry about version problems using something like npm shrinkwrap which will lock down the version of a package that your module depends on.
NPM packages that contain native addons/dependencies are tied to the OS + NodeJS version and have to be rebuilt specifically for the system you intend to use them on. This is why you're seeing the error mentioning bson.node, it is a native addon that has to be rebuilt, this can be done with the npm rebuild command.
I am facing challenges while creating package for all the clientside components of my project in a system which does not have internet connection (for windows).
I have installed NPM for windows in the system. I need to manage the node_modules and run gulp commands in the system to create the package.
Since there is no internet connection in the box, I decided to copy the node_modules from my local box to the box which does not have internet connection.
For copy task, I am using msbuild script but somehow it is not working for me. Also, I see that when I am trying to copy the node_modules manual from one folder to another folder I am unable to copy.
Installed Node version : v0.12.2
NPM version: 2.7.4
Can anyone help me to provide any working sample to fix the above problem.
I think your approach is generally sound. You can build on the internet connected computer no problem! Copy files over with a usb stick if you like.
Here are some tips that might help solve your problems.
If the box is offline, develop on your local machine and deploy to the box
If possible do the gulp steps also on your local machine
Otherwise you need to run the local directory install of gulp. e.g.
node node_modules\gulp\bin\gulp.js build
because you won't be able to npm install -g gulp
Point 3 will work for other global npm_modules too btw
If you are facing long paths (> 256 characters) which can cause copy problems
Try using npm dedupe to remove duplicates
Or try copying to a shorter path e.g to c:\proj instead of to c:\very\long\path\proj
Or explicity install a dependency which has a long path
e.g
npm install deepdep#1.2.4
and then prune that folder from the original
rm node_modules\package\node_modules\package_with_too_many_nested_folders
Or Install the latest npm (v3.0 or higher) which solves this issue once and for all
e.g. npm install -g npm
which will build a much flatter hierarchy for your packages. Requires removing and reinstalling all packages.
Point 5 is a notorious issue on windows which is not a problem on linux because paths can be extremely long. (unless you are mounting a windows a windows directory from linux)
Personally I would go straight for the latest npm version but you have a number of ways of getting around the issue if this is not possible.