I am facing challenges while creating package for all the clientside components of my project in a system which does not have internet connection (for windows).
I have installed NPM for windows in the system. I need to manage the node_modules and run gulp commands in the system to create the package.
Since there is no internet connection in the box, I decided to copy the node_modules from my local box to the box which does not have internet connection.
For copy task, I am using msbuild script but somehow it is not working for me. Also, I see that when I am trying to copy the node_modules manual from one folder to another folder I am unable to copy.
Installed Node version : v0.12.2
NPM version: 2.7.4
Can anyone help me to provide any working sample to fix the above problem.
I think your approach is generally sound. You can build on the internet connected computer no problem! Copy files over with a usb stick if you like.
Here are some tips that might help solve your problems.
If the box is offline, develop on your local machine and deploy to the box
If possible do the gulp steps also on your local machine
Otherwise you need to run the local directory install of gulp. e.g.
node node_modules\gulp\bin\gulp.js build
because you won't be able to npm install -g gulp
Point 3 will work for other global npm_modules too btw
If you are facing long paths (> 256 characters) which can cause copy problems
Try using npm dedupe to remove duplicates
Or try copying to a shorter path e.g to c:\proj instead of to c:\very\long\path\proj
Or explicity install a dependency which has a long path
e.g
npm install deepdep#1.2.4
and then prune that folder from the original
rm node_modules\package\node_modules\package_with_too_many_nested_folders
Or Install the latest npm (v3.0 or higher) which solves this issue once and for all
e.g. npm install -g npm
which will build a much flatter hierarchy for your packages. Requires removing and reinstalling all packages.
Point 5 is a notorious issue on windows which is not a problem on linux because paths can be extremely long. (unless you are mounting a windows a windows directory from linux)
Personally I would go straight for the latest npm version but you have a number of ways of getting around the issue if this is not possible.
Related
My node app will deploy in a bank that can't access the internet. Can I download all the node modules in a local directory, and install them from it, or just copy the node_modules to the bank machines? Any safe and practical advise will be highly appreciated, thank you.
OR
How to add node_modules dependencies without using internet .
Add offline-npm to your project to serve a npm compatible tgz file wich contains all dependencies for offline installation with npm install.
Additionally you can use offline-npm -n to install packages from your local npm cache directory
For installation
npm install -g offline-npm
Usage
1) Open terminal and go to your project you want to prepare for offline use. This folder needs to contain a package.json file.
2) Prepare your project for offline use
offline-npm --add
3) This changes the package.json file and adds a offline folder which will contain all your dependencies.
Pack your project
npm pack
For more details , visit the Link
Hope this answer gets useful to you .
Typically you will want to bundle your node_modules into your deployable artifact.
There are some gotchas here around any native dependencies as you need to match the nodejs version of the build environment with the target system.
The easiest way to achieve this in my experience is by using docker to build and package your deployment. Though it is possible to do when running directly on host machines, you may find it safest to just avoid the usage of native dependencies to remove any risk of things breaking from a nodejs or os update.
I've also successfully achieved this packaging the nodejs binary into my deployment artifact that was deployed directly on centos hosts, however we had a mixture of centos 6 and centos 7 hosts at the time, which brought additional complexity related to different glibc versions causing nodejs to fail to start with the system provided library.
In short if you can I would use docker to package your application into a completely self contained image.
you can just copy the node_modules into the deployment machine but it is bad practice.
there are other solutions like using local-npm package... your npm installs are fetched from the registry and then modules and their dependencies get stored in a local PouchDB database. This caches them so subsequent npm installs use the local cache rather than calling to the network. local-npm also takes care of keeping modules updated when they change. It does this by listening for changes to the remote registry so you don't have to worry about staleness.
or bundling your packages to use them offline you could visit this link for more details for offline installation of npm packages:
https://addyosmani.com/blog/using-npm-offline/
How to install node modules in local network?
1. Solution: yarn offline and 'Offline Mirro'
One of the main advantages of Yarn is that it can install node_modules from files located in file system. We call it “Offline Mirror” because it mirrors the files downloaded from registry during the first build and stores them locally for future builds.
2. New problem: How to use yarn in local network?
Download yarn.tar.gz into the local repository, and install it in local node_modules directory.
npm install yarn.tar.gz --no-save
3. Usage
# run yarn install, and download the node modules (.tar.gz) into the offline mirror directory '$REPOSITORY/yarn/yarn-offline-mirror'.
npm run online-install
# with yarn.lock file, install node_modules from offline mirror directory '$REPOSITORY/yarn/yarn-offline-mirror'
npm run offline-install
4. Problems
4.1. problem 1
4.1.1. description
error can`t make a request in offline mode("http://....")
4.1.2. reason
the indirect dependencies could not be downloaded into offline mirror directory
4.1.3. resolution
yarn config set yarn-offline-mirror-pruning false
5. Github demo
yarn-offline-deploy-demo
6. Reference
1. Running Yarn offline
There is a way to install npm package to a machine which doesn't have internet acccess is, using npm pack in machine with internet acces, copying it to machine without internet and running npm install <tar> in it. But npm pack, packs whole project.
But I want to manage and install the modules myself, without the opportunity for the developers to add/remove any modules. So I just want node_modules to be packaged. And then want to install it to machine without internet.
For example when developer push his/her commits to origin, I want to get node_modules from ftp etc. and codes from GitLab then go on continuous integration with this static node_modules.
How can I do that?
There is a solution to manage the modules yourself: you can store your node_modules in its own repo in which your developers will only be able to clone/get the repo and not contribute/modify it.
Hope this helped you
This can be done, please look at Installing a local module using npm? . You can FTP or whatever to get the packages and install them using npm.
I'm working on an offline network and want to install angular-cli using npm.
I have a zip file of angular-cli and using the latest node and npm version.
I'm using the command: npm install ./angular-cli-master to install angular-cli from the folder.
But I keep getting this error telling me I don't have an internet connection (which is ok).
So how can I install this angular-cli while offline using the zip I downloaded from Github?
Thanks for your help.
You simply copy the package and all dependencies in your node_modules folder, inside the project for local installation, or in the global folder (npm config get prefix to see where it is located) for a global installation.
The behavior of npm install is to check for the dependencies, and install them first. When it doesn't find them installed, nor the local file containing them, it tries to download them.
Since all of those steps fail (you don't have the dependency installed, it isn't available on the expected location, and it can't download it), the installation fails.
You can find the dependency list in the package.json of each module, but since it is recursive, it can take a long time to have everything set right if you do it manually, npm does it by recursion.
For you, the easiest way would be to create a new folder on the connected PC, and inside it npm install angular-cli, zip the folder and transfer it on the offline machine.
Jan 2016 - check out Addy Osmani's recommendations for offline installation of npm packages
May 2017 - as of npm 5, you can pass the --prefer-offline flag to npm install
yarn does this out of the box.
In 2019, I found none recommended approaches were applicable to an "air gapped" server with no internet access.
I found the only solution was to, on windows, using artillery.io as an example:
install the package on a machine with internet access, e.g local dev machine. npm install -g artillery
Browse to C:\Users\{username}\npm
zip up the \node_modules\artillery (e.g artillery.7z)
Copy the zip and the files artillery, artillery.cmd (at root of npm folder) to the server
Paste the two artillery, artillery.cmd to the root of the servers npm folder (C:\Users\{serverusername}\npm)
Extract the zip to C:\Users\{serverusername}\npm\node_modules
This is the complicated version for just one tool. If your local machine's npm folder is relatively light on tools, you could always just zip the whole npm folder and copy + extract it on the server.
I still think it's odd that npm insists on trying to connect to the registry even when using npm pack and npm install -g <tarfile>
Problem: I'd been in similar situation where I can't install the express.js and all other dependencies specifies by package.json on my local machine (offline) using npm due to unavailability of internet connectivity.
Solution: I've a solution that works on Windows(not so sure of other platforms) through which I installed express framework with all the dependencies I required for my project which include cookie-parser, jade, morgan etc.
Steps :
Install all the package(s) on a remote machine which has an internet access.
In my case I'm using Windows on both remote as well as local machines and my requirement was of installation of express.js on local machine . So I run below command on my remote machine to install express.js
C:\Users>npm install -g express-generator`
After installation of express.js I created an app on my remote machine using:
C:\Users\Name\Desktop>express Project`
C:\Users\Name\Desktop\Project>npm install -g =>to install all other dependencies globally*
Now browse to location where npm's global modules are stored, you can view the location by
C:\Users>npm config get prefix
Generally in Windows its
C:\Users\{Username}\AppData\Roaming\
Simply copy the npm and npm-cache folder of your remote machine.
And place both copied folders viz. npm and npm-cache into your local machine on same location thats
C:\Users\{Username}\AppData\Roaming\
the short answer, you can't. Most NPM packages such as #angular/cli need other dependencies and those need child dependencies which get installed when you run npm install
You can, however, install the cli when on the network and use it when offline.
You can find the npm install command documentation here: https://docs.npmjs.com/cli/install
I am not quite sure and unfortunately, I do not have the chance to test it myself right now, but I would try to either unzip the folder and remove the dot, like that:
npm install /angular-cli-master
(= installing a folder not a zip file)
or just add the zip file ending like that:
npm install ./angular-cli-master.tgz
(= installing a zip-file not a folder, file ending may be .zip or something else, though)
Was test success with node 18.x.x.
The following step guild how to install http-server package
On Online PC:
npm install -g http-server
After finish install, copy http server folder. (Usually locate at: C:\Users[UserName]\AppData\Roaming\npm\node_modules)
On offline PC:
Paste http-server folder. e.g. D:\http-server
npm install -g D:\http-server
Online computer:
npm install -g offline-npm
copy the npm-module to the offline computer and thats it !
is there any way to route npm install to a specific part of hard drive and when i do npm install it make node_module folder in that part of drive, and when i run any project it look for dependencies in that part of drive,
just like single pool for every project.
then if i have two projects with similar dependencies then i only need to npm install in one project so dependencies become available in pool, and no need to do npm install in another project just npm start
Thank you,
Inzamam Malik
You can achieve something close to what you are describing with the link option.
From https://docs.npmjs.com/misc/config#link:
If true, then local installs will link if there is a suitable globally installed package.
Note that this means that local installs can cause things to be installed into the global space at the same time. The link is only done if one of the two conditions are met:
The package is not already installed globally, or
the globally installed version is identical to the version that is being installed locally.
So you will still have some files in each project's node_modules, but you shouldn't have as large a folder.
To turn this behavior on, run:
npm config set link -g
Edit: There is no way you can avoid running npm install and having a node_modules folder. Node.js always looks in node_modules for dependencies (this behavior pre-dates npm itself). The link option will make npm create symlinks in node_modules, pointing to a common pool. That will reduce disc usage, but you cannot do away with node_modules.
You can use PNPM Package manager, It uses a global pool for dependencies.
I have a Linux server that runs some virtual Machines using KVM. Host machine is a Debian derivate with kernel 2.6.32-32(-pve).
In roughly 75% of the times I run npm install, it hangs while installing modules. I ran it in silly debug level and it always hangs at the line
npm info postinstall [module name] (older npm versions)
or
npm verb unlock done using [...] (newer npm versions)
forever.
It happens in every project on the host machine and on the virtualized systems for every node and every npm version I have tested (a lot, including the newest ones (2.7.x atm)).
It also is not deterministic. Sometimes it works, sometimes not and most of the times it hangs on a different module.
On my dev machine (which is in the same network, running OS X 10.10) it works fine.
I am not behind any kind of proxy.
What could possibly be wrong here?
EDIT: For the time being I solved this problem by checking in all my dependencies as gzipped files using https://github.com/JamieMason/shrinkpack
I was having the same issue for several hours, and couldn't work out what the problem was. I tried re-installing everything a few times, manually installing individual modules, switching from x64 to x86 versions of NodeJS, etc, and got nowhere.
In the end, I changed networks, from WiFi over ADSL2+ [~2mbps] to 4G Hotspot [~20mbps] and it worked in 30 seconds. No idea why, since it didn't seem like a network speed issue, but it solved the problem.
Hope this helps someone else, too!
I had the same problem on our Jenkins slaves based on Ubuntu, and it was solved by upgrading npm.
Default npm installed by apt-get has version 1.3.10 now, which is very old (shown by npm --version). There are at least two ways to upgrade it:
Run sudo npm install -g npm command. If you want to install a specific version you can run append it at the end of command: sudo npm install -g npm#2.1.3.
Or you could add NodeSource repositories using these instructions and run usual sudo apt-get update.
I have the same problem on Windows. I deleted my npm-cache folder and reinstalled npm. Everything started working normally after that.
The npm-cache folder can be found at
<your-drive>\Users\<your-name>\AppData\Roaming\npm-cache
Remember to turn on 'View hidden files because the AppData folder by default is hidden
You can reinstall npm by using the command,
npm i npm -g
On Windows make sure you are in a native CMD (not in VSCODE or whatever IDE). Also, try increasing the max memory limit for node:
set NODE_OPTIONS=--max_old_space_size=8096
I did the same, switched my network from WiFi to 3G and it worked.
I'm experiencing the same problem, this is the issue on Github:
https://github.com/npm/npm/issues/7862
There are some useful tips that seemed to help a few people.
For me doing $ npm install -g node-gyp (as suggested here: http://gangmax.me/blog/2013/05/13/resolve-npm-update-node-gyp-hung-problem/) fixed the problem.
I had the same problem and just run
npm cache clean
And it works!
Had this same problem in Windows. The solution was to restart after installing node and npm and then run command prompt as administrator.
In my case, a system restart did the work.
I ran npm install --verbose which suggested some lock on a file in npm-cache. And I think the system released the file (which was under use) after the system restart. I am using windows.
Try cloning the project again and then run npm install to install the packages in your terminal.
I ran into this with npm v6, and it turned out I was just being impatient. I just needed to wait a little longer and it proceeded to the next step.
npm cache clean --force
works for me :)
=> this method working with me When npm block in installation Package for IONIC installation and ReactNative and another package npm.
you can change temporary :
npm config set prefix C:\Users[username]\AppData\Roaming\npm\node_modules2
change the Path in Environment Variables set C:\Users[username]\AppData\Roaming\npm\node_modules2
Run the your command to install your package .
open file explorer copy the link C:\Users[username]\AppData\Roaming\npm\node_modules
ok file yourpackage.CMD created another folder Created "node_modules2" in node_modules and contain your package folder.
copy your package file CMD to parent folder "npm"
copy your package folder to parent folder "node_modules"
now run npm config set prefix C:\Users[username]\AppData\Roaming\npm
change the Path in Environment Variables set C:\Users[username]\AppData\Roaming\npm
now package working correctly with Command line