I setup a development environment with Windows 8 and Ubuntu as a virtual machine. For that I use VirtualBox.
I also manage to create a shared folder in VirtualBox.
In this shared folder I try to start a project with ember-generator of Yeoman.
yo ember --skip-install --karma
npm install --no-bin-links
For installing modules NPM I use the option "--no-bin-links" not to create symbolic links. Unfortunately, I still have errors creations symbolic links ... Is what I use although this option ? There he has a bug ?
The NPM docs about parameter "--no-bin-links" say:
will prevent npm from creating symlinks for any binaries the package
might contain.
Which will just cause NPM to not create links in the node_modules/.bin folder. I also searched for a way to prevent NPM from creating symlinks when using npm install ../myPackage, but can't find any solution...
Update: The npm support team said this will reproduce the old behaviour (no symbolic links):
npm install $(npm pack <folder> | tail -1)
Works for me in git-bash on Windows 10.
This Stack Overflow page comes up in Google search results when trying to solve the issue of installing local modules (ie. npm install ../myPackage) and not wanting symbolic links. So I'm adding this answer below to help others who end up here.
Solution #1 - For development environment.
Using the solution proposed by the NPM support team as mentioned in the other answer works...
# Reproduces the old behavior of hard copies and not symlinks
npm install $(npm pack <folder> | tail -1)
This is fine in the development environment for manual installs.
Solution #2 - For build environment.
However, in our case, the development environment doesn't quite matter as much though because when committing our changes to Git, the ./node_modules/ folder is ignored anyway.
The files ./package.json and ./package-lock.json is what is important and is carried into our build environment.
In our build environment (part of our automated CI/CD pipeline), the automation just runs the npm install command and builds from the dependencies listed in the package.json file.
So, here is where the problem affects us. The locally referenced files in the dependencies list of the package.json causes symlinks to appear. Now we are back to the old problem. These symlinks then get carried into the build's output which move onto the Stage and Production environments.
What we did instead is use rsync in archive mode with the --copy-links option that turns symbolic links into copies of the original.
Here is what the command looks like in the automated build:
# Install dependencies based on ./package.json
npm install
# Make a copy that changes symlinks to hard copies
rsync --archive --verbose --copy-links ./node_modules/ ./node_modules_cp/
# Remove and replace
rm -r ./node_modules/
mv ./node_modules_cp/ ./node_modules/
I have a similar environment. Apparently the Virtualbox (vagrant) synchronisation has problems when renaming or moving files, which happens when updating modules.
If you do a file listing (ls -alhp) on the command line and see ??? for the file permissions, then it is time to reboot your virtualbox. This will set the permissions to valid values. Then use the --no-bin-links option when installing a module.
Related
I'm getting errors with npm while trying to install/update packages without SU permissions on Linux.
The easy way to solve the problem is execute sudo npm install <package>, but I'm not sure if it is a good idea.
Best way is to become the owner of .npm folder, as I found into StackOverflow's questions and blog posts.
My question is: why run npm as SU it isn't a good idea?
Running npm as a super user has a risk of running some untrusted code as a super user which can potentially mess with your entire system. Running npm as an unprivileged user has a risk of running that code with less privileges and it won't be able to mess with the entire system - just with your own files (which can be equally bad, depending on how you look at it).
What I often do and recommend is to install Node in your home directory instead of globally on the system if it's your own computer. That way you don't have to run with sudo or su for npm or even for make install of Node itself.
I run a lot of versions of Node that I compile from sources sometimes with different switches and the convention that I use is to install Node in versioned directories, either globally in /opt (but then you need sudo) or locally in my home directory in ~/opt.
I do it like this:
wget https://nodejs.org/dist/v7.1.0/node-v7.1.0.tar.gz
tar xzvf node-v7.1.0.tar.gz
cd node-v7.1.0
./configure --prefix=$HOME/opt/node-v7.1.0
make && make test && make install
Then I create a symlink ~/opt/node pointing to ~/opt/node-v7.1.0 and I have:
PATH="$HOME/opt/node/bin:$PATH"
in my .profile or .bashrc.
That way I don't have to run as super user for installing Node or for running npm.
As a bonus I can quickly switch my default Node version just by changing the symlink, and at any time I can run any other version if I change the PATH or run Node with a full path like ~/opt/node-v7.0.0/bin/node.
I explained that installation process in more detail in my other answers:
node 5.5.0 already installed but node -v fetches with “v4.2.1” on OS X & homebrew?
NodeJS API with external deps in other language
I don't want to go into too much detail here since this answer is about why running npm as a superuser is not a good idea - this installation process is just one solution to not have to run npm as a superuser.
Other options of setting your npm permissions to avoid running as a superuser are described in Fixing npm permissions in npm docs (thanks to RyanZim for pointing it out in the comments).
I'm trying to put together a development environment and npm is causing me problems. Here is my scenario:
I have a development machine running Windows and VMWare Player. I have a Ubuntu Server VM (no UI) which is configured with Apache, PHP, NodeJS etc. As the VM has no UI I want to use the host OS for development. I set up a shared directory which in the VM is accessed as /mnt/hgfs/source/<project name>.
The problem comes when I attempt to run npm install within this directory. I see a lot of errors like Error: UNKNOWN, symlink '../requirejs/bin/r.js'. I know that my package.json file is OK because if I copy all files out of the share and into a regular unix directory (/var/www/<project name>) npm install works fine. So npm has a problem installing modules in the shared directory.
I thought I could get around this by installing the node packages globally but, for whatever reason, the GruntJS enthusiasts don't like that and it must be present locally. I then tried to create an npm link from global to local but that just results in a new error: Error: May not delete: /usr/lib/node_modules/grunt. I have full permissions on the /usr/lib/node_modules directory and all sub-directories.
I really don't want to write the entire project using a command-line text editor in the VM but it looks like I cannot have my code-base in a directory available to both the host and guest OS through VMWare.
I would very much appreciate any suggestions on how to either 1) allow npm modules to be installed in my shared directory, 2) run Grunt globally, or 3) solve the npm link error I'm seeing.
EDIT: Shortly after posting this I realised the fundamental issue here - it's not possible to create symbolic links within a VM shared directory when the host OS is Windows. As npm install uses symlinks by default it didn't work, and this is why the accepted solution does work.
Try the following:
npm install --no-bin-links
Grunt should be local since the plugins and gruntfile.js may require a certain version of Grunt in order to run your tasks. If another developer would like to run your tasks, they could just issue an npm install and they are set. (See this for more info.) grunt-cli is global which is used to run the local version of grunt
https://github.com/tejas-manohar/itnerary-civic-hacking -- I cloned this git repo down locally. Installed grunt-cli globally, and ran npm run nss (script written by author of original server template -- look in package.json, not complex) and faced numerous errors at the traceur install + git cloning. I've included the trail from terminal window in the pastebin linked below. The npm debug/error log mentioned is not present at the suggested location. npm install alone does not help the situation and returns nothing back. NodeJS v0.10.29 is installed via Node Version Manager (NVM) on OS X 10.9.3.
http://pastebin.com/UJFL3k2E
I'm not the most adept with some of the technologies discussed here. Would someone please attempt to walk me through (a) solution(s) and/or steps to get better clues? All assistance is appreciated.
There's a couple of problems going on which are causing the problems you have seen. The first of which is the command that is run when you run npm run nss, which fails for you with this:
rm: ../../app/static/js/vendor/traceur.js: No such file or directory
This is because, well, the file simply does not exist (yet). If you look at the commands which are run when you run npm run nss, you'll see that the remove file is followed by the copy file of traceur.js to that very location. So I would guess that whoever wrote the script intended for this to be run after it was initially setup, and the remove/copy would be done once the file was initially copied there. Therefore, you must first copy the traceur.js file to app/static/js/vendor before you can successfully execute npm run nss.
But then the next problem, the file should be copied from tools/traceur-compiler/bin doesn't exist either. This is because the traceur-compiler project has recently deleted this (compiled) file from their project, which you can see via this commit: https://github.com/google/traceur-compiler/commit/429c3d850dcb7636320e81fd782c61a06de0fbf1
So you need to regenerate this file, which you can do by (from the itnerary-civic-hacking root directory):
cd tools/traceur-compiler
make bin/traceur.js
cp bin/traceur.js ../../app/static/js/vendor
This will get you in the same state as if you successfully ran npm run nss.
You won't be able to re-run the npm run nss command, but you really shouldn't need to since its more of a setup command than anything (it creates a directory, clones a github repo, etc). If you wanted to update the traceur-compiler in the future (though you may never have to do this) then I would recommend running the following steps (from the itnerary-civic-hacking root directory):
cd tools/traceur-compiler
git pull origin master
npm install
make bin/traceur.js
cp bin/traceur.js ../../app/static/js/vendor
In the process of setting up MEAN Stack,
After installing meanio module on node 0.10.26, I can not pass through the command "mean init myApp".
1. npm install -g "meanio'
2. mean init myApp
This always gives me "Prerequisite not installed: undefined".
Could you pls look at it ?
Regards
Ram
Sometimes it is neccessary to clear npm cache to make sure you really get the latest version you can use the npm cache clear command. There have been a few releases over the last few days so clearing npm cache might help you
Here is a summary of the install procedure.
sudo npm install -g meanio#latest
mean init <your app name>
cd <your app name> && npm install
grunt
Checkout http://www.mean.io/#!/docs for full documentation and make sure you meet all the prerequites.
If you are a windows user and already installed Git on your machine, first add git to path in environment variable, then try it. It should work.
I too encountered the same issue.Though I have installed GIT on my mcahine, i was getting the error
"Prerequisite not installed: GIT" while running the command "mean init myApp".
This is because your command prompt doesnt know the path of GIT exe file.
I resolved this by adding the GIT path under Environment variables as shown here http://blog.countableset.ch/2012/06/07/adding-git-to-windows-7-path/
Right-Click on My Computer
Click Advanced System Settings link from the left side column
Click Environment Variables in the bottom of the window
Then under System Variables look for the path variable and click edit
Add the path to git's bin and cmd at the end of the string like this:
;C:\Program Files (x86)\Git\bin;C:\Program Files (x86)\Git\cmd
Please make sure that your give the correct path of GIT folder in the path under Environment variables.in your case it might not be same as mine.check where GIT files were created while installing.
Make sure you have a recent version of Git installed. That error should read, "Prerequisite not installed: git". I'll take a look at why it says undefined instead.
As user3211907 meantioned, please take a look at http://www.mean.io/#!/docs for full documentation.
Ram,
It is problem with git
install git and this problem should solve
To install git in Cent OS or any RedHat flavors use
yum install git
I am trying to speed up the npm install during the build process phase. My package.json has the list of packages pretty much with locked revisions in it. I've also configured the cache directory using the command
npm config set cache /var/tmp/npm-cache --global
However, on trying to install using npm install -g --cache, I find that this step isn't reducing the time to install by just loading the packages from cache as I would expect. In fact, I doubt if it's even using the local cache to look up packages first.
Proposing two more modern approches:
1) npm ci
Use npm ci, which is available from npm version 5.7.0 (although I recommend 5.7.1 and upwards because of the broken release) - this requires package-lock.json to be present and it skips building your dependency tree off of your package.json file, respecting the already resolved dependency URLs in your lock file.
A very quick
boost for your CI/CD envs (our build time was cut down to a quarter of the original!) and/or to make sure all your developers sit on the same versions of dependencies during development (without having to hard-code strict versions in your package.json file).
Note however that npm ci removes the node_modules/ directory before installing, so it won't benefit from any caching strategies.
2) npm i --prefer-offline
Use the --prefer-offline flag with your regular npm install / npm i. With this approach, you need to make sure you've cached your node_modules/ directory between builds (in a CI/CD environment). If it fails to find packages locally with the specific version, it falls back to the network safely.
You can also add --no-audit --progress=false to reduce pre-install checks and remove the progress bar (latter is only a very slight improvement)
For pure npm solution, you may try
npm install --prefer-offline --no-audit --progress=false
Prefer offline may not be useful for the first run.
As suggested by #Daniel Serodio
You could also include your node_modules folder inside your repository but you should probably zip it first than add to repo, and while installing you can unzip it and just
npm rebuild
(which works cross platform) it is quite fast.
This would also give you the benefit of full control over all your dependencies.
Also you can set the process flag to false to increase your speed by 2x.
npm set progress=false
Read source for more info
Update:
You can also use pnpm for this
npm i -g pnpm
This basically use local cached modules (i have heard its better then YARN)
It's better to install pnpm package using the following command:
npm i -g pnpm
pnpm uses hard links and symlinks to save one version of a module only ever once on a disk. When using npm or Yarn for example, if you have 100 projects using the same version of lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be saved in a single place on the disk and a hard link will put it into the node_modules where it should be installed.
As an example I can mention that whenever you want to install the dependencies of package.json file, what you should do is simply that enter the pnpm i and it handles the other things by itself.
UPDATE: The original answer is from 2014. I wouldnt recommend checking in node_modules, as there are definitly better options around speeding up the install especially for a ci pipeline, eg. npm ci --only=production
You could also include your node_modules folder inside your repository (you are probably using git), and just npm rebuild (which works cross platform) on build/deploy processes, and is pretty fast.
This would also give you the benefit of full control over all your dependencies (I know that's what shrinkwrap usually should be used for)
Edit:
Also you can set the progress flag to false to increase your speed by at least 20%. This works only with npm#v3.x.x, and there will be hopefully fixes for that soon (see second link)
npm set progress=false
Tweet about finding
Github Issue Cause identification
As very modern solution you can start to use Docker.
Docker allows you virtualize and pre-define as image the current state of your code, including installed npm-modules and other goodies.
Once the docker image for your infrastructure/env is built locally, or retrieved from remote repository, it will be stored on the host machine, and you can spin server in seconds.
Another benefit of it is that you use same virtualized code infrastructure on any machine where you deploy your code.
Docker speeds up install/deployment processes and is widely used technology.
To start using docker is enough to (all the snippets are just mock/example for pre-setup and are not by any means most robust/elegant solution) :
1) Install docker and docker-compose using manuals and get some basic understanding of it at docker.com
2) Write Dockerfile file in root of your application
FROM node:6.9.5
RUN mkdir /usr/local/app
WORKDIR /usr/local/app
COPY package.json package.json
RUN npm install
3) create docker-compose.yml in the root of your project with such content:
version: "2"
server:
hostname: server
container_name: server
image: server
build: .
command: sh -c 'NODE_ENV=development PORT=8080 node app.js'
ports:
- "8080:8080"
volumes: #list of folders and files to use
- ${PWD}/server:/usr/local/server
- ${PWD}/app.js:/usr/local/app.js
4) To start server you will need to docker-compose up -d. To see the logs docker-compose logs -f server. If you will restart your server it will do it in seconds once it built the image already at once.
Then it will cache build layers locally so next run will take only few seconds.
I know this might be bit of a robust solution, but I am sure it is have most potential/flexibility and is widely used in industry. And while it requires some learning for anyone who did not use Docker before, in my humble oppinion, it is the best one for your problem.
Nothing helped me more than disabling antivirus (Windows Defender in my case) I got from 2:30 to 1 minute.
With npm-cache package I got to ~30 secs.
I tried to use yarn, which is very fast, but was randomly failing in my case.
We have been trying to solve this problem to speed up our deployments.
We have settled on using pac, which follows the principles in the other answers. It zips the npm modules and includues them in your repo so you don't have a million files in your commits and code reviews and you can just unzip/rebuild for the target machine.
https://www.npmjs.com/package/pac