I just watched a talk where the speaker recommended running:
npm config set ignore-scripts true
so that post-install scripts and pre-install scripts of a package don't run. That way, you would avoid a virus in a malicious package.
My question is: After running this command, must I do anything differently to npm install packages and get them to work within a project?
If running this command comes with no additional inconvenience when using npm, then running it would have no downside. It would only help you avoid viruses.
If this was the case, why wouldn't this be the default setting?
I ask because I assume that by ignoring package scripts, npm packages would behave differently and one would have to do more things manually.
I agree with #RobC here. It also disactivated running custom scripts in my package.json completely for me, which obviously is a deal breaker since you can't define and run your custom scripts anymore.
Although it's probably useful to think about these security concerns, I don't think running npm config set ignore-scripts true is the right option. I ran it as well and ended up turning it back off to keep running my custom package scripts.
So the advice from the video ended up being not all too sound, I guess...
If you want to be safe, use '--ignore-scripts' or the config setting, but also use can-i-ignore-scripts.
It helps you find out which scripts exist (especially when you install new dependencies), but prevents automatically executing new scripts which appear with a new version of a library you already use.
I faced a similar problem when some dependencies need running scripts to build platform-specific code with node-gyp.
Would be nice to have an option in ignore scripts per project to enable specific ones to build.
So far decided to stay on ignore-scripts = true globally in .npmrc and using an extra script in my project that basically does this:
#!/bin/bash
set -e
npm explore sqlite3 -- yarn run install
npm explore bcrypt -- yarn run install
p.s. yarn does not have explore
Related
Basically what's to prevent me from publishing an NPM module with arbitrary installation script that steals everything from your computer when you npm install my-malicious-package if the installation is not running in sandbox?
In this article they suggest that most of the attackers would place their malicious script in the pre/post install hooks. That's easy to detect and filter out. I'm mostly concerned with the actual installation of the package where arbitrary could be ran.
There's nothing preventing those scripts from performing any actions as current user. You need to avoid running install scripts.
Most malicious packages are using those, but in rare cases (like lofygang recently) the packages may carry malicious code in the functionality too.
How to protect a project from malicious packages
make sure you don't run lifecycle (postinstall) scripts unless they're known and necessary (see my talk on this topic)
put 3rdparty code in a compartment, lock down the environment, decide on which powerful APIs to pass to each package.
The second step requires the use of Compartment, which is a work-in-progress in TC39 https://github.com/tc39/proposal-compartments/
But a shim exists. And Some tooling was built on top of that shim.
You could use the SES-shim directly and implement your own controls, or use the convenience of LavaMoat
LavaMoat lets you generate and tweak a per-package policy where you can decide which globals and builtins it should have access to.
LavaMoat also offers a tool to manage install scripts.
Here's my talk on SES and LavaMoat with a demo at the end.
How to set up LavaMoat
See LavaMoat docs for more details
disable/allow dependency lifecycle scripts (eg. "postinstall") via #lavamoat/allow-scripts
npm i --ignore-scripts -D #lavamoat/allow-scripts
npx --no-install allow-scripts setup
npx --no-install allow-scripts auto
then, edit the allow-list in package.json
after every insstall/reinstall run allow-scripts
run your server or build process in lavamoat-node
npm i -D lavamoat
in your package.json add something like:
"scripts": {
"lavamoat-policy": "lavamoat app.js --autopolicy",
"start": "lavamoat app.js"
run lavamoat-policy every time you make changes to your dependency tree and review the policy (see also: policy override)
run npm start to start your app
Disclaimer: I contribute to LavaMoat and Endo. They are Open Source projects on permissive licenses.
The only way that npm itself runs package code is in install hooks.
If you disable install hooks, no untrusted code can run until you actually load it in your application (at which point you're hosed).
I've made node-safe, which allows you to use the native macOS sandbox when using node, npm and yarn:
# Allow reading files, but only in the current folder
node --enable-sandbox --allow-read="./**" myscript.js
When using the sandboxed package managers rogue dependencies are not able to compromise your system anymore through postinstall scripts and other means.
I am currently working on a project at a large company, and according to the project I am working on, every time I want to quickstart the app, I would need to first run the command npm install and then run all the additional compiling instructions, but the problem is that running npm install can take a long time, and that is why I am wondering if it is necessary to run this command every time I make a change to the code, and then want to compile and run it.
What exactly does npm install do? If you could explain to me in terms of how we compile and run java code i.e. javac bob.java && java bob and try to make analogies on that basis, that would greatly help me understand the concept. The way I am currently thinking about it right now is that npm install kind of runs like how javac runs, but I am not sure if that is correct. Thanks in advance!
NPM Install
npm install simply reads your package.json file, fetches the packages listed there from (usually) https://www.npmjs.com/, and sometimes engages in the build steps for those packages.
So you only have to run npm install when you change your package.json file, and need to fetch new dependencies.
Keep in mind that npm install --save <packagename> (or npm install -S <packagename>) will update your package.json and run npm install all in one line!
You can view the results of your npm install inside ./node_modules/.
To compare to java
This might be a helpful resource if you're trying to get stuff done: Getting Started with Node.js for the Java Developer
Javascript is not a compiled language, unlike java. When you invoke javac, the java compiler reads in all your .java files, compiles them to java bytecode, and then writes them to .class files, which can then be bundled together into a .jar for execution.
Javascript doesn't do any of this! When you invoke node foo.js, the node executable wakes up, reads foo.js, and gets to work invoking it line by line**. Node does other cool things, including maintaining an event loop (which allows it to operate "asynchronously", and allows it to be very efficient as a webserver-- it doesn't sit around waiting for requests to complete, it carries forward with the next event in the queue.
Node also performs JIT and optimization, these details allow it to improve the performance of sections code it notices are running "hot".
Note also that node.js uses the V8 javascript engine (also used in Google Chrome). Really everything I've said above is handled by V8.
(** Technically there is a syntax checker which is run first, before execution. But this is not a compile step!)
It is not necessary to do "npm install" each time you want to compile. You just need to do it when you change the dependencies of your project.
NPM basically is the package manager for node. It helps with installing various packages and resolving their various dependencies. It greatly helps with your Node development. NPM helps you install the various modules you need for your web development and not just given you a whole bunch of features you might never need.
When you start an app, it comes with a package.json file. That package contains the list of node_modules you are gonna need. Whenever you enter npm install, what you are doing is to download that list of node_modules. So yeah, you have to download the modules all over again.
#NOTE: In your project, you have a file called package.json. this file is responsible for holding track of your project's dependencies. That's why you have to install it every time#.
I have gotten better at using the command line. I now use it regularly for foundation, gulp, ionic, etc. One thing I always struggle with is should I be installing modules globally?
I just got a new computer and I am starting fresh. When I do my first Foundation Project, or set up Gulp, when it prompts me to run 'npm install' should I be installing that gloabally? And if I do, will I need to run npm install the next time I set up a Foundation Project?
I think that NPMs documentation really sums it up quite nicely:
If you want to use it as a command line tool, something like the grunt CLI, then you want to install it globally. On the other hand, if you want to depend on the package from your own module using something like Node's require, then you want to install locally.
https://docs.npmjs.com/getting-started/installing-npm-packages-globally
In this case, you should install locally since they are dependencies of the application and not general command line utilities.
This doesn’t fit the Q&A format too well, but I would in general advise against that. Installing modules globally may lead to side effects in other projects that depend on the same packages.
I don’t know what a “Foundation Project” is (and Google does not yield a clear answer), so I think you’d have to add a link explaining what that is.
I have a project, which consists of one root node package containing subpackages linked together by npm link - these subpackages depend on each other (listed in package.json dependencies) and the structure basically looks like this:
-rootpackage
--subpackageA
--subpackageB
Lets say subpackageA has dependency on subpackageB, so I link them to avoid publishing/reinstalling subpackageB in subpackageA after every change in the source of subpackageB.
The link works just fine until I run npm update in subpackageA, which causes the subpackageB to be unlinked.
Now, I see two options:
I can theoretically run the npm link operation after each npm install or npm update to ensure the links are always present. This works with postinstall in case of installation, but in case of an update the postinstall is not called. I don't know any postupdate command for npm, which is to be called after update.
Maybe there is a way to do this more cleverly, perhaps with yarn, which I am also using, in a way, that it kind of prevents unlinking or excludes the update for my subpackages, so I don't lose the links between my subpackages, but right now I am not aware of such a way.
Is there any way to make one of those options work or any other way to solve this problem ? I need to keep this and other links so we don't have to run npm link after every installation/update. I can't really find information about this issue anywhere. Btw I am using Node 6.4.0 and NPM 3.10.3.
So the solution is to use Yarn Workspaces or maybe project like Lerna.
Yarn Workspaces is a utility that expects a structure similar to what was described in the question and which maintains the linking subpackages and root automatically. It is very easy to set up (just 2 lines in root package.json and executing yarn for the first time) and after it you don't have to worry about upgrade or install at all, the links stay in place unless you delete them manually.
Lerna expands on that and provides you with additional tooling for managing multipackage projects. It can use Yarn Workspaces internally for the linking if you use yarn but it is not a requirement and works fine with npm. Just make sure to have Git because last time I checked Lerna didn't work with SVN or other VCSs.
And if so, I'm interested in knowing if there is any way to run them from /usr/local/lib folder instead of installing those modules in each project's folder I'm working on.
The node documentation says that installing global modules should be relegated to those that require command line access ( such as nodemon etc.. ).
https://nodejs.org/en/blog/npm/npm-1-0-global-vs-local-installation/
Which to choose
Just like how global variables are kind of gross, but also necessary
in some cases, global packages are important, but best avoided if not
needed.
In general, the rule of thumb is:
If you’re installing something that you want to use in your program,
using require('whatever'), then install it locally, at the root of
your project.
If you’re installing something that you want to use in
your shell, on the command line or something, install it globally, so
that its binaries end up in your PATH environment variable.
The only other related issue I can think of to your question is that you can install express generator globally as it allows you to create an instance of their web scaffolding.
npm install express-generator -g
But that is different and not the same thing as installing express itself globally.
So as far as I can tell the answer is no - their is no benefit to this.
Well, personally, no. I ended up messing up my repos trying to globally install express and other modules. I really can't see the benefit of sparing a few extra seconds when you know it can't mess up your system :D And just to add, don't try globally installing mongoDB, REALLY ended up badly.