Basically what's to prevent me from publishing an NPM module with arbitrary installation script that steals everything from your computer when you npm install my-malicious-package if the installation is not running in sandbox?
In this article they suggest that most of the attackers would place their malicious script in the pre/post install hooks. That's easy to detect and filter out. I'm mostly concerned with the actual installation of the package where arbitrary could be ran.
There's nothing preventing those scripts from performing any actions as current user. You need to avoid running install scripts.
Most malicious packages are using those, but in rare cases (like lofygang recently) the packages may carry malicious code in the functionality too.
How to protect a project from malicious packages
make sure you don't run lifecycle (postinstall) scripts unless they're known and necessary (see my talk on this topic)
put 3rdparty code in a compartment, lock down the environment, decide on which powerful APIs to pass to each package.
The second step requires the use of Compartment, which is a work-in-progress in TC39 https://github.com/tc39/proposal-compartments/
But a shim exists. And Some tooling was built on top of that shim.
You could use the SES-shim directly and implement your own controls, or use the convenience of LavaMoat
LavaMoat lets you generate and tweak a per-package policy where you can decide which globals and builtins it should have access to.
LavaMoat also offers a tool to manage install scripts.
Here's my talk on SES and LavaMoat with a demo at the end.
How to set up LavaMoat
See LavaMoat docs for more details
disable/allow dependency lifecycle scripts (eg. "postinstall") via #lavamoat/allow-scripts
npm i --ignore-scripts -D #lavamoat/allow-scripts
npx --no-install allow-scripts setup
npx --no-install allow-scripts auto
then, edit the allow-list in package.json
after every insstall/reinstall run allow-scripts
run your server or build process in lavamoat-node
npm i -D lavamoat
in your package.json add something like:
"scripts": {
"lavamoat-policy": "lavamoat app.js --autopolicy",
"start": "lavamoat app.js"
run lavamoat-policy every time you make changes to your dependency tree and review the policy (see also: policy override)
run npm start to start your app
Disclaimer: I contribute to LavaMoat and Endo. They are Open Source projects on permissive licenses.
The only way that npm itself runs package code is in install hooks.
If you disable install hooks, no untrusted code can run until you actually load it in your application (at which point you're hosed).
I've made node-safe, which allows you to use the native macOS sandbox when using node, npm and yarn:
# Allow reading files, but only in the current folder
node --enable-sandbox --allow-read="./**" myscript.js
When using the sandboxed package managers rogue dependencies are not able to compromise your system anymore through postinstall scripts and other means.
Related
I just watched a talk where the speaker recommended running:
npm config set ignore-scripts true
so that post-install scripts and pre-install scripts of a package don't run. That way, you would avoid a virus in a malicious package.
My question is: After running this command, must I do anything differently to npm install packages and get them to work within a project?
If running this command comes with no additional inconvenience when using npm, then running it would have no downside. It would only help you avoid viruses.
If this was the case, why wouldn't this be the default setting?
I ask because I assume that by ignoring package scripts, npm packages would behave differently and one would have to do more things manually.
I agree with #RobC here. It also disactivated running custom scripts in my package.json completely for me, which obviously is a deal breaker since you can't define and run your custom scripts anymore.
Although it's probably useful to think about these security concerns, I don't think running npm config set ignore-scripts true is the right option. I ran it as well and ended up turning it back off to keep running my custom package scripts.
So the advice from the video ended up being not all too sound, I guess...
If you want to be safe, use '--ignore-scripts' or the config setting, but also use can-i-ignore-scripts.
It helps you find out which scripts exist (especially when you install new dependencies), but prevents automatically executing new scripts which appear with a new version of a library you already use.
I faced a similar problem when some dependencies need running scripts to build platform-specific code with node-gyp.
Would be nice to have an option in ignore scripts per project to enable specific ones to build.
So far decided to stay on ignore-scripts = true globally in .npmrc and using an extra script in my project that basically does this:
#!/bin/bash
set -e
npm explore sqlite3 -- yarn run install
npm explore bcrypt -- yarn run install
p.s. yarn does not have explore
I am currently working on a project at a large company, and according to the project I am working on, every time I want to quickstart the app, I would need to first run the command npm install and then run all the additional compiling instructions, but the problem is that running npm install can take a long time, and that is why I am wondering if it is necessary to run this command every time I make a change to the code, and then want to compile and run it.
What exactly does npm install do? If you could explain to me in terms of how we compile and run java code i.e. javac bob.java && java bob and try to make analogies on that basis, that would greatly help me understand the concept. The way I am currently thinking about it right now is that npm install kind of runs like how javac runs, but I am not sure if that is correct. Thanks in advance!
NPM Install
npm install simply reads your package.json file, fetches the packages listed there from (usually) https://www.npmjs.com/, and sometimes engages in the build steps for those packages.
So you only have to run npm install when you change your package.json file, and need to fetch new dependencies.
Keep in mind that npm install --save <packagename> (or npm install -S <packagename>) will update your package.json and run npm install all in one line!
You can view the results of your npm install inside ./node_modules/.
To compare to java
This might be a helpful resource if you're trying to get stuff done: Getting Started with Node.js for the Java Developer
Javascript is not a compiled language, unlike java. When you invoke javac, the java compiler reads in all your .java files, compiles them to java bytecode, and then writes them to .class files, which can then be bundled together into a .jar for execution.
Javascript doesn't do any of this! When you invoke node foo.js, the node executable wakes up, reads foo.js, and gets to work invoking it line by line**. Node does other cool things, including maintaining an event loop (which allows it to operate "asynchronously", and allows it to be very efficient as a webserver-- it doesn't sit around waiting for requests to complete, it carries forward with the next event in the queue.
Node also performs JIT and optimization, these details allow it to improve the performance of sections code it notices are running "hot".
Note also that node.js uses the V8 javascript engine (also used in Google Chrome). Really everything I've said above is handled by V8.
(** Technically there is a syntax checker which is run first, before execution. But this is not a compile step!)
It is not necessary to do "npm install" each time you want to compile. You just need to do it when you change the dependencies of your project.
NPM basically is the package manager for node. It helps with installing various packages and resolving their various dependencies. It greatly helps with your Node development. NPM helps you install the various modules you need for your web development and not just given you a whole bunch of features you might never need.
When you start an app, it comes with a package.json file. That package contains the list of node_modules you are gonna need. Whenever you enter npm install, what you are doing is to download that list of node_modules. So yeah, you have to download the modules all over again.
#NOTE: In your project, you have a file called package.json. this file is responsible for holding track of your project's dependencies. That's why you have to install it every time#.
I have a custom npm module that I am working on, and it has a GitHub repo. I'm also working on a project that uses the custom module. When working on the larger project, it is nice to use npm link so I can make changes to the module and see them right away in the main project.
To deploy to staging or production, I use shrinkwrap and shrinkpack so I can do an npm install after every deploy (some of the dependencies need binaries, and dev systems aren't the same as production systems, so they do need to be installed and not just kept in source control). Edit: I'm crossing this out as the answer below technically solves my issue, even though it doesn't solve for this particular point, but that wasn't as important as the rest of it.
Of course, since the module is linked to my main project and not listed in package.json, a deploy and install misses it entirely. I can go ahead and list it in package.json and have it point to the appropriate GitHub repo, but then every time I need to test a change in the main project I would have to commit and push those changes, then update the main project, kill and restart the app...that would get tiresome pretty quickly.
I guess I need something like the opposite of "devDependencies"; something where I can have it not install the module on dev, but do install it from GitHub when doing npm install on staging or production. Other than remembering to manually change package.json every time I need to go back and forth, is there a better way to do this?
you can specify a github repository as your package to install, in your package.json file:
{
dependencies: {
"my-library": "githubusername/my-library"
}
}
this will work in your production environment.
in your development environment, use "npm link".
from within the "my-library" folder, run npm link directly. that will tell npm on your local box that "my-library" is avaialable as a link.
now, in your project that uses "my-library", run npm link my-library. this will create a symlink to your local development version of "my-library", allowing you to change code in that repository and have it work in your other project that needs it.
once you are ready to push to production, push "my-library" to your github repository, and then you can npm install on your servers, like normal.
I'm used to deploy code depending on Composer (PHP's NPM cousing), that sports .json and .lock files. The first one describes the package and your version constraints, and the second one lists exactly what was installed. Always there's a lock file and you run composer install you're sure to receive the same set of packages; running composer update will re-read the json file, install new versions, and update the lock file.
That's awesome for production deployment, since you don't need to checkout your dependencies to your versioning system and you're sure to have the exact same set of dependencies in production as you have in development.
My question is: how to best automate deployment of NPM-dependent code? Is it possible to achieve a method similar to Composer? I've noticed that npm install only installs what's first available in the package.json file. After the first run, i.e. if you change a version constraint you must manually npm update that package - and that would render automate deployment useless, as there's no way to check in to versioning "update this package here to a new version"...
npm shrinkwrap is a analog of composer.lock file. It will generate a npm-shrinkwrap.json, that have all deps with version in it, so you can use it to deploy to production env. Also you can try a various libs from npm to lock versions or search for updates of it without changing packages.json.
It happens occasionally that the development version of a module works in my development workspace and passes on Travis-CI but after publishing to npm it turns-out the end-user package is broken.
For example if you use a sub module that should be in dependencies but had it in devDependencies then CI will pass (but there are plenty other possible breakages).
How do you automate testing this? Do you use external rigging? Is there a secret module? Do you have a user acceptance test suite?
I use Github with Travis-CI but the standard setup uses the development install.
Once upon a time I discovered that npm would let me publish packages that are uninstallable. So I've added a target to my Gruntfile that does this:
Issue npm pack to create a package from my source.
Into a directory created (automatically by my Gruntfile) just for testing install the new package using npm install <path to the package created in the previous step>.
I have a target for publishing a new version that will publish only if the steps above are successful.
The steps above would not catch the dependency problem you mentioned in the question but they could easily be extended to catch it. To do this, I'd add one or more tests that cause the package installed in step 2 above to call require with all that it depends on.
I would suggest to set up your own CI server that does essentially one thing, npm install package ; cd node_modules/package ; npm test. This would ensure that your package is installable at least on your server.
I heard that Jenkins is good for this (at least, that's what node.js core team seems to be using), but don't have any first hand experience yet. We're just planning to set in up in a couple of weeks.
Also, having some external module that depends on you and testing it helps a bit. :)