I may be a little bit behind, but I just upgraded NPM yesterday and now I'm getting all sorts of vulnerabilities with module dependencies like, prototype pollution, memory exposure and regular expression denial of service.
All I'm using the packages for is compiling, renaming, "uglifying" and compressing my assets using a build process like gulp or webpack.
So, I guess I'm wondering how important it is to fix these vulnerabilities or how relevant they are to my process given they are only used locally to build my assets (scss -> css, babel and image compression).
My hunch is that while you always want to error on the side of caution, since they're not being used on a node server or any live process, user input, etc...it may not matter as much as I had originally thought. Am I way off?
Your hunch is right ...but be cautious.
Use NPM's cli command, npm-audit to run a security audit and check each of the listed vulnerabilities in order to conclude the dangers according to your situation.
You can try to fix these vulnerabilities by running:
npm audit fix
...but often times, this will resolve only some of your vulnerability warnings by trying to updating these dependencies in question to the next possible semver version without breaking functionality.
Basically, UNTIL you have these packages with notices and cautions running as just generator scripts, build tasks, and automations, you'll be fine.
But as soon as possible, update them to their latest versions. I would recommend npm-check-updates for checking more recent versions of the installed packages.
Related
I'm using React (v17.0.2), with NodeJs (v16.14.0) and NPM (v8.5.1) and I've been using antd#4.18.8 in my project (it is listed as a dependency in package.json). The antd package has it's own dependencies as described in package-lock.json. One of these dependencies is copy-to-clipboard#3.3.1.
An AWS vulnerability scan marked some issues with our project. Running an "npm audit" in our project resulted in 0 vulnerabilities listed. However after cloning the copy-to-clipboard#3.3.1 project and running an "npm audit" from there the result was:
16 vulnerabilities (1 low, 4 moderate, 6 high, 5 critical)
It turns out that copy-to-clipboard#3.3.1 has dependencies and/or sub dependencies that have vulnerabilities associated to it. Mostly or possibly all to do with outdated versions of packages used. However copy-to-clipboard#3.3.1 is the latest version and the project doesn't appear to be particularly active (latest version was produced well over a year ago).
Solutions I thought to try were to use the "override" flag in package.json or npm-force-resolutions or "npm shrinkwrap". The problem is that the dependencies of copy-to-clipboard do not have their versions specified in package-lock.json. Instead these dependencies are defined in their own "lock" file located in node_modules/copy-to-clipboard/yarn.lock. These are not "lock" files that I have control over (i.e. they will be overwritten). Therefore I can't use any of those solutions.
I've created an issue with the copy-to-clipboard project to update their dependencies although it doesn't appear to be a particularly active project. So I don't know when that will happen if ever.
Swapping out antd for something else isn't really an option either.
What are my other options for trying to address this? Fork copy-to-clipboard to try and update the dependencies myself and hope I know what I'm doing? antd is widely used so I don't know how others aren't seeing this in their AWS (or whatever) vulnerability scans. Or perhaps they have solved this in a way that I haven't thought of.
I have a package.json that gives a load of security warnings. Looking at the first critical item I see its open#0.0.5 which hasn't been updated for five years. Looking at npm ll it is included by npm#6.5.0 where I am using the latest that was updated about two weeks ago.
I would like to remove the insecure dependencies. In the Java world the maven package manager lets you put exclude certain transitive dependencies. Ideally, with npm or another node package manager, I should be able to exclude dependencies with vulnerabilities. Then I can retest that my app works and not see any security errors. Is there a way to quickly exclude anything that has a security vulnerability from my package.json? If there isn't a way to do this what approaches can a take to ensure that no insecure packages are used by my application?
Update: Although "npm": "^6.5.0" is specified in the package.json I was building it with an older npm which was picking up the critical issue mentioned above. I fixed all the issues with ./node_modules/.bin/npm audit fix --force
By definition, you can't exclude a package that a dependency you are using relies on. In other words, if you require package A, and package A claims it is dependent on package B, then removing package B will cause A to either stop working altogether or begin behaving erratically.
Unfortunately this does happen, and your options include:
Ignoring the security warning.
Replacing package A with something else (applies in some cases and not others).
Asking the maintainer of package A to upgrade the version of package B they rely on, possibly opening a pull request yourself.
In your case, though, I'm not sure if your investigation is complete yet - I don't see open in npm's dependency list. Might be worth scrapping your node_modules and re-running npm install, then check again to see who is using open.
This specific warning is targeting at your lockfile, and can be easily fixed by removing the yarn.lock or package-lock.json and reinstall dependencies.
Tarn package manager has feature resulution by which you can set fixed libraries to insecure thirdparties.
See
How do I override nested dependencies with `yarn`?
NPM has something similar.
I generally run the npm install command on my local dev machine (OSX) in order to collect all of my node modules in one place so I can look through them, get completion in my IDE, etc.
However, I'm new to using yarn, and it seems with the lockfile that some versions of packages are locked to be specific to my development environment, so when I yarn down my package.json in an alpine docker container it tells me things like
warning fsevents#1.1.2: The platform "linux" is incompatible with this module.
Which I'm guessing means that the package was locked to a specific version or set of dependencies which is specific to OSX.
This doesn't seem to be breaking any functionality yet, but I'm concerned that it might down the line. Any ideas on how to maintain yarn for cross-platform development? Or is what I'm seeing here totally benign?
Not really an answer but...
Your example is not an issue specific to yarn, yarn.lock or npm and package-lock.json for that matter. The fsevents package is specifically for MacOS file system events. As such, it won't work on any other platform (e.g. Linux).
If you are using this package directly, you may want to consider a replacement that is cross-platform.
If this warning is coming up because of a dependency to your project, you may want to make sure that it (and it's dependencies) are cross-platform as well.
Oftentimes platform-specific packages exist to make up for a feature that is either incompatible or not present on that platform. So long as you (or your dependency) are handling this correctly, in a per-platform way, you should be fine. These packages are listed in package.json as optional because they are expected to fail on platforms they are not designed for (hence the warning).
For more information on this, Yarn has a good explanation of dependency types which includes optional dependencies:
This is useful for dependencies that won’t necessarily work on every
machine and you have a fallback plan in case they are not installed...
https://yarnpkg.com/lang/en/docs/dependency-types/
I spent about 12 hours getting my application working again after days and days of not building. I could only do so with npm shrinkwrap
Are there any significant downsides to this? I'd also be open to using yarn if there is some way that could possibly assist with this issue.
Thanks very much!
Well there aren't really downsides to shrinkwrapping your package. I believe it's the preferred way to ship your package. That way you won't have issues with dependencies accidentally being upgraded and possibly messing up your package.
As to the differences between yarn and npm, it seems that yarn is currently the new hot stuff, but for basic usage npm shrinkwrap using npm >= 3 will do the job.
Side Note
Even though it's not a part of the question I would still suggest you re-check the reason your app doesn't work without shrinkwrap. It smells not so good.
Shrinkwrap it looks like would be a nightmare to deal with long term. I came across a feature of Yarn, just added, that creates a yarn.lock file from your node_modules folder (as opposed to from a package.json)
Using yarn import I was able to create a yarn.lock which can be used to reliably install the application's dependencies.
npm allows us to specify bundledDependencies, but what are the advantages of doing so? I guess if we want to make absolutely sure we get the right version even if the module we reference gets deleted, or perhaps there is a speed benefit with bundling?
Anyone know the advantages of bundledDependencies over normal dependencies?
For the quick reader : this QA is about the package.json bundledDependencies field, not about the package.
What bundledDependencies do
"bundledDependencies" are exactly what their name implies. Dependencies that should be inside your project. So the functionality is basically the same as normal dependencies. They will also be packed when running npm pack.
When to use them
Normal dependencies are usually installed from the npm registry.
Thus bundled dependencies are useful when:
you want to re-use a third party library that doesn't come from the npm registry or that was modified
you want to re-use your own projects as modules
you want to distribute some files with your module
This way, you don't have to create (and maintain) your own npm repository, but get the same benefits that you get from npm packages.
When not to use bundled dependencies
When developing, I don't think that the main point is to prevent accidental updates though. We have better tools for that, namely code repositories (git, mercurial, svn...) or now lock files.
To pin your package versions, you can use:
Option1: Use the newer NPM version 5 that comes with node 8. It uses a package-lock.json file (see the node blog and the node 8 release)
Option2: use yarn instead of npm.
It is a package manager from facebook, faster than npm and it uses a yarn.lock file. It uses the same package.json otherwise.
This is comparable to lockfiles in other package managers like Bundler
or Cargo. It’s similar to npm’s npm-shrinkwrap.json, however it’s not
lossy and it creates reproducible results.
npm actually copied that feature from yarn, amongst other things.
Option3: this was the previously recommended approach, which I do not recommend anymore. The idea was to use npm shrinkwrap most of the time, and sometimes put the whole thing, including the node_module folder, into your code repository. Or possibly use shrinkpack. The best practices at the time were discussed on the node.js blog and on the joyent developer websites.
See also
This is a bit outside the scope of the question, but I'd like to mention the last kind of dependencies (that I know of): peer dependencies. Also see this related SO question and possibly the docs of yarn on bundledDependencies.
One of the biggest problems right now with Node is how fast it is changing. This means that production systems can be very fragile and an npm update can easily break things.
Using bundledDependencies is a way to get round this issue by ensuring, as you correctly surmise, that you will always deliver the correct dependencies no matter what else may be changing.
You can also use this to bundle up your own, private bundles and deliver them with the install.
Other advantage is that you can put your internal dependencies (application components) there and then just require them in your app as if they were independent modules instead of cluttering your lib/ and publishing them to npm.
If/when they are matured to the point they could live as separate modules, you can put them on npm easily, without modifying your code.
I'm surprised I didn't see this here already, but when carefully selected, bundledDependencies can be used to produce a distributable package from npm pack that will run on a system where npm is not configured. This is helpful if you have e.g. a system that's not networked / not on the internet: bring your package over on a thumb drive (or whatever) and unpack the tarball, then npm run or node index.js and it Just Works.
Maybe there's a better way to bundle up your application to run "offline", but if there is I haven't found it.
Operationally, I look at bundledDependencies as a module's private module store, where dependencies is more public, resolved among your module and its dependencies (and sub-dependencies). Your module may rely on an older version of, say, react, but a dependency requires latest-and-greatest. Your package/install will result in your pinned version in node_modules/$yourmodule/node_modules/react, while your dependency will get their version in node_modules/react (or node_modules/$dependency/node_modules/react if they're so inclined).
A caveat: I recently ran into a dependency that did not properly configure its dependency on react, and having react in bundledDependencies caused that dependent module to fail at runtime.