I am running the snyk test command on my project to identify vulnerabilities with third party libraries and came across the following vulnerabilities in react-scripts#5.0.1 ->#svgr/webpack": "^5.5.0"
Regular Expression Denial of Service (ReDoS) - nth-check#1.0.2
Regular Expression Denial of Service (ReDoS) - loader-utils#2.0.3
To mitigate this, I added a dependency override( or dependency resolution in my case as the project is using yarn) in my package.json as shown below to replace the nested dependencies with a non vulnerable version :
"resolutions": {
"loader-utils": "^2.0.3",
"nth-check": "2.0.1",
"#svgr/webpack": "^6.2.1"
},
and did yarn install. I confirmed in the project bundle that with these change , the latest version of loader-utils and nth-check was being installed. I also checked with Snyk Extension on VS Code and it seemed to resolve the vulnerability issues.
But when I run the snyk test in github actions pipeline as shown below :
- name: Run Snyk test scan
uses: snyk/actions/node#master
with:
command: test
args: --severity-threshold=high --fail-on=all
it still reports the vulnerability on nth-check and loader-utils. My assumption is that, this is because Snyk github actions doest really install your dependencies before running a code analysis. Instead it examines the package.json file and unwrap the dependencies layer by layer causing it to think I still have the vulnerable dependencies as nested dependencies within react-scripts package, while in reality that gets overriden by the resolutions section in package.json.
Is there a way to circumvent this or force snyk to consider the nested dependency overrides ?
After some research I found that there is no such issue within Snyk itself. The problem was in our snyk configuration. The scan configured to run on the package.json in our main branch while the fix was on a separate branch. Once we updated the snyk scan to run on the correct package.json, it did identify the issue as resolved.
Related
We're using a private npm package that we'll call design-lib. design-lib had a number of vulnerabilities that were discovered running yarn audit. These were corrected using the "resolutions" field in package.json.
We update design-lib in our main app using yarn install and run yarn audit and still see many vulnerabilities listed coming from design-lib. I'm curious why this is happening since when we run yarn audit directly inside of design-lib there are no vulnerabilities.
Do I need to also put the resolutions inside of the main app as well? I figured that since theyre inside of the dependency itself it would be a non-issue..
Yes, you need to put the resolutions inside the main app as well.
The main app is using the version of design-lib specified in its dependencies, and it's not aware of any changes made to design-lib directly.
By including the resolutions field in the main app's package.json, you are telling it which versions of the dependencies to use specifically, even if it goes against what is specified in the design-lib's own dependencies. This ensures that the vulnerabilities are not present in the main app.
see example
{
"dependencies": {
"design-lib": "1.0.0"
},
"resolutions": {
"design-lib/package-a": "2.0.0",
"design-lib/package-b": "3.0.0"
}
}
In above example the main app is using version 1.0.0 of design-lib, but it's specifying the versions of package-a and package-b that should be used, even if these versions are not specified in the design-lib's own dependencies. This helps to ensure that the vulnerabilities are corrected in the main app.
I'm using React (v17.0.2), with NodeJs (v16.14.0) and NPM (v8.5.1) and I've been using antd#4.18.8 in my project (it is listed as a dependency in package.json). The antd package has it's own dependencies as described in package-lock.json. One of these dependencies is copy-to-clipboard#3.3.1.
An AWS vulnerability scan marked some issues with our project. Running an "npm audit" in our project resulted in 0 vulnerabilities listed. However after cloning the copy-to-clipboard#3.3.1 project and running an "npm audit" from there the result was:
16 vulnerabilities (1 low, 4 moderate, 6 high, 5 critical)
It turns out that copy-to-clipboard#3.3.1 has dependencies and/or sub dependencies that have vulnerabilities associated to it. Mostly or possibly all to do with outdated versions of packages used. However copy-to-clipboard#3.3.1 is the latest version and the project doesn't appear to be particularly active (latest version was produced well over a year ago).
Solutions I thought to try were to use the "override" flag in package.json or npm-force-resolutions or "npm shrinkwrap". The problem is that the dependencies of copy-to-clipboard do not have their versions specified in package-lock.json. Instead these dependencies are defined in their own "lock" file located in node_modules/copy-to-clipboard/yarn.lock. These are not "lock" files that I have control over (i.e. they will be overwritten). Therefore I can't use any of those solutions.
I've created an issue with the copy-to-clipboard project to update their dependencies although it doesn't appear to be a particularly active project. So I don't know when that will happen if ever.
Swapping out antd for something else isn't really an option either.
What are my other options for trying to address this? Fork copy-to-clipboard to try and update the dependencies myself and hope I know what I'm doing? antd is widely used so I don't know how others aren't seeing this in their AWS (or whatever) vulnerability scans. Or perhaps they have solved this in a way that I haven't thought of.
I am a beginner (~15 days into learning web-development) and I am currently learning React among other things and I am sorry if this sounds too trivial.
I am trying to understand the difference between devDependencies and dependencies and the correct usage of the same.
I have tried to figure it out from the docs and stackoverflow questions but I am not a 100% sure if I have this right. So kindly review my understanding as of now and let me know if I have this right so far.
Definition
dependencies: only the packages which are finally going to be used by the production build. These will be there in the final package.json file.
devDepndencies: the packages which eases my development efforts and finally will not be used by the product / application. These will not be included in the package.json folder of the final build.
Importance of correct usage
Fairly important as correctly excluding devDependencies from dependencies can make the app lighter. At the same time, incorrectly excluding required dependencies will cause my app to break.
Practical example
In the package.json file created during my tutorial, I had the following packages and I am mentioning the type of dependency that the package should have according to my current understanding. Please let me know if I am erring somewhere:
babel-cli : devDependency
babel-core: devDependency
babel-loader: devDependency
babel-plugin-transform-class-properties: devDependency
babel-preset-env: devDependency
babel-preset-react: devDependency
css-loader: devDependency
node-sass: dependency
react: dependency
react-dom: dependency
react-modal: dependency
sass-loader: dependency
style-loader: dependency
validator: dependency
webpack: dev-dependency
webpack-dev-server: dev-dependency
Please let me know if I have any of these wrong
devDependencies are dependencies only required within your development environment or that are required for you to build your UI, for example nodemon is a dev dependency because you'll never run your application with it.
One of the advantages of splitting up your devDependencies from your normal dependencies is a smaller docker image size when building your final layer.
For example, in my dockerfile I'll run a suite of tests and also build the UI which requires an npm install but when building the final image that is going to actually run I will simply copy over the built UI files via docker then I'll run an npm install --production so that my devDependencies won't install and bloat my node_modules folder.
Hope this helps.
devDependencies are the module dependencies which are only required during the active development of your web application. For example, when you're coding new features into your web application. A lot of devDependencies will make development easier on your end and can provide functionality such as linting, bundling, transpiling, etc...
In contrast, regular dependencies are the modules which are necessary during the runtime of your web application. I.e. these are the necessary dependencies for your web application to work correctly when other users want to interact with your web application.
Note: By module I mean the underlying code from the library which you're leveraging. A more complete definition can be found here.
Your concerns about including more code than necessary for your production bundle are valid, and I would recommend reading more about the Cost of JavaScript. However, in the beginning I would encourage one to first get a working code base, and to keep iterating and improving your code as your skillset grows. Improving performance along the way.
Lastly, some common type of devdependencies include libraries for testing your code base, building, minifying, bundling, transpiling, and linting your code as well.
Hopefully that helps!
We're starting to adopt a monorepo setup using yarn workspaces and we'd like to have our firebase functions inside it. The repo structure is something like:
repo
node_modules <- all dependencies
packages
core
commom
functions <- firebase functions
So, I have 2 problems with this setup:
The dependencies of the functions don't live on the same folder as the entry file from functions
The functions depends on other packages such as core and commom that are in the repo so yarn symlinks from node_modules to the packages in the repo.
Is there anyway I can handle this?
With Yarn 2 node_modules aren't fetched and placed into in the respective functions directory (as it would be the case with calling npm i in the functions directory). So when calling firebase deploy --project default --only function the node_modules folder is missing and firebase will complain about this and abort the deployment process with the following error (or similar):
Error parsing triggers: Cannot find module [...]
Try running "npm install" in your functions directory before deploying.
There are two github issues that are tracking this issue at the moment:
Support mono-repos in deployment
Functions deployment fails when firebase-functions has been hoisted by a monorepo manager like yarn/workspaces or lerna
In the two issues above, several clever workarounds are presented by firebase users, e.g. using webpack to create a build that contains all the local packages in the release or using rsync or other tools that rewire the packages before release.
Another solution is not hoisting your project packages, if that is possible. You can do this, be adding the following two directives to your .yarnrc.yml file.
# yarnrc.yml
# disables yarn's plugnplay style and uses node_modules instead
nodeLinker: node-modules
# makes sure the node_modules are not hoisted to the (monorepo) project root
nmHoistingLimits: "dependencies"
The two directives above are explained in the yarnrc configuration docs as follows:
nmHoistingLimits Defines the highest point where packages can be hoisted. One of workspaces (don't hoist packages past the workspace that depends on them), dependencies (packages aren't hoisted past the direct dependencies for each workspace), or none (the default, packages are hoisted as much as possible). This setting can be overriden per-workspace through the installConfig.hoistingLimits field.
nodeLinker Defines what linker should be used for installing Node packages (useful to enable the node-modules plugin), one of: pnp, node-modules.
The solution I found for this is Yarn's nohoist option in your root package.json file.
By default Yarn hoists dependencies to the root directory so they can be shared between your packages. Unfortunately this will not work with Firebase. This means you need to tell Yarn not to hoist the dependencies used by your Firebase functions.
The documentation for nohoist is less than ideal, but here is an official blog post about it here:
https://yarnpkg.com/blog/2018/02/15/nohoist/
You probably want something like this:
{
"workspaces": {
"packages": [
"packages/*"
],
"nohoist": [
"functions/core",
"functions/common",
"functions/**"
]
}
}
Keep in mind that this uses the name field used in the package.json files of each workspace package. So in this example, it is assume that the functions directory has a package.json with "functions" as it's name.
functions/** tells yarn not to hoist any of the dependencies specified in packages/functions/package.json. This doesn't work for your shared yarn packages though, so functions/core and functions/common need to be specified separately.
You also need to include your workspaces as dependencies in your functions project, so add them to your package.json:
{
"name": "functions",
"dependencies": {
"core": "*",
"common": "*",
}
}
Once you have added all that, you should delete your packages/functions/node_modules directory and run yarn install. After doing this, you should see all your dependencies included in packages/functions/node_modules (not symlinks).
I am not sure I understand the question exactly, but I could give you my two cents on yarn workspaces based on whatever I understood from your question and from my experience using it.
Yarn workspaces consolidate all your dependencies into the node_modules present in project root as well as in a single package-lock.json to reduce conflicts and enables yarn to optimize the installation process giving you a faster yarn install. And also another advantage of it is, with a single pass yarn install can install dependencies of all packages under the workspace.
Edit: I think for some reason yarn link is not being called and instead only yarn install is being run, which will search the npm registries and throws the error mentioned in comment since it can't find the mentioned package on npm registry. So for a solution try creating an entry in the firebase's package.json like
"dependencies": {
"a": "file:../dependency-package-name/",
}
I have a package.json that gives a load of security warnings. Looking at the first critical item I see its open#0.0.5 which hasn't been updated for five years. Looking at npm ll it is included by npm#6.5.0 where I am using the latest that was updated about two weeks ago.
I would like to remove the insecure dependencies. In the Java world the maven package manager lets you put exclude certain transitive dependencies. Ideally, with npm or another node package manager, I should be able to exclude dependencies with vulnerabilities. Then I can retest that my app works and not see any security errors. Is there a way to quickly exclude anything that has a security vulnerability from my package.json? If there isn't a way to do this what approaches can a take to ensure that no insecure packages are used by my application?
Update: Although "npm": "^6.5.0" is specified in the package.json I was building it with an older npm which was picking up the critical issue mentioned above. I fixed all the issues with ./node_modules/.bin/npm audit fix --force
By definition, you can't exclude a package that a dependency you are using relies on. In other words, if you require package A, and package A claims it is dependent on package B, then removing package B will cause A to either stop working altogether or begin behaving erratically.
Unfortunately this does happen, and your options include:
Ignoring the security warning.
Replacing package A with something else (applies in some cases and not others).
Asking the maintainer of package A to upgrade the version of package B they rely on, possibly opening a pull request yourself.
In your case, though, I'm not sure if your investigation is complete yet - I don't see open in npm's dependency list. Might be worth scrapping your node_modules and re-running npm install, then check again to see who is using open.
This specific warning is targeting at your lockfile, and can be easily fixed by removing the yarn.lock or package-lock.json and reinstall dependencies.
Tarn package manager has feature resulution by which you can set fixed libraries to insecure thirdparties.
See
How do I override nested dependencies with `yarn`?
NPM has something similar.