The snyk wizard documentation recommends adding snyk protect to your project as a post-install step. How exactly do you do that? I referred to the entire package.json documentation here and couldn't find any reference to a post-install keyword.
There's some small difference between snyk wizard and snyk protect. Adding snyk protect is indeed a good practice, as it applies any potential patches to make your node app more secure. The snyk wizard will guide you through also upgrading the versions of your dependencies if they solve any vulnerabilities.
More info on snyk protect. When using the Fix PR functionality for a npm project, it will be automatically part of the PR, if it involves any patches.
An example can also be found here, Line14-21
Related
I'm maintaining some large NodeJS applications(micro-services, apps), which consume many(at least 100+) dependencies.
Updating those dependencies to fix a security vulnerability forces me to spend valuable development time.
Is there any way to automate package updates across repositories?
We're using GitLab for SCM.
I had thought of some options like:
Separating out core dependencies into a base level library.
Maintain a repository that specifies security-cleared dependency versions and repositories to be updated whenever the cleared list changes.
Any additional suggestions/examples?
Just to help shortlist approaches mentioned in the question:
1. Separating core dependencies into a "Core" library.
This can reduce the number of updates. But the libraries based on "Core" sill still have their own dependencies, which may be prone to vulnerabilities and the update cycle must be repeated for them manually. This approach just procrastinate the manual updates.
2. Maintain a repository that specifies security-cleared dependency versions and repositories to be updated whenever the cleared list changes.
Couple of options here:
Article on automating via pipeline scripts
GitHub: Use next-update + Hub CLI
GitLab: Use next-update + Lab CLI
Implement dependabot-core(GitHub) or dependabot-gitlab(GitLab)
Implement renovate.
I try using preinstall npm scripts, but it only run when I checkout the project into a new space, and run "npm i" standalone
I need a solution to run a script before the new dependency writed into package.json. It doesn't depend the type of dependency: dev or prod. All of them need to check.
For example, when a new developer join to the team, and want to add new dependency which has known vulnerability, this script stops the action before the package.json was changed, and show warning message for the developer
There isn't a way to do that with npm scripts. So, unless you feel like implementing one you're going to have to adjust your process. Start by identifying all the problems you're trying to address with an on-dependency-install hook.
You give the example of preventing the installation of a dependency or dependency version. That's not a problem: it's a solution you've identified for the problem. Figure out what the actual problem is, and then reevaluate your solution to see if it's really the most appropriate measure to take.
Possibly (probably) you are afraid of vulnerable code making it up to production. That's a problem definition you can work with. What possible solutions exist? You've already identified the blacklist. But not only is that not supported by your tooling, even if it were the onus is then on you to keep the blacklist up to date. Given just how quickly the Node world moves, that's enough work to keep several people employed fulltime. And that's not even getting into deploying it to your developers.
The good news is that that's not the only solution: you could establish procedural safeguards against integrating vulnerable code. If you're using a distributed VCS like Git, pull requests are right there: disable pushing commits to the master or development branch, have developers work in feature branches and submit pull requests, then review those pull requests and screen any new dependencies for vulnerabilities when they show up. If you're using something like SVN, you can use feature branches with code reviews to similar effect. Your developers get extra eyes on their code looking for vulnerabilities, optimizations, edge cases, and so forth; you don't waste time screening dependencies that nobody ever tries to integrate. And nobody has to worry about getting the latest copy of the blacklist. For this particular scenario, everybody wins with a process solution over a technical solution.
If you have other reasons for wanting to fire scripts when dependencies are installed, try working back to the root of the problem the same way. The way Node dependency management and module interactions work, you'll probably discover it's preferable to develop better process habits.
If you are using git, you can use pre-commit/push hooks, the result is pretty much the same, no vulnerabilities in code base.
For exemple with husky and nsp you could do something like this :
{
"scripts": {
"prepush": "nsp check"
}
}
Riffing off Gabriel's suggestions, since you are concerned about devs wasting time when the lib they add fails an nsp check... You can use an editor extension to run the nsp check as they code. Then have Husky do a pre-commit nsp check as well.
I would also recommend Greenkeeper.io to prevent vulnerabilities before they are found.
If the main concern is that these vulnerable packages are being run within your network (since there's no way to prevent those devs from using those packages in general), you could mirror a subset of the npm registry that you consider safe, or manually add known safe dependencies to that mirror, and block access to the main registry https://registry.npmjs.org/ at the network level. This would mean your developers are stuck waiting for the mirror to be updated, but would require somebody to at least stop and think before they're able to install a problematic module.
I lately helped out on a project, where I added a really small dependency - in fact, it only contained a regular expression (https://www.npmjs.com/package/is-unc-path).
The feedback I got from the developer of the project was that he tries to minimize third-party dependencies if they can be implemented easily - whereby he - if I understand it correctly - asks me to just copy the code instead of adding another dependency.
To me, adding a new dependency looks just like putting some lines of code into an extra file in the repo. In addition, the developers will get informed by an update if the code needs a change.
Is it just a religious thought that drives a developer to do this? Are there maybe any costs (performance- or space-wise, etc) when adding a dependency?
I also had some disputes with my managers once concerning the third party libraries, the problem was even greater he got into believing that you should version the node_modules folder.
The source of any conflict usually is the ignorance.
His arguments were:
you should deliver to the client a working product not needing for him to do any other jobs like npm install
if github, npm is down in the moment when you run npm install on the server what you will do ?
if the library that you install has a bug who will be responsible
My arguments were:
versioning node_modules is not going to work due to how package dependencies work, each library will download his own node_modules dependencies and then your git repository will grow rapidly to hundreds of mb. Deploy will become more and more slow, downloading each time half a gb of code take time. NPM does use a module caching mechanism if there are no changes it will not download code uselessly.
the problem with left-pad was painfull but after that npm implemented a locking system and now for each package you just lock to a specific commit hash.
And Github, and npm does not have just a single instance service, they run in cloud.
When installing a dependency you always have some ideas behind and there are community best practices, usually they resume to: 1. Does the repo has unit tests. 2. The download number 3. When was the latest update.
Node.js ecosystem is built on modularity, it is not that node is so popular cause of some luck, but cause of how it was designed to create modules and reuse them. Sometimes working in node.js environment feels like putting lego pieces together and building your toy. This is the main cause of super fast development in node.js. People just reuse stuff.
Finally he stayed on his own ideas, and I left the project :D.
I'm making app with using Xamarin.forms. (PCL Project)
Today, I added new three solution packages named SVG.Forms.Plugin.Abstractions, SVG.Forms.Plugin.iOS, SVG.Forms.Plugin.Android on workspace that downloaded from github.
I have used realm for Xamarin.
But After I added new packages, "Realms.RealmException has been thrown".
Message is "Fody not properly installed. allbX.Baby is a RealmObject but has not been woven."
Is it Fody's problem or Realm's or new packages'(SGV Control)?
And could you let me know how to solve it?
Better Answer
The check which is delivering that message is because Fody is not running.
So, they may have a RealmObject in their component but Fody doesn't get run building in your solution so weaving doesn't occur.
The easiest fix is to just use NuGet to add Fody to your main application project. That should install it in the right place for the solution.
Background
NuGet manages dependencies so if a package relies on Realm, it will go on in turn and install Realm. Realm itself relies on Fody, for example, so will in turn trigger a Fody installation.
You can manually install Realm but it is a little fiddly, having to add a couple of lines to your csproj to specify imports. We have chosen to only document installation via NuGet at this stage.
If you want to manually add Realm to another solution without using NuGet, I suggest you take a new clean solution, save a copy, and diff with the changes made to that solution by adding Realm via NuGet. You will then see the lines to copy into your existing solution.
I'm using NuGet to create a 'web framework' package containing code, master pages, css, javascript, etc.
In an attempt to speed up the build / test process I'm running nuget.exe update packages.config but I've noticed that it behaves differently than the package manager console's Update-Package command.
nuget.exe update seems to leave the previous version of the package still installed, resulting in multiple versions of the package installed. This usually doesn't cause problems but the Package-Manager Get-Package command shows many versions installed and sometimes the project will fail to build.
Update-Package actually uninstalls the package then reinstalls it, this is cleaner but slower
My questions are:
1. Is there documentation about the difference / relationship between these commands
2. Is the nuget.exe update behavior of installing multiple versions a bug?
3. Is there a better method for creating a package in one project and updating it in another project in a fast & automated manner?
Unfortunately, there's not much official guidelines or documentation except from piecing together forum and work item threads.
Current package manager console behavior was first included as a result from discussion in this thread, which later derived in a work item (sorry, apparently not enough rep to post more links).
However, as others already noted, behavior is not consistent with nuget.exe, where there's no such switch.
So, in answer to your questions:
VS Package Manager Console and nuget.exe do have different behaviors and seem to be updated independently (which is very unfortunate).
nuget.exe update behavior of installing multiple versions side-by-side has been a design feature from the start, as you can find from a comment on David Ebbo's blog about NuGet command line (again, I would have given you the link, but SO still doesn't trust me).
Unfortunately I haven't found anything about using package manager console cmdlets during build. What you could try is manually deleting all folders with your packageId on a build event and then packaging and installing using nuget.exe. Essentially replicate what Update-Package does manually, since as David Ebbo says, the way you uninstall a package through the command line interface is by, well, deleting the folder (again, can't post a reference, this is a bit annoying...)