One of the top-level projects on nixos.org is Disnix, "a distributed deployment extension for Nix". Another project is NixOps, "a tool for deploying NixOS machines in a network or cloud".
The feature sets sound very similar. Are they competitors, or is one a successor to the other, or are they solving different problems? If they are "competitors", as they seem to be, then what are the differences, and which should I be using?
EDIT: if it's any indication, NixOps appears to be under active development, but I can't see when Disnix was last worked on.
NixOps is a tool for "spinning up" nixos machines. Think: ec2 API.
Disnix is a toolset for managing the services running on a network of machines. Think: Chef/Puppet for multiple hosts.
Using DisnixOS (see the latest builds) the two can work together well, delegating to NixOps for instantiation of servers and delegating to Disnix to configure all your services on those servers.
If you are just deploying to a single server or the relationships between your services are simple, then NixOps can probably do everything you need. If you have multiple machines, running multiple services that are all subtly inter-related... then you will want Disnix/DisnixOS.
Sander van der Burg (the author of Disnix) has a rather detailed blog post explaining the relation/differences.
The short answer is: while both tools perform distributed deployment
tasks and use the Nix package manager as underlying (local) deployment
technology, they are designed for different purposes and address
different concerns. Furthermore, they can also be effectively used
together to automate deployment processes for certain kinds of
systems.
[...]
[Nixops] does infrastructure deployment, while [Disnix] does service
deployment. Infrastructure deployment is a prerequisite of doing
service deployment and both tools can actually be combined to automate
both concerns.
Service deployment is particularly useful for distributed systems that
can be decomposed into "distributable units" (such as service-oriented
systems), but not all kinds of distributed systems.
Moreover, NixOps is a tool that has been specifically designed to
deploy NixOS configurations, while Disnix can deploy services to
machines running any operating system capable of running the Nix
package manager.
Related
Currently, our product is a web application with SQL Server as DBMS, ASP.NET backend, and classic HTML/JavaScript/CSS frontend. The product is actively developed and each month we have to deploy a new version of it to production.
During this deployment, we update all the components listed above (apply some SQL scripts, update binaries, and client files) but we deploy only the delta (set of files which were changed since the last release). It has some benefits like we do not reset custom data/configs/client adjustments.
Now we are going to move inside clouds like Azure, AWS, etc. Adjust product architecture to be compliant with the Docker/Kubernetes and provide the product as SaaS.
And now the question itself: "Which approach of deployment is recommended in the clouds?" Can we keep applying the delta only? Or we have to reorganize the process to always deploy from scratch?
If there are some Internet resources I have missed, please share.
This question is extremely broad but maybe some clarification could steer you in the right direction anyway:
Source code deployments (like applying delta's) and container deployments are two very different directions in the sense that the tooling you invest in during the entire SLDC CAN differ substantially. Some testing pipelines/products focus heavily (or exclusively) on working with one or the other. There will be tools that can handle both of course.
They also differ in the problems they're attempting to solve and come with some pro's and con's:
Source Code Deployments/Apply Diffs:
Good for small teams and quick deployments as they're simple to understand and setup.
Starts to introduce risk when you need to upgrade the Host OS or application dependencies
Starts to introduce risk when the Host's in production begin to drift (have more differing files then expected) more dramatically over time
Slack has a good write up of their experience here.
Container deployments
Provides isolation from the application (developer space) and the Host OS (sysadmin/ops space). This usually means they can work with each other independently.
Gives an "artifact" that won't change between deployments, ie the container tagged v1 will always be the same unless you do something really funky. You can't really guarantee this
The practice of isolating stateless components makes autoscaling those components very easy, and you can eventually spend more time on the harder ones (usually stateful).
Introduces a new abstraction with new concerns that your team will have to mature into. Testing pipelines, dev tooling, monitoring/loggin architectures might all need to be adjusted over time and that comes with cost and risk.
Stateful containers is hardly a solved problem (ie shoving an existing database in a container can be a surprising challenge).
In order to work with Kubernetes, you need to have a containerized application. That doesn't mean you need to containerize your entire product over night. Splitting out the front end to deploy with cloudfront/s3, and containerizing a stateless app will get your feet wet.
Some books that talk about devops philosophies (in which this transition plays a part)
The Devops Handbook
Accelerate
Effective Devops
SRE book
I am using HF for some time and trying different things regarding business network specification and configuration.
But, I have couple of question regarding best practices (if there are any yet) in using HF in production.
When we talk about using HF in production, should we use docker-compose-base.yaml, docker-compose-cli.yams, cofigtx.yaml.... etc. as files used to setup and configure our business network, and if not, can you please specify what is the best practice use-case?
Thank you for your answers.
You could use Docker Swarm/Compose with derivatives of the sample compose files you referenced, or you could use Kubernetes to manage a network (or subset of same). Project Cello is working on delivering such capability. The Ansible driver in particular has been demonstrated to work effectively - though it is far from a 1.0 level of maturity.
The reality is that you'll want to manage (likely) more than just four peer nodes all on the same VM or host, but manage multiple peers on multiple VMs/hosts even across multiple networks for a production deployment.
Further, you will obviously need to add management and monitoring to the deployed containers for a true production experience. The Hyperledger chat and mailing lists can be good sources of help and insight.
When using Azure web/worker roles users can specify osVersion to explicitly set "Guest OS image" version. This ensures that when Microsoft issues new critical updates they are first shown up on a newer "OS image" which users can explicitly specify and test their service on.
How is the same achieved with Azure Service Fabric? Suppose I deployed my service into Azure Service Fabric and it's been running for a month, then Microsoft issues updates for the OS on the server where the service is running - how are they applied such that I can test them first to ensure they don't break the service?
Brett is correct. SF cluster is based on Azure VMSS and the expectation is that the customer is responsible to patch the OS. https://azure.microsoft.com/en-us/documentation/articles/service-fabric-cluster-upgrade/
We have heard from majority of the SF customers that this is not at all desirable and that they do not want to be responsible for OS patching.
The feature to enable an OPT-IN automatic OS patching is indeed a very high priority within Azure Compute team. The exact details on how best to offer this is still in design, however the intent is to have this functionality enabled before the end of the year.
Although that is the right long term solution, to mitigate this issue in the short term, SF team is working on a set of steps that will enable the customers to opt into having the their VMs patched using WU in a safe manner. Once the steps are tested out, we will blog about it and will publish a document detailing the steps. Expect that in the next couple of months.
As I understand it you are currently responsible for managing patching on SF cluster nodes yourself. Apparently moving this to be a SF managed feature is planned but I have no idea how far down the road it might be.
I personally would make this a high priority. Having used Cloud Services for many years I have come to rely on never having to patch my VM's manually. SF is a large backwards step in this particular area.
It'd be great to hear from an Azure PM on this...
Automatic Image based patching like cloud services in service fabric.
Today you do not have that option. The image based patching capability is work in progress. I posted a road map to get there on the team blog : https://blogs.msdn.microsoft.com/azureservicefabric/2017/01/09/os-patching-for-vms-running-service-fabric/ Try out the script and report any issues you hit. Looking forward to your feedback.
Lots of parts of Service Fabric are huge rolling dumpster fires backwards. Whole new hosts of problems have been introduced that the IIS/WAS/WCF team have already solved that need to be developed for once again. The concept of releasing a PAAS platform while requiring OS patch management is laughable. To add insult to injury there is no migration path from "classic cloud PAAS" to this stuff. WEEEE I get to write my very own service host. Something that was provided out of the box for a decade by WAS. Not all of us were scared by the ability to control all aspects of service host communication options via configuration. Now we get to use code so a tweak channel configuration requires a full patch/release cycle!
This question does not necessarily pertain to the organization of Node project structure, and more of how to represent separate, logical services. Within our team, we have requirements to create and support several services (i.e., a set of API endpoints). These services aren't directly related, so my initial reaction is they should be separate projects with separate code bases running in separate Node (or Express) servers. I'm wondering if this approach would complicate deployment and management. The alternative would be to have a single "entry point" (i.e., a single Node server) that delegates to the respective services depending on which context root or URL is seen. I'm curious which approach seems more logical and how people are handling these "microservices" in the wild now?
These services aren't directly related
These services should be separate projects/repos with distinct entry points.
I'm wondering if this approach would complicate deployment and management.
Yes, absolutely. I have several NodeJS JSON APIs in production and for each, I have 2-3 environments (canary, staging, production). When you get to about 3 production services in the wild, things can get unwieldy without some discipline.
You can manage this with documentation (via wiki or in repo) about each service and their environments as well as any other dependencies (services that this service depends on).
This also helps with emergencies where a service is slow or not responding. Sometimes, the service itself is fine but a service's dependency could be down. For example, the github API may be a dependency...it goes down.
The alternative would be to have a single "entry point" (i.e., a single Node server) that delegates to the respective services depending on which context root or URL is seen.
In some cases, you may have to also build a "gateway" service which consumes your other single-purpose services. One reason to do this is to support authentication and authorization (i.e. OAuth).
In other words, you may need multiple micro-services and a gateway service.
I'm not very skilled on Azure, but googling hasn't give me more answer on this topic.
I have an ASP.NET web page that use R-(D) COM Interface for doing some complex calculus. I'm evaluating to move everything to the Azure platform.
I saw that it's easy to move webpages on Azure however being that I need that RSERVER is installaled on the machine I need to move everything.
I was thinking of creating a VHD machine and publish the entire image on Azure but I'm not sure this is the best solution.
I am not familiar with RSERVER, but here are some guidelines you may follow:
By default all Windows Azure servers run in 64bit mode. This is
important for the COM interfaces.
You may run any executable as a Startup Task in regular
Windows Azure Web/Worker role. Frankly you can create vey complex
startup scripts. You may use the Windows Azure Bootstrapper to
ease the solution. The trick is that RSERVER must support
unattended/silent install.
I would stick to the least friction solution - which would be using a normal Windows Azure Web Role and a Startup Task.
If that is not working for you, you may consider preparing a VHD image and use the Windows Azure VM Role.
I've written a very similar answer to what I'd write to you here. The thing is, the Azure VM role is technically a good solution, depending on what you need to do with it. You can generally create really good solutions with a fairly minimal amount of effort to let legacy code work with Azure & all the shortcomings of the VM role.
In general, if you have a lot of custom installation you need to do, create the Azure VM role, absolutely. But make sure you make the communication with it proper. It's not going to behave exactly like a web or worker role. Although, if I remember correctly, you still have endpoints and configuration there, so you can expose your programming to the outside. Personally however, my architectures are way more queue based (as described in the answer highlighted above) so I'd opt for writing a bridge program in the VM.