I'm currently attempting to implement a continuous delivery pipeline for NodeJS, and want to have a tool that is capable of;
deploying and managing packages of an application
rolling back deployments
monitoring the deployments for potential rollbacks
has a REST API.
is not a SaaS solution.
I have tried go.cd, but it didn't have monitoring capabilities.
I think the product that may suit your needs is Codeship. There is a very good presentation about using this tool to deploy a simple web application, showing its capabilities.
A you can desume from its features it can:
Automate your development and deployment workflow
Run your automated tests and get notified
Speed up your tests with ParallelCI, that runs your test
Configure powerful deployment pipelines that run after successful tests to deploy your application to multiple environments
access debug builds via SSH
the API and Webhook enable you to integrate Codeship with the tools you are currently using
Check this out.
Related
Is it possible to pull / clone bitbucket repository within Azure Logic Apps?
I am curious if it is possible to set up some backend tests within Azure Logic Apps. So to pull repo with tests first and then execute them within CLI. I see that there is bitbucket connector in Logic Apps but there is no option to pull the repo. Or should I check some custom connector to run commands from hand like "git clone" etc. - if yes which one?
Azure Logic Apps is a cloud platform where you can create and run automated workflows with little to no code. By using the visual designer and selecting from prebuilt operations, you can quickly build a workflow that integrates and manages your apps, data, services, and systems.
From: What is Azure Logic Apps?
The key concepts here are "little to no code" and "prebuilt operations". Building your code with a CLI and running its tests is not something Logic Apps is targeting. It would also make executing Logic Apps a LOT more complex on the Azure end, since it would mean installing any and all frameworks, tools, etc. that are needed for building the code/running the tests.
If you look at Bitbucket actions that are supported, you can kind of make out that they're all API calls.
What you're looking for is available for free with GitHub workflows
A workflow is a configurable automated process that will run one or more jobs. Workflows are defined by a YAML file checked in to your repository and will run when triggered by an event in your repository, or they can be triggered manually, or at a defined schedule.
or Azure Pipelines.
Azure Pipelines automatically builds and tests code projects. It supports all major languages and project types and combines continuous integration, continuous delivery, and continuous testing to build, test, and deliver your code to any destination.
Potentially interesting read: Build Bitbucket Cloud repositories.
I am investigating ways to automate deployment of a specific build of a product to a specific Azure Cloud Service or VM.
The following steps would be automated, with as little manual intervention as possible:
Create a Cloud Service or VM
Install a specific build of the product (as a standalone exe or
Windows service, not IIS)
Tweak the configuration files(s)
Set up user account(s)
Run the exe/service
The code is currently in Visual Studio Online / TFS. We have Cruise Control .NET CI set up and we are looking at moving to TeamCity.
This will be used for the usual QA & Production type environments, but also for ad-hoc deployment e.g. if a trial feature has been added to the product and we want to deploy that to a new VM for a specific customer to play around with. Ideally we would be able to use the command line or a UI to pick the build, create the VM and specify any configuration changes.
One possible solution might be Octopus Deploy although I don't think this would be able to actually create an Azure VM. I will probably also look at the Azure API, and also TFS deploy.
Basically is this feasible, and are there any proven alternatives that I'm missing, in order to narrow down my research?
Thanks in advance!
While Octopus Deploy can do many things, in this particular scenario of yours, you're asking it to do three types of work - release management, automated provisioning and configuration management. It's a fine line between automation awesomeness and a really sticky situation.
Of the tasks you're asking, almost all of them can be done within Octopus today. I'd argue that it may be possible to Create a cloud service or VM. If there's some PowerShell cmdlet/library that allows you to spin up VMs with authentication, odds are you can do it Octopus - but it may not be the right tool to do that job today. Why?
In my opinion, it distorts the barrier between Developers, DevOps and SysAdmins. Whether you use Chef, Puppet, Salt, etc. whatever configuration management you have, that needs a whole layer of users with the expertise to back it up - often said expertise of system which the very developers who want such flexibility may not have. Secondly, right now this isn't a focus within Octopus (yet). I'd be hard pressed to say whether to use a tool such as Octopus on what it can do vs what it should do or not.
It's really nice that Azure now has support for preinstalling the Octopus tentacle for VMs. But that requires additional info such as, the Server thumbprint, port other supplementary configuration info in order to automate vm provisioning. That configuration management - should it be under Octopus's control, or something like Chef or Puppet? I honestly don't have an answer to this but my feeling as of now is not Octopus. Someday, perhaps, but until this is really ready and fully tested and vetted, I'd wait it out (a little) at least with Octopus.
If you're the adventurous type, then by all means try out Octopus. I may do a PoC (proof of concept) of this infrastructure automation later this year, but to rely on it today for business/production usage as the primary means of infrastructure automation will be risky and require a lot of work and experimentation. Again, I'm not saying it cannot be done, I'm questioning whether it should be done within Octopus as of this response today.
If anything, from the Octopus Deploy side of things is this feasible? Yes - it just hasn't quite been worked out yet. Looking at what you want to do, I'd say it's a two-phase process: 1. spinning up the new VM, attaching the tentacle to the environment and 2. running the deployment process on that new VM.
I'd also recommend checking out the Octopus blog. They're publicly talking about infrastructure automation. You can read about it here: http://octopusdeploy.com/blog/rfc-cloud-and-infrastructure-automation-support
I hope this response helps in some way.
The solution to the automated deployment in Azure is use ElasticBox.
I will skip the details of all the configuration options for Azure supported by ElasticBox, as they are detailed in the documentation section: http://elasticbox.com/documentation/deploying-and-managing-instances/using-azure/.
You only need to create a box (abstraction unit that ElasticBox uses to define the installation and configuration of the deployment of a service or application in any cloud) that takes care of the steps you need to be automated. So finally you will deploy the vm with near no manual intervention, just one click or a command with some parameters.
A box includes the variables necessary for your deployment and your scripts (In this case probably PowerShell, but they could be bash, python, perl, java, etc.)
When you deploy the box you create to deploy your application, ElasticBox will:
Create a Cloud Service or VM. (ElasticBox takes care of provision the vm in your Azure provider, or any of your preferred cloud provider).
Install a specific build of the product (as a standalone exe or Windows service, not IIS) -> This should be your install event script.
Tweak the configuration files(s) -> This should be part of your configure event script.
Set up user account(s) -> This should be part of your configure event script.
Run the exe/service -> This should be part of your start event script.
ElasticBox has a command line tool that enables to do VM deployments of your boxes and also you can manage your deployed vms with it: https://pypi.python.org/pypi/ebcli
It also support automatic termination of the vm after a custom time value.
This is quite a broad question, but certainly the goal is achieveable via one of a number of methods. While a bit old, Tom Hollander's blog on automated deployments is a good starting place. I've seen a lot of OctopusDeploy used as well as TeamCity but they all ultimately rely on Azure's PowerShell Cmdlets, Management Libraries in custom code or pure REST API calls.
Just an FYI; One option is to do everything by using the Azure Management API. I also like to reference the Azure Client Libraries in a VS project and do everything is C# code.
I'm working with Atlassian Bamboo on Demand for Continuous Integration and it works great.
Now I'm trying to use the "Deploy" feature and the problem is that I'm working with Azure (ftp, publish, git, mercurial... I really don't care how) and I can't find a "task" which could perform it.
Has anyone achieved this?
I do automated deployments to AWS from bamboo, but the concept is pretty much the same.
Bamboo has no specific options for deploying to the public cloud, so you have to build or call an existing deployment tool. At the end of the day bamboo deployments provide you with meta-data over which build has been deployed to which environment, and security over who can do deploys, but its up to you have to make the actual deploy work. Bamboo does give you a totally extensible engine for controlling the "how" via scripting. The deployment engine is basically a cut down version of the CI engine with a subset of tasks.
I resolved to build our deployment tooling due to it being fairly simple to get started and a worthwhile investment in time because this will be used often and improved over time. Bamboo gives me authorization and access control, and my scripts give me fine grained control of my deployments.
I'm assuming you are running a bamboo agent on a windows image like me. So powershell scripts are your friend . If you're running in linux you'll want to do the same with bash.
I have a powershell scripts controlling my deployments through a controller/agent model.
The controller script is source controlled and maintained in mercurial repo. This is pulled by the repository task.
The agent is a powershell script wrapped by a simple webapi rest service with a custom authentication mechanism. The agent is setup when an app server instance is provisioned in ec2. We use puppet for server provisioning.
The controller does the following for a deployment
connects to the vpc
determines the available nodes in my web farm using ec2
selects the first node and sends the node an "upgrade database" command
then proceeds to send "upgrade app server" command to each node
The logic for doing the deploy is parameterized so it can be re-used for deployment to different environment. I use bamboo deploy variables to manage feeding parameters for the different environments.
DEV is deployed automatically, test, staging and prod are all manual click deploys which are locked down to specific users.
One option I considered but did not invest the time to look at as aws elastic beanstalk as a deployment tool. It has a rich api for deploys. On the Azure side it looks like web deploy supports deployment to Azure IIS sites.
We have built a small MVC4 application using Azure Cloud Services. It has been deployed through Visual Studio. Now we are going add a test environment where the application should be tested, before being deployed into production.
I would like to have our CI server to build, test and create a deployable package, This package could then be deployed to any environment, providing correct configurations.
But I have not found a convenient way to do this. It is easy to build a package for a specific environment, with configuration transformations for .config and .cscfg files.
Is having the CI server to build a separate package for each environment the way to go, or have I missed something?
There are ways described how the web.config could be modified when the WebRole is starting, but this feels a bit hacky, and not the way the guys at Microsoft intended when creating Cloud Services.
Using the CI server to deploy the specific configuration has been the easiest in my experience. I think using the Visual Studio "Build" section in Team Explorer is what your looking for. We use Team Foundation Service as our Continuous Integration and Continuous Delivery server. In Visual Studio we've created Production and Testing build configurations. In the Build tab we've created a Continuous Integration Build which will kick off unit tests on every checkin, and a Continuous Delivery Build That will deploy newly tested checked in code on a regular schedule. These Build Events can be set to use a specific (Production/Testing) build configuration.
I would like use a continuous integration system in my projects. However, I don't want to use a server sitting in my office, instead I'd like my continuous integration server to run on Windows Azure. Has anyone set this up? Are there recipes to host Hudson or CruiseControl.Net (or any other CI system) within Azure?
We use the Build Manager in Team Foundation Server to push our automated builds to Azure. We set up our Azure hosted services to allow for Web Publish through remote desktop using this plugin - http://dunnry.com/blog/2010/12/20/UsingWebDeployWithWindowsAzure.aspx
You do not need to involve Team Foundation Server to use the plugin - you can set up your instances to publish to azure instantaneously through Visual Studio Web Publish without CI.
I think there are two parts to the Azure CI solution - automated builds from your source control and actually pushing the bits to azure. The plugin makes the publish to Azure much, much faster. So if you want real time access to changes on Azure, you will need to look into the plugin.
It sounds like you are trying to use Azure as an infrastructure provider, which runs a VM where your builds happen.
This is not a particularly good use-case for Azure. Azure is a platform to run your custom-built applications. While it does provide VM's similarly to Amazon or other IaaS cloud providers, those VM's are "stateless", can go up and down at will and meant to act as application servers where more than one can be up at any given time.
You can probably get this to work on an Azure VM but I am not sure if the pain would be worth it.
Azure's instances are application servers, not "windows servers".
HTH
I have installed Jenkins on Windows Azure, it works very well for me.
http://blogs.msdn.com/b/gongcheng/archive/2013/02/27/jenkins-on-windows-azure-the-missing-manual.aspx