I use Azure pipelines to deploy to different environments (dev, test, prod) that have been running for 2 1/2 years without issue.
Recently the pipeline has stopped asking for 'approval' after the test deployment stage.
It has been working with the following steps for 2.5 years
Build
Deploy to 'dev' without approval
Wait for manual approval
Deploy to 'test'
Wait for manual approval
Deploy to 'prod'
The Azure pipeline is a template within a separate repo - there have been no changes to the .yaml files but now the process is
Build
Deploy to 'dev' without approval
Wait for manual approval
Deploy to 'test'
Deploy to 'prod'
This behaviour is mirrored on every service that deploys using this template.
I am at a loss as to what is causing this
The solution was this:
I 'environment' production had lost its approval - adding it back did not work.
So deleted and recreated and excepted the loss of deployment history. Working now.
Why this would happen I do not not know
Related
As written deploy pipelines can be pushed manually. I want to know if there is a way to stop the deploy from happening unless all pervious stages are done.
For example if I push deploy on Production, the Production deploy should check that Dev, Test, and Pre-Prod completed successfully before continuing.
I looked at deploy gates, but there does not seem to be anything in there for this condition.
I am actually surprised this is not part of the tool design. As written out of the box someone can push a deploy to Prod even if Dev, Test, and Pre-Prod aren't completed at all.
Does Microsoft just want us to rely on approvers to eyeball the situation? I am not sure how to constrain deploys for this particular condition.
You can link the stages together, so that the deployment happens first on dev, and only if that succeeds it will go to pre-prod, and then prod.
However, even with stage dependencies, I think you can still deploy at any stage at any time directly. To have more safeguards, you can add a script to your deployment steps that will check that previous steps are completed using Azure DevOps API.
I am working with 3 pipes based off 3 different repos within Azure DevOps. All 3 pipes deploy to the same cloud VM and all 3 have a set of E2E automated tests which are run after the deployment stage. I am currently having issues where 1 pull request is completed kicking off a deployment, then while the E2E tests are running a different pipe is kicked off, this causes services to stop on the VM during deployment making the E2E tests fail unusually.
Can I setup my pipe to only start a deployment + test if no other builds are already running against the target VM?
Cheers
If these stages in release pipeline are set to run one after another, there will be no issues.
However, new push will trigger a new release and break the ongoing deployment, we suggest that you review this doc: Release gates and approvals overview, so you can try to set up release deployment control using gates or define approvals and checks.
I am implementing azure devOps in Azure Data Factory.
My development environment ADF is integrated to a git repository.
I have created a build pipeline for building artifacts when there is a change in adf_publish branch.
My next step is to deploy the ARM template artifact to the PROD environment.
Suppose there are many triggered pipelines in the PROD environment and few pipelines were running in the PROD env at the time when the DEV changes are getting deployed to PROD.
My questions are:
How the changes are deployed from DevOps to PROD environment? Will it delete and replace all the pipelines and triggers from DevOps or It will pick only the changes and update the PROD env without touching the unchanged pipelines?
Suppose a pipeline is running in PROD env at the time of deployment, what will happen to that pipeline run?
Suppose there is a triggered pipeline at 3:00 O clock and deployment starts at 2:55 and ends at 3:05. What will happen to that triggered pipeline which was supposed to run at 3:00?
Any one having a clear idea about above questions or Is there any documentation for reference to obtain answers for these questions?
I have gone through Keeping deployment mode as Incremental instead of keeping it as Complete at the time of ARM template deployment. Will that option do all the tasks which I asked above?
How the changes are deployed from DevOps to PROD environment? Will it delete and replace all the pipelines and triggers from DevOps or It will pick only the changes and update the PROD env without touching the unchanged pipelines?
It depends on the deployment mode you choose.
Deployment Mode: This specifies the deployment mode in which the Azure resources specified in the template have to be deployed.
Incremental mode handles deployments as incremental updates to the
resource group . It leaves unchanged resources that exist in the
resource group but are not specified in the template. Complete mode
deletes resources that are not in your template. Validate mode
enables you to find syntactical problems with the template before
creating actual resources. By default, incremental mode is used.
Suppose a pipeline is running in PROD env at the time of deployment,
what will happen to that pipeline run?
Your deployment will fail, and you may receive a 409 conflict error message. You could check the document Best practices for CI/CD：
Pre- and post-deployment script. Before the Resource Manager
deployment step in CI/CD, you need to complete certain tasks, like
stopping and restarting triggers and performing cleanup. We recommend
that you use PowerShell scripts before and after the deployment task.
For more information, see Update active triggers. The data factory
team has provided a script to use located at the bottom of this page.
Suppose there is a triggered pipeline at 3:00 O clock and deployment
starts at 2:55 and ends at 3:05. What will happen to that triggered
pipeline which was supposed to run at 3:00?
Similar to your second question, the pipeline should not be triggered (or in the pending state), but I still don't have an instance to test such a scenario.
I'm trying to create a CI/CD pipeline for an example prototype. Thus, I've started simple enough to test my infrastructure - I'm using an almost untouched boilerplate of ASP.NET Framework Web App (targeting 4.6.1). The steps I've completed are:
App is deployed to an Azure App Service.
Its version control is hosted with Azure DevOps.
A build pipeline with the following tasks has been created, set-up and tested if it executes (tasks and their order, come from a template):
Azure Deployment Options/Settings are bound to the repository DevOps, thus builds are also displayed in Azure, and should be deployed there if successful.
The Build Pipeline is bound to the correct repository inside DevOps
Builds get triggered by pushing to the master branch
The next step was to verify that a broken build, because of failed tests or any other reason is not deployed to production in Azure. I've created a failing test for this reason.
And this is where I'm left stumped. Builds do fail as expected and the "App Service Deploy" task is skipped, because the build tasks before it have a failure:
And yet those broken builds still get deployed to Azure and to production without even waiting for the pipeline to finish. I'm verifying that a change has actually happened with small visual updates.
Build started and finished in Azure as soon as a push occurs before the pipeline in DevOps is fully traversed (or even started, if finding an agent takes longer):
(DevOps still not finished):
What am I doing wrong here? Am I understanding the pipeline wrong? Have I missed a set-up step somewhere? I'm lost.
Edit: As asked by Josh, here's my trigger as well:
Edit 2.2 A bit more clarification of my deployment options in my App Service in Azure, related to Daniel's comments:
This turned out to be the issue.
This is the only option I'm allowed to choose when tying my deployment to DevOps. I'm not allowed to choose a pipeline, just a project and a branch. In a tutorial I've compared with, the settings are the same (at least in this menu), but the build does not get triggered from the repository, but expects the pipeline to reach the appropriate step first, which is why I haven't considered it to be the culprit. Is there some additional setting up, I've missed to do, to indicate that it must look for a pipeline, rather than fire straight away from branch changes?
The deployment you have set up in the Azure portal is tied to source control only, not your build definition. So every time you commit to source control, two things happen that are totally disconnected from each other and start in parallel since they listen to the same repository for changes:
A build fires off in the pipeline.
The Azure website is updated with the version you just pushed to source control, since its deployment options are bound to it.
Remove #2 and your problem will go away. You set the App Service you want updated in the pipeline, you don't need an additional hook in the App Service itself.
I am learning CI/CD concepts and trying to set up a pipe line in Azure to deploy my sample api to dev, int,qa and prod environments. I was able to deployment the build successfully to all environments. But what is the correct way of doing CI/CD in the companies? We cannot have build on every check-in to all environments even though we have the unit testing run in the build. The usual flow should be Unit test then deploy to Stage/QA and once QA signs off then we promote the build to PROD right? How does this fit into the CI/CD pipeline? Also we can have multiple builds in Dev, can we select which build to deploy to stg and prod?
The ideal way would be to link each of your check-ins to some bug/task so that it will be linked to appropriate test case .After the check in automated test cases are run simultaneously and test whether the check in is valid. If it is valid you can line it up to deployment which only will be deployed after getting the necessary approvals.It is recommended to run continuous integration pipeline to each check in so that you will have a history of build success and failures which is a big relief when you have to find/track errors.
To the prod environment you have multiple approvals as requirement to deploy and provide all the approvals when the sign off is provided.
For the build to map to stg and prod .yes you can do this .If you are using web app in azure and you have prod and stage slots then you can point the branch to respective slots in the deployment configuration.
Please let me know if you need anything else.
Hope this helps.