I'm using the github rest api to create repo environments:
https://docs.github.com/en/rest/reference/repos#create-or-update-an-environment
The api allows us to select deployment_branch_policy.custom_branch_policies: (boolean) but I can't find a way of creating a deployment branch rule with a custom pattern, like you would do on the UI:
Is there any way of creating this through the REST api?
Many thanks
It should be possible to do via the Deployment branch policies API:
The Deployment branch policies API allows you to specify custom name patterns that branches must match in order to deploy to an environment. The deployment_branch_policy.custom_branch_policies property for the environment must be set to true to use these endpoints. To update the deployment_branch_policy for an environment, see "Create or update an environment."
For more information about restricting environment deployments to certain branches, see "Using environments for deployment."
Related
I use github for hosting my projects and have multiple projects in github. And I use Azure devops for CICD alone. I have a single project in Azure devops, where I create individual pipeline corresponding to each project in my github repo. All these github projects would need to use the same azure-pipeline.yml for build. So instead of keeping the same yml file in each project, is there a way I can keep this yml centrally. So that in future, if at all a change is required, I need not do it for all individual projects, instead, update the main yml template.
A single yml file where I have all the code is even possible for my usecase? Any help is much appreciated
Have you considered using templates? Essentially you have would end up with a single template containing the main build steps that is reusable and individual yaml for each pipeline that can pass parameters to the template for any differences you have between them (such as different triggers or variable values). This way you can update all pipelines by making changes to the template
Template documentation
According to your description, you may setup a repo contains all the YAML files for pipelines. Kindly also be advised that we can also keep the templates in other repositories, if we have defined the repository resources in the core YAML pipeline. Kindly refer to the sample Core and template YAML files below.
#Core YAML in Azure Repos
trigger: none
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: GitHub_REPO_1
type: github
name: GitHubAccountName/GitHubRepo1
endpoint: GitHubServiceConnectionName
- repository: GitHub_REPO_2
type: github
name: GitHubAccountName/GitHubRepo2
endpoint: GitHubServiceConnectionName
steps:
- checkout: none
# - checkout: GitHub_REPO_1
- template: GHREPO1.yml#GitHub_REPO_1
# - checkout: GitHub_REPO_2
- template: GHREPO2.yml#GitHub_REPO_2
#Template YAML from GitHub Repo
steps:
- script: echo "This YAML template is from GitHubRepo1"
displayName: 'Template From GitHubRepo1'
By the way, we could also checkout the code from one or multiple repository resource(s) and trigger the pipeline by the commits from the repository resources. Please refer to the following documents for more information.
Define YAML resources for Azure Pipelines - Azure Pipelines | Microsoft Docs
Check out multiple repositories in your pipeline - Azure Pipelines | Microsoft Docs
I'm new to Azure Devops. I would like to have devops pipeline environments to be created automatically during pipeline flow. So the 5th line below should create environment if it does not exist:
- deployment: Deploy
displayName: Deploy job
pool:
vmImage: $(vmImageName)
environment: 'production'
Instead I'm getting:
What am I missing?
To automate environment creation I could also use Terraform but this time I cannot find terraform resource config responsible for that.
I had a similar problem and found that the documentation lists some possible reasons for why this can happen:
Quote from learn.microsoft.com:
Q: Why am I getting error "Job XXXX: Environment XXXX could not be
found. The environment does not exist or has not been authorized for
use"?
A: These are some of the possible reasons of the failure:
When you author a YAML pipeline and refer to an environment that does not exist
in the YAML file, Azure Pipelines automatically creates the
environment in some cases:
You use the YAML pipeline creation wizard in the Azure Pipelines web
experience and refer to an environment that hasn't been created yet.
You update the YAML file using the Azure Pipelines web editor and save
the pipeline after adding a reference to an environment that does not
exist.
In the following flows, Azure Pipelines does not have
information about the user creating the environment: you update the
YAML file using another external code editor, add a reference to an
environment that does not exist, and then cause a manual or continuous
integration pipeline to be triggered. In this case, Azure Pipelines
does not know about the user. Previously, we handled this case by
adding all the project contributors to the administrator role of the
environment. Any member of the project could then change these
permissions and prevent others from accessing the environment.
If you are using runtime parameters for creating the environment, it
will fail as these parameters are expanded at run time. Environment
creation happens at compile time, so we have to use variables to
create the environment.
A user with stakeholder access level cannot create the environment as
stakeholders do not access to repository.
In our case, the problem was using runtime parameters for creating the environment.
You have the environment name 'production' hardcoded, so your problem might be related to one of the other cases.
From the MS docs:
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/maven?view=azure-devops
mavenFeedAuthenticate Authenticate built-in Maven feeds (Required)
Automatically authenticate Maven feeds from Azure Artifacts. If
built-in Maven feeds are not in use, deselect this option for faster
builds. Default value: false Argument aliases: mavenAuthenticateFeed
But everytime I want to use a Maven feeds from Azure Artifacts and I try this by setting mavenFeedAuthenticate to true I always get 401 (Unauthorized).
Could someone please show me the correct usage of this value?
Please refer to this doc to check if you have given Contributor/Owner role to identities: Project Collection Build Service ({OrgName}) and {Project Name} Build Service ({Org Name}) as this doc described.
In addition, this mavenFeedAuthenticate argument in the Maven task is set to false by default, which means that Azure Pipelines will automatically authenticate Maven feeds from Azure Artifacts in the backend. But if it is set to true, we need to specify PAT with full scopes in settings.xml and add file path pointing to this settings.xml for authentication in Options menu in this Maven task.
I am currently planning on a type of multi-tenant system, were different resource groups with a set of AppServices are deployed for customers via ARM Templates. Hence, each customer has its own Resource Group and set of AppServices. Currently we use Azure DevOps to deploy to a set of AppServices used for Development and Quality Assurance before it gets to Production. I am now trying to incorporate DevOps into the mix, automating a pipeline creation of some sort... (it would be a copy of an existing pipeline but only changing the Target AppServices). Which is were my question comes from, Is there a way to dynamically create or edit a Release pipeline to add the deployment of those new AppServices, without the need of manually edit or create a pipeline an adding those newly created AppServices, I was thinking something around the lines of being able to copy a yaml file template then replacing the necessary info to point to those AppServices after they have been created, but I am not totally sure where could I store the new yaml file so that it is picked up by Azure DevOps, or how could I would accomplish these, with the main idea being that all of this continues to be part of an automated process (if possible).
Thanks a lot for any help, any suggestion is appreciated.
EDIT:
The question is not about how to Deploy an ARM Template through the DevOps release pipeline (I plan on using a PowerShell Script/REST API to accomplish that), instead, is about when the AppServices Resources are created, I need to deploy code to those newly created AppServices and also update that code when necessary (Hopefully through a Release Pipeline), somehow generate a new release pipeline each time I deploy a new set of Resources. So that, when there is a new update, I could easily have that pipeline triggered and that set if AppServices can be updated (created as part of the automation process "dynamically"). (I Already have a similar pipeline that deploys to a "static" set of AppServices).
This is possible as you eluded to with YAML Pipelines. Based upon the scenario you have subscribed each repository would have it's own pipeline.yml file that will define the trigger, pool etc. It would also reference a repository that will house your yaml template.
The template would accept whichever parameters you may required (resource group, app service name, etc...) The triggering pipeline associated with each repository would pass this information leveraging the teamplate.
By doing this CI/CD can be set up to trigger on the individual pipelines and deploy the appropriate code all while leveraging the same YAML template.
The repository reference would be similar to:
resources:
repositories:
- repository: YAMLTemplates
type: git
name: OrginazationName/YAML Project Name
With the call to the template being similar to:
- template: azure-ARM-template.yml#YAMLTemplate
parameters:
appServiceName: 'AppServiceName'
resourceGroupName: 'ResourceGroupName'
UPDATE
At a high level the YAML pipeline would consist of the following. If all App Services are similar as stated and ARM Templates are similar this how it could be constructed and triggered based on a folder path:
Build necessary artifacts
Publish Pipeline
Deploy Azure Resource Group Task
Deploy App Settings Task (if applicable)
Deploy App Service
Release the deployment pieces for each environment in appropriate stages to help alleviate the amount of copying and pasting each of the above tasks can be part of a template either individually at a task, combination of tasks, or all in one. This would allow for defining the YAML once and referencing it and including app specific components as needed as parameters to the templates.
GitLab now has nice feature called "Releases". You can define "release" as combination of "tag + some description + some URLs" and it will be shown on "Releases" and "Tags" pages of your project. GitLab doc says:
we recommend doing this as one of the last steps in your CI/CD release pipeline
But, wait! CI/CD job by default has no access to API calls or write to git repository. We can configure "deploy token" or "deploy key" for access to repository and use them (via "secret variables") in build scripts. But neither "deploy token", nor "deploy key" give access to API.
So, we can't create release from CI/CD job using its environment variables, we can't use deploy tokens, we can't use deploy keys. So, what exactly GitLab suggests us to do when it says: "we recommend doing this as one of the last steps in your CI/CD release pipeline" ?
This previous question highlighted the same issue, pointing out you need to access in your CI/CD release pipeline to (from doc)
either OAuth2 tokens
Personal access tokens
Session cookie
This is not limited to release.
As seen in gitlab-ce issue 61108: "Allow tags to be managed with CI_JOB_TOKEN"
However, it turns out that tags cannot be removed by simply using the CI_JOB_TOKEN.
Instead I would need to have create an access token and pass this as CI variable to be able to call this API from within the CI jobs.
Other examples:
gitlab-ce issue 60643: "Download releases using deploy tokens."
gitlab-ce issue 58235: "Allow JOB_TOKEN to access Releases API"
However, it turns out the call to this REST API does not work with the JOB_TOKEN header but only with the PRIVATE_TOKEN.
Is this limitation intended?
I don't want to maintain extra Private tokens just for manipulating the assets of the release.
That means for now (June 2019), maintaining an extra Private token, and passing it as CI variable might be the only available workaround, pending those issues to be resolved.
That would use, I supposed, a masked variable (GitLab 11.0+)
2019, I commented:
for now there is no direct way to use the GitLab Releases API from a CI/CD release pipeline.
Update 2022:
See GitLab 15.5 (October 2022)
Update a release using the Release CLI
In this milestone we added the ability to update a release using the Release CLI. You can use this to automate releases by updating any of the release attributes directly from the .gitlab-ci.yml file, and leveraging the CI/CD pipeline to do so.
See Documentation and Issue.
So it is now possible.
Plus (still GitLab 15.5 (October 2022))
Access release description from tag in CI/CD pipeline variable
In past releases, there was no easy way to configure a pipeline that refers to a release or to release notes associated with a tag.
Now, you can refer to this information using two predefined environment variables: $CI_COMMIT_TAG_MESSAGE and $CI_RELEASE_DESCRIPTION.
See Documentation and Issue.