GitLab now has nice feature called "Releases". You can define "release" as combination of "tag + some description + some URLs" and it will be shown on "Releases" and "Tags" pages of your project. GitLab doc says:
we recommend doing this as one of the last steps in your CI/CD release pipeline
But, wait! CI/CD job by default has no access to API calls or write to git repository. We can configure "deploy token" or "deploy key" for access to repository and use them (via "secret variables") in build scripts. But neither "deploy token", nor "deploy key" give access to API.
So, we can't create release from CI/CD job using its environment variables, we can't use deploy tokens, we can't use deploy keys. So, what exactly GitLab suggests us to do when it says: "we recommend doing this as one of the last steps in your CI/CD release pipeline" ?
This previous question highlighted the same issue, pointing out you need to access in your CI/CD release pipeline to (from doc)
either OAuth2 tokens
Personal access tokens
Session cookie
This is not limited to release.
As seen in gitlab-ce issue 61108: "Allow tags to be managed with CI_JOB_TOKEN"
However, it turns out that tags cannot be removed by simply using the CI_JOB_TOKEN.
Instead I would need to have create an access token and pass this as CI variable to be able to call this API from within the CI jobs.
Other examples:
gitlab-ce issue 60643: "Download releases using deploy tokens."
gitlab-ce issue 58235: "Allow JOB_TOKEN to access Releases API"
However, it turns out the call to this REST API does not work with the JOB_TOKEN header but only with the PRIVATE_TOKEN.
Is this limitation intended?
I don't want to maintain extra Private tokens just for manipulating the assets of the release.
That means for now (June 2019), maintaining an extra Private token, and passing it as CI variable might be the only available workaround, pending those issues to be resolved.
That would use, I supposed, a masked variable (GitLab 11.0+)
2019, I commented:
for now there is no direct way to use the GitLab Releases API from a CI/CD release pipeline.
Update 2022:
See GitLab 15.5 (October 2022)
Update a release using the Release CLI
In this milestone we added the ability to update a release using the Release CLI. You can use this to automate releases by updating any of the release attributes directly from the .gitlab-ci.yml file, and leveraging the CI/CD pipeline to do so.
See Documentation and Issue.
So it is now possible.
Plus (still GitLab 15.5 (October 2022))
Access release description from tag in CI/CD pipeline variable
In past releases, there was no easy way to configure a pipeline that refers to a release or to release notes associated with a tag.
Now, you can refer to this information using two predefined environment variables: $CI_COMMIT_TAG_MESSAGE and $CI_RELEASE_DESCRIPTION.
See Documentation and Issue.
Related
I have two CI pipelines in azure devops:
CI pipeline to train models
CI pipeline to score/predict/inference new data
Both of these pipelines are triggered when a PR is created on a specific branch. I have enabled "Tag Builds" on succeed with $build.BuildNumber format. I beleive if the builds are successful, they are given some tags.
I have a release pipeline, what I want to do is to check if the tag/buildNumber for 1st and 2nd CI pipelines are same or not. If not, the release pipeline should fail.
The problem is I cant find any tag information of the CI pipelines here is what I see after a build is succeeded.
I found out that. It is not possible to check if two or more tags are valid based on some logic in devops. So, we ended up using bash task and git commands to check if tags are valid (using regex).
I'm using the github rest api to create repo environments:
https://docs.github.com/en/rest/reference/repos#create-or-update-an-environment
The api allows us to select deployment_branch_policy.custom_branch_policies: (boolean) but I can't find a way of creating a deployment branch rule with a custom pattern, like you would do on the UI:
Is there any way of creating this through the REST api?
Many thanks
It should be possible to do via the Deployment branch policies API:
The Deployment branch policies API allows you to specify custom name patterns that branches must match in order to deploy to an environment. The deployment_branch_policy.custom_branch_policies property for the environment must be set to true to use these endpoints. To update the deployment_branch_policy for an environment, see "Create or update an environment."
For more information about restricting environment deployments to certain branches, see "Using environments for deployment."
I'm working on my first Azure DevOps Services release pipeline. I've got a secret in a variable group in the Azure DevOps | Library area. I've been struggling with getting to that secret so I can use it in the release pipeline. I see by this article Add & use variable groups on Microsoft's Docs that I must add that in the variables section of the YAML file. However, I've not seen any way to edit the release pipeline YAML file. I can view it, but there appears to be all. If there's a way of editing the YAML file, how do I get to it?
You should be able to do that, earlier To enable YAML release pipeline, you need to activate Multi-stage YAML pipelines in Preview features:
From the Docs,
To help make it even easier to edit and update your pipeline, we’ve
created an in-product editor with IntelliSense smart code completion,
and an easy task assistant
From the documentation you're pointing at.
If you're using YAML pipelines, you can reference the variable group in your YAML pipeline:
variables:
- group: my-variable-group
If you're using the classic mode, you can link the variable group to your pipeline through the UI:
I'm trying to deploy an ADF pipeline using CICD (Azure DevOps Deployment) using a release pipeline.
Here I'm trying to merge stuff from my Collaboration branch to master (usig GIT).
I'm also using:
Azure Deployment:
Create Or Update Resource Group action on SiteOpsConsolidatedProd (Agent Job) as a part of the Continuous Deployment Process
I'm getting the following error:
[error]At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
[error]Details:
##[error]BadRequest: The document creation or update failed because of invalid reference 'irslinked'.
##[error]Check out the troubleshooting guide to see if your issue is addressed: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
##[error]Task failed while creating or updating the template deployment.
irslinked is the name of a self-hosted IR (Linked)
this IR: irslinked is also included in the overwrite template parameters list.
As suggested by Microsoft in their ADF CICD best practices, the self-hosted IR should be hosted on a dedicated ADF instance and should be of type shared and linked to the dev/test/prod instances.
Link to the documentation: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#best-practices-for-cicd
Please respond if it works.
This may be helpful for someone. I had a similar invalid reference error. Actually, my ADF was imported from git. I have reconfigured the git with "import existing resource to repository" option.
Azure documentation https://learn.microsoft.com/en-us/azure/data-factory/ci-cd-github-troubleshoot-guide#recover-from-a-deleted-data-factory
If customer had a Self-hosted Integration Runtime in deleted ADF, they
will have to create a new instance in new ADF, also uninstall and
reinstall the instance on their On-prem machine/VM with the new key
obtained. After setup of IR is completed, customer will have to change
the Linked Service to point to new IR and test the connection or it
will fail with error invalid reference.
I had this error message but it referenced a resource that didn't even exist.
I was able to solve it using steps from here:
https://learn.microsoft.com/en-us/azure/data-factory/source-control#troubleshooting-git-integration
Notably:
Disconnect from existing git repo.
Reconnect back to the same repo, but select "import existing resources" into a "Create new git branch".
Then use git to create more commits on top of that branch, which remove all the extra crud that got generated, until the codebase is back to the desired state.
Note that I had to have sequential commits deleting stuff in dependency order, before it would work. i.e. Pipelines, then dataflows, then datasets, then linked services, then Int.Runtimes.
I have a problem on my build/release pipeline with Azure Container Reigstry.
I use a Azure Resource Group Deployment task to deploy Azure Container Registry (and other stuff) and it works perfectly.
I have the loginServer, username and password in output variables to reuse it.
Then I want to build and push image to ACR but I can't set the name of the registry (that I get from output variable) with a variable. I have to choose the registry when I setup the definition, but it is not created at this moment.
Is there a way to do this ?
As a workaround, I use the Azure Resource Group Deployment the create the registry and then I send output variables to a powershell script which build, tag and push my images to the registry.
If nobody has a better way, I think I will post a uservoice to change that.
When you say you use an Azure Resource Group Deployment task, are you referring to VSTS?
If you could provide more specific repro steps, I might be more helpful.
I'd also suggest you might take a look at https://aka.ms/acr/build as easy way to natively docker build images with your registry. ACR Build is now available in all regions and simplifies may of the experiences you may be hitting.
Daniel just made this post that helps with the VSTS integration: https://www.danielstechblog.io/building-arm-based-container-images-with-vsts-and-azure-container-registry-build/
Steve
Sorry for the delay, I was off the office.
I just retry to fix my problem and it seems that I can now enter a free text (and so, a release variable) to the VSTS docker task to specify the ACR I just created before with a Azure Resource Group Deployment task.
So no problem anymore.
Thank you for your response, I will take a look to ACR build :)
Bastien