I am trying to find a way to access my private github repo via my YAML pipeline which runs in azure devops. I can create PAT access tokens, but do i need to copy this token and paste it as a secret value within github -> repo -> settings -> secret? and how would i access this token to get access to my github private repo? i have the following steps though not sure if this works:
steps:
- shell: pwsh
env:
SUPER_SECRET: ${{ secrets.SuperSecret }}
run: |
example-command "$env:SUPER_SECRET"
what are the yaml tasks i need to access my gihuh repo? i know using below i can access public repos but not private ones:
repositories:
- repository: myrepo
endpoint: svc-github
name: projecta/privaterepo
type: github
Any ideas on this? thanks.
To use a private github repository from your YAML pipeline, you have to grant authorization for it :
Go to the settings of your Azure devOps project on Azure DevOps
Select "Service connections"
You have to create a new GitHub service connection and give access to your private repository
After that, specify the service connection created inside "endoint:"
So, you will obtain something like that :
resources:
repositories:
- repository: common
type: github
name: Contoso/CommonTools
endpoint: [your service connection name]
Azure DevOps documentation section about it : https://learn.microsoft.com/en-us/azure/devops/pipelines/process/resources?view=azure-devops&tabs=example#resources-repositories
I hope this will help you.
Related
I am trying to extend a template on a Azure DevOps pipeline which exists on a repository hosted on Azure Devops. Code looks like below.
resources:
repositories:
- repository: devops
type: git
name: otherProject/repositoryXYZ
ref: main
parameters:
- name: environment
type: string
values:
- "UAT"
- "Production"
default: "UAT"
trigger:
- none
pr: none
extends:
template: folder/template.yml#devops
parameters:
environment: ${{ parameters.environment }}
When I deploy this pipeline on the same project on which repository repositoryXYZ exists, I get a successful run.
For example I have a project A which holds 5 pipelines. One of the pipelines is the above and can download the repository and run successfully. This pipeline exists in project A where repositoryXYZ is located.
When I deploy the same pipeline from a different project project B within the same Azure Devops organization, I get the below error.
/azure-repo.yml: The repository DevOps in project f1809f72 could not be retrieved. Verify the name and credentials being used.
The id of the project on the logs is for project B. (f1809f72).
I tried to alter DevOps repository permissions and to append project
build collection administrators full access. (repositoryXYZ)
Then I tried to place the repository on github and I got the same
issue (added a PAT and changed the directories for the repository)
I also tried to edit project settings and deactivate the limit jobs
options. (all limit job aithorization settings have been deactivated for both projects)
Do I miss something? How can I use my pipeline to extend the template which will be downloaded from an Azure Devops repository of another project within the same devops organization?
The pipeline cannot start running at all, so I guess something should be wrong with the permissions.
My error was the referencing repository on the template.yml file. On the build pipeline I was pointing the correct repository, but on the template I was pointing a false one that could not be retrieved. I corrected that and I was able to trigger the pipeline.
I have a requirement of giving multiple teams access to a shared resource in azure. I therefore want to limit how people can publish changes to the shared resource.
The idea is to limit the use of a service connection to a specific pipeline, as per this documentation. However if the pipeline is stored in their own repo the developer could change it. This would not give me enough control. I therefore found that it was possible using a template from a central repo. Using a shared repo, would then allow me to have a service connection solely for the template?
So how I imagine doing the above is I need to grant project X a service connection for my BuildTemplates Repo. But this is basically just access to the repo and to be able to use the shared templates. Then in BuildTemplates repo I can have a service connection for my template A.
Now the developer in project X - creates her deployments and configurations for her pipeline with her own service connection scoped for her resources. Then she inherits a template from BuildTemplates Repo and passes relevant parameters for the template A.
She cannot alter the template pipeline A and only the template pipeline A can publish to the shared resource, because of the scoped service connection. I can therefore create relevant guards for the shared azure resource in the template pipeline A - so I restrict how developer X can publish to my shared azure resource.
does this make sense and is it viable?
The pipeline part in A cannot be edited by developer in X ?
The service connection in A will not propagate out so developer in X can use it in an inappropriate way?
Update
The above solution does not seem to be viable since the pipeline template is executed in the source branch scope.
Proposed Solution
The benefits I see with the above suggestion doe not seem possible, because of the issues. However one can utilise pipeline triggers, as a viable solution. This however results in a new issue. When a pipeline is triggered by Developer Y in Y's repository and it succeeds. Then a trigger is made in MAIN repository and the pipeline in MAIN fails e.g., because the artifacts from Y introduced an Issue. How does developer Y get notified about the issues in MAIN pipeline?
Here is my solution, in same Azure organization, we can create a Azure Project, then create a repo to save common pipeline template.
All the repos in other Azure project can access this pipeline template.
UserProject/UserRepo/azure-pipelines.yml
trigger:
branches:
include:
- master
paths:
exclude:
- nuget.config
- README.md
- azure-pipelines.yml
- .gitignore
resources:
repositories:
- repository: devops-tools
type: git
name: PipelineTemplateProject/CommonPipeline
ref: 'refs/heads/master'
jobs:
- template: template-pipeline.yml#devops-tools
PipelineTemplateProject/CommonPipeline/template-pipeline.yml
Since the inline script of pipeline has 5000 characters limitation,
you can put your script(not only powershell, but also other languages) in PipelineTemplateProject/CommonPipeline/scripts/test.ps1
# Common Pipeline Template
jobs:
- job: Test_Job
pool:
name: AgentPoolName
steps:
- script: |
echo "$(Build.RequestedForEmail)"
echo "$(Build.RequestedFor)"
git config user.email "$(Build.RequestedForEmail)"
git config user.name "$(Build.RequestedFor)"
git config --global http.sslbackend schannel
echo '------------------------------------'
git clone -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" -b $(ToolsRepoBranch) --single-branch --depth=1 "https://PipelineTemplateProject/_git/CommonPipeline" DevOps_Tools
echo '------------------------------------'
displayName: 'Clone DevOps_Tools'
- task: PowerShell#2
displayName: 'Pipeline Debug'
inputs:
targetType: 'inline'
script: 'Get-ChildItem -Path Env:\ | Format-List'
condition: always()
- task: PowerShell#2
displayName: 'Run Powershell Scripts'
inputs:
targetType: filePath
filePath: 'DevOps_Tools/scripts/test.ps1'
arguments: "$(System.AccessToken)"
Notes:
Organization Setting - Settings - Disable Limit job authorization scope to current project for release pipelines
Organization Setting - Settings - Limit job authorization scope to current project for non-release pipelines
Check some option in project setting as well.
So the normal user only access their own repo, cannot access DevOps project, and DevOps owner can edit template pipeline only.
For the notification issue, I use an Email extention "rvo.SendEmailTask.send-email-build-task.SendEmail#1"
I have checked this link Azure DevOps: 1 Solution Multiple Projects CI/CD which is related to one solution with multiple project.
Can we use multiple solution in single CI CD pipeline ? where we have different artifacts for each solution and app servers to deploy.
Please advise.
As long as the code is in the same repository there are no issues to using multiple .net solutions or any other type.
You can also publish multiple artifacts from the same pipeline
If you are using YAML pipeline, you can check out multiple repositories in your pipeline.
Pipelines often rely on multiple repositories. You can have different repositories with source, tools, scripts, or other items that you need to build your code. By using multiple checkout steps in your pipeline, you can fetch and check out other repositories in addition to the one you use to store your YAML pipeline.
Repository declared using a repository resource :
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
- repository: MyAzureReposGitRepository
type: git
name: MyProject/MyAzureReposGitRepo
steps:
- checkout: MyGitHubRepo
- checkout: MyAzureReposGitRepository
Repository declared using inline syntax :
If your repository doesn't require a service connection, you can declare it inline with your checkout step.
steps:
- checkout: git://MyProject/MyRepo # Azure Repos Git repository in the same organization
For details ,please refer to this official document.
I only seem able to integrate my ADFv2 instance with public repos in my organisation, which is undesirable.
In the documentation (https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/data-factory/source-control.md#author-with-github-integration), it states that:
..can use both public and private GitHub repositories with Data Factory as long you have read and write permission to the repository..
So, i complete the following steps:
Navigate to my GitHub Org.
Create a new "Private" repo, initialised w/ README.MD.
Create a new "Internal" repo, initialised w/ README.MD. (this is GitHub Enterprise Cloud)
In the Azure Portal, create a new ADFv2 instance without Git integration.
Click "Set up code repository" in the ADFv2 instance.
Choose, RepoType: GitHub - Authenticate w. GitHub pop-up.
Enter org name in to the GitHub Account section.
Git repo name: No results found
🤷🏼♂️
The solution turned out to be the need to grant access as an Authorized OAuth App to the organisation. I did not have permission to do this.
You can then find it under 'Applications' once done.. but i think when you first connect up your ADF to GitHub, it prompts you if you would like to grant it.
I was not seeing this due to permissions on the Org.
I want to create a central point repository with templates for all my rust projects. And in other projects I just wanna add link to them:
here is an example link to my templates repository. You can find them here.
trigger: ["master"]
pr: ["master"]
resources:
repositories:
- repository: templates
type: github
name: xoac/rust-azure-pipelines
# Test top level crate
- template: azure-test-stable.yml#templates
parameters:
name: test_tokio
displayName: Test tokio
cross: true
And I am getting an error here
I have found here that I need specify service connection.
If you choose github as your type, then name is the full name of the GitHub repo including the user or organization. For example, Microsoft/vscode. Also, GitHub repos require a service connection for authorization.
I don't know what type of connection this should be.
I want to make it accessible to everyone who want to use it.
It would be a Github service connection (when you create new service connection)
Permissions are granted on build level, not on user level, so anyone who can launch the build would be able to launch it and it would work
for github repo name, i think you are using the correct one, but you would also need to add endpoint: service_connection_name to the repositories definition.
Reading:
https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema#repository-resource