Is it possible for GitLab project to generate "Deploy Token" remotely, via command line?
I'd like to automate creation of my projects.
Not via the GitLab API at the moment.
There is an open issue which is looking to add a set of new API endpoints that will allow administrators to create, list and revoke GitLab Deploy Tokens.
Related
Context & Problem
I'm working on a DevOps platform for a company which wants to provide developers with tools for code versioning & automatic testing / build / deploy, using Gitlab + Jenkins + Azure (alongside other tools like Jira / Sonarqube / Grafana, but not relevant here).
For security reasons & ease of management, we want to provide users with only one account for all those tools : Azure Active Directory.
Currently, Gitlab users can connect through Azure AD, and a push to the Gitlab repository triggers a build in Jenkins.
When I configure Jenkins authentication through Azure AD, the Gitlab integration with Jenkins doesn't work anymore, since you can no longer use username / password to connect to Jenkins, and it seems the only way provided by Gitlab integrations.
Current Setup
Gitlab (14.6.0-ee) & Jenkins (2.319.1) are installed on 2 different virtual machines (CentOs 7.9) hosted on Azure
Jenkins integration is enabled on Gitlab (https://docs.gitlab.com/ee/integration/jenkins.html)
Gitlab plugin is enabled on Jenkins (https://plugins.jenkins.io/gitlab-plugin/)
Jenkins integration with Azure Active Directory works fine (https://plugins.jenkins.io/azure-ad/)
Groups & Users are configured on Azure AD to give roles on Jenkins (https://www.cloudbees.com/blog/securing-jenkins-role-based-access-control-and-azure-active-directory)
✅ When Azure AD is not enabled on Jenkins, Gitlab triggers build after each push, and you can track every step status in Gitlab
❌ When Azure AD is enabled on Jenkins, Gitlab cannot trigger Jenkins (using Webhooks or official Jenkins integration)
Questions
Is there a way for Jenkins to provide multiple auth strategies depending on the user ? (as Gitlab does)
Is there a way for Gitlab to use something else that user / password for Jenkins integration or Webhooks ?
Do we have to choose between Azure AD & Gitlab integration ? (if so, does this mean no one uses Jenkins Single Sign On & Gitlab ? Seems weird)
Leads & Ideas
Tried using webhooks instead of the Jenkins integration, but it fails since the webhooks also need username / password (and integration would probably be less "advanced")
Tried finding a way to enable multiple auth methods on Jenkins, but seems impossible right now (https://issues.jenkins.io/browse/JENKINS-15063)
Tried giving role permissions to Gitlab on Azure AD, but it won't change anything since I can't find how to make Gitlab using OAuth with Jenkins
I searched thorougly for answers, but still may have missed something trivial.
I didn't find much litterature on this specific setup, so I'm hoping someone around here did encounter this type of situation before 🙂
Thank you for your help
I have come across this problem that when I queue a build of my flutter code in Azure Pipeline, one of the packages in the pubspec.yaml tries to access a Private Azure repo for one of it's plugin. This is throwing a build error. So how can I solve this in the pipeline.
Thanks in advance.
You can go to the private azure repo, Click Clone and then Click Generate Git Credentials.
Then you will get a username and password. You can use the username and password to access this private repo. For below example:
git clone https://username:password#dev.azure.com/{org}/{proj}/_git/{repo}
You can also use Personal access token(PAT) with Code read scope to access to private azure repo
git clone https://PAT#dev.azure.com/{org}/{proj}/_git/{repo}
Please check here to get a Person access token.
Hope above helps!
The question is about Authentication in Build Pipelines, not for cloning the project. Cloning the project is fine whatever mechanism you consider it works. But how does the Build machine that is residing in cloud knows to authenticate?
So I solved this issue by following these steps:
1) create a PAT token for my account in Profile> Security tokens> personal access token
2) define the scopes / permission and copy that PAT token into my local machine (notepad)
3) use the private dependencies in pubspec yaml
Eg:
plugin_name:
git:
url: https://PAT#dev.azure.com/{org}/{proj}/_git/{repo}
This workflow is already automated and officially supported. You need to install the official Azure Pipeline in your GitHub account via Marketplace. This will trigger an OAuth authentication workflow to connect your Azure Devops and GitHub account.
After successful authentication you will see a GitHub installation token in your Azure DevOps Service Connections.
After this you will be able to access private repo securely in your pipeline.
You can achieve it by using an SSH key with the following steps,
Generate an ssh key pair
Add a public key to the github repo Upload
Private key to the Azure DevOps secure files
Configure the Azure DevOps pipeline via YAML
Here is a sample
My goal is to automatically register a shared Gitlab runner on our hosted Gitlab. To do this, I need to obtain the runners token via the Gitlab API.
Unfortunately, I haven't found a point in the API to fetch the shared runners token. On the website, the token is shown in Admin area / Overview / Runners / Set up a shared Runner manually.
As far as I know, Gitlab has 3 different types of runners token:
Specific (assigned to projects)
Group (assigned to a group)
Shared (for unassigned projects)
I am able to access the runners_token in the project details and the group details but I haven't found a place to obtain the shared runners_token.
I am thankful for every help!
Without an API endpoint that supports this, here's an alternative solution. The command has to be run on the server hosting your Gitlab instance. The line below will output the current shared runner registration token.
sudo gitlab-rails runner -e production "puts Gitlab::CurrentSettings.current_application_settings.runners_registration_token"
I have setup build/test/release on VSTS. And we have another pipeline (in Linux) to get the build artifact from the build system with a give build id. So is there a way to download the build artifact from VSTS on Linux? I found that you can't download it without login. We used to use Jenkins. It works good as Jenkins doesn't require login for that.
No, authentication is required, you can create personal access tokens to authenticate access.
You also can build a API app to get artifacts with authentication, then another pipeline can get build artifact from that API app.
In my current company we use GitHub Enterprise as revision control. Where I sign in with SAML to access my account. Now I want to deploy code that is there in GitHub to my Azure App service. So in azure deployment options, I selected GitHub option. But it is saying "Azure needs your permission to access your GitHub account" and asking me to login into GitHub. But I do not have GitHub user name and password as we are using SAML to access GitHub account. Could you please help me how can I authorize GitHub account with Azure?
Azure GitHub deployment process
The Azure deployment process is kinda tricky, trying to deploy from Github Enterprise, I manage to get it working like this:
In the Azure project menu, click on Properties then for the DEPLOYMENT TRIGGER URL, copy it.
Go to the project settings in Github Enterprise, on the option Hooks & Services and add a new webhook, set the DEPLOYMENT TRIGGER URL on the Payload URL, leave everything else the same.
Now we add the deployment process, in the Azure project menu click on Deployment Options then choose as source External Repository
On your Github Account go to settings, then on the menu click on Personal Access Tokens, now generate a new token and give it the following scopes: repo and admin:pre_receive_hook then copy the token
Finally, set the main branch that will activate the webhook when you push code on the repo, on the repository URL we have to set the following items:
https://<github_access_token>:x-oauth-basic#github.<enterprise>.net/<repo_name_from_root>
something like this:
https://c4207e6aae44ce086595c9abfcccb5123caf20cc:x-oauth-basic#github.mycompany.net/repolocation/deployrepo
Now push on your repo and the webhook will automatically trigger the deployment and build process.