gitlab checkout and status using Personal Access Tokens - gitlab

I am writing a php application that will need to commit and check status of files on a gitlab repository.
my application asks the user to create the personal Access Token on gitlab
All good so far.
I am using curl to run git commands and I am able to get the list of projects and branches on my project, but what I could not find is how to 'clone a repo' and check status on https://docs.gitlab.com/ee/api/README.html.
How do I check out/get status using personal access tokens?

Related

How to create a Git tag in Gitlab CI without using personal credentials?

I'm using GitLab Enterprise Edition 14.6.5-ee
I want to create a Git tag automatically when I merge a branch back to master. I'm fine with the actual Git commands; the problem is with the authentication: the build bot doesn't know how to authenticate back to the server. There's an answer here how to set up SSH keys. But this requires me to use my personal credentials, which is just wrong, because it's not me creating the tag; it's the build bot.
Seriously, it just doesn't make sense to say that the bot doesn't know how to authenticate. I mean, it just pulled the freakin' code from the repo! So why is it such a big leap from being able to pull code to being able to push code?
Any ideas how to automate the creation of tags without using my personal credentials?
CI jobs do have a builtin credential token for accessing the repository: the $CI_JOB_TOKEN variable. However this token only has read permissions, so it won't be able to create tags. To write to the repository or API, you'll have to supply a token or SSH key to the job. However, this doesn't necessarily have to be your personal token.
There are a few ways you can authenticate to write to the project without using a personal credential:
You can use project access tokens
You can use group access tokens -- these are only exposed in the UI after GitLab 14.7
You can use deploy SSH keys (when you grant read-write to the key)
So why is it such a big leap from being able to pull code to being able to push code?
This is probably a good thing. While it may require you to do extra work in this case, the builtin job authorization tries to apply the principle of least privilege. Many customers have even argued that the existing CI_JOB_TOKEN permissions are too permissive because they allow access to read other projects!
In any case, it is on GitLab's roadmap to make these permissions more controllable and flexible :-)
Alternatively, use releases
If you don't mind creating a release in addition to a tag, you could also use the release: keyword in the CI yaml as an easy way to create the tag.
It's somewhat of an irony that the releases API allows you to use the builtin CI_JOB_TOKEN to create releases (and presumably tags) but you cannot (as far as I know) use CI_JOB_TOKEN on the tags API to create a tag.
However, in this case, it will still have the effect that the releases/tag appear to be created by you.

Obtaining a "gitlab runner token"

I a gitlab repository that I would like to set up a gitlab runner for. In the documentation it states that I need to register the runner using the "registration token" from the gitlab website in the repository settings.
However, after issuing the "gitlab-runner start" command, I still need to create a config.toml file with the url and a "runner token" in order to be able to run the gitlab-runner run command, which I need to run a number of commands upon a push on a specified branch.
I am not sure where to find this "runner token". In the documentation it only states that the runner token is different from the registration token, but not where to obtain it.
I would highly appreciate any kind of help or suggestions.
The config.toml was not being created in the proper folder, see here

Deployment to Azure web app display Error: fatal: Authentication failed for 'https://xxx.scm.azurewebsites.net:443/xxx.git/' [duplicate]

I'm trying to clone my webapp in Azure.
When I run git clone https://username#appname.scm.azurewebsites.net:443/appname.git the terminal asks me for my password.
But when I fill in my password, it keeps saying that the authentication has failed.
Even though I changed my password plenty of times in the Portal (Settings -> Set deployment credentials).
Any reason it keeps saying that my authentication has failed?
Yes many of the answers here are pointing to the correct path, but my case:- I was given a url to access git repos on azure cloud for which I'm perfectly fine to access/browse the code on browser (Vs) when I'm trying to clone I've got the same issue.
So when you click on clone as shown in below image, you've got to Generate Git credentials; this is weird, not sure why, probably they've setup my account to access azure cloud, which can't be used to clone git repos(means can't be used as git credentials). This is something different from what I've seen with AWS/gitlab/bitbucket, maybe an issue with account setup, or if it's common thing for repos on Azure and if you're in same issue - give it a try.
Another one, as everyone has already mentioned, please don't try to connect to url which is produced over there (something looks like):
https://project-account#dev.azure.com/project-name/apps/_git/library/
Instead try to connect thru your GIT username and then use password:
https://username#dev.azure.com/project-name/apps/_git/library/
As a tip: if your username has special characters in it, Git cmd/bash will throw errors, so replace those with valid characters, ex. :-
# can be replaced with %40
+ can be replaced with %2B
something like : https://username%40xyz.com#dev.azure.com/project-name/apps/_git/library/
I had the same problem with my site and it turned out the issue is with the site url:
the automatically generated remote url was: https://user#site.scm.azurewebsites.net/site.git
On the other hand the portal showed: https://user#site.scm.azurewebsites.net:443/site.git
After updating the remote url in git with the following command:
git remote set-url azure
https://<user>#<site>.scm.azurewebsites.net/<site>.git
https://<user>#<site>.scm.azurewebsites.net:443/<site>.git
things started working as expected.
The morale of the story: check the deployment url as well as the password.
Accepted answer did not work for me but this worked.
Check your set Azure URL with
git config --get remote.<azure-remote-name>.url
Reset azure url with following command
git remote set-url <azure-remote-name>
https://<user>#<sitename>[_<slotName>].scm.azurewebsites.net:443/<site>.git
Try pushing your code using
git push <azure-remote-name>
It will open windows authentication screen which says to enter credential to connect to https://<user>#<sitename>[_<slotName>].scm.azurewebsites.net/<site>.git
Cancel this window. It will prompt the basic credential window which says to enter credential to connect to https://<user>#<sitename>[_<slotName>].scm.azurewebsites.net:433/<site>.git
Enter in your credential and it works.
Since azure does not support ssh you will have to use your password at least once (in some cases you will not be prompt for password next time)
Here is an article describe how to set it up
https://azure.microsoft.com/en-us/documentation/articles/web-sites-publish-source-control/
And here is how to set up git with Azure:
http://weblogs.asp.net/shijuvarghese/building-and-deploying-windows-azure-web-sites-using-git-and-github-for-windows
Here are the steps that you followed
If this is your first time setting up a repository in Azure, you need to create login credentials for it.
You will use them to log into the Azure repository and push changes from your local Git repository. From your web app's blade, click Settings > Deployment credentials, then configure your deployment username and password.
When you're done, click OK.
Now do this:
I had the same problem when try to git clone https://<username>#praat.scm.azurewebsites.net:443/<repo-name>.git . I solved the problem by removing the port.
So your new url should not have the port 443.
Click on Generate Credentials and then copy the password and paste it.
It works.
None of the above solution worked for me.
The reason why the same error was observed in my deployment was
because my password contained a character "#".
This malformed the Git clone url for my Azure web app. When I changed my password by removing #, it solved this issue. You can change # to %40 in your connection string as well.
Current Azure Structure for adding credential
Use the highlighted user, password, confirm password section to create an user credential. Use this to proceed where you were stuck.
Login into Azure Dev Ops portal.
Go to Project you wish to clone.
Select Repo from left menu.
Click on Clone from right top corner.
Click Generate Git Credential
Enter given username & password where you are cloning repository.
you are done!
Assuming you're entering the correct password each time, it might be a delay/propagation issue.
You can also use your Site-Level credentials as detailed on this wiki page. You can get these by downloading your publishing credentials in the Portal via the "Get Publish Profile" option on your main WebApp bar. The user name / password can then be found in the yourapp.PublishSettings file and will look something like userName="$yourapp" userPWD="ABC123". This is only mentioned as a workaround if your User-Level credentials are not working (which they should be if they are correct).
What worked for me is I created personal access token.
Click username in top right corner > Security > Select Personal Access token from left pane > New Token.
Provide token name and scope for your token and Click create.
Save that token and use it as password.
I had a similar problem where the automatically generated git clone url was:
https://group-admin-account-username#dev.azure.com/site.git
So instead I had to replace this with my own account username
https://my-username#dev.azure.com/site.git
This then prompted me for the password for my account correctly:
git clone https://my-username#dev.azure.com/site.git
I had to do it this way
Generate Git Credentials
git remote add origin https://username:password#dev.azure.com/xxx/_git/xxx
git add.
git commit -m init
push -u origin --all
I got this error in Visual studio because I used the sync feature when the branch hadn't been created in 'https://dev.azure.com/foo/bar' yet.
Instead of doing a sync, I did a push & then it prompted me for my credentials. After that it worked fine.
Hopefully this helps someone.
Adding token before dev.azure in URL settings worked for me. I am using SourceTree.
https://token#dev.azure.com/org/_git/repo
A straight forward method to get this over with is to Download/Get Publish profile (make sure you refresh the portal page) from main toolbar and use the password given in the .publishsettings file.
For me it was a matter of cloning the project with the remote url and cancelling the windows security prompt. Afterwards, it'll ask you for the password.
Once you're able to clone it, you can push it successfully.
Moral of the story: Clone it first before pushing.
On mine, the password was never created when I set the deployment user from the cli. I had to go to the web app, app services, for the app, then set the deployment credentials
Go to the Security Tab >> Personal Access Tokens >> New Toke >> Give it Full Access >> Note the Access token as you will use it as your password
this may help: with my first webapp, i had to go to Deployment Center, Deployment Credentials. And then use either the App Credentials or create User Credentials. After this, delete the cached credentials on the local machine (windows: Control Panel\User Accounts\Credential Manager). run "git push webapp master:master" again, enter either of the Deployment Credentials. That worked.
In my case none of the above methods solved my issue (but they directed me to find out where I'm going wrong)
I tried both auto generated App credentials and my custom created user credentials, and every time I was getting authentication failed message.
The description on the credentials page was advising me to have the username in the following format:
<app-name>\<username>
This was why authentication failed. As soon as I put my username without the <app-name> prefix the authentication passed and everything worked smoothly.
Hope this information helps
Navigate to the Deployment Center >> Deployment Credentials, there will be application credentials for local git. Use the username and password there will work.
Using SSH instead of HTTPS worked for me after adding my SSH key
Some of the above brought me close, such as the post by #Nilay Vishwakarma but I finally found the cause of my authentication issue to be that my password contained a '$' followed by a '..u7' which I eventually noticed in powershell was highlighted yellow when I typed.
I thus assumed it could be the CLi has an issue with this character in passwords (even though it wasn't mentioned as a reserved character in this tutorial).
I re-ran the 'az webapp deployment user set --user-name [user] --password [pwd]' command with a revised password without the '$' and was able to then enter my credentials and authenticate.
Using Mac I had the same issue. Using Git Credential manager worked.
Installed GCM using brew https://github.com/microsoft/Git-Credential-Manager-Core
Connected git to remote Azure Repo
On git push, pop up requested username and password
All connected.
Don't be surprised when you see PAT (Personal Access Token) in you settings.
"Once authenticated, the credential manager creates and caches a personal access token for future connections to the repo. Git commands that connect to this account won't prompt for user credentials until the token expires. A token can be revoked through Azure Repos."
Below link should help
https://learn.microsoft.com/en-us/azure/devops/repos/git/set-up-credential-managers?view=azure-devops
If you are using Sourcetree and you tried all of the answers here and nothing works.
try cloning your project again using this url structure as source:
https://<username>:<password>#dev.azure.com/project-name/apps/_git/library/
username and password comes from the Generate Git Credentials button when you clone your branch.
If you unable to clone the projects/repos,
Check proxy settings or VPN if provided
Check repository path access
Check your access or password expiry
If you are facing login issue,
Generate PAT (Personal Access token) for respective version control system like Git, Azure etc.
If PAT fails, create alternate credentials
If this both are right and still you are unable to access then
Goto your repository click on Clone button, there you can find Generate Credentials button. Try to login with those newly generated credentials.
Check for all above options first. If you are following everything and still facing issues then try to take help from colleagues/team leader/ manager or client.
I suddenly had the same problem. It turned out to be a problem with my local station and the Internet adapter being rejected leading to not authenticated. Disabling IPV6 solved the problem for me: See https://stackoverflow.com/a/69712045/7302498
I had a same problem because of saved login info in Macbook.
I entered Key Chain Access App of Mac.
Then, I found login info under login, opened it.
I updated password with my Credential Password of Azure Repo. (Click "Generate Git Credentials")
Done! Now, I can pull my branch in SourceTree.
In my case, I had to provide the Personal Access Token (PAT) that was created when I created a new Azure Organization, instead of any password. This PAT needs to be saved by you since it will not be available later after you close the PAT screen.
I had forgotten my PAT and had to regenerate it and provide it for the Git login, for this issue to resolve.

Gitlab CI Pipeline with Token

I created Personal Access Token in gitlab.com to read the gitlab api because I want to create some documentation about my repos in asciidoc automatically. Basically this works in a local script.
Now I want to turn this into a Giutlab CI pipeline. As of now the token is part of my local script. But I don't want this token to be readable in a public repo. Is there a way to get the token from the pipeline in a secure way without putting it into the .gitlab-ci.yml in plain text or any other reusable form?
Yes, in your gitlab project Settings > CI / CD > Variables
There you can add Key value pairs which can be "masked" so aren't visible in scripts.
But you should be aware this isn't fully "secure" take a look at this

Creating files in gitlab via api fails

I have admin token with all privileges but when I am trying to create files via api I am getting this message: You are not allowed to push into this branch. I checked and this repository master branch is not protected, I wonder what could be the case?
I forgot to add Sudo header, which is required when action is done on behalf of user.

Resources