I am trying to retrieve the file and folders from a gitlab project. I used the following command:
curl --header "PRIVATE-TOKEN: ${MY_TOKEN}" https://gitlab.com/api/v4/projects/${PROJECT_ID}
The problem is, my project is private. Even though I am a member of the project, I cant use my access token to retrieve the project information. But if I take out the id from the command, and search for all the projects, I get a list of all the public projects. Is there something I have to do differently to access the private projects?
Thanks.
Related
There is an unstable gitlab server and I am not sure that it will be able to work in the future. Therefore, I want to make a backup copy of all the repositories (projects) that are there.
Cloning the source code will be enough, but it will be great if there is a way to save issues as well. Are there any ways to do this?
It depends on what kind of access you have, but if you don't have administrator access to do a full backup, then the best thing to do is to use a couple of API endpoints to get the information you need and go from there.
Use the Projects API to get a list of all projects accessible to you.
Note the pagination limits.
What you store depends on how you want to get the information.
Store at least the ID number of each.
Filter by membership if you only want the ones you're a member of.
Filter by min_access_level = maintainer (or higher) if you want to export whole projects.
Use the Project export API to trigger a project export for each project you're a member of, and you're a maintainer (or higher).
For all other projects where you have a lower role, or where it's public, you could still use git clone for the repositories by storing the ssh_url_to_repo or http_url_to_repo from the Projects API and running through each.
For all other parts of a project, you could store the JSON version to recreate them later if you want to go through the hassle. For example, for issues, use the Issues API.
I am writing a script to automate the production of some trigger tokens for some projects in gitlab.
The script starts with a call to the gitlab group api:
curl --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "https://gitlab.com/api/v4/groups/12345 | jq > projects.txt "
This creates a list of projects.
Some projects are missing from this list. I know they are missing because I can see these missing projects via the gitlab web interface and they are clearly listed as part of my group (id = 12345; this is a dummy id for the purposes of this question).
Further, a call to the projects api for each specific missing project (using the same PRIVATE_TOKEN) successfully returns complete information about those projects.
Any input would be much appreciated!
Thanks,
Luke
You have to use pagination with the GitLab API to get complete results beyond the max page size.
Also note that if you have projects organized into subgroups, the Group List Projects API has a include_subgroups option (default false) that will need to be provided if you also want to list projects contained in subgroups.
I have a project filled with various pipeline templates that other projects can use for their work, keeping my CI DRY. These are not in a .gitlab-ci.yml file, but separate ./templates/language.yml files.
I'm already using yaml lint to make sure it is valid yaml, but I want to make sure I'm using valid GitLab CI syntax also.
I'd like to lint my CI templates when I'm merging. I have rejected running gitlab-runner exec shell because I can't figure out how to trigger specific copies. It looks like there might be something in the API with this, but I haven't been able to nail down the secret sauce.
How are you doing this?
We are using two different approach to achieve this.
via API - https://docs.gitlab.com/ee/api/lint.html
with a fake project setup within my templates
with gitlab-ci-local
via API
The first approach is using the linter from gitlab via API:
curl --header "Content-Type: application/json" --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/ci/lint" --data '{"content": "{ \"image\": \"ruby:2.6\", \"services\": [\"postgres\"], \"before_script\": [\"bundle install\", \"bundle exec rake db:create\"], \"variables\": {\"DB_NAME\": \"postgres\"}, \"types\": [\"test\", \"deploy\", \"notify\"], \"rspec\": { \"script\": \"rake spec\", \"tags\": [\"ruby\", \"postgres\"], \"only\": [\"branches\"]}}"}'
The problem here, is that you can not utilize the JOB_TOKEN to do this, therefore you need to inject a secret and generate a token to achieve this. there is even a linting library available - https://github.com/BuBuaBu/gitlab-ci-lint
fake project
The second approach mimics the setup, with an own .gitlab-ci.yml which includes the templates and executes it. Like normal merge request pipelines. This way we ensure the scripts do not have any failure and are save to use them. We do this for docker images as well for gradle build templates etc.
eg. for docker images we build the image, include the template, and overwrite the image property of the jobs to the temporary docker image.
gitlab-ci-local
The third approach is not as sufficient and depending on the feature, lacks functionality. There is the tool gitlab-ci-local https://github.com/firecow/gitlab-ci-local which can be used to verify gitlab ci builds and execute them. But it is not an official implementation and not all features are present. In the end you also need again some kind of project setup.
If i can choose i would go with the first approach. In our case it has proven to be useful. The initial effort of faking a project is not that much, for the benefit of a long term save solution.
I am trying to create a GitLab variable for a project on GitLab using a cURL command.
I am following the API and the relevant paragraph here.
Looking at the example on the docs, I cannot find any evidence on how Gitlab knows which repo I am trying to add the variable to.
I tried adding a private token and runnnig the command and I ended up with a 403.
Could someone please explain how to use the GitLab API to add variables to a repo through a cURL command?
Looking at the example on the docs, I cannot find any evidence on how Gitlab knows which repo I am trying to add the variable to
But... the all idea behind "instance-level" variables is to be "global", not tied to a specific project.
See issue 14108:
Global (instance-level) variables for CI builds
Less of an issue or more or a feature/question perhaps ... It would nice to support global variables in the CI builds; classic use-case being deployment keys / docker registry credentials.
Problem to solve
Without support for global variables users must manually enter the same credentials repeatedly for dozens of projects when migrating to GitLab for details like:
docker private registry credentials
kubernetes credentials)
deployment keys
Proposal
Implement support for global (instance-level) variables for CI builds by:
Re-use the refactor/re-design of CI variables in project/group settings
Place under CI/CD section in the admin settings
A better API for your case would be:
Create variable (project
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
"https://gitlab.example.com/api/v4/projects/1/variables" \
--form "key=NEW_VARIABLE" --form "value=new value"
I am a beginner with GitLab and GitLab api so there is alot of basic knowledge of that i might be missing.
I have a synology server where i want to make backups of the GitLab projects. Due to network limitations there is no GitLab Installation on Synology therefore i want to back-up the gitlab projects in this way.
Idea: Using GitLab api i want to extract a list of GitLab project ID's, loop through these id's and export and download the project(s) to a location on Synology.
The main problem i am having right now is getting all the Project ID's in a list that i can loop through or any similar data structure.
Does any of you have any idea about how i could accomplish this?
At the moment i am trying out getting all the projects owned using the following api call:
curl --header "PRIVATE-TOKEN: MyToken" "https://gitlab.example.com/api/v4/projects?owned=true"
At the moment i am just getting an empty pair of square brackets as a result. I am not sure why that is yet as i am kind of figuring things out as i go.
Is it possible to parse the information about the projects owned to extract only the project ID's?
Try adding --header 'Accept: application/json' to your curl command.
Try This instead:
curl --header "PRIVATE-TOKEN: MyToken" "https://gitlab.example.com/api/v4/projects"