Scenario: I've created two repositories inside my GitLab account: 'work' & 'project'. There are multiple branches in each of the following repos. I'm currently in the project repo and have created the .gitlab-ci.yml file.
Task: I want to copy two files 'A' & 'B' from the feature branch of 'work' repository to the current location (i.e., in the root of my project repository).
Thanks in advance :)
It's very easy, just add it to your script section
- git clone https://user:password#gitlab.com/group/project.git
Or to download specific files
- curl --header 'Private-Token: <your_access_token>' https://gitlab.com/api/v4/projects/:id/repository/files/:filename\?ref\=:ref
Also it will be a great practice to add Private-Token to project masked varible
You can get access token here
Related
For one of my project the TestResults.zip file is publishing on url https://dev.azure.com/MyProject/Project/_TestManagement/Runs?runId=8309822&_a=runCharts.
I want to change storage location for TestResults.zip file from above given URL to my own defined repository location.(Like: Myproject/target/surefire-reports.zip) How to do that?
Because in my azure pipeline the test are running and when it comes to create a zip for TestResults it's storing in given above URL and i want to store in one of my project sub-module under target directory so that i can create a zip file.
Please help me to resolve this issue.
Many Thanks
How to modify the repository / storage location for Test Results in azure pipeline
It is not recommended to publish the test results to the repo.
If you still want do that, you could use git command line to submit the test file to the repo in the pipeline after the test task, like:
cd $(System.DefaultWorkingDirectory)
copy Path\surefire-reports.zip target
cd target
git add surefire-reports.zip
git commit -m "Add a test file"
git push https://PATn#dev.azure.com/YourOrganization/YourProject/_git/xxx.test HEAD:master
You could check the similar thread for some more details.
I'm writing GitLab CI/CD pipeline script in .gitlab-ci.yml
I want to check if a specific file changed in another repo and if so I would like to copy the file, commit and push to the current repo.
everything works until I get to the 'git push' part
I tried several ways to fixed it:
stages:
- build
build:
stage: build
script:
- echo "Building"
- git checkout -b try
- git remote add -f b https://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab.{otherRepo}.git
- git remote update
- CHANGED=$(git diff try:mobile_map.conf b/master:mobile_map.conf)
- if [ -n "${CHANGED}" ]; then
echo 'changed';
FILE=$(git show b/master:mobile_map.conf > mobile_map.conf);
git add mobile_map.conf;
git commit -m "updating conf file";
git push;
else
echo 'not changed';
fi
- git remote rm b
for this code I get :
fatal: unable to access 'https://gitlab-ci-token:[MASKED]#gitlab.{curr_repo}.git/': The requested URL returned error: 403
also I tried to add this line in the beginning :
git remote set-url origin 'https://{MY_USER_NAME}:"\"${PASSWORD}\""#gitlab.{curr_repo}.git'
and I get this error message:
fatal: Authentication failed for 'https://{MY_USER_NAME}:"\"${PASSWORD}\""#{curr_repo}.git/'
also I added:
- git config --global user.name {MY_USER_NAME}
- git config --global user.email {MY_EMAIL}
please help me,
Thanks
Job-tokens only have read-permission to your repository.
A unique job token is generated for each job and provides the user read access all projects that would be normally accessible to the user creating that job. The unique job token does not have any write permissions, but there is a proposal to add support.
You can't use deploy-tokens because they can't have write-access to a repository (possible tokens).
You could use a project-access-token with read-write-access to your repository.
You can use project access tokens:
On GitLab SaaS if you have the Premium license tier or higher. Project
access tokens are not available with a trial license.
On self-managed instances of GitLab, with any license tier. If you
have the Free tier: [...]
Then you can use your project-access-token as an environment variable in the url.
git push "https://gitlab-ci-token:$PROJECT_ACCESS_TOKEN#$CI_SERVER_HOST/$CI_PROJECT_PATH.git"
At least that's how we use it in our pipelines.
I hope this helps you further.
Complete example of simple GitLab CI stage, that commits back to its own repo
ci_section_name:
# N.B. This stage produces an extra commit to the repo!
stage: stage_name
script:
- apt-get update && apt-get install -y git
- echo "hello" >> file_to_be_modified.txt # your real job may do smth else
after_script:
- git config user.name "Name On Your Choice"
- git config user.email "email_on_your_choice#$CI_SERVER_HOST"
- git pull "https://project_access_token_name:$PROJECT_VARIABLE_WITH_ACCESS_TOKEN_VALUE#$CI_SERVER_HOST/$CI_PROJECT_PATH.git" $CI_COMMIT_BRANCH --rebase --autostash
- git commit -a -m "Message on your choice"
- git push "https://project_access_token_name:$PROJECT_VARIABLE_WITH_ACCESS_TOKEN_VALUE#$CI_SERVER_HOST/$CI_PROJECT_PATH.git" HEAD:$CI_COMMIT_BRANCH
I want to stop on few important aspects:
file_to_be_modified.txt - any modified files in the example are supposed to already exist in the repo, if you need add smth new, you will need
include at least git add command
project_access_token_name - is the name of used Project Access Token, not the token value itself (see screenshots below); you should create it by hands for your GitLab project if necessary
PROJECT_VARIABLE_WITH_ACCESS_TOKEN_VALUE - is the name of Project Variable (see screenshots below), you should create this variable by hands for your GitLab project and populate it with value of chosen Project Access Token; btw, you can add literal value of the token into stage code instead, but it is probably bad for security reasons
--rebase --autostash - note that autostashing your changes instead of simply trying to pull-push them will help you to guarantee the push in case of conflicts (even our new GitLab stage may conflict between its launches) but files would be left with conflict trace instead of correct content; because it is hard to resolve conflicts automatically, it is supposed that you control such situation with another tools (for example, further build on conflict state will simply fail)
This simple example will lead to an infinite flow of commits, probably real stage should contain some only conditions, etc.
Screenshots
GitLab Project Access Tokens page
GitLab Project CI/CD Settings menu, where to find Variables settings
Add project Variable pop-up
I have a inventory.txt in one folder of my azure repos. I need to copy that to another folder. I have used "Copy File task" but it copying file to the required folder only in agent machine.Its not reflecting in azure repos. What Can I do.
My Main Task is to give a packages_list variable in a inventory file. But this variable is being used by two yaml files which are two different folders and used for two different pipeline. For that I have declared a packages_list variable in one of the folder and copy to another folder. ANy other alternativeees are much appreciated.
After copy the files to target folder, you need run git command to push changes to sync the changed folder back into Azure devops repo.
Please try with below steps to configure your pipeline:
(1) The first Command line task:
git config --global user.email "xxx#xx.com"
git config --global user.name "Merlin"
cd $(Build.SourcesDirectory)
git init
(2) In second task, execute the Copy file task.
(3) In next Command line task, do git push to push the changes:
git add .
git commit -m "File copied"
git remote rm origin
git remote add origin https://xxx#dev.azure.com/xxx/xxx/_git/xxxx
git push -u origin HEAD:master
Since above command is modifying Azure repos, please enable Allow scripts to access the OAuth token in agent job and ensure the corresponding Build service account has Contribute permission to Git repos. Just follow this doc to configure the permission setting.
I am working on a micro-services project, each service has its own pipeline because it gets deployed to a server of its own, we have each project in its own repository on gitlab with its own .gitlab-ci.yml but I want to collect all of these services in a single repository to make them easier to maintain and trigger a deployment of all the services when a commit is pushed.
The issue is I don't want to have a big fat yaml file that contains the build & deployment process of each service but instead keep the yaml files in the services folders and have a yaml file on the root that references them, i.e.:
| service1
| service1-code
| .gitlab-ci.yaml << build process for service1
| service2
| service2-code
| .gitlab-ci.yaml << build process for service2
| .gitlab-ci.yaml << reference to service1/yaml & service2/yaml
Is that doable?
There is currently no way for GitLab to do this, and there is an open issue to add this feature for monorepos.
(...) keep the yaml files in the services folders and have a yaml file on the root that references them
Just found this comment on GitLab by robindegen.
We create separate repositories, and a parent main repository that has
them as submodules. This works just fine for doing CI on subsets. When
we push to master on the main repo (to update the submodules), a full
CI run is done on everything, including integration tests.
I recon a CI clone includes submodules so this would just work. So if you already have a repo per project; have your cake and eat it too!
I'm trying to use a remote trigger for (re)building in ci.gitlab. For explaining this, I made up this scenario:
2 repository, "lib" and "app1"
app1 will successfully build only if lib is included (solved simply by .gitlab-ci.yml)
I need to trigger the build of app1 (only for the master branch, in best-case) on commit (or merge request) of lib
I tried to figure it out using web hooks, but I wasn't able to find a url for ci.gitlab.com. Is this possible in a gitlab environment?
You can do this with newly added triggers functionality.
In your CI's project, find the section "Triggers". Add a trigger and use its token like this:
curl -X POST \
-F token=TOKEN \
https://ci.gitlab.com/api/v1/projects/{project_id}/refs/REF_NAME/trigger
(https://about.gitlab.com/2015/08/22/gitlab-7-14-released/)
Obsolete:
we have the same problem, and the way we solved it is by pushing and subsequently deleting a tag.
The assumption is that you manage the machine with Gitlab-CI runner. First, clone the main repository, app1 for you. And in lib's .gitlab-ci.yml add the steps:
- cd /path/to/app1_repository
- git pull
- git tag ci-trigger master
- git push origin ci-trigger
- git push --delete origin ci-trigger
- git tag -d ci-trigger
Make sure that you have the option Tag push events checked in your Gitlab Services settings for Gitlab-CI.
This solution has drawbacks:
Gitlab-CI runner must have write permissions to the repository, so it won't work for shared runners
git history will be bloated with all this tagging (especially Gitlab UI)
I opened an issue for this (https://gitlab.com/gitlab-org/gitlab-ci/issues/223) so let's hope they add this functionality to the API (http://doc.gitlab.com/ci/api/README.html).