I have a inventory.txt in one folder of my azure repos. I need to copy that to another folder. I have used "Copy File task" but it copying file to the required folder only in agent machine.Its not reflecting in azure repos. What Can I do.
My Main Task is to give a packages_list variable in a inventory file. But this variable is being used by two yaml files which are two different folders and used for two different pipeline. For that I have declared a packages_list variable in one of the folder and copy to another folder. ANy other alternativeees are much appreciated.
After copy the files to target folder, you need run git command to push changes to sync the changed folder back into Azure devops repo.
Please try with below steps to configure your pipeline:
(1) The first Command line task:
git config --global user.email "xxx#xx.com"
git config --global user.name "Merlin"
cd $(Build.SourcesDirectory)
git init
(2) In second task, execute the Copy file task.
(3) In next Command line task, do git push to push the changes:
git add .
git commit -m "File copied"
git remote rm origin
git remote add origin https://xxx#dev.azure.com/xxx/xxx/_git/xxxx
git push -u origin HEAD:master
Since above command is modifying Azure repos, please enable Allow scripts to access the OAuth token in agent job and ensure the corresponding Build service account has Contribute permission to Git repos. Just follow this doc to configure the permission setting.
Related
I have created a release pipeline in Azure devops to generate the .dacpac file using CMD task in release pipeline. The .dacpac file is getting generated successfully when I am giving \tf: path as $(Agent.HomeDirectory). I want to give path of the folder in my repository where I would like to save the .dacpac file.
Below is the command which I tried to store the file on (Agent.HomeDirectory) and it runs fine.
C:\Program Files (x86)\Microsoft SQL Server\150\DAC\bin>sqlpackages.exe /Action:Extract /ssn:tcp:,1433 /sdn: /su: /sp: /tf:$(Agnet.HomeDirectory)/test.dacpac /p:Storage=File
Please guide me how we can store the file to repository folder from command line.
Thanks,
Sachin
Please refer to the following steps:
I created a release pipeline and added this repo: testproject, branch: test
Add CMD task, the script is as follows:
cd D:\a\r1\a_testproject
cd . > 001.txt // I create a test txt here. Please copy the file you need to the path. Or you can to use git to clone a repo in the path you want.
git config --global user.email "xxxx#mail.com"
git config --global user.name "xxxx"
git status
git add -A
git status
git commit -m "Latest updated"
git show-ref
git push -u origin HEAD:test //if the push step show the error: src refspec xxxx does not match any. please check the log and choose a rignt head
Allow scripts to access the OAuth token
set permission of the account {projectname} Build Service (orgname)
For more info ,please refer Copy file from CI Pipeline to Azure Devops Repository and How to add/update files in git repository from Azure DevOps Pipeline Dynamics 365 CE.
For one of my project the TestResults.zip file is publishing on url https://dev.azure.com/MyProject/Project/_TestManagement/Runs?runId=8309822&_a=runCharts.
I want to change storage location for TestResults.zip file from above given URL to my own defined repository location.(Like: Myproject/target/surefire-reports.zip) How to do that?
Because in my azure pipeline the test are running and when it comes to create a zip for TestResults it's storing in given above URL and i want to store in one of my project sub-module under target directory so that i can create a zip file.
Please help me to resolve this issue.
Many Thanks
How to modify the repository / storage location for Test Results in azure pipeline
It is not recommended to publish the test results to the repo.
If you still want do that, you could use git command line to submit the test file to the repo in the pipeline after the test task, like:
cd $(System.DefaultWorkingDirectory)
copy Path\surefire-reports.zip target
cd target
git add surefire-reports.zip
git commit -m "Add a test file"
git push https://PATn#dev.azure.com/YourOrganization/YourProject/_git/xxx.test HEAD:master
You could check the similar thread for some more details.
I'm writing GitLab CI/CD pipeline script in .gitlab-ci.yml
I want to check if a specific file changed in another repo and if so I would like to copy the file, commit and push to the current repo.
everything works until I get to the 'git push' part
I tried several ways to fixed it:
stages:
- build
build:
stage: build
script:
- echo "Building"
- git checkout -b try
- git remote add -f b https://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab.{otherRepo}.git
- git remote update
- CHANGED=$(git diff try:mobile_map.conf b/master:mobile_map.conf)
- if [ -n "${CHANGED}" ]; then
echo 'changed';
FILE=$(git show b/master:mobile_map.conf > mobile_map.conf);
git add mobile_map.conf;
git commit -m "updating conf file";
git push;
else
echo 'not changed';
fi
- git remote rm b
for this code I get :
fatal: unable to access 'https://gitlab-ci-token:[MASKED]#gitlab.{curr_repo}.git/': The requested URL returned error: 403
also I tried to add this line in the beginning :
git remote set-url origin 'https://{MY_USER_NAME}:"\"${PASSWORD}\""#gitlab.{curr_repo}.git'
and I get this error message:
fatal: Authentication failed for 'https://{MY_USER_NAME}:"\"${PASSWORD}\""#{curr_repo}.git/'
also I added:
- git config --global user.name {MY_USER_NAME}
- git config --global user.email {MY_EMAIL}
please help me,
Thanks
Job-tokens only have read-permission to your repository.
A unique job token is generated for each job and provides the user read access all projects that would be normally accessible to the user creating that job. The unique job token does not have any write permissions, but there is a proposal to add support.
You can't use deploy-tokens because they can't have write-access to a repository (possible tokens).
You could use a project-access-token with read-write-access to your repository.
You can use project access tokens:
On GitLab SaaS if you have the Premium license tier or higher. Project
access tokens are not available with a trial license.
On self-managed instances of GitLab, with any license tier. If you
have the Free tier: [...]
Then you can use your project-access-token as an environment variable in the url.
git push "https://gitlab-ci-token:$PROJECT_ACCESS_TOKEN#$CI_SERVER_HOST/$CI_PROJECT_PATH.git"
At least that's how we use it in our pipelines.
I hope this helps you further.
Complete example of simple GitLab CI stage, that commits back to its own repo
ci_section_name:
# N.B. This stage produces an extra commit to the repo!
stage: stage_name
script:
- apt-get update && apt-get install -y git
- echo "hello" >> file_to_be_modified.txt # your real job may do smth else
after_script:
- git config user.name "Name On Your Choice"
- git config user.email "email_on_your_choice#$CI_SERVER_HOST"
- git pull "https://project_access_token_name:$PROJECT_VARIABLE_WITH_ACCESS_TOKEN_VALUE#$CI_SERVER_HOST/$CI_PROJECT_PATH.git" $CI_COMMIT_BRANCH --rebase --autostash
- git commit -a -m "Message on your choice"
- git push "https://project_access_token_name:$PROJECT_VARIABLE_WITH_ACCESS_TOKEN_VALUE#$CI_SERVER_HOST/$CI_PROJECT_PATH.git" HEAD:$CI_COMMIT_BRANCH
I want to stop on few important aspects:
file_to_be_modified.txt - any modified files in the example are supposed to already exist in the repo, if you need add smth new, you will need
include at least git add command
project_access_token_name - is the name of used Project Access Token, not the token value itself (see screenshots below); you should create it by hands for your GitLab project if necessary
PROJECT_VARIABLE_WITH_ACCESS_TOKEN_VALUE - is the name of Project Variable (see screenshots below), you should create this variable by hands for your GitLab project and populate it with value of chosen Project Access Token; btw, you can add literal value of the token into stage code instead, but it is probably bad for security reasons
--rebase --autostash - note that autostashing your changes instead of simply trying to pull-push them will help you to guarantee the push in case of conflicts (even our new GitLab stage may conflict between its launches) but files would be left with conflict trace instead of correct content; because it is hard to resolve conflicts automatically, it is supposed that you control such situation with another tools (for example, further build on conflict state will simply fail)
This simple example will lead to an infinite flow of commits, probably real stage should contain some only conditions, etc.
Screenshots
GitLab Project Access Tokens page
GitLab Project CI/CD Settings menu, where to find Variables settings
Add project Variable pop-up
RESUME
When I push from my local workspace to the target repo in GitLab on my remote VPS, I want GitLab run a script and ask a beta repo on the same VPS to git checkout and pull in order to check my changes.
Current configurations
Gitlab configurations
Suppose you already have a project in your admin area, you just need to get it's relative path
Admin area > Projects
Select your project
Get the path in the project profil (hashed for this case)
Create a new directory in this location called custom_hooks.
sudo su
cd /var/opt/gitlab/git-data/repositories/hashed/path/of/project
mkdir custom_hooks
Inside the new custom_hooks directory, create a file with a name matching the hook type. For a post-receive hook the file name should be post-receive with no extension.
cd custom_hooks
nano post-receive
Write the code to make the server hook function as expected. Hooks can be in any language. Ensure the ‘shebang’ at the top properly reflects the language type. In that case the script code used is :
#!/bin/sh
unset GIT_INDEX_FILE
cd /var/repo/beta.git && git checkout master && git up # --[see the note 2 below]
Make the hook file executable and make sure it’s owned by Git.
chmod +x post-receive
Note 1 :
You can find more informations about git hooks here : GitLab Documentation : Server hooks
VPS configurations
Create new alias with #RichardHansen recommandations
git config --global alias.up '!git remote update -p; git merge --ff-only #{u}'
Note 2 :
During my researches, I've found an interesting answer about git pull on the forum.
I've decided to follow that advice and I made an alias named git up.
What is important is that :
it works with all (non-ancient) versions of Git,
it fetches all upstream branches (not just the branch you're currently working on), and;
it cleans out old origin/* branches that no longer exist upstream.
You can find more informations about "In what cases could git pull be harmful ?" here :
Link to answer
Create a directory for git repos only and access it to process Hooks configurations
# Create a repo for the project in apache area
mkdir /var/www/beta
# Create the git repo only folder
cd /var
mkdir repo && cd repo
# Create git repo and init
mkdir beta.git && cd beta.git
git init --bare # --bare means that our folder will have no source files, just the version control.
# Add gitlab remote
git remote add gitlab
# Accessing hooks folder to create script
cd hooks
cat > pre-receive
# On the blank line, write this script then 'ctrl + d' to save and press enter to exit
#!/bin/sh
unset GIT_INDEX_FILE
git --work-tree=/var/www/beta --git-dir=/var/repo/beta.git checkout -f
# Make the file executable
chmod +x post-receive
Note 3 :
'git-dir' is the path to the git repository. With 'work-tree', we can define a different path to where the files will actually be transferred to.
The 'post-receive' hook will be looked into every time a push is completed and will set the path where the files will be transferred to /var/www/beta in that case.
Local Workspace configurations
# Create in your workspace a folder to hold the project
cd /path/to/workspace
mkdir project && cd project
# Initialize git and add gitlab remote
git init
git remote add gitlab ssh://user#mydomain.com/gitlab/path/of/project
# Create an index.html file and send the initial commit
nano index.html
# copy this into the file then 'ctrl + x' then 'y' then 'enter' to save
<html>
<head>
<title>Welcome to Beta domain!</title>
</head>
<body>
<h1>Success! The beta virtual host is working!</h1>
</body>
</html>
# prepare the changes and then send the commit
git status
git add index.html
git commit -m "chore: add index.html :tada: :rocket:"
git push gitlab master
EXPECTED RESULTS
The expected result of this process is that when the git push gitlab master is done, the hook inside the gitlab hashed directory of the project, run a script who make something like this :
# Access the beta.git directory
cd /var/repo/beta.git
# Run command for updating repo
git checkout master && git up
# If we access the beta folder in apache area we should see index.html
cd /var/www/beta
ls
--index.html
ACTUAL RESULTS
No result.
ERROR MESSAGES
No error messages.
REQUEST
How can I set up a process like this one ?
There is something in my process I did not take in consideration ?
I'm trying to use a remote trigger for (re)building in ci.gitlab. For explaining this, I made up this scenario:
2 repository, "lib" and "app1"
app1 will successfully build only if lib is included (solved simply by .gitlab-ci.yml)
I need to trigger the build of app1 (only for the master branch, in best-case) on commit (or merge request) of lib
I tried to figure it out using web hooks, but I wasn't able to find a url for ci.gitlab.com. Is this possible in a gitlab environment?
You can do this with newly added triggers functionality.
In your CI's project, find the section "Triggers". Add a trigger and use its token like this:
curl -X POST \
-F token=TOKEN \
https://ci.gitlab.com/api/v1/projects/{project_id}/refs/REF_NAME/trigger
(https://about.gitlab.com/2015/08/22/gitlab-7-14-released/)
Obsolete:
we have the same problem, and the way we solved it is by pushing and subsequently deleting a tag.
The assumption is that you manage the machine with Gitlab-CI runner. First, clone the main repository, app1 for you. And in lib's .gitlab-ci.yml add the steps:
- cd /path/to/app1_repository
- git pull
- git tag ci-trigger master
- git push origin ci-trigger
- git push --delete origin ci-trigger
- git tag -d ci-trigger
Make sure that you have the option Tag push events checked in your Gitlab Services settings for Gitlab-CI.
This solution has drawbacks:
Gitlab-CI runner must have write permissions to the repository, so it won't work for shared runners
git history will be bloated with all this tagging (especially Gitlab UI)
I opened an issue for this (https://gitlab.com/gitlab-org/gitlab-ci/issues/223) so let's hope they add this functionality to the API (http://doc.gitlab.com/ci/api/README.html).