Is it possible to configure Microsoft AppCenter to set build frequency on "Build this branch on every push" but to build if commit message contains a specific string for each push.
you can add pre-build script, do your check then call exit 1 if needed,
reference: https://github.com/microsoft/appcenter/blob/1574250f1947cf2f595d1b63414199b167087bf0/sample-build-scripts/xamarin/app-displayname/appcenter-pre-build.sh#L64
You can use the following logic for this :
if ("git show $gitCommit || grep "regex" == false)
// { your code } do not deploy
else
// deploy
Related
I have a some spark jobs which are written on scala and build using maven.
Right now the follow a path like this
/src/job1/<<build data>>
/src/job2/<<build data>>
.....
This build data holds the code, pom.xml, tests and other thinks needed for each job pass to maven.
We have a root folder that has a father pom.xml that builds all the code, but I don't want to build all the code every time. I want to be able to test and build only the code for the jobs that changes.
The only way that I think is possible is to run build an tests only in the folders that changes was detected but I can't seem to do that without create one azure devops build for each folder, any ideas?
TL:DR - How to pass the path of folder that have changes in azure devops to the build process?
How to pass the path of folder that have changes in azure devops to the build process?
We could add a powershell scripts to check the folder or path, like:
$editedFiles = git diff HEAD HEAD~ --name-only
echo "$($editedFiles.Length) files modified:"
$editedFiles | ForEach-Object {
echo $_
Switch -Wildcard ($_ ) {
'/src/job1' {
Write-Output "##vso[task.setvariable variable=job1]True"
}
'/src/job2' {
Write-Output "##vso[task.setvariable variable=job2]True" }
}
}
This script sets variables which are then referenced in custom conditions in the pipeline build task or in the build pipeline:
and(succeeded(), eq(variables['job1'], 'True'))
You could check this similar thread for some details.
Hope this helps.
You can work around this by creating a script that will do the following:
Get the changeset/commit in the build to check which files are changed.
Based on the files changes, you define the logic to decide which folder it should build, run tests, etc.
This solution will require you to build all the logic to decide which folders to build and test.
Thanks for your inputs. I was able to do it with your inputs on git, I realized that we are using gitflow workflow and every time that we have pull request to the master branch (meaning deployment) a commit is created with starting "Merged PR", this way I can compare the latest two commits and check what changed.
How I'm doing:
# This part uses this to get the latest two pull requests hashes
$last_pr_commit=git log --grep="^Merged PR" -n 1 --pretty=format:"%h" --abbrev=200
$second_last_pr_commit=git log -n 1 --grep="^Merged PR" --oneline --abbrev=200 $last_pr_commit^...HEAD~2 --pretty=format:"%h"
# This get the path of the files
$result=git diff $last_pr_commit $second_last_pr_commit --name-only
And then I just build the ones that I'm interested.
ForEach ($line in $($result -split "`r`n"))
{
# In my case I only need the two first parts of path
$folderPath=$line.split('/')[0]+'/'+$line.split('/')[1]
if ($line.contains('.scala')){
# Running the tests and packaging
mvn package $folderPath
}
}
This is powershell but it can work with bash changing the for loop.
Thanks
I am looking for a help in Jenkins to trigger a build when specific tags(release.*) are pushed.
So far I got the way of skipping the deployment if there are no proper tags (https://jenkins.io/blog/2018/05/16/pipelines-with-git-tags/)
stage('Deploy') {
when { tag "release-*" }
steps {
echo 'Deploying only because this commit is tagged...'
sh 'make deploy'
}
}
Looks for something which will help me to trigger the builds from feature branch whenever a developer will push a tag.
On Microsoft docs it states you can set conditionals for a specific build task. For example to run the build on a specific version of a defined Git branch.
But is it also possible to make an conditional to make the VSTS to build the NPM package only if a commit contains a git-tag?
Update 1: The command $ git describe (source) seems like a possible part of the solution. This command takes the last commit hash and matches it to see if there were annotated tags. However, it doesn't equal it to a boolean value what we need in the custom condition of VSTS.
Update 2: Using npm git-describe you can return wether or not the latest commit has annotated tags. Example of a gulp task:
/** Example of a gulp task using git describe */
var gulp = require('gulp');
var {gitDescribeSync} = require('git-describe');
/**
* #function
* #name checkGitTag
* #description Returns wether or not the latest commit has a tag
* #returns {(String|null)} Git tag or null if no git tag exists on the commit
*/
var checkGitTag = function() {
var gitInfo = gitDescribeSync();
// output the result to debug
console.log(gitInfo.tag);
// gitInfo.tag seems to contain the logic needed
};
gulp.task('checkGitTag', checkGitTag);
module.exports = checkGitTag;
Maybe by installing the NPM package on the build server and using a similar function would work. Going to test it out.
Yes, it’s possible to conditional run a task when the build commit contains tag(s).
But since there is no such predefined variable to record the tag for the build commit $(BUILD.SOURCEVERSION), so you should add the steps (add a variable and a PowerShell task) to check if there has tag(s) on the build commit. Detail step as below:
1. Add a variable (such as result) with the default value 0. And the variable is used for check if the build commit contains tag(s).
2. Then add a PowerShell task before the task which you want to conditional run, and the powershell script as below:
$tag=$(git tag --contains $(BUILD.SOURCEVERSION))
if($tag)
{
Write-Host "##vso[task.setvariable variable=result]1"
echo "The build version $(BUILD.SOURCEVERSION) contains tag(s)"
}
else
{
Write-Host "##vso[task.setvariable variable=result]0"
echo "The build version $(BUILD.SOURCEVERSION) does not contain any tags"
}
3. Finally set custom condition for the task you want to run only when the commit contains tag as below:
and(succeeded(), eq(variables['result'], '1'))
We are working on integrating GitLab (enterprise edition) in our tooling, but one thing that is still on our wishlist is to create a merge request in GitLab via a command line (or batchfile or similar, for that matter). We would like to integrate this in our tooling. Searching here and on the web lead me to believe that this is not possible with native GitLab, but that we need additional tooling for that.
Am I correct? And what kind of tooling would I want to use for this?
As of GitLab 11.10, if you're using git 2.10 or newer, you can automatically create a merge request from the command line like this:
git push -o merge_request.create
More information can be found in the docs.
It's not natively supported, but it's not hard to throw together. The gitlab API has support for opening MR: https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/merge_requests.md#create-mr
You can use following utility.
Disclosure : I developed it.
https://github.com/vishwanatharondekar/gitlab-cli
You can create merge request using this.
Some of the features it has are.
Base branch is optional. If base branch is not provided. Current branch is used as base branch.
target branch is optional. If target branch is not provided, default branch of the repo in gitlab will be used.
Created pull request page will be opened automatically after successful creation.
If title is not supported with -m option value. It will be taken from in place editor opened. First line is taken as title.
In the editor opened third line onwards takes as description.
Comma separated list of labels can be provided with its option.
Supports CI.
Repository specific configs can be given.
squash option is available.
remove source branch option is available.
If you push your branch before this command (git push -o merge_request.create) it will not work. Git will response with Everything up-to-date and merge request will not be created (gitlab 12.3).
When I tried to remove my branch from a server (do not remove your local branch!!!) then it worked for me in this form.
git push --set-upstream origin your-branch-name -o merge_request.create
In addition to answering of #AhmadSherif, You can use merge_request.target=<branch_name> for declaring the target branch.
sample usage:
git push -o merge_request.create -o merge_request.target=develop origin feature
Simple This:
According to the Gitlab documents, you can define an alias for this command to simpler usage.
git config --global alias.mwps "push -o merge_request.create -o
merge_request.target=master -o merge_request.merge_when_pipeline_succeeds"
I made a shell function which opens up the GitLab MR web page with desired parameters.
Based on the directory with the git repo you are currently in, it:
Finds the correct URL to your repo.
Sets the source branch to the branch you're currently on.
As a optional first argument you can provide the target branch. Otherwise, GitLab defaults to your default branch, which is typically master.
gmr() {
# A quick way to open a GitLab merge request URL for the current git branch
# you're on. The optional first argument is the target branch.
repo_path=$(git remote get-url origin --push | sed 's/^.*://g' | sed 's/.git$//g')
current_branch=$(git rev-parse --abbrev-ref HEAD)
if [[ -n $1 ]]; then
target_branch="&merge_request[target_branch]=$1"
else
target_branch=""
fi
xdg-open "https://gitlab.com/$repo_path/merge_requests/new?merge_request[source_branch]=$current_branch$target_branch"
}
You can set more default values in the URL, like removing the source branch after merge:
&merge_request[force_remove_source_branch]=true
Or assignee to someone:
&merge_request[assignee_ids][]=12345
Or add a reviewer:
&merge_request[reviewer_ids][]=54321
You can easily find the possible query string parameters by searching the source of the GitLab MR webpage for merge_request[.
As of now, GitLab sadly does not support this, however I recently saw it on their issue tracker. It appears one can expect a 'native tool' in the upcoming months.
GitLab tweeted out about numa08/git-gitlab some time ago, so I guess this would be worth a try.
In our build script we just pop up the browser with the correct URL and let the developer write his comments in the form hit save to create the merge request. You get this url with the correct parameters by creating a merge request manually and copying the url of the form.
#!/bin/bash
set -e
set -o pipefail
BRANCH=${2}
....
git push -f origin-gitlab $BRANCH
open "https://gitlab.com/**username**/**project-name**/merge_requests/new?merge_request%5Bsource_branch%5D=$BRANCH&merge_request%5Bsource_project_id%5D=99999&merge_request%5Btarget_branch%5D=master&merge_request%5Btarget_project_id%5D=99999"
You can write a local git alias to open a Gitlab Merge Request creation page in the default browser for the currently checked-out branch.
[alias]
lab = "!start https://gitlab.com/path/to/repo/-/merge_requests/new?merge_request%5Bsource_branch%5D=\"$(git rev-parse --abbrev-ref HEAD)\""
(this is a very simple alias for windows; I guess there are equivalent replacements for "start" on linux and fancier aliases that work with github and bitbucket too)
As well as being able to immediately see&modify the details of the MR, the advantage of this over using the merge_request.create push option is that you don't need your local branch to be behind the remote for it to work.
You might additionally want to store the alias in the repo itself.
I use https://github.com/mdsb100/cli-gitlab
I am creating the MR from inside of a gitlab CI docker container based on alpine linux, so I include the install command in before-script (that could also be included in your image). All commands in the following .gitlab-ci.yml file, are also relevant for normal command line usage (as long as you have the cli-gitlab npm installed).
variables:
TARGET_BRANCH: 'live'
GITLAB_URL: 'https://your.gitlab.net'
GITLAB_TOKEN: $PRIVATE_TOKEN #created in user profile & added in project settings
before-script:
-apk update && apk add nodejs && npm install cli-gitlab -g
script:
- gitlab url $GITLAB_URL && gitlab token $GITLAB_TOKEN
- 'echo "gitlab addMergeRequest $CI_PROJECT_ID $CI_COMMIT_REF_NAME \"$TARGET_BRANCH\" 13 `date +%Y%m%d%H%M%S`"'
- 'gitlab addMergeRequest $CI_PROJECT_ID $CI_COMMIT_REF_NAME "$TARGET_BRANCH" 13 `date +%Y%m%d%H%M%S` 2> ./mr.json'
- cat ./mr.json
This will echo true if the merge request already exists, and echo the json result of the new MR if it succeeds to create one (also saving to a mr.json file).
Since GitLab 15.7 (Dec. 2022), the GitLab CLI glab is officially integrated to GitLab.
Introducing the GitLab CLI
The command line is one of the most important tools in a software engineer’s toolkit and the majority of their process and work revolve around tools available there. They customize their CLI with styles and extend it through applications to ensure maximum efficiency while performing tasks. The CLI is the backbone of scripts and workflows developers depend on to complete their work.
To support more developers where they’re already working, we’ve adopted the open source project glab, which will form the foundation of GitLab’s native CLI experience.
The GitLab CLI brings GitLab together with Git and your code, with no application or tab switching required.
You can read about our adoption of glab, our partnership with 1Password, and how to contribute to the project in our blog post.
A special thank you to Clement Sam for creating glab and trusting us with its future.
That means you can create a MR with glab mr create:
glab mr create -a username -t "fix annoying bug"
I'm running test cases with nosetests in Jenkins. In general, it will have 100 test cases and I want to mark the build unstable when less than 20 test cases failed. If more than 20 test cases failed, then, mark the build failed.
The command I ran:
nosetests test.py --tc-file config.yml --tc-format yaml
First of all, I tried to just change the status of the build to Unstable but it still failed.
The groovy script I used:
manager.addWarningBadge("Thou shalt not use deprecated methods.")
manager.createSummary("warning.gif").appendText("<h1>You have been warned!</h1>", false, false, false, "red")
manager.buildUnstable()
The first two lines of code are executed, but the job is still marked as Failed.
Is there anything wrong with my jenkins config? Or the groovy postbuild plugin does not work with nosetest?
This is the console output:
FAILED (failures=2)
Build step 'Execute shell' marked build as failure
Build step 'Groovy Postbuild' marked build as failure
Finished: FAILURE
As DevD outlined, FAILED is a more significant build state than UNSTABLE. This means calling manager.buildUnstable() or manager.build.setResult(hudson.model.Result.UNSTABLE) after a step failed will still leave the build result FAILED.
However, you can override a failed build result state to be UNSTABLE by using reflection:
manager.build.#result = hudson.model.Result.UNSTABLE
Below example iterates over the build log lines looking for particular regex. If found it which will change (downgrade) build status, add badges & append to the build summary.
errpattern = ~/TIMEOUT - Batch \w+ did not complete within \d+ minutes.*/;
pattern = ~/INSERT COMPLETE - Batch of \d+ records was inserted to.*/;
manager.build.logFile.eachLine{ line ->
errmatcher=errpattern.matcher(line)
matcher=pattern.matcher(line)
if (errmatcher.find()) {
// warning message
String errMatchStr = errmatcher.group(0) // line matched
manager.addWarningBadge(errMatchStr);
manager.createSummary("warning.gif").appendText("<h4>${errMatchStr}</h4>", false, false, false, "red");
manager.buildUnstable();
// explicitly set build result
manager.build.#result = hudson.model.Result.UNSTABLE
} else if (matcher.find()) {
// ok
String matchStr = matcher.group(0) // line matched
manager.addInfoBadge(matchStr);
manager.createSummary("clipboard.gif").appendText("<h4>${matchStr}</h4>", false, false, false, "black");
}
}
Note: this iterates over every line, so assumes that these matches are unique, or you want a badge & summary appended for every matched line!
Post-build result is:
Build step 'Execute Groovy script' marked build as failure
Archiving artifacts
Build step 'Groovy Postbuild' changed build result to UNSTABLE
Email was triggered for: Unstable
Actually It is the intended way to work.
Preference
FAILED -> UNSTABLE -> SUCCESS
using groovy post build we can change the lower result(SUCCESS) to higher preference(FAILED/UNSTABLE)..
not vise versa.
as workaround after your Nosetest ,add an execute shell and "exit 0". so always your result will be the lower preference. now by your post build groovy script decide your exit criteria based on test results. This is actually a tweak.. will explore more and update you on this.