I understand that gitlab has support to Jenkins CI, but what I need is a lot less than that.
I have a Rails application and get the coverage from the tests using simplecov. It generates HTML output in a directory by running a rake task. I would like to see the current coverage through gitlab. Is there a simple way to integrate this report with gitlab?
I fear there is still no easy way to integrate code coverage reports but Gitlab now supports (since Version 8.0 integrated) build jobs for your code. Unfortunately you have to implement your solution by writing a custom .gitlab-ci.yml to run your coverage tests. For viewing the reports, you can specify the generated "artifacts" or publish them on gitlab pages.
For more information, see here: https://about.gitlab.com/gitlab-ci/
Additionally you can parse a text output to display a short code coverage report:
(Enable builds and output test coverage)
Go to "Project Settings" -> Builds
Add to "Test coverage parsing" a regular expression (examples below, simplecov included)
See Publish Code Coverage Report with GitLab Pages
The short answer: Unfortunately there is no easy way to do this.
The longer answer:
GitLab not yet has a Jenkins support.
What you basically need is a service like GitLab CI or Jenkins CI, which starts simplecov and posts the output back to GitLab. Unfortunately GitLab does not offer such a functionality yet.
But I know other organizations which do have a Jenkins service for GitLab which automatically comment git pushes with the Jenkins result.
You now (June 2020, GitLab 13.1) have code coverage history, in addition of Test coverage parsing.
Graph code coverage changes over time for a project
All too often, a project has a code coverage target but development teams might not have much visibility into which direction that target value is trending over time.
There needs to be an easier way to track changes in code coverage over time without that extra hassle.
The Code Coverage graph now provides better visibility into how code coverage is trending over time.
It displays a simple graph of the coverage value(s) calculated in pipelines.
See Documentation and Issue
With GitLab 13.6 (November 2020), you also have (not for free though)
Display code coverage data for selected projects
In 13.4, we released the first iteration of Code Coverage data for Groups that enables you to compare the coverage of multiple projects and download the data in a single file from a single screen. However, to analyze the data, you had to open the file to check it manually, and probably imported it into a spreadsheet for further analysis.
In GitLab 13.6, you can now select specific projects in a group to see their latest coverage values directly in GitLab itself without needing to download a file or waste development time accessing code coverage data. We welcome feedback on the functionality and possible iterations for this feature in our feedback issue.
See Documentation and Issue.
Related
I have notified the complexity of my configuration for Gitlab CI/CD grows fairly fast. Normally, in a either a programming language or infrastructure code I could write automated test for the code.
Is it possible to write automated test for the gitlab-ci.yml file itself?
Does it exist any library or testing framework for it?
Ideally I would like to setup the different environments variables, execute the gitlab-ci.yml file, and do assertions on the output.
I am not aware of any tool currently to really test behaviour. But i ended up with two approaches:
1. testing in smaller chunks aka unit testing my building blocks.
I extracted my builds into smaller chunks which i can include, and test seperately. This works fine for script blocks etc. but is not really sufficient for rule blocks - but offers great value, eg i use the templates for some steps as verification for the generated docker image, which will be used by those steps.
2. verification via gitlab-ci-local
https://github.com/firecow/gitlab-ci-local is a tool which allows you to test it locally, and which allows you to provide environment variables. There are some minor issues with branch name resolution in pr pipelines but besides that it works great. I use it to test gitlab ci files in GitHub Actions.
I hope that this helps somehow
Gitlab have a linter for gitlab-ci.yaml
To access the CI Lint tool, navigate to CI/CD > Pipelines or CI/CD > Jobs in your project and click CI lint.
Docs
We have a test suite in our gitlab pipeline which produces Allure test reports. To make these results available after the test, we currently publish the results to artifacts and have an allure serve running which makes them available over a subdomain, based on the branch name.
We would like to host the test results with gitlab pages for each branch. However we can only ever host one version of a page through gitlab pages at one time. This is a problem since we want to host the test results for each branch, not only for the last executed branch. It seems like this is currently not possible without hacks.
I also found this 3 year old gitlab issue about the topic which indicates this is coming in some version of gitlab in the future.
Is there a better way to do this? Or is our best bet currently to wait until this becomes available in gitlab?
If you want to show a test report summary for each pipeline build, you can use GitLab's "Unit test reports" feature. It is slightly different to GitLab Pages, but it's easier to use, because you don't have to configure and host web pages yourself.
You only need to specify the paths to the XML files of the test results, something like this:
java:
stage: test
script:
- gradle test
artifacts:
when: always
reports:
junit: build/test-results/test/**/TEST-*.xml
It will show the summary on GitLab's web page of the pipeline build results, like this:
Source and guidance:
https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html
https://docs.gitlab.com/ee/ci/testing/unit_test_report_examples.html
https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#browsing-artifacts
and one HTML file that you can view directly online when GitLab Pages is enabled > (opens in a new tab). Select artifacts in internal and private projects can only > be previewed when GitLab Pages access control is enabled.
This sound to me like you just need GitLab pages enabled and then can browser HTML files from the artifacts repository.
(correct me if a m wrong)
From https://docs.gitlab.com/ee/user/project/pages/pages_access_control.html:
"You can enable Pages access control on your project if your administrator has enabled the access control feature on your GitLab instance. When enabled, only members of your project (at least Guest) can access your website"
As the standard pipeline badge from GitLab looks like this
you can tell pretty well that those are not really distinguishable.
Is there a way to change the pipeline text manually or programmatically to something else for each badge?
Btw, the badges were added with those links
https://gitlab.com/my-group/my-repository/badges/master/pipeline.svg
https://gitlab.com/my-group/my-repository/badges/dev/pipeline.svg
Additional facts:
The pipeline runs locally on my computer
My repo is private
I know it is a bit of an old post, but I was looking for the same and found that it is available now since GitLab 13.1.
The text for a badge can be customized to differentiate between multiple coverage jobs that run in the same pipeline. Customize the badge text and width by adding the key_text=custom_text and key_width=custom_key_width parameters to the URL:
https://gitlab.com/gitlab-org/gitlab/badges/main/coverage.svg?job=karma&key_text=Frontend+Coverage&key_width=130
The example is for the Coverage badge but this also works for Pipelines, so in your case:
https://gitlab.com/my-group/my-repository/badges/master/pipeline.svg?key_text=master&key_width=50
https://gitlab.com/my-group/my-repository/badges/dev/pipeline.svg?key_text=dev&key_width=50
(Found this via https://microfluidics.utoronto.ca/gitlab/help/ci/pipelines/settings.md#custom-badge-text)
There are multiple ways how you can achieve custom pipeline badges in GitLab.
One way could be to use Shields.io which provide a way to generate dynamic badges for your Gitlab repository via a jsonfile.But if your repository is private (only accessible from internal network) then you will get an inaccessible message in your badges.
Otherwise, if your build uses python Docker images or any other python installation with dependencies, you can simply install the anybadge package and generate svg badges to be used in the project from the artifacts directly.
It would be good in future that GitLab offers us more cleaner way to customize the badges, but for now I think those are the workaround solutions.
I am using GitLab for Git version control and GitLab CI / CD for my automated builds. Usually, the builds are triggered by Git repository activity but I also have a weekly build to ensure that projects not under active development continue to work. When there is only a "master" branch on a project, it is easy to ensure a weekly build is run on the latest code. When there are multiple branches in a project, I would like to repeat the pipeline work for each of them in turn.
What I would like to be able to do is schedule a build (weekly, fortnightly or monthly) that runs on all current branches visible in Git. Is that possible within GitLab's Continuous Delivery system?
The motivation behind doing this is to ensure that external activity, such as tool and library updates, do not introduce an issue without it being promptly visible. Assuming there are reasonable automated testing, coverage and comprehensive builds for target platforms, a monthly build with the latest tools should highlight the problem promptly. This is better than an invisible mountain to problems accumulating while a project is shelved for a few years (or months). Sometimes all that is required is occasional maintenance.
There are only a handful of feature branches and release lines on the projects currently. I would not expect that number to grow significantly. There is time enough over a weekend to run the required pipelines dozens if not hundreds of times at present.
Ideally, I would like something straightforward to set up. I cannot see anything in the admin GUI that would allow this at present. I did look at the API and I can see there is some scope there to script the addition and removal. Perhaps some script that is run once a month to create new Scheduled pipelines based on git branches is the only way. A pre-made solution on those lines would be perfectly acceptable. If nothing exists I might start work on something like that in time.
I am currently running GitLab Community Edition 11.2.3 06cbee3 (GitLab CE 11.2.3). If there is an Enterprise Edition only answer, that is fine and will add to the justifications of purchasing the EE version. I would pick at CE one above the EE one though.
You cannot set a schedule for all branches at once, you have to configure one schedule per branch yourself.
Perhaps some script that is run once a month to create new Scheduled
pipelines based on git branches is the only way.
I would go in that way.
I'm using Visual Studio Online, and looking at CI and automated deploys
My Web Test Project has three branches: DEV,UAT,LIVE
I have a build Definition that builds from the DEV branch and uses msbuild to deploy the Website to my UAT server, this works fine. What I would like to do is after the publish, branch the code from DEV to UAT
any help would be appreciated
You can use PowerShell post test to call "tf merge" to push code from DEV to UAT.
You are doing it the wrong way round. If you are promoting by quality that is a value of the build output and not the code.
A better way would be to not have branch by quality (dev->qa) but to have branch by release (1.0->2.0). Then have a single build on a version that progress through DEV->QA. This is the recommended method of controlling quality and minimises risk. The release management tools that are part of TFS and VSO use this method.
http://nakedalm.com/create-release-management-pipeline-professional-developers/
If you want to increase and control quality you will need to move away from branch by quality. It is not possible to achieve continuous delivery of any form with it.