I have been trying to create a CI CD using GitLab, Jenkins, and sonarqube for the last week. I have a project with multiple people working on it. and the pipeline is working fine. now I have a doubt about sonar scanning. Can we get user-wise scanning results if we create user logins in sonarqube? If possible, how?
Now sonar gives an overall report. one does not need to see others' bugs. That is why I am trying to separate it.
Related
I need to integrate security scans of my project files (SAST) in my Gitlab CI/CD pipeline, and it's easy to do with just another job in .gitlab-ci.yml, like:
security-scan:
stage: test
image: my_image:latest
script:
- scan run project/folder
But the problem is that developers can easily comment this part of the code and prevent the job run.
How can I create some kind of external job which will always be running by the trigger and developers would not be able to modify it?
I found this discussion on the Gitlab forum, but I don't get it.
We have multiple repos and whenever the developer runs the sonarQube scanner through Jenkins job, it is creating one project with build number along with the date, is there anyway I can re-use the same project name ..?
developers are running sonarQube reports through Jenkins jobs.
sonar.projectKey=portal1-sonar:1stiteration-${BUILD_NUMBER}
sonar.projectName=SonarQube nodeJS portal1 Build : ${BUILD_NUMBER}_${BUILD_TIMESTAMP}
anyway, I can change and use same project name, whenever developer runs sonarQube.
every project is defined in SonarQube with it's own key. That means if the key is the same, it is the same Project, and you will have a "history" of analyses and can compare parameters.
Although the idea of buildnumbers seems to be interesting, i recommend to use Branch names instead. There are currently two ways of doing this, for the first one, you need to have a SonarQube installation with a paid price. Than you are entitled to use the branch plugin. Which is actually the more superior way, because your project will show branches. The sonarQube docs are quiet helpful regarding this.
The old/Deprecated way will create a new project per branch, which you can than compare. the property you need to set is sonar.branch and this will be automatically added to your project key. So if the project key is Project and the sonar.branch is set to develop your new project will have the key Project:develop. This parameter is deprecated, and i am not sure, how long it will stay in the system.
We are trying to get an old legacy code base under control while simultaneously developing new features. Currently the code compiles with a hell of a lot of compiler warnings and warnings from static code analyzers. For that reason it is not uncommon that code introducing new warnings reaches production simply because the new warning got lost in the shuffle.
Currently we are using Jenkins for nightly builds and make the build fail on new warnings. However, when Jenkins detects the new warnings the code was already merged a few hours ago. So we would like to not only shorten the feedback cycle but also ensure to only merge changes that do not introduce new warnings.
As far as I know it is possible to trigger a Jenkins build on a push to GitLab. But Jenkins can only compare the count of warnings to the previous build. But we would need to compare to a build of a different branch.
Can GitLab CI or a combination of GitLab EE and Jenkins somehow be configured to detect if a merge request introduces new warnings?
Yes that is possible but that's rather an open-ended question that will depend a lot on how long a build takes and how you will compare the outcomes.
You don't have to run only the checks on the branch you have checked out. You may set up two jobs in parallel that run tests on current branch and the develop branch, pass them as artifacts to a third job and compare them there.
You may want to store the state of a build on your develop branch and download the artifact to your current job and compare it against the local results. You could also store them in a database, on a file server or wherever else it's comfortable.
Finally you may try an external code quality tool like SonarQube which has greater insight into what's new and what's old.
In the meantime tools got developed that allow a workflow which is not perfect but comes quite close.
Jenkins has the Warnings Next Generation Plugin which can compare the warnings found in one Jenkins job to the warnings found in another Jenkins job. So we set up a job to compile our develop branch each time a new commit is pushed to it. We then use the results as baseline. Another job that gets triggered for each merge request in GitLab then uses this baseline to determine the new warnings introduced by the merge request.
This works reasonably well.
We are looking at removing developers from production and want a simple kind of deployment management tool. One suggestion that some members are using with SalesForce is Jenkins. I have never used Jenkins or any kind of deployment tool before. I normally just copied my code from IDE and updated the file in the SuiteScript file cabinet.
Does Jenkins work for NetSuite? Or what do you recommend for this purpose?
We are planning to use Bit Bucket (which runs Git in the background) as our version control in case that matters.
Thank you for any help
IMO the greatest challenge in integrating with any CI environment(be it Jenkins or any other) is the fact that you can move code files from one system to another using code/APIs but, NOT things like scripts, custom records, fields its deployments , etc. for which you need a bundling process and hence, manual intervention.
NetSuite in recent Suiteworld 2015 said that its coming up "Change Management" which would allow you to put everything that is part of your app to version control system such as git. Please see SuiteAnswer Id 42387, when this feature is rolled out, you can integrate with your CI tool to automatically copy/deploy your app details to an another NetSuite account and run your tests there and accordingly pass/fail your build.
Why do you want to remove developers from Production? This will severely hamper their ability to create solutions for your NetSuite account and will create a ton of overhead for them.
If you must have them out of Production, then probably your "best" option would be to have them build their solutions in Sandbox and then use SuiteBundles for deployment to Production. A Production Admin would need to update the appropriate Bundle(s) for all Production migrations.
NetSuite has also built a SuiteCloud IDE plugin for Eclipse which allows uploading and downloading files (no copy-paste necessary), so if you're not using that I would recommend it.
We are using Jenkins for our own internal automated testing, but not for deployment into NetSuite. I do not know if someone has already built a NetSuite plugin for Jenkins; it is likely you would have to build your own file upload mechanism using the NetSuite Web Services SOAP API, but that would still only allow deployment of source files. Developers will most likely also need to be creating and updating custom records, fields, lists as well as Script records and Script Deployment records, which you will not be able to do through Jenkins or any other tool that I know of.
I understand that gitlab has support to Jenkins CI, but what I need is a lot less than that.
I have a Rails application and get the coverage from the tests using simplecov. It generates HTML output in a directory by running a rake task. I would like to see the current coverage through gitlab. Is there a simple way to integrate this report with gitlab?
I fear there is still no easy way to integrate code coverage reports but Gitlab now supports (since Version 8.0 integrated) build jobs for your code. Unfortunately you have to implement your solution by writing a custom .gitlab-ci.yml to run your coverage tests. For viewing the reports, you can specify the generated "artifacts" or publish them on gitlab pages.
For more information, see here: https://about.gitlab.com/gitlab-ci/
Additionally you can parse a text output to display a short code coverage report:
(Enable builds and output test coverage)
Go to "Project Settings" -> Builds
Add to "Test coverage parsing" a regular expression (examples below, simplecov included)
See Publish Code Coverage Report with GitLab Pages
The short answer: Unfortunately there is no easy way to do this.
The longer answer:
GitLab not yet has a Jenkins support.
What you basically need is a service like GitLab CI or Jenkins CI, which starts simplecov and posts the output back to GitLab. Unfortunately GitLab does not offer such a functionality yet.
But I know other organizations which do have a Jenkins service for GitLab which automatically comment git pushes with the Jenkins result.
You now (June 2020, GitLab 13.1) have code coverage history, in addition of Test coverage parsing.
Graph code coverage changes over time for a project
All too often, a project has a code coverage target but development teams might not have much visibility into which direction that target value is trending over time.
There needs to be an easier way to track changes in code coverage over time without that extra hassle.
The Code Coverage graph now provides better visibility into how code coverage is trending over time.
It displays a simple graph of the coverage value(s) calculated in pipelines.
See Documentation and Issue
With GitLab 13.6 (November 2020), you also have (not for free though)
Display code coverage data for selected projects
In 13.4, we released the first iteration of Code Coverage data for Groups that enables you to compare the coverage of multiple projects and download the data in a single file from a single screen. However, to analyze the data, you had to open the file to check it manually, and probably imported it into a spreadsheet for further analysis.
In GitLab 13.6, you can now select specific projects in a group to see their latest coverage values directly in GitLab itself without needing to download a file or waste development time accessing code coverage data. We welcome feedback on the functionality and possible iterations for this feature in our feedback issue.
See Documentation and Issue.