We have lot of python code residing in local git repository. Having installed gitlab locally, need to implement CI/CD pipeline. Need is to ensure, that all code is sanitized before being pushed to remote git repository. pre commit hooks that come by default with git, should help in doing so. Question is will it help to integrate git hooks with CI / CD pipeline? How ?
That hook is a client-side hook.
While CI/CD is done on the server side. Which means the hook itself is not integrated, but the script/command used by that hook can be reused in a gateway pipeline (on a runner configured to run Python).
(See also those CICD pipelines examples)
you push your topic/feature branch
the gateway pipeline is triggered (by the push event)
if it passes, it merges in turn your code to an integration branch (like development)
if it does not pass, your code does not end up on the dev branch, forcing you to fix whatever issue was highlighted by the gateway pipeline execution.
You also have Code Quality reports, to analyze how your improvements are impacting your code’s quality.
Related
I am planning to experiment building a pipeline using Azure DevOps. One thing that I noticed early on is, after azure-pipelines.yml created, I have to commit this first before being able to run it. But I want to experiment on it which revolves around trial and error. Doing multiple commit just to test things out are not feasible.
In Jenkins I can just define my steps and try to run it without committing the file.
Is this also possible to do in Azure DevOps?
But I want to experiment on it which revolves around trial and error. Doing multiple commit just to test things out are not feasible.
Yes it is - you just use a different code branch. That will allow you the freedom to make as many changes as you need, while putting the pipeline together and trying it out, without committing to the master branch.
Then when you're happy with the way the pipeline is running, you can merge your branch into the master branch which the pipeline normally uses.
You cannot run YAML pipelines without committing them, but you can create classic pipelines and run them without committing anything pipeline-related to the repository (except for the source code you want to build). Classic pipelines can later be turned (or copy-pasted, to be exact) into yaml pipelines with view YAML -option.
https://learn.microsoft.com/en-us/azure/devops/pipelines/get-started/pipelines-get-started?view=azure-devops#define-pipelines-using-the-classic-interface
If you're on your own branch, or in a repository without any other developers making changes then you can
Make a change
use git commit --amend to overwrite your previous commit with the new file
use git push --force-with-lease to push that up to Azure DevOps
That will hide your commit history while experimenting
we want to add a tag after merge_request approve/merged in default develop branch
after add this dev version tag, we also want to do source code complie
once complie passed, save target build to local server for test purpose.
seems gitlab still not official support it, is there any unofficial API or good solution to do it?
the most important part is how to trigger this job after merge_requst.
Thanks
To solve this flow I would suggest the following general approach:
create a job that only runs on develop branch and creates a git tag: git tag $TAG_NAME && git push --tags
create another job that also only runs on the develop branch, but needs the job from point 1 and does the code compiling
create a third job that needs job from point 2, only runs on develop and copies the artifact via scp, sftp or similar method to the local server you mentioned.
The essential part is running the CI job on the develop branch as this is the only option to hook into after the merge.
Some documentation that will help you create the specific solution you need:
only -> https://docs.gitlab.com/ee/ci/yaml/#only--except
needs -> https://docs.gitlab.com/ee/ci/yaml/#needs
about ssh (sftp/scp) on gitlab -> https://docs.gitlab.com/ee/ci/ssh_keys/#ssh-keys-when-using-the-shell-executor
git tag -> https://git-scm.com/docs/git-tag
git push -> https://git-scm.com/docs/git-push
about git push in gitlab-ci -> https://docs.gitlab.com/ee/topics/git/tags.html
we have azure devops repo setup with Master > TEST > Development
for last sprint we merged changes from Development to TEST and had merge conflicts
one of the developer created local TEST branch and Merged Development changes to resolve changes and committed changes using local TEST branch
After that pullrequest is created from local TEST branch to Remote TEST branch & completed pull request
Now if we create pullrequest from Development to remote TEST it shows remote TEST is having latest changes
Any help will be appreciated and what all options available to resolve this issue
Flow
Thank you
Dyanesh
Here is my reproduction of your question. Is the process you want to express as shown below?
First clone the remote repo to the local and merge from the local Development branch into the local Test branch. Then push the local Test branch to the remote Test branch.
Then when you create a pull request from the Development branch to the Test branch in azure devops, you get the following prompt.
If so, this is normal, because you have merged the changes from the development branch to the local Test branch locally, and then pushed the changes from the local Test branch to the remote Test branch. At this point, the remote development branch has been synchronized with the Test branch, so naturally there are no changes that need to be merged.
If I misunderstood the issue, please point it out. It would be much easier for people to understand and reply if you could attach screenshots of detailed steps.
Thank you for reply, just after some research i see
1) We merged remote local branch from remote development few times using pullrequest and completing it.
2) After that used same remote local branch to merge changes to remote development branch
3) Now even though remote DEV and TEST branch are in sync, when we do pullrequest from DEV to TEST (remote) it shows still some changes are pending.
how to solve this issue any help will help us resolve this. flow explained in this diagram
Flow
My requirement is whenever developer try to do check-in existing GitLab repository then before doing check-in in repository,build should trigger (Jenkins build) and Junit test case should run on new check-in and if passes then it should go forward and will allow developer to do check-in in main repository.
I am not sure but is pre-hook commit can achieve this requirement?
While you could achieve this with pre-commit hooks, it's more common to do so with post-commit hooks on the server-side.
You can achieve this by operating a branch based workflow, there are multiple to choose from - I would recommend reading through this guidance by Atlassian.
Developers will create branches from a 'main' branch (often master, but can be a 'dev' branch working towards a release for instance), then develop code on that branch. They will then push their branch and commits to the remote repository (GitLab). When ready to merge into the main branch, your developers can open a merge request onto the main branch.
On GitLab you can setup a webhook to trigger Jenkins builds when a push event occurs. I would recommend this guide to guide you through it.
In the GitLab project settings you can require a passing build before merge requests are allowed to merge.
Furthermore, your understanding of Git seems incorrect - check in is not a term used in Git. Please take a look at the Git documentation. In Git a developer creates commits against a local copy of the repository, then pushes these to a remote repository (GitLab/GitHub etc.). There is no direct equivalent of the 'check in' used in various centralised version control systems e.g. SVN.
We have setup and perfectly running gitlab + gitlab-ci installation. We are now looking how to do cross-project builds. Our project is divided into several repositories and everything is joined during build process via composer.
What I would like to achieve is - when you commit to any of those repositories, they trigger main repository to get built. I was trying to achieve this via webhooks, unfortunately I need a lot of information about commit from the main repository, that I don't have.
Any idea how to do it?
I updated gitlab-ci code a little bit: https://github.com/gitlabhq/gitlab-ci/commit/7c7066b0d5a35097a04bb31848d6b622195940ed
I can now call the api.