I have setup a Git project + CI (using Gitlab-runner) on Gitlab v12.3.5. I have a question about issues and pipelines. Let's say I create an issue and assign it to myself. So this create a branch/merge request. Then, I open up the WebIDE to modify some files in an attempt to fix the issue. Now I want to see what if my changes will fix the issue. In order to run the pipeline, is it necessary to commit the changes into the branch or is there some other way?
The scenario I have is that it may take me 20 times to fix the files to make the pipeline 'clean'. In that case, I would have to keep committing on each change to see the results. What is the preferred way to accomplish this? Is it possible to run the pipeline by just staging the changes to see if they work?
I am setting up the gitlab-ci.yaml file. Hence it is taking a lot of trials to get it working properly.
You should create a branch and push to that. Only pushed changes will trigger pipeline runs. After you're done, you can squash and merge the branch so that the repo's history will be clean.
Usually though, you won't have to do this because you'll have automated tests set up to check whether your code works. You should also try testing the Linux commands (or whichever commands you're running in your GitLab CI scripts) locally first. If you're worried about whether your .gitlab-ci.yml syntax is correct, you can navigate to the file in your repository and check there (there's a button at the top which lints it).
Related
I am planning to experiment building a pipeline using Azure DevOps. One thing that I noticed early on is, after azure-pipelines.yml created, I have to commit this first before being able to run it. But I want to experiment on it which revolves around trial and error. Doing multiple commit just to test things out are not feasible.
In Jenkins I can just define my steps and try to run it without committing the file.
Is this also possible to do in Azure DevOps?
But I want to experiment on it which revolves around trial and error. Doing multiple commit just to test things out are not feasible.
Yes it is - you just use a different code branch. That will allow you the freedom to make as many changes as you need, while putting the pipeline together and trying it out, without committing to the master branch.
Then when you're happy with the way the pipeline is running, you can merge your branch into the master branch which the pipeline normally uses.
You cannot run YAML pipelines without committing them, but you can create classic pipelines and run them without committing anything pipeline-related to the repository (except for the source code you want to build). Classic pipelines can later be turned (or copy-pasted, to be exact) into yaml pipelines with view YAML -option.
https://learn.microsoft.com/en-us/azure/devops/pipelines/get-started/pipelines-get-started?view=azure-devops#define-pipelines-using-the-classic-interface
If you're on your own branch, or in a repository without any other developers making changes then you can
Make a change
use git commit --amend to overwrite your previous commit with the new file
use git push --force-with-lease to push that up to Azure DevOps
That will hide your commit history while experimenting
I need to set up a pipeline for a branch that will do the following,
On push, it will trigger a pipeline (which happens as per GitLab)
1st Job to trigger a build (.net5) for the branch (new code pushed to the branch). If the build succeeds, it will begin the next job. However, I am to trigger build (MSBuild) on the runner location check out when the pipeline runs. But is there any way to directly run a build on the branch?
If the above build fails, the push should be reverted on a branch. I applied git revert and reset commands, but it only gets used to the build location of the runner. I need to revert commit at the branch level.
When GitLab starts a pipeline, it is a "detached HEAD" which basically means it creates a temporary branch that points to your specific commit. It does this because there might be a commit after it which the current build doesn't know about.
To get out of the detached head, you need to switch to the branch directly.
git checkout branch-name
To get the branch name, you find an appropriate predefined variable which works for your.
Once you have that, you need to get the latest version so you can revert. I usually needed the reset here to make things work.
git reset --hard
git pull
From there, you can revert. Now, the CI process doesn't have write permissions, so you need to have a GitLab access token with write permissions, which means you have to set up a new origin or parent.
git remote add changes https://oauth2:$PUSH_GITLAB_TOKEN#gitlab.com/group-name/project-1
git push -f changes branch-name
Now... that said, this is probably not what you want to do. Since you said you were new to CI, I'm going to say I'm 99.99999% sure you don't want to go down this path because it only leaves to heartache and frustration.
The reason is the branch may have a second or additional commit. It could be you missed something at the last minute and threw up a new one, an automated process starts up, or just your typical race conditions. Reverting a commit means you either will blow up with those changes or you end up erasing every other commit as you have to force push to get it up.
This also means that this is a very fragile process and will break many times, usually at the worst times (when you have a rapid series of commits), and then you have to tell everyone to stop working while you fix it.
(A good sign is how much work you have to do to fight the CI process.)
Instead, I recommend you create a merge request (MR) against your branch and set up the rules to make sure that is valid before you merge it into your branch. That way, the process doesn't have to worry about future or past commits, it just says what you have in your merge request (which is another branch) can't be applied after merging it with branch-name.
In our team, we have many branches (main, next, release/2.3.4, release/2.4.0, etc.) and doing MRs against each one has worked out well. We do the merge requests against each of the release branches, and the CI process says it can merge and pass tests before it tries to commit, but then we can commit knowing that 98+% of the time, it will be fine.
And it doesn't require jumping through hoops.
I’m trying to set up GitLab CI/CD for an old client-side project that makes use of Grunt (https://github.com/yeoman/generator-angular).
Up to now the deployment worked like this:
run ’$ grunt build’ locally which built the project and created files in a ‘dist’ folder in the root of the project
commit changes
changes pulled onto production server
After creating the .gitlab-ci.yml and making a commit, the GitLab CI/CD job passes but the files in the ‘dist’ folder in the repository are not updated. If I define an artifact, I will get the changed files in the download. However I would prefer the files in ‘dist’ folder in the to be updated so we can carry on with the same workflow which suits us. Is this achievable?
I don't think commiting into your repo inside a pipeline is a good idea. Version control wouldn't be as clear, some people have automatic pipeline trigger when their repo is pushed, that'd trigger a loop of pipelines.
Instead, you might reorganize your environment to use Docker, there are numerous reasons for using Docker in a professional and development environments. To name just a few: that'd enable you to save the freshly built project into a registry and reuse it whenever needed right with the version you require and with the desired /dist inside. So that you can easily run it in multiple places, scale it, manage it etc.
If you changed to Docker you wouldn't actually have to do a thing in order to have the dist persistent, just push the image to the registry after the build is done.
But to actually answer your question:
There is a feature request hanging for a very long time for the same problem you asked about: here. Currently there is no safe and professional way to do it as GitLab members state. Although you can push back changes as one of the GitLab members suggested (Kamil Trzciński):
git push http://gitlab.com/group/project.git HEAD:my-branch
Just put it in your script section inside gitlab-ci file.
There are more hack'y methods presented there, but be sure to acknowledge risks that come with them (pipelines are more error prone and if configured in a wrong way, they might for example publish some confidential information and trigger an infinite pipelines loop to name a few).
I hope you found this useful.
I am having a build issue on travis with my node.js project. The issue stems from the fact that I have a rather complex test that I want to run, which requires building and running some test scaffolding framework on the VM before I get to 'npm test'. Somewhere along the line it is failing, and I find myself adding debugging statements to my .travis.yml to try to root out the problem, but its annoying to have my commit history littered with these changes/attempted fixes.
I guess I want to be able to either (a) get on the travis box at the time the test is running (or afterwards) so I can inspect what is going on/went wrong, or (b) at least be able to tweak and run my .travis.yml file and associated scripts somehow and re-run immediately without having to formally check those changes in in order to kick off travis again.
I find myself adding debugging statements to my .travis.yml to try to root out the problem, but its annoying to have my commit history littered with these changes/attempted fixes.
If the history is important, maybe because your changelog is generated from it, then my suggestion is, to create a private sandbox for experiments by cloning the repo.
clone a organization repo to a user repo.
activate travis on the user repo
try and error commit as long as you need to your .travis.yml
when everything is working like you want, squash the git commits into 1
do a pull request of this single commit from user repo to company repo
et voila: history stays clean
Big Warning: When you have no contributors with forks to worry about, then you could simply commit till you get it right and squash the history into a single commit and do a force push.
get on the travis box at the time the test is running (or afterwards) so I can inspect what is going on/went wrong
That's not possible. But you can view or download the log from the builds.
If you view the build log directly after a push, then you get live view of the processing steps on the Travis env. You can also cancel it manually.
at least be able to tweak and run my .travis.yml file and associated scripts somehow and re-run immediately without having to formally check those changes in in order to kick off travis again.
When you are logged in on Travis and you will find a button to rerun a build.
You could try executing your build commands inside a normal Ubuntu VM.
Back in the days box images were available over at http://files.travis-ci.org/boxes/provisioned/travis-ruby.box
But Travis switched from Vagrant to BlueBox and stopped providing the downloads.
You could try on IRC and ask to get access to your “box” for debugging.
I'm not sure if you get access.
We have setup and perfectly running gitlab + gitlab-ci installation. We are now looking how to do cross-project builds. Our project is divided into several repositories and everything is joined during build process via composer.
What I would like to achieve is - when you commit to any of those repositories, they trigger main repository to get built. I was trying to achieve this via webhooks, unfortunately I need a lot of information about commit from the main repository, that I don't have.
Any idea how to do it?
I updated gitlab-ci code a little bit: https://github.com/gitlabhq/gitlab-ci/commit/7c7066b0d5a35097a04bb31848d6b622195940ed
I can now call the api.