I am working on a micro-services project, each service has its own pipeline because it gets deployed to a server of its own, we have each project in its own repository on gitlab with its own .gitlab-ci.yml but I want to collect all of these services in a single repository to make them easier to maintain and trigger a deployment of all the services when a commit is pushed.
The issue is I don't want to have a big fat yaml file that contains the build & deployment process of each service but instead keep the yaml files in the services folders and have a yaml file on the root that references them, i.e.:
| service1
| service1-code
| .gitlab-ci.yaml << build process for service1
| service2
| service2-code
| .gitlab-ci.yaml << build process for service2
| .gitlab-ci.yaml << reference to service1/yaml & service2/yaml
Is that doable?
There is currently no way for GitLab to do this, and there is an open issue to add this feature for monorepos.
(...) keep the yaml files in the services folders and have a yaml file on the root that references them
Just found this comment on GitLab by robindegen.
We create separate repositories, and a parent main repository that has
them as submodules. This works just fine for doing CI on subsets. When
we push to master on the main repo (to update the submodules), a full
CI run is done on everything, including integration tests.
I recon a CI clone includes submodules so this would just work. So if you already have a repo per project; have your cake and eat it too!
Related
Scenario: I've created two repositories inside my GitLab account: 'work' & 'project'. There are multiple branches in each of the following repos. I'm currently in the project repo and have created the .gitlab-ci.yml file.
Task: I want to copy two files 'A' & 'B' from the feature branch of 'work' repository to the current location (i.e., in the root of my project repository).
Thanks in advance :)
It's very easy, just add it to your script section
- git clone https://user:password#gitlab.com/group/project.git
Or to download specific files
- curl --header 'Private-Token: <your_access_token>' https://gitlab.com/api/v4/projects/:id/repository/files/:filename\?ref\=:ref
Also it will be a great practice to add Private-Token to project masked varible
You can get access token here
For one of my project the TestResults.zip file is publishing on url https://dev.azure.com/MyProject/Project/_TestManagement/Runs?runId=8309822&_a=runCharts.
I want to change storage location for TestResults.zip file from above given URL to my own defined repository location.(Like: Myproject/target/surefire-reports.zip) How to do that?
Because in my azure pipeline the test are running and when it comes to create a zip for TestResults it's storing in given above URL and i want to store in one of my project sub-module under target directory so that i can create a zip file.
Please help me to resolve this issue.
Many Thanks
How to modify the repository / storage location for Test Results in azure pipeline
It is not recommended to publish the test results to the repo.
If you still want do that, you could use git command line to submit the test file to the repo in the pipeline after the test task, like:
cd $(System.DefaultWorkingDirectory)
copy Path\surefire-reports.zip target
cd target
git add surefire-reports.zip
git commit -m "Add a test file"
git push https://PATn#dev.azure.com/YourOrganization/YourProject/_git/xxx.test HEAD:master
You could check the similar thread for some more details.
As of now I have a simple solution structure(WebAPI projects, which are gonna be microservices) with default generated dockerfiles for each project in the solution, like:
Solution X
| Project A
| | Dockerfile
| Project B
| | Dockerfile
| Project C
| | Dockerfile
| Project D
| | Dockerfile
| azure-pipeline.yml
From the development and debuggind point of view everything works(through "Docker" as launcher), but after creating with the Azure wizard the first pipeline for the "Project A" my build always fails at a COPY instruction at the build step:
COPY ["Project A/ProjectA.csproj", "Project A/"]
With the error from the pipeline run as:
COPY failed: stat /var/lib/docker/tmp/docker-builder196561826/Project A/ProjectA.csproj: no such file or directory
##[error]COPY failed: stat /var/lib/docker/tmp/docker-builder196561826/Project A/ProjectA.csproj: no such file or directory
Am not an expert in docker neither in azure but I guess I'm setting up this solution in the wrong way to accomplish such thing.
What could be a better setup or fix?
no such file or directory
This is a very common error people encountered after they migrate the Docker project from Visual studio into Azure Devops, even they can build docker very successfully in local.
This caused by the different work logic between Visual Studio(local) and Azure Devops. For local, the docker runs at the Repos/solution level. BUT, for Azure devops CI, it running the docker in the directory where the dockerfile lives, which is at project level. At this time, the relevant path which work fine locally, will not suitable in Azure devops any more.
I guess you may not want to make changes to your dockerfile. So here you just need specify the build context in Docker task:
Specify $(Build.Repository.LocalPath) to the Docker 2.* argument Build context.
Check my previous answer.
I have GitLab running on computer A, development environment (Visual studio Pro) on computer B and Windows Server on computer C.
I set up GitLab-Runner on computer C (Windows server). I also set up .gitlab-ci.yml file to perform build and run tests for ASP.NET Core application on every commit.
I don't know how can I get code on computer C (Windows server) so I can build it (dotnet msbuild /p:Configuration=Release "%SOLUTION%"). It bothers me that not a single example .gitlab-ci.yml I found on net, doesn't pull code form GitLab, before building application. Why?
Is this correct way to set-up CI/CD:
User create pull request (a new branch is created)
User writes code
User commit code to branch from computer B.
GitLab runner is started on computer C.
It needs to pull code from current branch (CI_COMMIT_REF_NAME)
Build, test, deploy ...
Should I use common git command to get the code, or is this something GitLab runner already do? Where is the code?
Why no-one pull code from GitLab in .gitlab-ci.yml?
Edited:
I get error
'"git"' is not recognized as an internal or external command
. Solution in my case was restart GitLab-Runner. Source.
#MilanVidakovic explain that source is automatically downloaded (which I didn't know).
I just have one remaining problem of how to get correct path to my .sln file.
Here is my complete .gitlab-ci.yml file:
variables:
SOLUTION: missing_path_to_solution #TODO
before_script:
- dotnet restore
stages:
- build
build:
stage: build
script:
- echo "Building %CI_COMMIT_REF_NAME% branch."
- dotnet msbuild /p:Configuration=Release "%SOLUTION%"
except:
- tags
I need to set correct variable for SOLUTION. My dir (where GitLab-Runner is located) currently holds this folder/files:
- config.toml
- gitlab-runner.exe
- builds/
- 7cab42e4/
- 0/
- web/ # I think this is project group in GitLab
- test/ # I think this is project name in GitLab
- .sln
- AND ALL OTHER PROJECT FILES #Based on first look
- testm.tmp
So, what are 7cab42e4, 0. Or better how to get correct path to my project structure? Is there any predefined variable?
Edited2:
Answer is CI_PROJECT_DIR.
I'm not sure I follow completely.
On every commit, Gitlab runner is fetching your repository to C:\gitlab-runner\builds.. on the local machine (Computer C), and builds/deploys or does whatever you've provided as an action for the stage.
Also, I don't see the need for building the source code again. If you're using Computer C for both runner and tests/acceptance, just let the runner do the building and add Artifacts item in your .gitlab-ci.yaml. Path defined in artifacts will retain your executables on Computer C, which you are then able to use for whatever purposes.
Hope it helps.
Edit after comment:
When you push to repository, Gitlab CI/CD automatically checks your root folder for .gitlab-ci.yaml file. If its there, the runner takes over, parses the file and starts executing jobs/stages.
As soon as the file itself is valid and contains proper jobs and stages, runner fetches the latest commit (automatically) and does whatever script item tells it to do.
To verify that everything works correctly, go to your Gitlab -> CI / CD -> Pipelines, and check out whats going on. You should see something like this:
Maybe it would be best if you posted your .yaml file, there could be a number of reasons your runner is not picking up the code. For instance, maybe your .yaml tags are not matching what runner is created to pick up etc.
I'm trying to use a remote trigger for (re)building in ci.gitlab. For explaining this, I made up this scenario:
2 repository, "lib" and "app1"
app1 will successfully build only if lib is included (solved simply by .gitlab-ci.yml)
I need to trigger the build of app1 (only for the master branch, in best-case) on commit (or merge request) of lib
I tried to figure it out using web hooks, but I wasn't able to find a url for ci.gitlab.com. Is this possible in a gitlab environment?
You can do this with newly added triggers functionality.
In your CI's project, find the section "Triggers". Add a trigger and use its token like this:
curl -X POST \
-F token=TOKEN \
https://ci.gitlab.com/api/v1/projects/{project_id}/refs/REF_NAME/trigger
(https://about.gitlab.com/2015/08/22/gitlab-7-14-released/)
Obsolete:
we have the same problem, and the way we solved it is by pushing and subsequently deleting a tag.
The assumption is that you manage the machine with Gitlab-CI runner. First, clone the main repository, app1 for you. And in lib's .gitlab-ci.yml add the steps:
- cd /path/to/app1_repository
- git pull
- git tag ci-trigger master
- git push origin ci-trigger
- git push --delete origin ci-trigger
- git tag -d ci-trigger
Make sure that you have the option Tag push events checked in your Gitlab Services settings for Gitlab-CI.
This solution has drawbacks:
Gitlab-CI runner must have write permissions to the repository, so it won't work for shared runners
git history will be bloated with all this tagging (especially Gitlab UI)
I opened an issue for this (https://gitlab.com/gitlab-org/gitlab-ci/issues/223) so let's hope they add this functionality to the API (http://doc.gitlab.com/ci/api/README.html).