Present we kept multiple pipelines in separate yml files and all of them are included in the mail ci yml file.
using include statements.
Is there anyway to make it conditional.
Like I will have pipeline1 and pipeline2 yaml files.
So I can create another pipeline.yml where I can choose among the above two files based on some condition.
Gitlab is designed for deterministic pipelines, all other solutions add-on that looks implicit.
Web service
Include in .gitlab-ci.yml link to the web server, which could be a dynamically generated yml file.
include:
- 'https://your-service.com/gitlab-ci-generate'
Unfortunately, It's not possible to use CI variables in link to server. gitlab issue link
Dynamic child pipelines
You have complete control of the pipeline. Write a simple bash script, which renames your pre-pushed .yml files and executes that pipeline.
Related
I have a great experience of pipeline creating automation (in case of huge amount of repos).
For example, a project has 20 similar repos with Java app (like a microservice) and a pipeline for each of them is differing only by repo url (and a few more minor attributes). The CI/CD process for each of them is the same.
So, we can create a separated devops-repo with declaration configuration for our services. Also we can create a single pipeline which will pull the devops repo and create all needed pipelines for each repo in the configuration (this operation is going to be executed only once in the beginning and in case if we want to change the devops-configuration)
I have implemented that using Jenkins. Now, I am going to do so using GitLab CI. But I can't get how is it possible.
Is it possible to create a pipeline from another one (dynamically)?
Any suggestions?
You can use include and put the generic pipeline in your devops repo.
In your java repos you can include the devops pipeline and set the variables which are specific for the respective java repo.
So the pipeline for your java repos can be as short as this:
include:
- project: 'your-group/devops-repository'
file: '.generic-ci.yml'
variables:
FOO: bar
We have hundreds of similar projects in GitLab which have the same structure inside.
To build these projects we use a one common TeamCity build. We trigger and pass project GitLab URL along with other parameters to the build via API, so TeamCity build knows which exact project needs to be fetched/cloned. TeamCity VCS root accepts target URL via parameter.
The question is how to replace existing TeamCity build with a GitLab pipeline.
I see the general approach is to have CI/CD configuration file(.gitlab-ci.yml) directly in project. Since the structure of the projects the same this is not the option to duplicate the same CI/CD config file across all projects.
I'm wondering is it possible to create a common pipeline for several projects which can accept the target project URL via parameter ?
You can store the full CICD config in a repository and put in all your projects a simple .gitlab-ci.yml which includes the shared file.
With thus approach there is no redundant definition of the jobs.
Still, you can add specific other jobs to specific projects (in the regarding .gitlab-ci.yml files or define variables in a problem and use some jobs conditionally) - you can also include multiple other definition files, e.g. if you have multiple similar projects.
cf. https://docs.gitlab.com/ee/ci/yaml/#include
With latest GitLab (13.9) there are even more referencing methods possible: https://docs.gitlab.com/ee/ci/yaml/README.html#reference-tags
As #MrTux already pointed out, you can use includes.
You can either use it to include a whole CI file, or to include just certain steps. in Having Gitlab Projects calling the same gitlab-ci.yml stored in a central location - you can find detailed explanation with examples of both usages
I have read some tutorials about Azure DevOps. There are 3 things i do not really understand:
Can we say azure-pipeline.yml on Azure is the equivalent of .gitlab-ci.yml on gitlab ?
I have read some tutorials talking about azure-pipeline.yml files and others talking about azure-pipelines.yml ? What is the good syntax for this file name ?
I have create a "devops project" from Azure Services page. I have choose ASP.Net Core Application and Windows Web App. I can see a pipeline on dev.azure.com but there is no yml file in source code. So i am wondering where is this file...
Thanks
Can we say azure-pipeline.yml on Azure is the equivalent of .gitlab-ci.yml on gitlab
YAML defines the way to code your configuration management by defining build and release pipelines in the code.It is named as azure-pipelines in Azure Devops and .gitlab-ci.yml on gitlab
have read some tutorials talking about azure-pipeline.yml files and others talking about azure-pipelines.yml ? What is the good syntax for this file name ?
azure-pipelines.yml is the default name, but if you need you canhange the name of the yaml file by clicking on "Edit in the visual designer".
I have create a "devops project" from Azure Services page. I have choose ASP.Net Core Application and Windows Web App. I can see a pipeline on dev.azure.com but there is no yml file in source code. So i am wondering where is this file...
There are two ways to create the pipeline one is using the classic editor and using the YAML code. It should definitely be there if you create it using YAML
Answering your question in the same order they were asked.
Yes, azure-pipeline.yml is the equivalent of gitlab-ci.yml. In both cases you bundle together a number of commands you want to execute.
It is the same. The file is called azure-pipelines.yml. Good thing if you try to edit the file in azure-devops is that there is a really nice check and auto-completion tool, that helps a lot especially in indentation (spaces before the command)which is a common issue with yaml files.
If you created the Ci/CD pipelines by the development center in azure portal you see a UIed version of the yaml. But for every step you can still see the yaml code if you wish. If you try to create a new build pipeline the default way you get is to use the yaml file.
I have a template repository for build pipelines say 'azure-templates-repo', I have python task template as mentioned below:
steps:
- task: PythonScript#0
inputs:
scriptSource: 'filepath'
scriptPath: 'python_test.yml' #this is located on repo: 'azure-templates-repo'
My question is on scriptPath, when I use this template in a build pipeline azure-pipeline.yml in a repository my-great-app, azure attempts to find the file in my-great-app instead where it really located azure-templates-repo.
So, Is there a way to mention repository for a filePath parameter of azure task 'pythonScript'?
When you use a template in a pipeline, you are only consuming the .yml template file, not the entire repository that contains the template. So by default no other additional files (besides the template itself) that may exist in the template repository will be available when the primary pipeline is composed.
If you need access to scripts or other files that exist in your template repository you will need to use the checkout task and actually checkout the template repository.
- checkout: git://MyProject/MyTemplateRepo
One thing to be aware of if you go down the path of checking out multiple repositories is that it will cause the structure of your $(Build.SourcesDirectory) to change. In practice this can cause pain as you have to update any tasks that expected your primary repository location to be at the root of $(Build.SourcesDirectory).
That mutation of the $(Build.SourcesDirectory) might not be a big deal for you for new pipelines. It can turn into a pain if you have lots of pipelines that you want to consume a new template in, that require supporting scripts.
One option is to package the supporting template scripts, and publish them to an internal package feed. Then within your template block pull down the required script as a package. I have used this strategy before with templates that needed supporting powerShell scripts. We pipeline those scripts and publish them as a universal packages, then consume them at the template level.
I'm trying to setup Azure Pipelines for a CI setup and I'm using the YAML syntax to get started. However, I was wondering if it is possible to script the flow at "runtime"? Like you can do in Jenkins script: spawn builds etc.
Depending on the commit I want to have a vastly different flow.
This is because I currently have a mono-repo setup with Conan libraries and I want to rebuild the libraries that are necessary depending on the commit, thus the build-flow is not the same for each commit. I want to spawn jobs so I can take advantage of parallel building on several agents.
For your issue ,do you refer to trigger builds based on specified commits? If so, you can trigger builds by adding tag trigger in yaml. You can create tags on the commits. If the tag created meets the trigger condition of the tag trigger in yaml , then the build will be triggered.
trigger:
tags:
include:
- v2.*