I am running my GitLab CI pipelines in an offline environment with none of my repositories containing the pipeline YAML files. Due to the sheer number of repositories I enabled AutoDevOps to use a baseline image until I can define pipelines for each specific repository as necessary, however, the application dependencies and pipeline images cannot be downloaded. I have a means to resolve this, but it requires configuration in the .gitlab-ci.yml file meaning the default AutoDevOps templates cannot be used and I have to make changes in every single repository, defeating the point of enabling AutoDevOps in the first place.
Is there a way to override the default AutoDevOps templates with my own?
From the documentation :
Auto DevOps is completely customizable because the Auto DevOps
template is just an implementation of a .gitlab-ci.yml file.
So AutoDevOps pipelines are only .gitlab-ci.yml files and you need to include them to overrides jobs and anything in this pipeline. You need to include a .gitlab-ci.yml file to override the default.
AutoDevOps is useful even if you include a ci file because it add a lot of jobs without writing them.
Related
In my project I'd like to have two separate Gitlab CI/CD configurations, each for specific Gitlab instance. These file names could look like:
.gitlab-ci-1.yml
.gitlab-ci-2.yml
Is that possible to specify in the Gitlab which configuration file it should use instead of using default .gitlab-ci.yml?
Yes, this is possible. See Specify a custom CI/CD configuration file.
I have a great experience of pipeline creating automation (in case of huge amount of repos).
For example, a project has 20 similar repos with Java app (like a microservice) and a pipeline for each of them is differing only by repo url (and a few more minor attributes). The CI/CD process for each of them is the same.
So, we can create a separated devops-repo with declaration configuration for our services. Also we can create a single pipeline which will pull the devops repo and create all needed pipelines for each repo in the configuration (this operation is going to be executed only once in the beginning and in case if we want to change the devops-configuration)
I have implemented that using Jenkins. Now, I am going to do so using GitLab CI. But I can't get how is it possible.
Is it possible to create a pipeline from another one (dynamically)?
Any suggestions?
You can use include and put the generic pipeline in your devops repo.
In your java repos you can include the devops pipeline and set the variables which are specific for the respective java repo.
So the pipeline for your java repos can be as short as this:
include:
- project: 'your-group/devops-repository'
file: '.generic-ci.yml'
variables:
FOO: bar
We have hundreds of similar projects in GitLab which have the same structure inside.
To build these projects we use a one common TeamCity build. We trigger and pass project GitLab URL along with other parameters to the build via API, so TeamCity build knows which exact project needs to be fetched/cloned. TeamCity VCS root accepts target URL via parameter.
The question is how to replace existing TeamCity build with a GitLab pipeline.
I see the general approach is to have CI/CD configuration file(.gitlab-ci.yml) directly in project. Since the structure of the projects the same this is not the option to duplicate the same CI/CD config file across all projects.
I'm wondering is it possible to create a common pipeline for several projects which can accept the target project URL via parameter ?
You can store the full CICD config in a repository and put in all your projects a simple .gitlab-ci.yml which includes the shared file.
With thus approach there is no redundant definition of the jobs.
Still, you can add specific other jobs to specific projects (in the regarding .gitlab-ci.yml files or define variables in a problem and use some jobs conditionally) - you can also include multiple other definition files, e.g. if you have multiple similar projects.
cf. https://docs.gitlab.com/ee/ci/yaml/#include
With latest GitLab (13.9) there are even more referencing methods possible: https://docs.gitlab.com/ee/ci/yaml/README.html#reference-tags
As #MrTux already pointed out, you can use includes.
You can either use it to include a whole CI file, or to include just certain steps. in Having Gitlab Projects calling the same gitlab-ci.yml stored in a central location - you can find detailed explanation with examples of both usages
Is it possible to specify no-sources files that should not trigger the Gitlab CI?
When I make changes in README.md, the pipeline triggers, thought that file is only the documentation inside the gitlab and is not packaged in anz output artifact.
You can control when each of your jobs is added to a running pipeline using the only, except, or rules keywords. The easiest way to prevent jobs from running when only the README is changed is with except:
build_job:
stage: build
except:
changes:
- README.md
With this job syntax, if the only file that changes in a push is README.md, this job will not run. Unfortunately you can only set these rules at a job level, not the pipeline level so you'd have to put this in each of your jobs to prevent them all from running.
I have a template repository for build pipelines say 'azure-templates-repo', I have python task template as mentioned below:
steps:
- task: PythonScript#0
inputs:
scriptSource: 'filepath'
scriptPath: 'python_test.yml' #this is located on repo: 'azure-templates-repo'
My question is on scriptPath, when I use this template in a build pipeline azure-pipeline.yml in a repository my-great-app, azure attempts to find the file in my-great-app instead where it really located azure-templates-repo.
So, Is there a way to mention repository for a filePath parameter of azure task 'pythonScript'?
When you use a template in a pipeline, you are only consuming the .yml template file, not the entire repository that contains the template. So by default no other additional files (besides the template itself) that may exist in the template repository will be available when the primary pipeline is composed.
If you need access to scripts or other files that exist in your template repository you will need to use the checkout task and actually checkout the template repository.
- checkout: git://MyProject/MyTemplateRepo
One thing to be aware of if you go down the path of checking out multiple repositories is that it will cause the structure of your $(Build.SourcesDirectory) to change. In practice this can cause pain as you have to update any tasks that expected your primary repository location to be at the root of $(Build.SourcesDirectory).
That mutation of the $(Build.SourcesDirectory) might not be a big deal for you for new pipelines. It can turn into a pain if you have lots of pipelines that you want to consume a new template in, that require supporting scripts.
One option is to package the supporting template scripts, and publish them to an internal package feed. Then within your template block pull down the required script as a package. I have used this strategy before with templates that needed supporting powerShell scripts. We pipeline those scripts and publish them as a universal packages, then consume them at the template level.