We are building a set of serverless functions in Azure, but having difficulty deciding how to structure our source (Azure GIT) and DevOps to support them.
I am thinking of a single GIT repo, with all function apps housed independently within projects. We may have a lot of these function apps, we see great value in small code segments to do utility type of work, and I don't want dozens and dozens of independent repos just because of DevOps deployments. Is there a way to have a unique build and release process for each project, not the repo entirely? We aren't clear how this can be done and searches have come up empty on this. I thought it was possible to have unique build YAMLs per project across many projects in a single repo - but unclear how to implement the DevOps build and release pipleines to support this approach - ie; only a single function gets updated and we need to deploy - any guidance if this is possible and how to approach it would be great.
I haven't done this myself, but I'm in a similar situation where I'd like to have multiple functions (and other stuff) in a single Git repo for simplicity, but only build/deploy them as needed when they change. It looks like you can have multiple pipelines on a single repo with a different YAML file for each pipeline. The steps are documented in this link, and summarized below
In Azure DevOps, create a new Pipeline.
For the "Where is your code?" page, at the bottom choose the Use the classic editor option.
Select your source repo and branch.
On the "Select a template" screen, choose the YAML option at the top. Hit Apply.
There is a YAML file path field where you can specify the path and name of your YAML file for the pipeline.
You may want to set the pipeline to run manually if you don't want a build each time there's a commit to the repo.
EDIT There may be an easier way to do this now. If you go through the New Pipeline wizard, select your source location, on the Configure tab, at the bottom you can choose the Existing Azure Pipelines YAML file option. This lets you select a custom YAML file directly.
Related
I have a number of solutions, each of which have a mixture of applications and libraries. Generally speaking, the applications get built and deployed, and the libraries get published as NuGet packages to our internal packages feed. I'll call these "apps" and "nugets."
In my Classic Pipelines, I would have one build for the apps, and one for the nugets. With path filters, I would indicate folders that contain the nuget material, and only trigger the nuget build if those folders had changes. Likewise, the app build would have path filters to detect if any app code had changed. As a result, depending on what was changed in a branch, the app build might run, the nuget build might run, or both might run.
Now I'm trying to convert these to YAML. It seems we can only have one pipeline set up for CI, so I've combined the stages/jobs/steps for nugets and apps into this single pipeline. However, I can't seem to figure out a good way to only trigger the nuget tasks if the nuget path filters are satisfied and only the app tasks if the app path filters are satisfied.
I am hoping someone knows a way to do something similar to one of the following (or anything else that would solve the issue):
Have two different CI pipelines with their own sets of triggers and branch/path filters such that one or both might run on a given branch change
Set some variables based on which paths have changes so that I could later trigger the appropriate tasks using conditions
Make a pipeline always trigger, but only do tasks if a path filter is satisfied (so that the nuget build could always run, but not necessarily do anything, and then the app build could be triggered by the nuget build completing, and itself only do stuff if path filters are satisfied.
It seems we can only have one pipeline set up for CI
My issue was that this was an erroneous conclusion. It appeared to me that, out of the box, a Pipeline is created for a repo with a YAML file in it, and that you can change which file the Pipeline uses, but you can't add a list of files for it to use. I did not realize I could create an additional Pipeline in the UI, and then associate it to a different YAML file in the repo.
Basically, my inexperience with this topic is showing. To future people who might find this, note that you can create as many Pipelines in the UI as you want, and associate each one to a different YAML file.
I have a great experience of pipeline creating automation (in case of huge amount of repos).
For example, a project has 20 similar repos with Java app (like a microservice) and a pipeline for each of them is differing only by repo url (and a few more minor attributes). The CI/CD process for each of them is the same.
So, we can create a separated devops-repo with declaration configuration for our services. Also we can create a single pipeline which will pull the devops repo and create all needed pipelines for each repo in the configuration (this operation is going to be executed only once in the beginning and in case if we want to change the devops-configuration)
I have implemented that using Jenkins. Now, I am going to do so using GitLab CI. But I can't get how is it possible.
Is it possible to create a pipeline from another one (dynamically)?
Any suggestions?
You can use include and put the generic pipeline in your devops repo.
In your java repos you can include the devops pipeline and set the variables which are specific for the respective java repo.
So the pipeline for your java repos can be as short as this:
include:
- project: 'your-group/devops-repository'
file: '.generic-ci.yml'
variables:
FOO: bar
I'm using a deployment slot called staging on my Azure web.
My application use some specfic files settings.I would replace these file after swapping.
Is there a way to do a post-swap ? I'd like to replace some files automatically when the swap is finish ( to put the new settings )
Thanks
I think that the path of least resistance for you is going to be to use an Azure DevOps Pipeline. A pipeline is made up of one or more tasks, and there are lots of tasks to choose from.
You can check out source code from a specific branch, build, publish to a slot, swap slots, and copy files using scripted pipelines.
Here is a full list of available tasks. If you would like to copy files from within a repo from one place to another, just take a look at the command line task. The pipeline runs on a Windows or Linux machine (your choice when you choose a Build Agent), so you can use normal DOS copy commands to move files around.
I have a release pipeline which i use to deploy my resources to other environments. All works fine but the problem is that every time i deploy, all the resources even if no modification is made, are deployed. Is there a way through which i can do selective deployment; i.e. I deploy only those resources which have been modified. Any help would do. Thanks.
That`s a broad question. There is no out-of-box feature to select units to deploy. But you can use variables in the release pipeline:
Define a variable for each resource/unit and set some default value and "Settable at release time" property.
For each resource, define a separate task to deploy and define the custom condition, like: and(succeeded(), eq(variables['Custom.DeployUnit1'], 'YES'))
You can update these variables at the release creation time:
Is there any way to do selective deployment in azure devops?
There is no such out of box way to selective deployment in azure devops.
That because Azure devops release does not support release only changed files since only release changed files not always meaningful and could not archive what the project intend to release (such as the config file only changed in a commit).
But you could create a PowerShell script to compare timestamp for all files:
Create XML file that stores the last upload/publish information of
each files (e.g. file name, date time, changeset/commit version).
Create a PowerShell script file that included the logical to compare
files (get files metadata and compare with that XML file) and copy
updated files to specific folder
Publish the files in that folder
Check the similar thread for some more details.
Besides, if deploying via the deploy.cmd or MSDeploy.exe, you could also use the the -useChecksum WebDeploy flag:
WebDeploy/MSDeploy Quick Tip: Only Deploy Changed Files
Hope this helps.
Suppose we have 100 static websites of similar type. It will have similar build pipeline tasks. So instead of creating build and release pipelines one by one using visual designer, is there a way to automate it so that it will get created automatically?
You can do that via rest api, also, if all the pipelines are in different repos you can use azure-pipelines.yaml in the root of the repo, it will pick it up automatically.
go to builds > edit > top right >
on the next screen you can rename it: