I am trying to find a good example of the json body for Create Build Definition in Azure Devops. Most of the documentation I find has api definitions, but I haven't been able to see an example json body to work from.
Microsoft Documentation:
https://learn.microsoft.com/en-us/rest/api/azure/devops/build/definitions/create?view=azure-devops-rest-5.1
I have found this article that describes doing something similar to what I hope to accomplish. However, they are trying to duplicate the same build definition across different projects.
Similar Example:
https://www.nebbiatech.com/2018/11/29/automating-build-pipeline-creation-using-azure-devops-services-rest-api/
Ultimately, I would like to be able to generate (either create new or clone/modify) as many standard build definitions within a single project as are necessary by my automation. Each one of these build definitions will pull from a different repository within the project and have a different cosmetic name for the pipeline, but will be otherwise identical.
Any suggestions are greatly appreciated. Thanks!
For the usage of YAML build as comment suggested, it will meet your requirements. It letting you define your build in a YAML file that lived with your code. This meant you could use the same branching and code review practices for your build definitions as you did for your code.
The best way to get started with YAML pipelines is through the quickstart guide and Customize your pipeline . After that, to learn how to configure your YAML pipeline the way you need it to work, see conceptual topics such as Build variables and Jobs.
As for a sample of application/json body when you use Rest API to create build definition. You could also refer below links:
How to create Build Definitions through VSTS REST API
Create VSTS Build Definitions using PowerShell
Related
my ideal case is to build a secure CI/CD pipeline using cloud build applying SAST and other security test in order to prevent the developer to deploy code that has a smell or have any vulnerabilities
I currently using Github as my version control platform and hosting my app on google cloud
and with the help of google cloud build i did build a CI/CD pipeline to deploy my app each time we have an update , the current flow is whenever create a pull request in one of the main branches that reflects our environments prod/staging/dev a check status is triggered which is cloud build CI/CD.
the pipeline currently just build, scan and then push the image also starting the managed insatnce group update but its now my focus for this question, i am trying to follow the secure methodologies for DevSecOps by applying some security tests in the early stages like SAST,SCA and image scanning
so the issue is that google does not have the full ecosystem currently and no clear path for DevSecOps using cloud build as the Ci tool for such as what third party tools to integrate with in order to fill the gaps? this was the only clear path i did found until now https://cloud.google.com/blog/products/devops-sre/devsecops-and-cicd-using-google-cloud-built-in-services a blog to follow but not answering all the questions
what i have achieved until now is applying an IDE SAST tool like sonarlint also i found snyk
Although i mainly focused on the free tools as a start in order to test because as i said no clear integrations with cloud build is out there except for snyk.io i did found an integration with cloud build but its even not a native support yet
the reason for using cloud build not github actions that i don't want to store any service account key outside our project because it will be a general behavior in my company and its a more safer to keep our keys inside , so i need some ideas what tools i can integrate with cloud build in order to achieve that and how to apply it and is there a way to use github actions and cloud build at the same time while orchestrating the pull request check status order ? for example check the source code before start building the docker image
I want to create CI on Github Action for QA Automation. But there is multiple language are use to install dependecies. Can i use NodeJS and Golang at the same file?
I read the documentation of Github Action, but there is configuration for each language not both. Any reference or idea i can use?
In short, you write a manifest file (in YAML) and tell GitHub Actions build agent(s) to execute the commands you wanted in an automatic way. See, there is nothing there bind to a single programming language.
You see per language samples/tutorials, simply because that's how new users/developers to get started with a CI/CD system, and it is easy to write up the necessary steps if focusing on the ecosystem of a single programming language.
The underlying GitHub Actions build machines (if managed by GitHub), however, have almost everything pre-installed, so of course you can use Node.js and Golang tools in the same manifest and you don't need any specific reference.
Open the image pages and learn what tools are preinstalled if you like.
Try it out by combining multiple manifests into the single one, and you will see how it works out.
My company is in the process of migrating all our pipelines over to Databricks from AzureML, and I have been tasked with refactoring one of our existing pipelines made with azureml-sdk (using functions such as PipelineData, PythonScriptStep etc.), and converting it into a dbx pipeline which uses a deployment.yml file.
I have found this "Deployment file reference" on dbx documentation page, and I think it's quite adequate compared to some of AzureML's documentation. However, if I had an example project to compliment that page, it would help me greatly to put it into practice.
Is there any repos/sources which gives an example of building a dbx pipeline which uses .py-files instead of notebooks?
However, if I had an example project to compliment that page, it would help me greatly to put it into practice.
Please take a look at the Quickstart doc which generates a sample project and walks you through it step by step.
If you're looking for more profound and in-depth example with orientation towards MLOps practices, take a look at the following session - MLOps on Databricks: A How-To Guide. It also links to an example repo with dbx.
I've requested to my Team Lead that we start integrating a CI/CD pipeline into most, if not all, of our projects. Our newest project relies heavily on our own, external class library that is referenced in the solution ; it is under "Dependencies" as a project reference.
The project runs fine when I build it in my machine using Visual Studio 2019, and before we needed to integrate an external library, it would build and release fine using our Azure DevOps pipelines.
However, with the addition of an external class library, when I try to run a build through Azure DevOps, I get the following error:
The project file ....csproj was not found.
I fully understand why it can't find it - because I need to pull in the external class library and build that first! There doesn't seem to be a lot of online material (not that I could find anyway!) that describes solutions to this other than "use nuget" ; unfortunately, it is a requirement from my Team Lead that this is not a route we go down - which has lead to a long couple of days!
With this in mind, I can't find another way to do this in Azure DevOps. I have looked into some sort of PowerShell command but to no avail thus far.
Has anyone run into this issue before with external class libraries in DevOps and can give me advice on the best way to approach it?
Generally speaking in 99,99% cases keeping a direct reference to the project is not a good idea. You can end up with really unmaintainable CI/CD logic and/or with dll versions mismatches during deployments. Actually I am an Architect in the project where I tried to fix that issue by migrating all dependencies to the NuGet server.
Azure Artifacts
You mentioned, that you are using Azure DevOps as main CI/CD tool, so this is a great opportunity to introduce Azure Artifacts as internal nuget server which is a part of Azure DevOps. For the first 2 GB it is free, here you have pricing details.
Alternatives
If for some reason you cant use Azure Artifacts, I recommend some alernatives:
MyGet
ProGet
Own nuget server
More information about alternatives you can find in this article.
I want to evaluate ARM template file which contains actual values passed by the user before it passed to deployment engine.
Is there any way to do that?
I have started creating evaluation code through PEGJS library of nodejs by using which I can evaluate only particular condition or expression present in AZURE ARM functions but can't evaluate actual template which is passed to deployment engine to create any service.
Also, I have checked sdk azure-rest-client but can't find any way please help me to find the solution for the above issue?
There is no built-in way of doing that, you can use validate deployment api call (its also implemented in different sdk's\cli's) but it doesnt actually guarantee that the template will work, it does some basic sanity checks.
You best bet is to write a script that would deploy the template and a set of tests that would validate the outcode.