I want to add pipeline details(status, id, who triggered it) to my SIEM solution. For that, can you suggest me how to get that information.
Some rough Ideas
Invoke a lambda from Pipeline and supply pipeline related information through SNS
If the above is possible, can you tell me how. I couldn't find a way to do it dynamically for all pipelines. I don't want to hardcore Project and Organization details.
Not very familiar with SIEM, but for azure devops pipelines, you can get their detailed information programmatically by REST APIs.
You can use Pipelines - Get to get information about pipeline's status, id, who triggered it and so on.
GET https://dev.azure.com/{organization}/{project}/_apis/pipelines/{pipelineId}?api-version=6.0-preview.1
You can click this link for more resources of Azure DevOps Pipelines REST APIs.
It uses PAT or OAuth2 as the authentication method and returns information as JSON.
Updates:
Here is a REST API Pipelines - List that can get a list of pipelines in a project.
GET https://dev.azure.com/{organization}/{project}/_apis/pipelines?api-version=6.0-preview.1
Related
I'd like to send an email when we deploy on staging, and the email should contain the tasks, or the Pull Requests (title) that will be added to the environment. Is there a way to do this? I thought about comparing the Pull Requests' creation data, with the latest deploy's date, but I don't know if is this possible. I'm not too experienced with Azure DevOps pipelines, and I'm curious if this is possible?
By default, Azure DevOps sends an email notification to team members after the compilation succeeds or fails.
To create it for CI/CD pipeline check here the complete steps to configure email setting.
For more information please refer this SO THREAD: Best way to send email notification in an Azure DevOps build pipeline
I would like to call an Azure Pipeline API from within a stage of a Pipeline. Specifically to get the status of one pipeline from another so that a job can be forced to wait until the other pipeline is not busy.
I can call the API with a PAT locally. I am just not sure of the best way of passing auth from within the pipeline. Does the agent have some kind of built in auth mechanism with devops apis? Does the agent itself need a PAT and if so what's the best way of providing it one?
The System.AccessToken as detailed here https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml was whart I required to call pipeline apis.
Can anyone please share the details how to create a test case in azure devops using rest api, preferably in python code. I couldn't find a direct API in azure devops but they shared an API for work item. But I don't have an idea how the body should be while creating a test case. Please suggest.
https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/work%20items/create?view=azure-devops-rest-6.1
How to create test case in azure devops using rest api in python
The test case is also a work item. That is the reason why they shared an API for work item for you.
When you create a workitem with TestCase type:
Then we could find it display on the test case tab:
So, we could use the REST API Create a work item to create a workitem with Testcase type to create the test case:
PATCH https://{instance}/DefaultCollection/{project}/_apis/wit/workitems/${workItemTypeName}?api-version={version}
On the other hand, we could check the REST API about the Test cases, there are Get a test case, Delete a test case but no create the test case API directly.
To create a workitem in python, you could refer this official Azure DevOps Python API doc.
It contains Python APIs for interacting with and managing Azure
DevOps. These APIs power the Azure DevOps Extension for Azure CLI. To
learn more about the Azure DevOps Extension for Azure CLI, visit the
Microsoft/azure-devops-cli-extension repo.
Here is some example code for creating work item in python.
You could check this thread for some more details.
I don't know where else to ask this question so would appreciate any help or feedback. I've been reading the SDK documentation for azure machine learning service (in particular azureml.core). There's a class called Pipeline that has methdods validate() and publish(). Here are the docs for this:
https://learn.microsoft.com/en-us/python/api/azureml-pipeline-core/azureml.pipeline.core.pipeline.pipeline?view=azure-ml-py
When I call validate(), everything validates and I call publish but it seems to only create an API endpoint in the workspace, it doesn't register my pipeline under Pipelines and there's obviously nothing in the designer.
My question: I want to publish my pipeline so I just have to launch from the workspace with one click. I've built it already using the SDK (Python code). I don't want to work with an API. Is there any way to do this or would I have to rebuild the entire pipeline using the designer (drag and drop)?
Totally empathize with your confusion. Our team has been working with Azure ML pipelines for quite some time but PublishedPipelines still confused me initially because:
what the SDK calls a PublishedPipeline is called as a Pipeline Endpoint in the Studio UI, and
it is semi-related to Dataset and Model's .register() method, but fundamentally different.
TL;DR: all Pipeline.publish() does is create an endpoint that you can use to:
schedule and version Pipelines, and
re-run the pipeline from other services via a REST API call (e.g. via Azure Data Factory).
You can see PublishedPipelines in the Studio UI in two places:
Pipelines page :: Pipeline Endpoints tab
Endpoints page :: Pipeline Endpoints tab
I have created Azure Data Factory with Copy Activity using C# and Azure SDK.
How can deploy it using CI/CD ?
Any URL or link will help
Data Factory continuous integration and delivery is now possible with directly through the web user interface using ARM Templates or even Git (Github or Azure DevOps).
Just click on "Set up Code Repository" and follow the steps.
Check the following link for more information, including a video demostration: https://aka.ms/azfr/401/02
One idea that I got from Microsoft was that using the same Azure SDK you could deserialize the objects and save down the JSON files following the official directory structure into your local GitHub/Git working directory
In other words you would have to mimic what the UI Save All/Save button does from the portal.
Then using Git bash, you can just commit and push to your working branch (i.e. develop) and from the UI you can just publish (this will create an adf_publish release branch with the ARM objects)
Official reference for CI using VSTS and the UI Publish feature: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment
Unfortunately, CI/CD for ADF is not very intuitive at first glance.
Check out this blog post where I'm describing what/how/why step by step:
Deployment of Azure Data Factory with Azure DevOps
Let me know if you have any questions or concerns and finally - if that works for you.
Good luck!
My resources on how to enable CI/CD using Azure DevOps and Data Factory comes from the Microsoft site below:
Continuous integration and delivery (CI/CD) in Azure Data Factory
I am still new to DevOps and CI/CD, but I do know that other departments had this set up and it looks to be working for them.