I want to be notified via slack whether my scheduled pipeline runs are failing or passing. I also do not want a trigger every time the pipeline fails or passes, I want it to be specifically limited to the scheduled pipelines
Related
How to schedule a one time run, non-repeating pipeline in AzurDevOps. I want to create this pipeline for our UAT environment, but I don't want to run it manually, so I was thinking is there a way I can put multiple specific dates to run the pipeline?
In short, we can't schedule a non-repeating pipeline in DevOps because it defines a schedule using cron syntax.
Each Azure Pipelines scheduled trigger cron expression is a space-delimited expression with five entries(Minutes, Hours, Days, Months, Days of week).
If you need to run pipeline at some specific days, as a workaround, please schedule it on your end and call the Rest API to run your pipeline.
There are the detailed steps: https://blog.geralexgr.com/cloud/trigger-azure-devops-build-pipelines-using-rest-api.
I have pipeline need to run at every one hour.
Using tumbling window trigger.
If pipeline is successfull it should continue to run every one hour.
If pipeline fails with some reason, next instance, I mean pipline should not run for next hour.
How can we stop trigger if pipeline fails.
Currently, this feature is not available in the Azure data factory. You can raise a feature request from the Azure data factory feedback.
Alternatively, you can add a web activity in your pipeline upon the failure of the previous activity. In web activity, you can make a call to an HTTP request to Stop the trigger.
Refer to this document to Stop a trigger.
I had around 30 pipelines (each doing its own build, deploy, tests) , all in same project.
Instead of having to manually trigger all 30 pipelines each time, I wanted to create a separate pipeline YAML which on running it can trigger all the 30 individual pipelines.
Is there a way to achieve this?
I understand from documentation there is concept to add the pipeline triggers. However, I was not able to understand if single yaml can trigger individual pipelines - and if so, whether it is getting triggered at the completion of the pipeline or at the start of it.
Flow I was looking for is -
There are 30 individual pipelines each having complete flow for services:
stages:
stageA
stageB
stageC
Now, I was trying to create a pipeline yaml all_apps.yml which triggers all the 30 individual pipelines at once.
Configure pipeline to trigger multiple pipelines
There are several ways to accomplish it, you can choose the one that suits you.
First, we could set the Build completion for those 30 pipelines:
Go the edit page of the triggered yaml pipeline(Deploy pipeline), Click the 3dots and choose Triggers :
Go to Triggers--> Build completion and click add--> Select your triggering pipeline(all_apps.yml pipeline):
Second, there is an extension Trigger Azure DevOps Pipeline, we could use this task to trigger those 30 pipelines.
Third, you could it with both the Runs API and Build Queue API, both work with Personal Access Tokens. You can also use loops to make REST API calls more graceful. Check this thread for some more details.
I have a published and scheduled pipeline running at regular intervals. Some times, the pipeline may fail (for example if the datastore is offline for maintenance). Is there a way to specify the scheduled pipeline to perform a certain action if the pipeline fails for any reason? Actions could be to send me an email, try to run again in a few hours later or invoke a webhook. As it is now, I have to manually check the status of our production pipeline at regular intervals, and this is sub-optimal for obvious reasons. I could of course instruct every script in my pipeline to perform certain actions if they fail for whatever reason, but it would be cleaner and easier to specify it globally for the pipeline schedule (or the pipeline itself).
Possible sub-optimal solutions could be:
Setting up an Azure Logic App to invoke the pipeline
Setting a cron job or Azure Scheduler
Setting up a second Azure Machine Learning pipeline on a schedule that triggers the pipeline, monitors the output and performs relevant actions if errors are encountered
All the solutions above suffers from being convoluted and not very clean - surely there must exist a simple, clean solution for this problem?
This solution reads from the logs of your pipeline and let's you do something within a Logic App capability, I used it to email the team when a scheduled pipeline failed.
Steps:
Create Event Namespace and Event Hub
Create Service Bus Namespace and Service Bus Queue
Create a Stream Analytics Job using EventHub as Input and Service
Bus Queue as Output
Create Logic App with a trigger to any event coming into the Service
Bus Queue then, add an Outlook 360 send an email (v2) step
Create an Event Subscription inside ML Service that sends filtered
events to the Event Hub
Start Stream Analytics Job
Two fundamental steps while creating the Event subscription:
Subscribe to the 'Run Status Changed' event to get the log when a pipeline fails
Use the advanced filters section to specify which pipeline you want to monitor (change 'deal-UAT' to your specific ml experiment), like this:
It looks like a lot of setup but it's super easy and quick to do, it would look something like this in the end:
I created with my user, one trigger in the Pipeline triggers of my project. I successfully executed the trigger and it appears on the job page. But since I want this trigger to be used by all developers on my team, I need to see who triggered the job. But always appears on the job page, that I, who created the trigger, triggered all jobs and not the other user who actually triggered it.
There is any way that can I show who triggered the job?
I tried to use, as another option the creating pipeline via API and setting a private token to each user. But when I pass a variable in the request, apparently the Gitlab API is not working with the variables option. Because, when we create a pipeline trigger and request it, we don't pass a user token, we pass the trigger token.
https://gitlab.com/gitlab-org/gitlab-ce/issues/52926
If you navigate to the repository that contains the CI Pipeline, under CI/CD>Pipelines there should be a table showing all past instances of the pipeline. There is a "Triggerer" column which should show you who triggered it.
If you are still unable to see who triggered it, perhaps you need to adjust your own permissions