I want add a functionality in my ADF pipeline which will send me email notification in case of failure. On searching the internet, I came to know that Azure Logic Apps helps with this. I am trying to follow below link to achieve this.
https://microsoft-bitools.blogspot.com/2018/03/add-email-notification-in-azure-data.html
I have tried searching up many tutorials, guides and the official docs as well. However, all of them have some templates already there in the Logic Apps Designer. I cannot find the templates and the 'When a HTTP request is received' trigger is also not available in the drop-down.
Please let me know how to proceed.
EDIT :
If you start with a blank Logic App, search for 'HTTP' or 'Request' and select Request.
On the next screen under triggers, select "When a HTTP request is received" and you should be good to go.
EDIT:
It looks like you created a Standard Logic App, which works in a slightly different way. For instance it can contain multiple workflows, which means you create workflows yourself. In the Consumption model, there's one workflow withing a Logic App, so you can open up the editor for that one directly. If there's no explicit reason for you to use Standard, a Consumption Logic App will be easier to work with.
If you really need a Standard Logic App, go to 'Workflows' and create a new workflow:
Then click the newly created workflow to edit it, go to 'Designer' and search for 'HTTP' to add an HTTP trigger:
Here's some information on the Consumption model for Logic Apps:
Resource type
Benefits
Resource sharing and usage
Limits management
Logic App (Consumption) Host environment: Multi-tenant Azure Logic Apps
- Easiest to get started - Pay-for-what-you-use - Fully managed
A single logic app can have only one workflow. Logic apps created by customers across multiple tenants share the same processing (compute), storage, network, and so on.
Azure Logic Apps manages the default values for these limits, but you can change some of these values, if that option exists for a specific limit.
See Resource type and host environment differences for a comparison with the other hosting options.
I was able to solve this. I wasn't able to view a few functionalities because of another error : Functions runtime error Microsoft.WindowsAzure.Storage: Value cannot be null. (Parameter 'connectionString').
AzureWebJobsStorage App Setting was missing which caused the error. I added that and now I can see the triggers and other stuff.
Thanks #rickvdbosch
Related
With Azure Data Factory I have built a pipeline to orchestrate the processing of my Azure Analysis Services model trough a dedicated Logic App as explicated in this article, and it works properly.
Now, always using Azure Data Factory (through Logic App), I wish I could also update the list of the user in a specific roles.
In the article mentioned above, to process the Azure Analysis Services models, the Logic App calls a specific API that has the following format:
https:// <rollout>.asazure.windows.net/servers/<serverName>/models/<resource>/refreshes
but this API doesn't seem to work for update the model's roles.
Is there anyone who knows the correct method to be able to update model roles using a specific Logic App?
Thanks for any suggestions
If you don't necessarily need to use the logic app for this, I think it might be possible using Azure automation and the powershell cmdlets for managing azure analysis services:
https://learn.microsoft.com/en-us/azure/analysis-services/analysis-services-refresh-azure-automation
https://learn.microsoft.com/en-us/azure/analysis-services/analysis-services-powershell
https://learn.microsoft.com/en-us/powershell/module/sqlserver/Add-RoleMember?view=sqlserver-ps
One alternative approach might be to have fixed AD groups as members of the tabular model roles and add / remove members from those AD groups. Therefore the tabular model roles would not need to be refreshed, it would simply be a matter of adding or removing members from the AD groups as part of your governance process.
A second approach would be to use dynamic row-level security. Adding records to a Azure SQL DB table is perfectly possible with Logic Apps and could be used to drive security, depending on your requirements. You can then refresh your security dimension with the Logic App. See here for more details:
https://learn.microsoft.com/en-us/power-bi/desktop-tutorial-row-level-security-onprem-ssas-tabular
To answer your question however, the Azure Analysis Services REST API is useful but is not that fully featured, ie it does not contain all possible operations for tabular models or the service. One other missing example I found was backups, ie although it is possible to trigger a pause or resume of the service, it is not possible to trigger a backup of a tabular model via the REST API. I do not believe it is possible to alter role members or at least, the operation is not listed in the REST API, although happy to be corrected if I am wrong. To be more specific, Roles is not mentioned in the list of available objects which can be passed in to the Objects array using the POST / Refreshes eg here. table and partition are the only ones I'm aware of.
There are also no examples on the MS github site:
https://github.com/microsoft/Analysis-Services
Finally, consider calling TMSL via Powershell in an Azure Function, which you can call from Azure Data Factory.
HTH
I'm having a problem similar to this one. I'm attempting to deploy an API connection via an ARM template, and I'm getting 'ParameterNotDefined' errors when I deploy.
However, instead of creating an API connection to an azure table, I'm connecting to an azure queue.
Instead of getting a specific answer, I'd like guidance on how I can use documentation or other methods to answer this myself: I'm unable to find documentation on the existing APIs that are supported via the "MICROSOFT.WEB/CONNECTIONS" type, as well as the associated 'parameterValues' that each API expects you to supply.
Searching for a solution here is quite difficult because the terms are very overloaded. Here's what I've already tried:
I've tried using the Azure Resource Explorer, but it looks like this browses resources, not available APIs. If I can use it to view APIs, it's not clear how.
I've browsed through samples on the azure quickstart project. I see examples for azure blobs but not queues.
I've read some documentation on this page about creating API connections for logic apps. This appears relevent, and the author mentions making GET requests to "https://management.azure.com/.../" to view API details, but I get authorization header errors when doing this and I'd hope I can get documentation without operating this low on the stack.
I'm going to be creating a lot of API connections with logic apps in the near future and I need a more sophisticated method of discovery then google-searches, stack-overflow, and reuse of code samples. How do I view the entire set of API connections, along with the supported 'parameterValues' that are to be used with each one?
You can use ARMClient to retrieve the needed paremeters.
armclient.exe get https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Web/locations/{region}/managedApis/{Api}?api-version=2016-06-01
For more information https://www.bruttin.com/2017/06/13/deploy-logic-app-with-arm.html
For anyone, checking it now, you can create the Api Connection manually and check the JSON View of it, it contains all the parameters.
How do I change the Azure Logic App Frequency and Request type (Not programmatic ally)
Frequncy and Request type
Thank you for providing a screenshot! So the REQUEST and FREQUENCY info that you're seeing on the Overview section relate to the specific Trigger that you're using in your Logic App. Different triggers will show different info here depending on how they're designed to operate & work.
In your case, it looks like you're using one of Logic Apps' built-in connector called Request which will trigger workflow in your Logic Apps whenever a request is received to its endpoint hence manual and on-demand.
To change this trigger, navigate to Logic Apps Designer view under DEVELOPMENT TOOLS in the Azure Portal as shown below:
Moreover, I recommend reading Logic Apps' documentation here and finding more about triggers and actions; overall concepts behand Logic Apps. Hope this helps and goodluck with your journey on Logic Apps!
I understand that Microservices is about independent loosely coupled services. I have read https://en.wikipedia.org/wiki/Microservices.
When it comes to Azure, I understand there are many components like Azure Service Fabric, AKS and also have the option of deploying containers within Azure VMs using Docker or any other containerization tools. However, since Microservices is about developing atmoic individually scalable services, can this also be achieved by deploying each service as an Azure Web API APP within an App Service Plan and configure Auto-Scale based on Performance metrics (though each API APP may not be individually scalable, they can still be individually manageable in terms of deployment, configuration etc)?
Can someone please suggest if this thought process is correct?
Microservices aren't a platform or technology so if you can make small independently deployable services then they are microservices. Sure - some tech helps but it depends on your situation.
If you only need a few services you probably don't need anything complex. Make sure services are well modeled, own their own data and ideally have a good monitoring and deployment pipeline setup. Design for service failure where possible.
Do you need to scale each part independently? Ideally, you should be able to but do services have very different requirements? You could have many small App service plans but that comes at cost of unused resources so split when you need to.
This question and of course the answers are going to be opinion based, but generally when thinking in terms of micros services, think not in terms as things like loads of API's and VM's etc. Instead think in terms of. When i upload an image, its needs to be resized, and the table updated to give a url for the thumb. or when XXX record is updated in database, Run XXX in order to create a report, or update Azure search. and that each service, just knows how to do a single thing only. I.E Resize an image.
Now one could say. I have a system, A repo library, and some functions library. When an image is posted, I upload, then call this, and that etc.
With Micor services. You would instead just add the image to a queue. Create an azure function that has a queue trigger. that would resize and save both the large and the thumb to storage. this would then either update the database, or in true micro service, it would add a queue to store the new info, another function would watch that queue and insert into the database.
You can use the DB queue from anything. You can use the Blob queue from anything. Your main API, does not care how images are handled. You can change your functions one day to maybe save to dropbox, instead of azure blob. All really easy, with no re-build of the API, because the API does not care.
A good example I use it for is email and SMS. My systems dont know how to send an email, or an SMS. They only know how to add to a queue. My microservices. SendEmail and SendSMS do know how to do it, and I can change how and who i send that content with, really easy. I can tomorrow change from Twilio to send grid, without ever telling the API that i've done it.
On a more complex thing. I have approval, at the moment that approval sends an email or SMS to either user or admin, and that can change over time. So I have an SMS server, Email Service and and approvalService. when approval happens, it just adds a config to the queue, The rest is done by a logic app, that knows to send an email to XXX and an SMS to XXX and then update database. My api, is just a post, that creates a queue.
Basically what I am saying here is to get started, maybe porting an existing app. Start with the workflow stuff, like send an email, resize an image, create a report, create a PDF, email 50 subscribers etc. and take all that code out and put into there own micro service that just knows how to do one thing. Then when you grow with confidence, create a workflow from all of these services with Logic Apps, let azure take care of the rest, thats what they want to do.
I currently have a couple of WebApi projects that use a few class libraries such as address lookup, bank validation, image storage etc.
Currently they are all in a shared solution but I'm planning to split them up. I thought about moving the libraries into NuGet packages so that they are separate from the API projects and is properly shared.
However, if I make a change to one of these components I will need to build and redeploy the API service even though it's a separate component which has changed.
I thought about putting these components into a separate service but seems a bit of overhead for what it is.
I've been looking at Azure WebJobs and think I may be able to move these components into this instead. I have two questions related to this:
Are WebJobs suitable for calling on demand (not using a queue)? The request will be activated from a user on a web site which calls my API service which then calls the Web Job so it needs to be quick.
Can a WebJob return data? I've seen examples where it does some processing and updates a database but I need a response (ideally Json) back to my API service.
Thanks
According to your requirement, I assume that you could try to leverage Azure Functions by creating a function using the HTTP trigger, which could be triggered by accessing the Function URL with parameters and return the response as you expected. You could follow this tutorial for getting started with Azure Functions.