I am trying to figure out if Azure LogicApp can be used for files/documents migration from Azure Blob Storage to a custom service, where we have REST API. Here is the shortlist of requirements I have right now:
Files/documents must be uploaded into Azure Storage weekly or daily, which means that we need to migrate only new items. The amount of files/documents per week is about hundreds of thousands
The custom service REST API is secured and any interaction with endpoints should have JWT passed in the headers
I did the following exercise according to tutorials:
Everything seems fine, but the following 2 requirements make me worry:
Getting only new files and not migrate those that already moved
Getting JWT to pass security checks in REST
For the first point, I think that I can introduce a DB instance (for example Azure Table Storage) to track files that have been already moved, and for the second one I have an idea to use Azure Function instead of HTTP Action. But everything looks quite complicated and I believe that there might be better and easier options.
Could you please advise what else I can use for my case?
For the first point, you can use "When a blob is added or modified" trigger as the logic app's trigger. Then it will just do operation on the new blob item.
For the second point, just provide some steps for your reference:
1. Below is a screenshot that I request for the token in logic app in the past.
2. Then use "Parse JSON" action to parse the response body from the "HTTP" action above.
3. After that, your can request your rest api (with the access token from "Parse JSON" above)
Related
I would like to send a welcome email to the user when the Azure AD admin assigns the user to the application (enterprise or custom). It would be nice to use a custom template to define an access URL and maybe some additional info (how to use, some rules, etc.). I haven't found anything similar in the Azure portal.
Does Azure AD provide such functionality? Or should I build custom implementation (Graph API, EventGrid + Azure Functions, SCIM protocol, etc.)?
From what I understand, at the very base, you want an email to be sent whenever a user is assigned to an enterprise application (or custom - assuming that also falls under same). I assume you do not need to know much about who actually assigned the permissions. Either way, with the query below you should have enough to get going.
As far as I know there's no native support for this in AD (I couldn't find anything at all that's available but I could be wrong) but a workaround for us was to do it using Log Analytics + Azure Logic App. While our use case is slightly different (we use it to monitor and be alerted for logins to a specific account), the same logic might apply to you - I put together a few lines to query log analytics. but I couldn't get the alert part working - Azure could just be sleepy right now.
EDIT: Alert worked the following day. Just needed some time to warm up I guess.
Give it a try:
Make sure Azure AD has diagnostic settings configured to send logs to a Log Workspace
Query the workspace using the following:
AuditLogs
| where TimeGenerated > ago(5m) //Change as required
| where ActivityDisplayName has "Add app role assignment grant to user"
| project Time = TimeGenerated, Activity = ActivityDisplayName, Application=parse_json(TargetResources)[0].displayName, User=parse_json(TargetResources)[1].userPrincipalName
| where Application contains "myapp"
Create an alert from Log Analytics (hopefully you get it working right away - each alert cost USD 1.50/month).
(a) You will need to create an Action Group
(b) Under Action Group, configure your email in the Notifications.
(c) You will need to come back to reconfigure an Action field to the Logic App you will create below.
Create a Logic App and start with an HTTP connector as the trigger to receive the content in JSON format.
Setup a Send an Email (v2) action with all variables and such, or another connector if applicable in your case. Customise the email in HTML
One alternative to using Log Analytics and Alerts could also be to use PowerShell to query AAD logs and then parse the information to Logic App through the HTTP POST Url that shows up when you save the HTTP Connector.
Another alternative to using the HTTP Connector could be to use the O365 connector with trigger When a new email arrives (v3)
Things to consider:
There's at least a 5 minutes delay between the time the event is logged and triggered. This is just the way the alert query works.
You will need to login to Logic App using the mailbox from which you want to send that email. That's something you might want to manage separately - expiring credentials etc.
Hope this gives you some ideas.
I am trying to dynamically add/update linked service REST based on certain trigger/events to consume a RESP API to be authenticated using cookie which provides telemetry data. This telemetry data will be stored in Data Lake Gen2 and then will use Data Bricks to move to secondary data storage/SQL Server.
Have someone tried this? I am not able to find the cookie based Auth option while adding the linked service REST.
Also how to create data pipes dynamically or to have the parameters of the rest api to be dynamic ?
Currently, unfortunately this is not possible using Azure data factory native components/activities. For now at least, you cannot get access to the response cookies from a web request in data factory. Someone has put a feature request for this or something that might help, see here
It might be possible to do this via an Azure function to get/save the cookie and then send it as part of a following request. I had a similar problem but resorted to using Azure functions for all of it, but I guess you could just do the authentication part with a function! ;-)
EDIT: update
Actually, after I wrote this I went back to check if this was still the case and looks like things have changed. There now appears (never seen this before) in the web response output, a property called "ADFWebActivityResponseHeaders" and as you can see there is property for the "Set-Cookie"
See example below:-
Want to try out the Text Translation API from , I created my text translation resource, as described at https://www.microsoft.com/en-us/translator/business/trial/#get-started, and was presented with a subscription key and an endpoint https://<resource name>.cognitiveservices.azure.com/sts/v1.0/issuetoken.
I proceeded to replicate the sample code from https://learn.microsoft.com/en-us/azure/cognitive-services/translator/quickstart-translate?pivots=programming-language-csharp (also on Github).
That code wants the subscription key and endpoint added as environment variables, and I did so, using the strings provided to me when creating the resource. However, trying to run the sample code it fails with a 404 when going to the endpoint. Looking more carefully at https://learn.microsoft.com/en-us/azure/cognitive-services/translator/quickstart-translate?pivots=programming-language-csharp, I noticed that it told me to use https://api.cognitive.microsofttranslator.com/ as endpoint, not the URL above. After that change, the code worked.
So what is the https://<resource name>.cognitiveservices.azure.com/sts/v1.0/issuetoken endpoint for? Certainly one gets the impression it's the endpoint to use for translation requests, so maybe clarify the documentation there?
I am currently trying out Azure Logic Apps and am now trying to create
a first simple flow that acquires data from an external REST API.
I'm using FRED, a hosted Node-Red instance, to create an REST API end-point.
In FRED you use https://x_y.fred.sensetecnic.com/api/hello to access my mock API (where x_y is my user name).
Accessing "https://x_y.fred.sensetecnic.com/api/hello" in a browser returns the intended data as specified in the Node-Red flow. Same thing goes when accessing
"https://x_y.fred.sensetecnic.com/api/hello" using GET in Postman (then adding two required Headers, required by FRED).
The strange thing is that in my Azure Logic App, I am not able to even save the flow with my URI stated as "https://x_y.fred.sensetecnic.com/api/hello". It states "Enter a valid URI". I enter the headers as was required.
So I can get to the REST API directly from a browser or using Postman, but are not able to access it from within an Azure Logic App.
I test the problem is there is a special characters _ in your URI. You could have a try if you delete _ then it won't state error.
So you coudl Initialize a variable to store your name x_y, then insert the variable in the URI. The flow would be like this. If you don't find the name in Dynamic content, you could use variables('name') this expression to insert it.
With this way the GET method will run correctly.
I'm trying to use the Computer Vision API from Microsoft's Cognitive Services. However, my keys don't seem to be working. I created an account using the free trial of that API and got the two keys from it. Trying to use the key with the ProjectOxford.Vision SDK always yields:
Access denied due to invalid subscription key. Make sure to provide a valid key for an active subscription.
I tried the API console, however I get the same error with my key in the Ocp-Apim-Subscription-Key field. I tried both keys and neither of them work. I even got the free version of Face API and tried its console, but encountered the same issue with its keys. I even tried different datacenters, but they all seem to return the same error.
This would need to be some problem with the key then right? This can't be a problem with my C# code, since the console doesn't work either. And since it's failing in the API console, there's nothing more I can do to rule out any other possibilities is there? I'm not sure what else I can do to debug this. I'd like to regenerate my keys (I saw a tutorial video which showed an older UI of getting the API keys and they used to have a "regenerate" link) but I don't see a way of doing that anymore.
I only just made the account and registered for the APIs, so there's no way I'd be over quota. Is there something else I need to do to enable these keys or something?
I managed to skirt around the issue of 'Access Denied' by performing the following actions:
I created a free Azure account
I set up an instance of the Cognitive Services Api (this generated a pair of new keys for me to use)
Utilizing the new key, I had to use the following link:
https://westus.api.cognitive.microsoft.com/vision/v1.0/ocr
Instead of
https://westus.api.cognitive.microsoft.com/vision/v1.0/recognizeText
(I obtained this link from the Cognitive Services Test Dashboard).
Look at the request pattern on the test dashboard and you should be able to tell how to use the api.
Even when #Xuan Hu response states correctly to the solution, I scratched my head some time trying to figure out how to change the end point. Here are my 2 cents:
Go to portal.azure.com, in the dashboard of your subscription to the Cognitive Services > General Information > End Point take note of the URL. You need it.
Find in the code of your VisionAPI samples where the VisionServiceClient is instantiated:
VisionServiceClient VisionServiceCliente = new VisionServiceClient(SubscriptionKey);
and change including the URL that you found in Azure:
VisionServiceClient VisionServiceCliente = new VisionServiceClient(SubscriptionKey, StringOfMyURLTakedFromPortal);
That worked for me.
If you are using the free trial keys got from azure.microsoft.com. You need to change the API endpoint region to westcentralus. The previous default region is westus and I think that is the reason of the invalid key problem.
FYI, there is a blog post that covers all of the 401 Access Denied scenarios, including this one regarding the free API keys and region specific API endpoint. Adding it here for folks in the future who find this SO post - https://blogs.msdn.microsoft.com/kwill/2017/05/17/http-401-access-denied-when-calling-azure-cognitive-services-apis/.
Using the incorrect regional endpoint
Most of the Cognitive Services APIs are region specific, which means that during API account creation you select which region you want to create the account in. These APIs have region specific endpoints such as westus.api.cognitive.microsoft.com or eastus2.api.cognitive.microsoft.com, and an API key for an account created in one region will only work using the endpoint for that specific region. This means that if you create an API account in West US you will not be able to call the eastus2.api.cognitive.microsoft.com API endpoint.
You can verify the region and endpoint in the Azure management portal.
Trial API Keys
The free trial API keys have 30 day expiration dates, and the same restrictions for region and version. If you are using the trial keys you can go to https://azure.microsoft.com/en-us/try/cognitive-services/my-apis/ to manage your API keys (if you are not already logged in then just click one of the ‘Create’ buttons and you can go through the wizard to login and see your existing API keys), and you will also see the expiration date and endpoint.
One thing to remember if using Postman to get the results is to use GET and put your keys in the Header.