I want to create Azure Logic App which will constantly request a specific website on the Internet and parse received HTML.
I've created Logic App and set up interval and HTTP request action.
Which action should I choose as the next step for simple regex operation on HTML code?
What comes to my mind is creating Azure Function which will do the job, but I wonder if there is any other solution, more suitable for such task.
I want it the be simple as possible.
Edit:
Just found out some cool feature. Logic Apps contain some basic expressions for primitive types.
Unfortunetly it lacks of any regex or string.contains.
For now, I'll try with Azure Functions.
I've managed to solve my problem with use of Workflow Definition Language and building blocks provided by Azure.
The Azure Function idea was not that bad and would fit perfectly for any more complex case, but as I mentioned, I wanted it as simple as possible, so here it is.
This is how my flow looks now.
For sake of completeness, here is the flow in JSON format
{
"$connections": {
"value": {
"wunderlist": {
"connectionId": "/subscriptions/.../providers/Microsoft.Web/connections/wunderlist",
"connectionName": "wunderlist",
"id": "/subscriptions/.../providers/Microsoft.Web/locations/northeurope/managedApis/wunderlist"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Condition": {
"actions": {
"Create_a_task": {
"inputs": {
"body": {
"completed": false,
"list_id": 000000000,
"starred": true,
"title": "#{variables('today date')}"
},
"host": {
"connection": {
"name": "#parameters('$connections')['wunderlist']['connectionId']"
}
},
"method": "post",
"path": "/tasks",
"retryPolicy": {
"type": "none"
}
},
"limit": {
"timeout": "PT20S"
},
"runAfter": {},
"type": "ApiConnection"
},
"Set_a_reminder": {
"inputs": {
"body": {
"date": "#{addHours(utcNow(), 3)}",
"list_id": 000000,
"task_id": "#body('Create_a_task')?.id"
},
"host": {
"connection": {
"name": "#parameters('$connections')['wunderlist']['connectionId']"
}
},
"method": "post",
"path": "/reminders",
"retryPolicy": {
"type": "none"
}
},
"limit": {
"timeout": "PT20S"
},
"runAfter": {
"Create_a_task": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"expression": "#contains(body('HTTP'), variables('today date'))",
"runAfter": {
"Initialize_variable": [
"Succeeded"
]
},
"type": "If"
},
"HTTP": {
"inputs": {
"method": "GET",
"uri": "..."
},
"runAfter": {},
"type": "Http"
},
"Initialize_variable": {
"inputs": {
"variables": [
{
"name": "today date",
"type": "String",
"value": "#{utcNow('yyyy/MM/dd')}"
}
]
},
"runAfter": {
"HTTP": [
"Succeeded"
]
},
"type": "InitializeVariable"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"Recurrence": {
"recurrence": {
"frequency": "Day",
"interval": 1,
"startTime": "2017-08-01T23:55:00Z",
"timeZone": "UTC"
},
"type": "Recurrence"
}
}
}
}
You can use inline code action in logic app to run javascript regex code (preview- May 2019) (Not supported on Flow).
Iniline Code
Logic App Inline Code Ref
You're probably on the right track. An Azure Function would be the most appropriate way to implement this right now. An API App is an option but that's a heavier platform than you need.
create an Azure Function along the lines of:
{
log.Info("C# HTTP trigger function processed a request.");
// Get request body
dynamic data = await req.Content.ReadAsAsync<object>();
// Set name to query string or body data
string input = data?.input.ToString();
var regexJson = data?.regexList;
var regexes = regexJson.ToObject<List<RegexReplace>>();
foreach (var regex in regexes)
{
var re = Regex.Replace(regex.Regex, "\\\\","\\");
var replace = Regex.Replace(regex.Replace, "\\\\","\\");
input = Regex.Replace(input, "\\\"","\"");
input = Regex.Replace(input, re, replace);
}
input = Regex.Replace(input, "[\\r\\n]", "");
return data.regexList == null
? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a name on the query string or in the request body")
: req.CreateResponse(HttpStatusCode.OK, input, "application/json");
}
public class RegexReplace
{
public string Regex { get; set; }
public string Replace { get; set; }
}
This is my function to use for replacing text in a string. this is reusable and the approach can be used for many similar type of aspects of working in Logic Apps:
using System.Net;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
dynamic data = await req.Content.ReadAsAsync<object>();
string removeme = data?.removeme;
string replacewith = data?.replacewith;
string value = data?.value;
return req.CreateResponse(HttpStatusCode.OK, value.Replace(removeme, replacewith));
}
I would then post an object like this from my logic app:
{
"removeme": "SKU-",
"replacewith": "P-",
"value": "SKU-37378633"
}
... to get the result: "P-37378633"
Related
Here is my Scenario,
There will be a drop of csv file into blob storage every day ,that will be processed by my dataflow in ADF and generate a csv in output folder.
Now Using logic apps, I need to send that csv file (less than 10 mb ) as an attachement to customer via Outlook connector.
Besides ,My body of the email must have dynamic value coming from that blob csv .
For example 'AppWorks' is the column value in column 'Works/not'. Sometimes it may be "AppNotWorks".So How to handle this scenario in Azure logic apps
You can use the combination of both data factory and logic apps to do this. Use look up activity to get the first row of the file (Since the entire column value will be same, we can get the required value from one row).
Now use web activity to trigger the logic app. Pass the logic app's HTTP request URL to web activity. In the body, pass the following dynamic content:
#activity('Lookup1').output.firstRow
When you debug the pipeline, the logic app will be successfully triggered. I have given the Request Body JSON schema to get values individually. For the sample I have taken, it would look as shown below:
{
"properties": {
"customer": {
"type": "string"
},
"id": {
"type": "string"
}
},
"type": "object"
}
Create a connection to storage account to link the required file.
Now, using the Outlook connector, send the Email.
The following is the entire Logic app JSON:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Get_blob_content_(V2)": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/v2/datasets/#{encodeURIComponent(encodeURIComponent('AccountNameFromSettings'))}/files/#{encodeURIComponent(encodeURIComponent('JTJmZGF0YSUyZnNhbXBsZTEuY3N2'))}/content",
"queries": {
"inferContentType": true
}
},
"metadata": {
"JTJmZGF0YSUyZnNhbXBsZTEuY3N2": "/data/sample1.csv"
},
"runAfter": {},
"type": "ApiConnection"
},
"Send_an_email_(V2)": {
"inputs": {
"body": {
"Attachments": [
{
"ContentBytes": "#{base64(body('Get_blob_content_(V2)'))}",
"Name": "sample1.csv"
}
],
"Body": "<p>Hi #{triggerBody()?['customer']},<br>\n<br>\nRandom description</p>",
"Importance": "Normal",
"Subject": "sample data",
"To": "<to_email>"
},
"host": {
"connection": {
"name": "#parameters('$connections')['office365']['connectionId']"
}
},
"method": "post",
"path": "/v2/Mail"
},
"runAfter": {
"Get_blob_content_(V2)": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"manual": {
"inputs": {
"schema": {
"properties": {
"customer": {
"type": "string"
},
"id": {
"type": "string"
}
},
"type": "object"
}
},
"kind": "Http",
"type": "Request"
}
}
},
"parameters": {
"$connections": {
"value": {
"azureblob": {
"connectionId": "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.Web/connections/azureblob",
"connectionName": "azureblob",
"id": "/subscriptions/xxx/providers/Microsoft.Web/locations/westus2/managedApis/azureblob"
},
"office365": {
"connectionId": "/subscriptions/xxx/resourceGroups/v-sarikontha-Mindtree/providers/Microsoft.Web/connections/office365",
"connectionName": "office365",
"id": "/subscriptions/xxx/providers/Microsoft.Web/locations/westus2/managedApis/office365"
}
}
}
}
}
The following is the resulting Mail image for reference:
How can you create a child resource (e.g. container for Blob Storage, or consumer group for Eventhub) using Logic App?
To create a resource with the ARM connector in Logic App, you need to specify provider and short resource id which are used to construct the path to the new service. However, they do not correspond to the "type" and "name" parameters from ARM template (which would be in the example "Microsoft.Eventhub/namespaces" and "vvtesteventhub").
"inputs": {
"body": {...},
"host": {
"connection": {
"name": "#parameters('$connections')['arm']['connectionId']"
}
},
"method": "put",
"path": "/subscriptions/#{variables('subscriptionId')}/resourcegroups/#{variables('resourceGroup')}/providers/Microsoft.EventHub/namespaces/vvtesteventhub",
"queries": {
"x-ms-api-version": "2021-06-01-preview"
}
}
For a child resource, it is necessary to somehow construct the full path including the parent resource name. However, I am not able to construct it even when editing the directly through the code view (see below). The run fails with error message "Resource not found", despite the fact that it includes the correct path to the existing eventhub where I want to create the consumer group.
{
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['arm']['connectionId']"
}
},
"method": "put",
"path": "/subscriptions/#{variables('subscriptionId')}/resourcegroups/#{variables('resourceGroup')}/providers/Microsoft.EventHub/namespaces/eventhubs/#{variables('eventhubNamespacesName')}/#{variables('eventhubName')}/consumergroups/#{variables('platformName')}",
"queries": {
"x-ms-api-version": "2021-06-01-preview"
}
}
}
We have tested this in our local environment it is working fine, Below statements are based on the analysis.
In our local environment, we have created an event hub Namespace & a logic app.
Using logic app Azure resource Manager connector actions Create or Update a Resource we are able to create EventHub & followed it by a consumer group in it.
Here is the logic app :
Here is the code view of the logic app:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Compose": {
"inputs": "Microsoft.EventHub/namespaces",
"runAfter": {},
"type": "Compose"
},
"Create_a_ConsumerGroup_to_existing_EventHub": {
"inputs": {
"body": {
"location": "westus"
},
"host": {
"connection": {
"name": "#parameters('$connections')['arm']['connectionId']"
}
},
"method": "put",
"path": "/subscriptions/#{encodeURIComponent('<sub-id>')}/resourcegroups/#{encodeURIComponent('<rgName>')}/providers/#{encodeURIComponent(outputs('Compose'))}/#{encodeURIComponent('/<EventHubNamespacesName>/eventhubs/<EventHubName>/consumergroups/<ConsumerGroupName>')}",
"queries": {
"x-ms-api-version": "2021-11-01"
}
},
"runAfter": {
"Creates_a_EventHub_to_existing_EventHubNamespaces_": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Creates_a_EventHub_to_existing_EventHubNamespaces_": {
"inputs": {
"body": {
"location": "westus"
},
"host": {
"connection": {
"name": "#parameters('$connections')['arm']['connectionId']"
}
},
"method": "put",
"path": "/subscriptions/#{encodeURIComponent('<sub-id>')}/resourcegroups/#{encodeURIComponent('<rgName>')}/providers/#{encodeURIComponent(outputs('Compose'))}/#{encodeURIComponent('/<EventHubNamespacesName>/eventhubs/<EventHubName>')}",
"queries": {
"x-ms-api-version": "2021-11-01"
}
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"Recurrence": {
"recurrence": {
"frequency": "Hour",
"interval": 3
},
"type": "Recurrence"
}
}
},
"parameters": {
"$connections": {
"value": {
"arm": {
"connectionId": "/subscriptions/<sub-id>/resourceGroups/<rgName>/providers/Microsoft.Web/connections/arm",
"connectionName": "arm",
"id": "/subscriptions/<sub-id>/providers/Microsoft.Web/locations/eastus/managedApis/arm"
}
}
}
}
}
Here is the sample output for reference:
Note:
In order to create a consumer group to EventHub you need to have an existing EventHub or you Need to create a New Event Hub.
We are building nested logic apps where a logic app will call another logic app to aggregate some data.
Currently we have Logic Apps A, B and C all being setup with a GET request.
Logic App ABC that aggreagtes all the data will be exposed by the API over a GET request and internally call A to C to gather all the data.
When we use Postman however we get an error message:
{
"error": {
"code": "TriggerRequestMethodNotValid",
"message": "The HTTP method for this request is not valid: expected 'Get' and actual 'POST'."
}
}
Which would imply that ABC calls A with a POST instead of a GET.
We've got the hunch, that this is due to us calling the Logic App with a body element.
We temporarly fixed the problem by setting all logic app calls to POST, but we would like to avoid this, since we might expose A as system layer API and would like to keep it as a GET.
The code for A:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Filter_is_null": {
"actions": {
"Get_locations": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['salesforce']['connectionId']"
}
},
"method": "get",
"path": "/datasets/default/tables/#{encodeURIComponent(encodeURIComponent('Location__c'))}/items"
},
"runAfter": {},
"type": "ApiConnection"
},
"Response": {
"inputs": {
"body": "#body('Get_locations')?['value']",
"statusCode": 200
},
"kind": "Http",
"runAfter": {
"Get_locations": [
"Succeeded"
]
},
"type": "Response"
}
},
"else": {
"actions": {
"Get_locations_filtered": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['salesforce']['connectionId']"
}
},
"method": "get",
"path": "/datasets/default/tables/#{encodeURIComponent(encodeURIComponent('Location__c'))}/items",
"queries": {
"$filter": "#triggerBody()?['filter']"
}
},
"runAfter": {},
"type": "ApiConnection"
},
"Response_error": {
"inputs": {
"body": {
"component": "sf-locations-get",
"message": "bad request - validate filter"
},
"statusCode": 400
},
"kind": "Http",
"runAfter": {
"Get_locations_filtered": [
"Failed"
]
},
"type": "Response"
},
"Response_filtered": {
"inputs": {
"body": "#body('Get_locations_filtered')?['value']",
"statusCode": 200
},
"runAfter": {
"Get_locations_filtered": [
"Succeeded"
]
},
"type": "Response"
}
}
},
"expression": {
"and": [
{
"equals": [
"#triggerBody()?['filter']",
"#null"
]
}
]
},
"runAfter": {},
"type": "If"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"request": {
"inputs": {
"method": "GET",
"schema": {
"properties": {
"filter": {
"type": "string"
}
},
"type": "object"
}
},
"kind": "Http",
"type": "Request"
}
}
},
"parameters": {
"$connections": {
}
}
}
Image of A for reference:
Image of ABC for reference:
Is there a possibility to call nested logic apps, containing a body using GET?
You can call another logic app from one logic app and pass a body to it.
If you are calling another logic app which triggers with a http request the method expected is generally POST. Try to pass the body and check if that helps in your case.
Please refer to this document
You may also check these and see if these suffices your requirement:
https://www.serverlessnotes.com/docs/nested-logic-apps
https://blog.sandro-pereira.com/2016/04/19/the-ability-to-call-nested-logic-apps-directly-from-logic-apps-designer/
Using a body in a GET request breaks the internet.
In short, this breaks the semantics that a resource-uri in combination with the headers is the cacheable key. When you add a body to the GET request, the caching ability is removed.
It is by design that you cannot and should not add a body to a GET request.
You should rethink your call setup and when you need to send a body, then you need to use other HTTP methods that do not break proper semantics. e.g. POST, PUT, etc.
More info: https://stackoverflow.com/a/983458/1581925
Is it possible to save the response from the HTTP request(First Step) into the Blob Storage(Second Step) while using Azure Logic App.
Thank you.
Yes, You can achieve it by using Http and Create blob task.
Code
{
"$connections": {
"value": {
"azureblob": {
"connectionId": "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.Web/connections/azureblob",
"connectionName": "azureblob",
"id": "/subscriptions/xxx/providers/Microsoft.Web/locations/xxx/managedApis/azureblob"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Create_blob": {
"inputs": {
"body": "#triggerBody()",
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/files",
"queries": {
"folderPath": "/testing",
"name": "Test",
"queryParametersSingleEncoded": true
}
},
"runAfter": {},
"runtimeConfiguration": {
"contentTransfer": {
"transferMode": "Chunked"
}
},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"HTTP": {
"inputs": {
"method": "GET",
"uri": "https://reqres.in/api/users?page=2"
},
"recurrence": {
"frequency": "Minute",
"interval": 3
},
"type": "Http"
}
}
}
}
Update 1:
Just update your blob name with the expression like utcNow('yyyyMMdd')
Presuming you have no extraordinary circumstances, yes, you for sure can save the Request content to a Blob. Any Content really.
You will start with the Create Blob Action specifying the Trigger Body as the content.
Don't over think it, it's the same pattern as saving to a local file.
Using Logic App in Azure to post message to slack. This works fine with standard text message.
When I change to also post attachment nothing gets sent:
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['slack']['connectionId']"
}
},
"method": "post",
"path": "/chat.postMessage",
"queries": {
"attachments": [
{
"color": "danger",
"fallback": "Azure alert attachment.",
"fields": [
{
"title": "Check list"
},
{
"value": "Check services on VM0 and VM1"
},
{
"value": "If you cannot fix this issue make sure someone else can"
}
],
"pretext": "<!channel> Action required",
"text": "`'#{triggerBody()['context']['name']}'` API down - '#{triggerBody()['context']['resourceName']}' Details: #{body('Http')['id']}",
"ts": 123456789
}
],
"channel": "#devops",
"text": "SYST ALERT"
}
}
Looking into this, it seems that Logic Apps does not support Attachment type. Please upvote in uservoice #
https://feedback.azure.com/forums/287593-logic-apps/suggestions/31896379-add-support-for-attachments-with-slack-post-messag
So with that being the case, how do we do this today. Slack Supports Incoming Webhooks as well as APIs. I enabled this in Logic Apps using chat.PostMessage API for more details on the API look at :
https://api.slack.com/methods/chat.postMessage/test
The basic problem in this approach is the requirement for a token, in my sample i used a test token from
https://api.slack.com/custom-integrations/legacy-tokens
This isn't the best approach (but i was having some issues using the Incoming WebHook will continue trying that approach and post if i have success there) from a Security PoV but does get the Job done. Final working code is as following :
{
"$connections": {
"value": {
"office365": {
"connectionId": "<ConnectionID",
"connectionName": "office365",
"id": "<ID>"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Condition_3": {
"actions": {
"HTTP": {
"inputs": {
"body": "?token=<Token>&channel=C8270DY6L&attachments=%5B%7B%22fallback%22%3A%22Requiredplain-textsummaryoftheattachment.%22%2C%22color%22%3A%22%2336a64f%22%2C%22pretext%22%3A%22Optionaltextthatappearsabovetheattachmentblock%22%2C%22author_name%22%3A%22BobbyTables%22%2C%22author_link%22%3A%22http%3A%2F%2Fflickr.com%2Fbobby%2F%22%2C%22author_icon%22%3A%22http%3A%2F%2Fflickr.com%2Ficons%2Fbobby.jpg%22%2C%22title%22%3A%22SlackAPIDocumentation%22%2C%22title_link%22%3A%22https%3A%2F%2Fapi.slack.com%2F%22%2C%22text%22%3A%22Optionaltextthatappearswithintheattachment%22%2C%22fields%22%3A%5B%7B%22title%22%3A%22Priority%22%2C%22value%22%3A%22High%22%2C%22short%22%3Afalse%7D%5D%2C%22image_url%22%3A%22http%3A%2F%2Fmy-website.com%2Fpath%2Fto%2Fimage.jpg%22%2C%22thumb_url%22%3A%22http%3A%2F%2Fexample.com%2Fpath%2Fto%2Fthumb.png%22%2C%22footer%22%3A%22SlackAPI%22%2C%22footer_icon%22%3A%22https%3A%2F%2Fplatform.slack-edge.com%2Fimg%2Fdefault_application_icon.png%22%2C%22ts%22%3A123456789%7D%5D&pretty=1",
"method": "POST",
"uri": "https://slack.com/api/chat.postMessage"
},
"runAfter": {},
"type": "Http"
}
},
"expression": "#equals(triggerBody()?['HasAttachment'], True)",
"runAfter": {},
"type": "If"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"When_a_new_email_arrives": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['office365']['connectionId']"
}
},
"method": "get",
"path": "/Mail/OnNewEmail",
"queries": {
"folderPath": "Inbox",
"importance": "Normal"
}
},
"recurrence": {
"frequency": "Minute",
"interval": 3
},
"splitOn": "#triggerBody()?['value']",
"type": "ApiConnection"
}
}
}
}