What are the logging options in Azure API Management? From what I understand, Azure API Management can generate logging of errors that occur in it (like invalid subscription key, invalid endpoint, etc). How can identify and configure loggable events in APIM? How can that logging output be wired into a custom logging service hosted through APIM?
Microsoft's recommendation is to send the events to EventHub. You can then process the events there and log them however you want. This approach minimizes the performance impact of logging on your APIs.
https://azure.microsoft.com/en-us/updates/azure-api-management-and-azure-event-hubs-integration/
Related
I am using API management services to expose my APIs, i am using consumption based instance for API management.
I have published 4-5 APIs and products in that. I want to analytics the behavior of my APIs exposed by viewing the http responses and if possible request and response.
What is the best way to do this as in consumption based, i cannot see api analytics in left nav of api management services
There are number of options one can perdue with Consumption tier:
Use diagnostic settings to store gateway logs in Log Analytics and use it's capabilities to analyze the data: https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-use-azure-monitor#resource-logs
Use APIMs Application Insights integration https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-app-insights
Build custom pipeline sending data from policy using log-to-eventhub or even send-request policies https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-log-event-hubs
As per this Microsoft Document, Built-in API Analytics are not supported for the consumption tier if you want to use analytics you need to increase your plan.
I am looking for some design guidance in using Azure APIM as a forward proxy between backend services (such as function apps or logic apps) and external clients.
We have a function app that processes invoice requests (for an order previously placed) from a mainframe system, does some transformations and sends them to an external party via HTTP. I have two options with the sending process.
Store external HTTP endpoint as an app setting in function app and let function app send the
transformed message directly to external party.
Send the transformed message to Azure APIM with the external HTTP endpoint set as the backend service (using set-backend-service APIM policy) and let APIM forward it to external party.
Now, are there are any real benefits with the second approach if the requirement is only to forward requests to external party from a security or operational standpoint? There are no requirements to enrich the outgoing message (add or remove headers, modify message body etc).
Personally, all I see is an extra hop with the second approach. But some of my teammates seem to think the second approach can offer some benefits in the way of analytics. There are some metrics available under APIM --> Analytics tab that lets you see the number of successful, failed requests etc. But I think this information is also available to Application Insights tied to the function app.
Just interested to see some insights around this so we can make an informed decision.
I guess using Azure API Management would be an over-engineering in your case, assuming the Azure Function is only going to be the caller of the Azure API Management instance.
Even though Azure API management provides analytics, url-rewrites, transformations, analytics etc - it makes more sense when you use it to govern the API. Ask yourself a question on, how many of the Azure API management functionalities are going to be leveraged fully in your case.
Remember Azure API management by design is an API governance tool where you can perform throttling, monetizing your API by giving access to it to different 3rd party with segregated subscription keys. In my opinion, Azure API management comes into the scene where you need to distribute some API to 3rd party applications, which clearly not in your case.
I want to create logging for API and azure functions.
As thinking to use, "service bus" to create logging.
Logging needs for each request, response and error.
Would it be correct approach to do logging for api management and azure functions through service Bus. and appreciate if any example to create service bus and call from azure or api management to log the requests/responses.
Note: Regarding Application Insights, as found its hit the performance and its more for performance monitoring then logging. https://blogs.msdn.microsoft.com/apimanagement/2018/01/12/application-insights-integration/
I would still use Application Insight for that. You want to track requests and errors which Applicatin Insight will offer out of the box and also provides you a query language to query your logs or to build dashboards. Regarding your performance concern, you should just test the impact on your system - it most likely isn't that relevant.
How can I correlate a single request across multiple Azure services in Application Insights?
Say we expose a "Create Case" API endpoint in API Management, using an API App.
The API App does some work, including triggering a Logic App.
How can I see the "flow" of the request throughout all the various Azure services to give a single "view" of the state of a particular case?
And I'm only saying Application Insights as we sort of use it in Web Apps / API Apps, so any other Azure based tool is fine
What we do:
in API Management inbound policy we create a GUID (CorrelationId)
which we then pass on either in HTTP headers to API backends or in Message
Properties through Service Bus Queues/Topics
API Management logging (including the CorrelationId) is pushed to EventHub which then is logged to AppInsights with an Azure Functions
all APIs and Functions that handle Queue etc. messsages also log to AppInsights
In summary: all logic components that can log to AppInsights take the CorrelationId and put it into the CustomDimensions. This way we keep track of all steps a request takes.
You can use Client Tracking Id for this.
The client tracking ID is a value that will correlate events across a logic app run, including any nested workflows called as a part of a logic app. This ID will be auto-generated if not provided, but you can manually specify the client tracking ID from a trigger by passing a x-ms-client-tracking-id header with the ID value in the trigger request (request trigger, HTTP trigger, or webhook trigger).
i have question regarding Azure API Management again : ).
i am using API management which is API Gateway doing HTTPS to Azure Storage REST API directly
and is there any way that i cant get email notification when there are massive requests or high latency at response ??
Thanx for reading : )
You can configure alert notifications either in the portal or via the REST API or .NET SDK to monitor for specific Azure Storage Metrics that you want.
See https://azure.microsoft.com/en-us/documentation/articles/insights-receive-alert-notifications/ for more details.
For massive requests, you might want to consider using "TotalRequests" or "TotalBillableRequests" in a specific time period.
For high response latency, you can track "AverageE2ELatency" or "AverageServerLatency" in a specific time period.
See https://azure.microsoft.com/en-us/documentation/articles/storage-monitoring-diagnosing-troubleshooting/#monitoring-performance for more details on these specific metrics and how they relate to performance monitoring.
Hope this helps.
Sriprasad's answer makes sense for configuration from the Storage side. From the API Management side, you cannot currently set a notification on any event other than the built-in ones (subscription requests, new subscriptions, application gallery requests, new issues/comments, approaching of user subscription quota limit).
You can use Log-To-Eventhub policy to log a message to event hub for every request and consume it in a custom or third party solution like AppInsights/Runscopee to fire an alert.
Refer
https://azure.microsoft.com/en-us/documentation/articles/api-management-log-to-eventhub-sample/
If your requirement is to get report/metrics from API Management you can use the management rest api's for APIM.
https://msdn.microsoft.com/en-us/library/dn781421.aspx
Specifically you might want to look at reportByAPI (which gives you useful metrics in response like callcounts, apiTimeAvg) based on which you can setup alerts/email notification.
https://msdn.microsoft.com/en-us/library/dn781421.aspx#ReportByAPI