I want to create logging for API and azure functions.
As thinking to use, "service bus" to create logging.
Logging needs for each request, response and error.
Would it be correct approach to do logging for api management and azure functions through service Bus. and appreciate if any example to create service bus and call from azure or api management to log the requests/responses.
Note: Regarding Application Insights, as found its hit the performance and its more for performance monitoring then logging. https://blogs.msdn.microsoft.com/apimanagement/2018/01/12/application-insights-integration/
I would still use Application Insight for that. You want to track requests and errors which Applicatin Insight will offer out of the box and also provides you a query language to query your logs or to build dashboards. Regarding your performance concern, you should just test the impact on your system - it most likely isn't that relevant.
Related
Is there a supported/standard way to send container logs (info, errors etc.) to Azure Monitor, as there is for metrics with the Metrics collector module in Azure IoT Edge?
I know about this solution, but is there a supported/standard way to solve this?
https://github.com/veyalla/logspout-loganalytics
/Jonas
There is IoT ELMS, which is mentioned in the docs. It leverages an OpenTelemetry module that uploads logs to Log Analytics. It also describes a second way of doing it, which utilizes a timed Azure Function to send Direct Methods to the device, asking for the logs and then uploading them to Log Analytics.
But the repository you linked is by Venkat Yalla; his team is responsible for the Metrics collector, so I'd say that module is as close to standard as it'll get.
I have several Azure WebJobs (.Net Framework, Not .Net Core) running which interact with an Azure Service Bus. Now I want to have a convenient way to store and analyze their Log-Messages (incl. the related Message from the Service Bus). We are talking about a lot of Log Messages per Day.
My Idea is to send the Logs to an Azure Event Hub and store them in an Azure SQL Database. Later I can have for example a WebApp that enables Users to conveniently browse and analyze the Logs and view the Messages.
Is this a bad Idea? Should I instead use Application Insights?
Application insight charges are more than your implementation. So i would say this is good idea. Just one change i would send each logs to logic apps and do some processing like sorting error logs, info logs etc differently. Also why are you thinking about SQL when this can be stored in non SQL Azure tables and fetch them from there.
I am working on Azure web API which uses Log4Net Application Insight appender to track traces. We are planning to use the Application Insight SDK directly (TrackTrace(), TrackEvent()) in order to use built-in alerting features.
However it does seem like, Azure is not supporting trace or event alerts but only metric alerts. So we got some issue there.
If I go one step back, web API is invoked by number of Logic Apps runs in x time intervals. Logic App simply calls web API (business logic is here) and it log all information/managed exceptions.
The main requirement is to be proactive when an exception happens like sending a mail to technical inbox. Secondary requirement is notify sources, if it got any data issues.
Any suggestion on our approach please, what we can do more to fulfill our requirements?
Please refer to this App Insights Exception Alerts. It is possible to set alerts for rates of exceptions in a defined time period.
Set up Exception alerts
You can also invoke webhooks to perform additional actions when the alert is fired.
Thanks Sreejit for your suggestion.
To conclude the answer, what's the preferred engine to process alerts please, through application insight or log analytics?
application insight only support metric alerts only, so the application exceptions need to be flagged as custom metrics as opposed to traces.
[TelemetryClient --> TrackTrace() vs TrackMetric()]
If we go for log analytics, we can use custom events. Then use alert management solution in OMS workspace. Perhaps even use of a separate logic app to built the logic of sending alerts.
[TelemetryClient --> TrackEvent()]
What are the logging options in Azure API Management? From what I understand, Azure API Management can generate logging of errors that occur in it (like invalid subscription key, invalid endpoint, etc). How can identify and configure loggable events in APIM? How can that logging output be wired into a custom logging service hosted through APIM?
Microsoft's recommendation is to send the events to EventHub. You can then process the events there and log them however you want. This approach minimizes the performance impact of logging on your APIs.
https://azure.microsoft.com/en-us/updates/azure-api-management-and-azure-event-hubs-integration/
We are planning to host our NServiceBus application in Azure web and worker roles (using Azure Service Bus as transport). Existing documentation is a bit unclear when it comes to monitoring and logging of Azure based endpoints.
What would be recommended practices to set up logging and monitoring of such setup?
Would you recommend creating a custom logging module inheriting from ILog interface?
Does ServiceControl work well for monitoring endpoints in Azure? If yes, where do we deploy it?
EDIT:
Igorek's answer helped me to solve logging issue. Can anyone comment on ServiceControl part of the question?
NServiceBus can hook into and utilize log4net. log4net can send its data to Windows Event Logs thru standard log4net appender. Azure Diagnostics can move the data from Event Logs to WADLogs table for inspection by on-demand tools such as Cerebrata's Diagnostic Manager or 24x7 monitoring services such as AzureWatch. http://www.paraleap.com
Alternatively, log4net can also send the logs directly to a custom Azure Table if you want to build your own monitoring tools.
Also, consider monitoring queue/topic depths of your Service Bus, to make sure that stuff is moving thru correctly. Also, consider possibly scaling by the number of messages in SB queues or topics if your NSB setup supports dynamically adding/removing instances.
Disclaimer: I work for company that makes AzureWatch and we happen to have a number of customers who utilize AzureWatch to monitor their NSB-enabled apps.
ServiceControl can indeed be used to monitor your Azure endpoints, just set up ServiceControl for the transport you use, drop the ServiceControl plugins into your endpoint bin folder, and you're good to go.
ServiceControl is best deployed locally, or on an azure virtual machine.