Is it possible to collection traces from classic ETW providers? Or is XPerf limited to manifest-based ones? I have a custom classic provider and was wondering if I could collect its event using XPerf.
You can collect traces from any ETW provider.
Related
Similar past question of mine: Azure Service Bus Queue: Can I manage/cancel scheduled messages? - accepted answer here details how to cancel scheduled messages.
I'm wondering now if there's a way to view scheduled messages using the AMQP APIs. The Azure UI (and this method https://learn.microsoft.com/en-gb/java/api/com.microsoft.azure.management.servicebus._message_count_details?view=azure-java-stable) offer message counts but I can't see anything that lets me actually view those messages.
Is this a limitation of the service bus/Azure architecture? Or is there an API that will allow it? My goal would be to build a nice UI that displays scheduled tasks, and ideally I'd like to do that without maintaining that list of tasks myself outside of Azure if possible.
As of today, all messages can be peeked, but there's no way to peek those based on message status. There's an open issue on GitHub for the broker to add this feature. You can leave use-case scenario to help product team to have an idea why a feature as such would help customers.
Adding some information I found useful while reading about this.
Azure Service Bus Queues allows browsing/peeking scheduled messages.
Azure Service Bus Subscriptions does not support this feature yet.
Read More
View Example
Can the windows diagnostic do the same as the OMS extension in terms of getting performance counter information and event details? Is there a reason to use the OMS extension over WAD for event/performance information?
Can the windows diagnostic do the same as the OMS extension in terms
of getting performance counter information and event details?
WAD: This uses Azure Diagnostics agent for single VM.
OMS: When you use OMS to monitor your VM or multiple Vs, The OMS collect data from Microsoft Monitoring Agent by default. However, OMS can collect data from managed resources into a central repository. This data could include events, performance data, or custom data provided through the API. So, OMS can also use the WAD's data through the storage account which contains the agent's data.
So, OMS has more features than WAD. Also as #4c74356b41 said, if you want to monitor one VM, WAD is enough to achieve that.
Is there a reason to use the OMS extension over WAD for
event/performance information?
OMS focus on collecting data from different place and Log Analytics.
It can collect data from Windows/Linux VMs, Azure services and Data Collector API.
Reporting and analyzing data is the most important feature of OMS. Alerting can also be configured in OMS.
Azure Automation provides process automation and configuration management to OMS.
Over all, if you want to do Analytics from multiple service, OMS is the best choice. If you only want to monitor a single VM rather doing other things, WAD is enough.
if you are looking at performance from one vm - no.
but there is a chance you have more than one vm oms extension suddenly gets more interesting, as you can look at all of the vms at the same time, setup alerts, actions etc.
Is Azure diagnostics only implemented through code? Windows has the Event Viewer where various types of information can be accessed. ASP.Net websites have a Trace.axd file at the root that can viewed for trace information.
I was thinking that something similar might exist in Azure. However, based on the following url, Azure Diagnostics appears to require a custom code implementation:
https://azure.microsoft.com/en-us/documentation/articles/cloud-services-dotnet-diagnostics/#overview
Is there an easier, more built-in way to access Azure diagnostics like I described for other systems above? Or does a custom Worker role need to be created to capture and process this information?
Azure Worker Roles have extensive diagnostics that you can configure up.
You get to them via the Role configuration:
Then, through the various tabs, you can configure up specific types of diagnostics and have them periodically transferred to a Table Storage account for later analysis.
You can also enable a transfer of application specific logs, which is handy and something that I use to avoid having to remote into the service to view logs:
(here, I transfer all files under the AppRoot\logs folder to a blob container named wad-processor-logs, and do so every minute.)
If you go through the tabs, you will find that you have the ability to extensively monitor quite a bit of detail, including custom Performance Counters.
Finally, you can also connect to your cloud service via the Server Explorer, and dig into the same information:
Right-click on the instance, and select View Diagnostics Data.
(a recent deployment, so not much to see)
So, yes, you can get access to Event Logs, IIS Logs and custom application logs without writing custom code. Additionally, you can implement custom code to capture additional Performance Counters and other trace logging if you wish.
"Azure diagnostics" is a bit vague since there are a variety of services in Azure, each with potentially different diagnostic experiences. The article you linked to talks about Cloud Services, but are you restricted to using Cloud Services?
Another popular option is Azure App Service, which allows you many more options for capturing logs, including streaming them, etc. Here is an article which goes into more details: https://azure.microsoft.com/en-us/documentation/articles/web-sites-enable-diagnostic-log/
We are planning to host our NServiceBus application in Azure web and worker roles (using Azure Service Bus as transport). Existing documentation is a bit unclear when it comes to monitoring and logging of Azure based endpoints.
What would be recommended practices to set up logging and monitoring of such setup?
Would you recommend creating a custom logging module inheriting from ILog interface?
Does ServiceControl work well for monitoring endpoints in Azure? If yes, where do we deploy it?
EDIT:
Igorek's answer helped me to solve logging issue. Can anyone comment on ServiceControl part of the question?
NServiceBus can hook into and utilize log4net. log4net can send its data to Windows Event Logs thru standard log4net appender. Azure Diagnostics can move the data from Event Logs to WADLogs table for inspection by on-demand tools such as Cerebrata's Diagnostic Manager or 24x7 monitoring services such as AzureWatch. http://www.paraleap.com
Alternatively, log4net can also send the logs directly to a custom Azure Table if you want to build your own monitoring tools.
Also, consider monitoring queue/topic depths of your Service Bus, to make sure that stuff is moving thru correctly. Also, consider possibly scaling by the number of messages in SB queues or topics if your NSB setup supports dynamically adding/removing instances.
Disclaimer: I work for company that makes AzureWatch and we happen to have a number of customers who utilize AzureWatch to monitor their NSB-enabled apps.
ServiceControl can indeed be used to monitor your Azure endpoints, just set up ServiceControl for the transport you use, drop the ServiceControl plugins into your endpoint bin folder, and you're good to go.
ServiceControl is best deployed locally, or on an azure virtual machine.
We are looking a solution for logging and tracing for our multi-tenant application with distributed architecture, that will be hosted on Azure.
We have already gone through these two articles – Troubleshooting Best Practices for Developing Windows Azure Applications and Enabling Diagnostics in Windows Azure. Is there anything other better solution?
We would like to know
• what are the best practices and approach for it?
o Storage strategy?
• Any third party / open source tool that helps us for the same?
EDIT:
We are looking for two things:
Best practice for storage strategy, where should we store log data? Since it's multi-tenant multi-tier application, should we keep data separate for each tier per tenant, combine them or any better solution? How do we store the data so that we can trace single request individually that spanned across multiple tiers?
A tool that helps us to view trace data, analyse them, filter, sort, etc. Since size of trace data will be comparatively huge, trace a flow of single task that spanned across multiple tiers.
I have used System.Diagnostic with XML listener, in on-premise application - with multiple tiers (web app, service layer 1, service layer 2, etc). I then, used Microsoft Service Trace Viewer to view the log data. SVCTraceViewer supports many features including combining log files of many tier, graphical representation, tracing individual request, etc.
So, some thing similar third party / open source tool for Azure. That also helps support engineer to drill down the issue and resolve it.
I would recommend looking into an open source library like log4net. It provides a pluggable/fully configurable and super flexible way to log messages with a lot of custom data and to a lot of sources. Configuration for it can be retrieved from external sources/xml, code, config files, etc.
You can create your own appender for Table Storage or find someone else's
HTH