I have a website hosted on Azure. I am using Trace.Error to output all my error logs to file system. However when I enable Application Logging on Azure website, it only remains enabled for 12 hours.
This is also confirmed in this article: http://www.hanselman.com/blog/StreamingDiagnosticsTraceLoggingFromTheAzureCommandLinePlusGlimpse.aspx
Now I will like to keep storing error logs indefinitely (i.e. till my website is live). I am not sure if I am missing the point here. How can I keep logging enabled forever?
You can store your logs in Azure Storage (tables or blobs). This doesn't have the 12 hour constraint that the file system does.
Related
I'm setting up Azure WebApp logging. My concern is that error logs are stored in webapp server level, the size increasing day by day from Elmah. Is there a best approach to maintaining the logs, both storing and automating archiving or deleting?
My web development is based on angular. Any suggestion for aggregating logs, like what kind of logs would be generated?
Yes, by default, logs are not automatically deleted (with the exception of Application Logging (Filesystem)). To automatically delete logs, set the Retention Period (Days) field. You could automate the deletion by leveraging KUDU Virtual File System (VFS) Rest API. For a sample script, checkout this discussion thread for a similar approach:
How can you delete all log files from an Azure WebApp using powershell?
Just to highlight, these are the logging that you could capture on WebApps:
• Detailed Error Logging
• Failed Request Tracing
• Web Server Logging
• Application logging - you can turn on the file system option temporarily for debugging purposes. This option turns off automatically in 12 hours. You can also turn on the blob storage option to select a blob container to write logs to.
For log directory information kindly refer to this document: https://learn.microsoft.com/azure/app-service/troubleshoot-diagnostic-logs
I have created nodejs Web API, hosted as Azure App Service.
What would be the best way to log errors/info/debug/warning?
We're using Azure Log Analytics with some custom node-winston middleware that sends an async request out to the ALA REST service. However, there is a bit of a lag between the event being sent from our node Web App and it appearing in the ALA dashboard so although it will be good for monitoring a production environment it's not great for rapid debugging or testing.
If you just write to console.log then everything does get stored in a log file that you can access through the Kudu console. Kudu also has the ability to do a live tail of the console, as does the azure command line interface. Because of this we're debugging using those and leaving ALA for the future.
Once we figure out what the pattern is for those logs being written (i.e. filename/size/time/etc.) we'll drop a scheduled Azure Function in to regularly archive those logs into cold blob storage.
I'll also add that according to the Twelve Factor App in XI they recommend writing logs to stdout, which is what console.log does. I always take these opinionated frameworks/methodologies as guidance and not strict rules, but at these seem to be grounded in reality and will at the very least spawn some interesting discussions among your team.
As you're using Azure, I would recommend the Application Insights:
https://learn.microsoft.com/en-us/azure/application-insights/app-insights-nodejs
I have an Azure WebApp that continually reports "Your app experienced failure(s) due to a transient storage access issue." The suggested solution is "Explore Local Cache feature for your web app." but my webapp exceeds the maximum storage (3GB) for this option.
The problem mostly occurs between midnight and 6am in the morning when the site is LEAST active, but there seems to be an increasing number of occurrences during the day.
What are the underlying causes of this problem? Is this something to do with my WebApp or is it the Azure Infrastructure. In either case, how do I determine the underlying issue(s) and resolve?
"Your app experienced failure(s) due to a transient storage access issue."
The Web Apps environment provides diagnostic functionality for logging information from both the web server and the web application. You could try to enable Logging and check the logs that generated within that period of time.
According to the error, it seems that a temporary issue causes app failure, and it suggests enabling local cache. You could follow the suggested solution and make sure if it helps resolve issue.
Besides, you could try to scale your web app (which would take additional charge) and check if it could mitigate the issue.
Updates:
As we know App Service offers a shared, persistent storage for the application. Maybe something wrong with the shared storage when the instances in farm access the storage, which may be the cause of the issue.
To determine the underlying issue, you may try to enable diagnostics logging for your web app. This should provide more information on what is happening at the storage level and what kind of activity is going on.
Is Azure diagnostics only implemented through code? Windows has the Event Viewer where various types of information can be accessed. ASP.Net websites have a Trace.axd file at the root that can viewed for trace information.
I was thinking that something similar might exist in Azure. However, based on the following url, Azure Diagnostics appears to require a custom code implementation:
https://azure.microsoft.com/en-us/documentation/articles/cloud-services-dotnet-diagnostics/#overview
Is there an easier, more built-in way to access Azure diagnostics like I described for other systems above? Or does a custom Worker role need to be created to capture and process this information?
Azure Worker Roles have extensive diagnostics that you can configure up.
You get to them via the Role configuration:
Then, through the various tabs, you can configure up specific types of diagnostics and have them periodically transferred to a Table Storage account for later analysis.
You can also enable a transfer of application specific logs, which is handy and something that I use to avoid having to remote into the service to view logs:
(here, I transfer all files under the AppRoot\logs folder to a blob container named wad-processor-logs, and do so every minute.)
If you go through the tabs, you will find that you have the ability to extensively monitor quite a bit of detail, including custom Performance Counters.
Finally, you can also connect to your cloud service via the Server Explorer, and dig into the same information:
Right-click on the instance, and select View Diagnostics Data.
(a recent deployment, so not much to see)
So, yes, you can get access to Event Logs, IIS Logs and custom application logs without writing custom code. Additionally, you can implement custom code to capture additional Performance Counters and other trace logging if you wish.
"Azure diagnostics" is a bit vague since there are a variety of services in Azure, each with potentially different diagnostic experiences. The article you linked to talks about Cloud Services, but are you restricted to using Cloud Services?
Another popular option is Azure App Service, which allows you many more options for capturing logs, including streaming them, etc. Here is an article which goes into more details: https://azure.microsoft.com/en-us/documentation/articles/web-sites-enable-diagnostic-log/
I have a website hosted in Azure as a cloud service (not as a website), and I need to get the hit count for every web page of the site.
I enabled Azure Diagnostics, and I see the IIS logs copied to my blob storage, however this logs contain very few data (only one hit to a javascript file).
Furthermore, putting "Verbose" or "All" in the diagnostics configuration of the web role doesn't seem to affect the results, I get only one line (an access to a css file, or an image file, etc).
I'm using Azure SDK 2.0.
Is it possible to use the included IIS logs generated by azure to get a hit count? What should I need to change in the diagnostics configuration?
Or should I need a different approach to achieve this?
The IIS logs it produces are the same ones you'd find on a Windows Server anywhere. Note that depending on the settings you provided to the diagnostics it might take a little while before the data is moved to the storage account. Setting the level of verbosity for the configuration determines what is moved from the instances over to the storage account. Did you give it plenty of time to move the data over before looking at the file in storage again? Sometimes it just brings over what it has, and of course, there could be buffering which means when the file was brought over not everything was in it, etc.
You should be able to get this information from the logs, and yes, you should be able to do it from the IIS logs. That being said, if what you are after is a hits per page I would suggest actually a different approach. Look at an analytics provider like Google Analytics or one of the competitors to that. You'll get a massive amount of information beyond just page hits and no need to worry about parsing log files, etc.