A seeming simple ask, but had no luck finding the best way.
So i need to log application events from a console app that will spin up inside a container and do some work then die.
How can i log custom data from inside?
I've tried Azure Monitor and created a workspace and used HTTP Data Collector API inside the app but no joy in working out where logs are being stored.
Is there a simple way to log to an Azure Storage account and then using Azure Monitor to manage the events?
I've been googling for hours but a lot of posts are 8 years old and not relevant and i cannot really find a simple use case in modern azure.
Perhaps it's so simple i just cannot see it
Any pointers or links greatly received!
thanks
Paul
Why not trace events using Application Insight custom events ?
https://learn.microsoft.com/en-us/azure/azure-monitor/app/api-custom-events-metrics
With that you can trace events with any metadata and check them in the Azure Application Insights Blade or reach them by the Application Insights SDK or The Api.
You just need to create an Application Insight instance and use the Telemetry Key to do that.
SDK: https://github.com/Microsoft/ApplicationInsights-dotnet
API : https://dev.applicationinsights.io/reference
Code sample to write events:
TelemetryClient client = new TelemetryClient();
client .InstrumentationKey = "INSERT YOUR KEY";
client.TrackEvent("SomethingInterestingHappened");
Also you can send more than just an string value:
tc.TrackEvent("PurchaseOrderSubmitted",
new Dictionary<string, string>()
{
{"CouponCode", "JULY2015" }
}, new Dictionary<string, double>()
{
{"OrderTotal", 68.99 },
{"ItemsOrdered", 5}
});
Related
I am using a console application that runs on on-premise servers triggered by a task scheduler. This console application performs the various actions and needed to be logged these. It would generate logs of around 200kb per run and the console app runs every hour.
Since the server is not accessible to us, I am planning to store the logs to Azure. I read about blob/table storage.
I would like to know what is the best strategy to store the logs in Azure.
Thank you.
Though you can write logging data in Azure Storage (both Blobs and Tables), it would actually make more sense if you use Azure Application Insights for logging this data.
I recently did the same for a console application I built. I found it incredibly simple.
I created an App Insight Resource in my Azure Subscription and got the instrumentation key. I then installed App Insights SDK and referenced appropriate namespaces in my project.
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
This is how I initialized the telemetry client:
var appSettingsReader = new AppSettingsReader();
var appInsightsInstrumentationKey = (string)appSettingsReader.GetValue("AppInsights.InstrumentationKey", typeof(string));
TelemetryConfiguration configuration = TelemetryConfiguration.CreateDefault();
configuration.InstrumentationKey = appInsightsInstrumentationKey;
telemetryClient = new TelemetryClient(configuration);
telemetryClient.InstrumentationKey = appInsightsInstrumentationKey;
For logging trace data, I simply did the following:
TraceTelemetry telemetry = new TraceTelemetry(message, SeverityLevel.Verbose);
telemetryClient.TrackTrace(telemetry);
For logging error data, I simply did the following:
catch (Exception excep)
{
var message = string.Format("Error. {0}", excep.Message);
ExceptionTelemetry exceptionTelemetry = new ExceptionTelemetry(excep);
telemetryClient.TrackException(exceptionTelemetry);
telemetryClient.Flush();
Task.Delay(5000).Wait();//Wait for 5 seconds before terminating the application
}
Just keep one thing in mind though: Make sure you wait for some time (5 seconds is good enough) to flush the data before terminating the application.
If you're still keen on writing logs to Azure Storage, depending on the logging library you're using you will find suitable adapters that will write directly into Azure Storage.
For example, there's an NLog target for Azure Tables: https://github.com/harouny/NLog.Extensions.AzureTableStorage (though this project is not actively maintained).
So I'm fairly new to working with Azure and there are some things I can't quite wrap my head around. One of them being the Azure Storage Account.
My web jobs keeps stopping with the following error "Unhandled Exception: System.InvalidOperationException: The account credentials for '[account_name]' are incorrect." Understanding the error however is not the problem, at least that's what I think. The problem lies in understanding why I need an Azure Storage Account to overcome it.
Please read on as I try to take you through the steps taken thus far. Hopefuly the real question will become more clear to you.
In my efforts to deploy a WebJob on Azure we have created the following resources so far:
App Service Plan
App Service
SQL server
SQL database
I'm using the following code snippet to prevent my web job from exiting:
JobHostConfiguration config = new JobHostConfiguration();
config.DashboardConnectionString = null;
new JobHost(config).RunAndBlock();
To my understanding from other sources the Dashboard connection string is optional but the AzureWebJobsStorage connection string is required.
I tried setting the required connection string in portal using the configuration found here.
DefaultEndpointsProtocol=[http|https];AccountName=myAccountName;AccountKey=myAccountKey
Looking further I found this answer that clearly states where I would get the values needed, namely an/my missing Azure Storage Account.
So now for the actualy question: Why do I need an Azure Storage Account when I seemingly have all the resources I need place for the WebJob to run? What does it do? Is it a billing thing, cause I thought we had that defined in the App Service Plan. I've tried reading up on Azure Storage Accounts over here but I need a bit more help understanding how it relates to everything.
From the docs:
An Azure storage account provides resources for storing queue and blob data in the cloud.
It's also used by the WebJobs SDK to store logging data for the dashboard.
Refer to the getting started guide and documentation for further information
The answer to your question is "No", it is not mandatory to use Azure Storage when you are trying to setup and run a Azure web job.
If you are using JobHost or JobHostConfiguration then there is indeed a dependency for Storage accounts.
Sample code snippet is give below.
class Program
{
static void Main()
{
Functions.ExecuteTask();
}
}
public class Functions
{
[NoAutomaticTrigger]
public static void ExecuteTask()
{
// Execute your task here
}
}
The answer is no, you don't. You can have a WebJob run without being tied to an Azure Storage Account. Like Murray mentioned, your WebJob dashboard does use a storage account to log data but that's completely independent.
I've been looking around and I can't find a concise example around getting this metric. I've installed WindowsAzure.Management.Compute and now I don't know where to begin. How can this be done?
I would recommend you to take a look at Azure Resource Explorer: https://resources.azure.com. You could find the instances request URL as following:
For the Authorization header please have a look at this article. We can use C# HttpRequest to write the code. Here is the result I tested in fildder.
the endpoint:
https://management.azure.com/subscriptions/<subscription id>/resourceGroups/jatestgroup/providers/Microsoft.Web/sites/testcore1/instances?api-version=2015-08-01
Result:
We can calculate the instance number from the response json. In addition, it will need a long time in this rest API to show all instances when scale the instance in azure portal.
[Update]
According with my comment, I tested with the article:http://blog.amitapple.com/post/2014/03/access-specific-instance/#.V9tLKyh95hF
The following is my result:
Please download the library at here. Refer to this article for more information about Windows Azure Management Certificates.
Here is the code snipped:
var cert = new X509Certificate2();
cert.Import(Convert.FromBase64String(""));
var _client = new Microsoft.WindowsAzure.Management.WebSites.WebSiteManagementClient(new Microsoft.WindowsAzure.CertificateCloudCredentials("****", cert));
var ids= await _client.WebSites.GetInstanceIdsAsync("EastAsiawebspace", "testcore1");
You cannot get the instance count from within one instance of the Web App. But you can get it from the portal, or from the Azure ARM API (it's the numberOfWorkers property on the Web Hosting Plan object).
Also, note that WindowsAzure.Management.Compute does not apply to Azure Web App.
Is there an easy way to send an alert or notification in the Azure Management portal if a worker role throws an exception or has an error?
I am using Azure 2.5
I have tracing and diagnostics all set up and can view the logs in the server explorer of Visual studio, but is there anyway to set an alert if for example and error message appears in the logs.
I know you can set up alerts for monitoring metrics in the Management portal is there an easy way to add metrics for errors and exceptions?
Or someway to get C# exception code to create notifications or alerts in the Azure Management portal?
I ended up using email alerts and Application insights to monitor my worker role in the Azure portal. I created an Application insight on the portal according to these instructions.
Using the Nuget package manager in Visual Studio I added the Application insights API, Application insights for website (even though my worker role is not a web app) and the Application insights trace listener.
I then created an Application insight instance by adding the following to the worker role.
private TelemetryClient tc = new TelemetryClient();
And then adding this to the onStart method.
tc.Context.InstrumentationKey = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX";
You can find your insturmentation Key in the Azure Portal.
After running, or deploying my worker role I could then view all of my Trace.TraceInformation and TraceError statements in the Azure portal as well as add tc.TrackError and tc.TrackEvent statements to track errors and events.
TrackError worked perfectly for notifying me when an exception was thrown.
I use SendGrid for sending emails through Azure because it's free. Here is what I would do something like below:
try
{
//....
}
catch(Exception ex)
{
MailMessage mailMsg = new MailMessage() {
//Set your properties here
};
// Add the alternate body to the message.
mailMsg.AlternateViews.Add(
AlternateView.CreateAlternateViewFromString(Body
, new System.Net.Mime.ContentType("text/html")));
SmtpClient smtpClient = new SmtpClient(
ServerGlobalVariables.SmtpServerHost
, Convert.ToInt32(587));
System.Net.NetworkCredential credentials =
new System.Net.NetworkCredential(
ServerGlobalVariables.SmtpServerUserName
, ServerGlobalVariables.SmtpServerPassword);
smtpClient.Credentials = credentials;
smtpClient.Send(mailMsg);
}
Please note that I store my credits in a globalvariables class called ServerGlobalVariables. Also, I send my emails formatted as HTML, but you don't have to do that.
I'm doing my first project but large one on developing Azure Application with Intergration Component.
Currently most of the integration are done using SSIS Packages and would like to transform them on to Worker Role in Azure.
Could someone please help me to understand the following queries regarding Worker Role please?
Is there way to start or stop the Worker role (just like SSIS or Windows Schedulers) via GUI? If not how to achieve this?
How do I know my worker role has been running or not running (including why it's not running ie. logs)
How do I spin multiple worker role based on time (i.e. (9:00AM to 11:00AM spin 4 roles and scale down on quiet period)
Does the following code creates any poison message or dead lock (if multiple there are 10,000 messages to process and every 5 seconds the new thread (Processsing.run) is started?
while(true)
{
var thread = new Thread(Run);
thread.start();
Thread.Sleep(5000);
Trace.WriteLine("Working", "Information");
}
public class PhotoProcessing
{
public static void Run()
{
// Read from queue
CloudQueueMessage msg =
Storage.Queue.GetNextMessage();
while(msg != null)
{
string[] message = msg.AsString.Split('$');
if(message.Length == 2)
{
AddWatermark(message[0], message[1]);
}
// Message has been read so remove it
Storage.Queue.DeleteMessage(msg);
// Get next message if any
msg = Storage.Queue.GetNextMessage();
}
}
Is there way to start or stop the Worker role (just like SSIS or Windows Schedulers) via GUI? If not how to achieve this?
There are actually many ways to achieve this. You can use Windows Azure Portal to do or you could use 3rd party tools (like our Cloud Storage Studio) or you could write your own application using Windows Azure Service Management API (http://msdn.microsoft.com/en-us/library/ee460799.aspx)
How do I know my worker role has been running or not running (including why it's not running ie. logs)
Again you could use one of the GUI based tools to see the status of your roles. As far as why the roles are not running, you would need to enable Windows Azure Diagnostics in your worker role (http://msdn.microsoft.com/en-us/library/gg433048.aspx)
How do I spin multiple worker role based on time (i.e. (9:00AM to 11:00AM spin 4 roles and scale down on quiet period)
You can write your own application using Windows Azure Service Management API to do so or you could make use of 3rd party tools like AzureWatch from Paraleap or Azure Management Cmdlets (both from Microsoft and our company). While the cmdlets will get the job done, I believe Azure Watch is much more sophisticated solution. We wrote a blog post for autoscaling some days back which you can find here: http://www.cerebrata.com/Blog/post/Scale-your-Windows-Azure-instances-with-Azure-Management-Cmdlets.aspx.