We are currently in the process of doing some overhauls to our WorkerRole on Azure. Our current implementation uses the DiagnosticsMonitor to automatically put all of the trace and error information into the WAD-Logs table in our storage account and works well. However, as we are implementing the Diagnostics portion of the role in our rewrite, ReSharper is diligently informing me that DiagnosticMonitor is now an obsolete API. However, I cannot find any information that shows what is meant to replace this API.
Some relevant information (all of these should be latest versions via NuGet):
Microsoft.WindowsAzure.Diagnostics :: version 2.5.0.0
Microsoft.WindowsAzure.Configuration:: version 3.0.0.0
Microsoft.WindowsAzure.ServiceRuntime:: version 2.5.0.0
Microsoft.WindowsAzure.Storage:: version 4.3.0.0
The code we are attempting to replicate
public static void ConfigureDiagnostics()
{
//warning here on DiagnosticMonitor
var config = DiagnosticMonitor.GetDefaultInitialConfiguration();
config.ConfigurationChangePollInterval = TimeSpan.FromMinutes(1d);
config.Logs.BufferQuotaInMB = 500;
config.Logs.ScheduledTransferLogLevelFilter = Microsoft.WindowsAzure.Diagnostics.LogLevel.Error;
config.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1d);
//warning here on DiagnosticMonitor
DiagnosticMonitor.StartWithConnectionString(ConfigurationManager.AppSettings.Get("LogStorageConnectionString"), config);
}
This was the "old" way of doing the diagnostics and we're deprecating this solution in favor of the new XML based one, meaning you can also remotely configure the Diagnostics infrastructure etc.
More info you can find here on how to migrate as well.
Related
I am new to Azure app insights and want to know why are Asynchronous HttpClient calls not shown as dependency automatically in Azure App insights for App service.
Also, what configuration changes or code changes should I make in ASP.NET project for tracking http dependencies ?
When I refer documentation it says "A dependency is a component that is called by your application. It's typically a service called using HTTP, or a database, or a file system. Application Insights measures the duration of dependency calls, whether its failing or not, along with additional information like name of dependency and so on. You can investigate specific dependency calls, and correlate them to requests and exceptions."
You can use the below example of code to track http dependency automatically
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DependencyCollector;
using Microsoft.ApplicationInsights.Extensibility;
using System.Net.Http;
using System.Threading.Tasks;
namespace ConsoleApp
{
class Program
{
static void Main(string[] args)
{
TelemetryConfiguration configuration = TelemetryConfiguration.CreateDefault();
configuration.InstrumentationKey = "removed";
configuration.TelemetryInitializers.Add(new HttpDependenciesParsingTelemetryInitializer());
var telemetryClient = new TelemetryClient(configuration);
using (InitializeDependencyTracking(configuration))
{
// run app...
telemetryClient.TrackTrace("Hello World!");
using (var httpClient = new HttpClient())
{
// Http dependency is automatically tracked!
httpClient.GetAsync("https://microsoft.com").Wait();
}
}
// before exit, flush the remaining data
telemetryClient.Flush();
// flush is not blocking when not using InMemoryChannel so wait a bit. There is an active issue regarding the need for `Sleep`/`Delay`
// which is tracked here: https://github.com/microsoft/ApplicationInsights-dotnet/issues/407
Task.Delay(5000).Wait();
}
static DependencyTrackingTelemetryModule InitializeDependencyTracking(TelemetryConfiguration configuration)
{
var module = new DependencyTrackingTelemetryModule();
// prevent Correlation Id to be sent to certain endpoints. You may add other domains as needed.
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.windows.net");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.chinacloudapi.cn");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.cloudapi.de");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.usgovcloudapi.net");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("localhost");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("127.0.0.1");
// enable known dependency tracking, note that in future versions, we will extend this list.
// please check default settings in https://github.com/microsoft/ApplicationInsights-dotnet-server/blob/develop/WEB/Src/DependencyCollector/DependencyCollector/ApplicationInsights.config.install.xdt
module.IncludeDiagnosticSourceActivities.Add("Microsoft.Azure.ServiceBus");
module.IncludeDiagnosticSourceActivities.Add("Microsoft.Azure.EventHubs");
// initialize the module
module.Initialize(configuration);
return module;
}
}
}
NOTE: -
Based on the MSDOC Azure Monitor Application Insights Agent currently supports ASP.NET 4.x only.
For more information please refer the below links:
MS DOC: Application Insights for .NET console applications(Source code) & Automatically dependency tracking
GitHub : LINK1 & LINK2
Getting Keyword not supported: 'authentication' error while trying to connect an azure DB through 'Active Directory Integrated' option in .NET core 2.1 project.
Note: I am using EF core to connect the Data source.
TL;DR
As called out by #Aamir Mulla in the comments, this has officially been added since Version 2.0.0
UPDATE - 16/08/2019
Active Directory Password Authentication has now been added for .NET Core in Microsoft.Data.SqlClient 1.0.19221.1-Preview
Unfortunately, the authentication keyword is not yet fully supported in .NET Core. Here is an issue which discusses this.
But .NET Core 2.2 has added some support for this use case as mentioned in this comment. The basic idea is to get the access token by any means (ADAL, REST, etc.) and set SqlConnection.AccessToken to it.
As for using this with EF Core, there's a good discussion about this in this github issue and in particular the comment by mgolois provides a simple implementation to the solution that cbriaball mentions in the thread.
Here is the same for reference
Note that this sample is using the Microsoft.Azure.Services.AppAuthentication library
// DB Context Class
public class SampleDbContext : DbContext
{
public SampleDbContext(DbContextOptions<TeamsDbContext> options) : base(options)
{
var conn = (System.Data.SqlClient.SqlConnection)this.Database.GetDbConnection();
conn.AccessToken = (new AzureServiceTokenProvider()).GetAccessTokenAsync("https://database.windows.net/").Result;
}
}
// Startup.cs
services.AddDbContext<SampleDbContext>(options =>
{
options.UseSqlServer(<Connection String>);
});
The connection string would be something like this
Server=tcp:<server_name>.database.windows.net,1433;Database=<db_name>;
If you're still having the issue, make sure you have
Microsoft.Data.SqlClient package installed, not System.Data.SqlClient. They both contain SqlConnection class, switching the package for the first one fixed the issue for me.
As of today 7/18/2022 , I am still getting the issue from Azure when trying to use it through ManagedIdentity.
The microsoft doc at https://learn.microsoft.com/en-us/azure/app-service/tutorial-connect-msi-sql-database?tabs=windowsclient%2Cefcore%2Cdotnetcore
to use managed identity we need to use the connection string in this format!
"Server=tcp:.database.windows.net;Authentication=Active Directory Default; Database=;"
But looks like Azure is not liking it!
However, adding the access token helped!
var connectionString =
"Server=tcp:yourazuresqlservername.database.windows.net; Database=yourazuresqldbname;";
var con = new SqlConnection(connectionString);
//And then
con.AccessToken = (new AzureServiceTokenProvider()).GetAccessTokenAsync("https://database.windows.net/").Result;
con.Open();
//Do sql tasks
con.Close();
I'm trying to follow this link to set up logging for my ASP.NET 5 app in azure https://docs.asp.net/en/latest/fundamentals/logging.html but can't make it work.
What is the way to do it?
You can configure logging through the Startup constructor. Here is a sample:
public Startup(ILoggerFactory loggerFactory)
{
var serilogLogger = new LoggerConfiguration()
.WriteTo
.TextWriter(Console.Out)
#if DNX451
.WriteTo.Elasticsearch()
#endif
.MinimumLevel.Verbose()
.CreateLogger();
loggerFactory.MinimumLevel = LogLevel.Debug;
loggerFactory.AddSerilog(serilogLogger);
}
That's all you need for configuration. From there, you can inject either ILoggerFactory or ILogger<T> (which is mostly the type for the class you want to inject the logger into) to the places you want to log stuff.
My sample configuration makes use of Serilog.Framework.Logging version 1.0.0-rc1-final-10071. Also, under dnx451, it will use Serilog.Sinks.ElasticSearch version 2.0.60.
In Azure Web App, there is no difference the way you configure it. You just need to choose the right provider.
You can see the entire sample here. Also check out ASP.NET 5 and Log Correlation by Request Id which might give you some more ideas.
At this point Azure AppService doesn't support ASP.NET 5 standard trace logging. Here is a potential workaround:
https://github.com/davidebbo-test/ConsoleInterceptor
I'm trying to work with Azure Webjobs, I understand the way its works but I don't understand why I need to use two connection strings, one is for the queue for holding the messages but
why there is another one called "AzureWebJobsDashboard" ?
What its purpose?
And where I get this connection string from ?
At the moment I have one Web App and one Webjob at the same solution, I'm experiment only locally ( without publishing anything ), the one thing I got up in the cloud is the Storage account that holds the queue.
I even try to put the same connection string in both places ( AzureWebJobsDashboard,AzureWebJobsStorage) but its throw exception :
"Cannot bind parameter 'log' when using this trigger."
Thank you.
There are two connection strings because the WebJobs SDK writes some logs in the storage account. It gives you the possibility of having one storage account just for data (AzureWebJobsStorage) and the another one for logs (AzureWebJobsDashboard). They can be the same. Also, you need two of them because you can have multiple job hosts using different data accounts but sending logs to the same dashboard.
The error you are getting is not related to the connection strings but to one of the functions in your code. One of them has a log parameter that is not of the right type. Can you share the code?
Okay, anyone coming here looking for an actual answer of "where do I get the ConnectionString from"... here you go.
On the new Azure portal, you should have a Storage Account resource; mine starts with "portalvhds" followed by a bunch of alphanumerics. Click that so see a resource Dashboard on the right, followed immediately by a Settings window. Look for the Keys submenu under General -- click that. The whole connection string is there (actually there are two, Primary and Secondary; I don't currently understand the difference, but let's go with Primary, shall we?).
Copy and paste that in your App.config file on the connectionString attribute of the AzureWebJobsDashboard and AzureWebJobsStorage items. This presumes for your environment you only have one Storage Account, and so you want that same storage to be used for data and logs.
I tried this, and at least the WebJob ran without throwing an error.
#RayHAz - Expanding upon your above answer (thanks)...
I tried this https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-get-started
but in .Net Core 2.1, was getting exceptions about how it couldn't find the connection string.
Long story short, I ended up with the following, which worked for me:
appsettings.json, in a .Net Core 2.1 Console app:
{
"ConnectionStrings": {
"AzureWebJobsStorage": "---your Azure storage connection string here---",
"AzureWebJobsDashboard":"---the same connectionstring---"
}
}
... and my Program.cs file...
using System;
using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
namespace YourWebJobConsoleAppProjectNamespaceHere
{
public class Program
{
public static IConfiguration Configuration;
static void Main(string[] args)
{
var builder = new ConfigurationBuilder()
.SetBasePath(Path.Combine(AppContext.BaseDirectory))
.AddJsonFile("appsettings.json", true);
Configuration = builder.Build();
var azureWebJobsStorageConnectionString = Configuration.GetConnectionString("AzureWebJobsStorage");
var azureWebJobsDashboardConnectionString = Configuration.GetConnectionString("AzureWebJobsDashboard");
var config = new JobHostConfiguration
{
DashboardConnectionString = azureWebJobsDashboardConnectionString,
StorageConnectionString = azureWebJobsStorageConnectionString
};
var loggerFactory = new LoggerFactory();
config.LoggerFactory = loggerFactory.AddConsole();
var host = new JobHost(config);
host.RunAndBlock();
}
}
}
I'm trying to understand the various ways to configure the Diagnostics in Windows Azure.
So far I've set a diagnostics.wadcfg that is properly used by Azure as I retrieve its content in the xml blob stored by Diagnostics in the wad-control-container (and the tables are updated at the correct refresh rate).
Now I would like to override some fields from the cscfg, in order to boost the log transfer period for all instances, for example (without having to update each wad-control-container file, which will be erased in case of instance recycle btw).
So in my WebRole.Run(), I get a parameter from RoleEnvironment.GetConfigurationSettingValue() and try to apply it to the current config ; but my problem is that the values I read from DiagnosticMonitor.GetDefaultInitialConfiguration() do not correspond to the content of my diagnostics.wadcfg, and setting new values in there doesn't seem to have any effect.
Can anyone explain the relationship between what's taken from diagnostics.wadcfg and the values you can set at run-time?
Thanks
GetDefaultInitialConfiguration() will not return you your current settings, becasue as its name states it takes a default configuration. You have to use the GetCurrentConfiguration method if you need to take the configuration that is in place.
However, if you need to just boost the transfer, you could use for example the Cerebrata's Azure Diagnostics Manager to quickly kick off on-demand transfer of your roles.
You could also use the Windows Azure Diagnostics Management cmdlets for powershell. Check out this article.
Hope this helps!
In order to utilize values in wadcfg file the following code code could be used to access current DiagnosticsMonitorConfiguration:
var cloudStorageAccount = CloudStorageAccount.Parse(
RoleEnvironment.GetConfigurationSettingValue(WADStorageConnectionString));
var roleInstanceDiagnosticManager = cloudStorageAccount.CreateRoleInstanceDiagnosticManager(
RoleEnvironment.DeploymentId,
RoleEnvironment.CurrentRoleInstance.Role.Name,
RoleEnvironment.CurrentRoleInstance.Id);
var dmc = roleInstanceDiagnosticManager.GetCurrentConfiguration();
// Set different logging settings
dmc.Logs....
dmc.PerformanceCounters....
// don't forget to update
roleInstanceDiagnosticManager.SetCurrentConfiguration(dmc);
The code by Boris Lipshitz doesn't work now (Breaking Changes in Windows Azure Diagnostics (SDK 2.0)): "the DeploymentDiagnosticManager constructor now accepts a connection string to the storage account instead of a CloudStorageAccount object".
Updated code for SDK 2.0+:
var roleInstanceDiagnosticManager = new RoleInstanceDiagnosticManager(
// Add StorageConnectionString to your role settings for this to work
CloudConfigurationManager.GetSetting("StorageConnectionString"),
RoleEnvironment.DeploymentId,
RoleEnvironment.CurrentRoleInstance.Role.Name,
RoleEnvironment.CurrentRoleInstance.Id);
var dmc = roleInstanceDiagnosticManager.GetCurrentConfiguration();
// Set different logging settings
dmc.Logs....
dmc.PerformanceCounters....
// don't forget to update
roleInstanceDiagnosticManager.SetCurrentConfiguration(dmc)