I'm trying to understand the various ways to configure the Diagnostics in Windows Azure.
So far I've set a diagnostics.wadcfg that is properly used by Azure as I retrieve its content in the xml blob stored by Diagnostics in the wad-control-container (and the tables are updated at the correct refresh rate).
Now I would like to override some fields from the cscfg, in order to boost the log transfer period for all instances, for example (without having to update each wad-control-container file, which will be erased in case of instance recycle btw).
So in my WebRole.Run(), I get a parameter from RoleEnvironment.GetConfigurationSettingValue() and try to apply it to the current config ; but my problem is that the values I read from DiagnosticMonitor.GetDefaultInitialConfiguration() do not correspond to the content of my diagnostics.wadcfg, and setting new values in there doesn't seem to have any effect.
Can anyone explain the relationship between what's taken from diagnostics.wadcfg and the values you can set at run-time?
Thanks
GetDefaultInitialConfiguration() will not return you your current settings, becasue as its name states it takes a default configuration. You have to use the GetCurrentConfiguration method if you need to take the configuration that is in place.
However, if you need to just boost the transfer, you could use for example the Cerebrata's Azure Diagnostics Manager to quickly kick off on-demand transfer of your roles.
You could also use the Windows Azure Diagnostics Management cmdlets for powershell. Check out this article.
Hope this helps!
In order to utilize values in wadcfg file the following code code could be used to access current DiagnosticsMonitorConfiguration:
var cloudStorageAccount = CloudStorageAccount.Parse(
RoleEnvironment.GetConfigurationSettingValue(WADStorageConnectionString));
var roleInstanceDiagnosticManager = cloudStorageAccount.CreateRoleInstanceDiagnosticManager(
RoleEnvironment.DeploymentId,
RoleEnvironment.CurrentRoleInstance.Role.Name,
RoleEnvironment.CurrentRoleInstance.Id);
var dmc = roleInstanceDiagnosticManager.GetCurrentConfiguration();
// Set different logging settings
dmc.Logs....
dmc.PerformanceCounters....
// don't forget to update
roleInstanceDiagnosticManager.SetCurrentConfiguration(dmc);
The code by Boris Lipshitz doesn't work now (Breaking Changes in Windows Azure Diagnostics (SDK 2.0)): "the DeploymentDiagnosticManager constructor now accepts a connection string to the storage account instead of a CloudStorageAccount object".
Updated code for SDK 2.0+:
var roleInstanceDiagnosticManager = new RoleInstanceDiagnosticManager(
// Add StorageConnectionString to your role settings for this to work
CloudConfigurationManager.GetSetting("StorageConnectionString"),
RoleEnvironment.DeploymentId,
RoleEnvironment.CurrentRoleInstance.Role.Name,
RoleEnvironment.CurrentRoleInstance.Id);
var dmc = roleInstanceDiagnosticManager.GetCurrentConfiguration();
// Set different logging settings
dmc.Logs....
dmc.PerformanceCounters....
// don't forget to update
roleInstanceDiagnosticManager.SetCurrentConfiguration(dmc)
Related
I have a webjob that I have set up to be triggered when a file is added to a directory:
[FileTrigger(#"<DIR>\<dir>\{name}", "*", WatcherChangeTypes.Created, autoDelete: true)] Stream file,
I have it configured:
var config = new JobHostConfiguration
{
JobActivator = new NinjectActivator(kernel)
};
var filesConfig = new FilesConfiguration();
#if DEBUG
filesConfig.RootPath = #"C:\Temp\";
#endif
config.UseFiles(filesConfig);
config.UseCore();
The path is for working locally and I was expecting that commenting out the FilesConfiguration object leaving it default would allow it to pick up the connection string I have set up and trigger when files are added. This does not happen it turns out that by default the RootPath is set to "D:\Home" and produces an InvalidOperationException
System.InvalidOperationException : Path 'D:\home\data\<DIR>\<dir>' does not exist.
How do I get the trigger to point at the File storage area of the storage account I have set up for it. I have tried removing the FilesConfiguration completely from Program.cs in the hope that it would work against the settings but it only produces the same Exception.
System.InvalidOperationException : Path 'D:\home\data\\' does not exist.
When you publish to azure, the default directory is D:\HOME\DATA, so when you run webjob it could not find the path so you get the error message.
How do I get the trigger to point at the File storage area of the storage account I have set up for it.
The connectionstring you have set have two applies: one is used for dashboard logging and the other is used for application functionality (queues, tables, blobs).
It seems that you could not get filetrigger working with azure file storage.
So, if you want to invoke your filetrigger when you create new file, you could go to D:\home\data\ in KUDU to create a DIR folder and then create new .txt file in it.
The output is as below:
BTW, it seems that you'd better not use autoDelete when you create file, if use you will get error like:
NotSupportedException: Use of AutoDelete is not supported when using change type 'Changed'.
We are hosting 3party sites in our webrole and to limit them access to the storage container I need to set the connection string from code instead of the connection string in serviceconfiguration?
Is this possible?
Based on answer i ran into a problem.
DiagnosticMonitorConfiguration dmConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();
DiagnosticMonitor.StartWithConnectionString(conn, dmConfig);
This resets the configuration to defaults and overrides the stuff that was deployed with the cloud service. I assume when using the StartWithConnectionString, you cant use the support they added in visual studio for setting these things.
Yes, I think you can. Do take a look at DiagnosticMonitor.StartWithConnectionString method. You would do something like this in your WebRole's OnStart() method:
DiagnosticMonitorConfiguration dmConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();
DiagnosticMonitor.StartWithConnectionString("DefaultEndpointsProtocol=https;AccountName=accountname;AccountKey=accountkey", dmConfig);
return base.OnStart();
However I would not recommend hard coding the connection string in the code itself. Instead take it from some database.
I am not able to find out how to see azure diagnostics logs. Code which I wrote is as follows.
DiagnosticMonitorConfiguration config = iagnosticMonitor.GetDefaultInitialConfiguration();
System.Diagnostics.Trace.Listeners.Add(new Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener());
config.Logs.ScheduledTransferLogLevelFilter = LogLevel.Information;
config.WindowsEventLog.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(1.0);
DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", config);
then I added Trace.WriteLine("some message"); in the code. Now where do I find those messages. I checked in Visual Studio server explorer, where I added my storage account reference. wad-control-container has only config files.
You might want to take a look at this blog post by Michael S. Collier. A setting in your cloud project might cause the logs to end up in an other place than you would expect them to:
http://michaelcollier.wordpress.com/2012/04/02/where-is-my-windows-azure-diagnostics-data/
Update:
Note that you'll have to take care of every small detail to make everything work.
Where are you writing Trace.WriteLine("some message"); ? Is it in your WebRole.cs? If that's the case, you'll need to configure the trace listener for the WebRole.cs (this runs in an other process, different than your actual web application).
Here is an example how you can set up the trace listener in the WebRole.cs class:
System.Diagnostics.Trace.Listeners.Add(new Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener());
System.Diagnostics.Trace.AutoFlush = true;
After setting that up, you can use Trace.WriteLine.
This is a good step by step on how to enable diagnostics:
http://www.windowsazure.com/en-us/develop/net/common-tasks/diagnostics/?sec=commontasks
I wrote a tool that allows you to view Azure Diagnostic information.. check it out
AzTools - Azure Diagnostic Viewer
Click here to see how you could use this tool
How do I create new instances of some role via C# using Azure emulator? Is there some guide about that? There are some manuals about creating instances in the cloud, not in emulator.
So far I know that:
I need to change config-file. Is it config in sln-file or in some temp-delpoyment folder?
I need to use csrun tool. How to pick params?
UPD
Got it.
To change count or instances on emulator, you have to:
update 'ServiceConfiguration.cscfg' file in bin-folder
run 'csrun' tool with params: string.Format("/update:{0};\"{1}\"", deploymentId, "<path to ServiceConfiguration.cscfg>")
where deploymentId:
// get id from RoleEnvironment with regex
var patternt = Regex.Escape("(") + #"\d+" + Regex.Escape(")");
var input = RoleEnvironment.DeploymentId;
var m = Regex.Match(input, patternt);
var deploymentId = m.ToString().Replace("(", string.Empty).Replace(")", string.Empty);
If you have troubles running csrun via code, read this:
http://social.msdn.microsoft.com/Forums/en/windowsazuredevelopment/thread/62ca1372-2388-4181-9dbd-8fbba470ea77
In local emulator, you need to modify the CSCFG file under the deployment .csx folder, instead of your source code folder, since the local emulator will fire your application up from that folder.
Once you modified the saved your CSCFG file, for example the count of the instances you can retrieve the new value from your code immediately. But if you want the local emulator detect this changes and perform the related actions, such as increase the VMs or invoke the Configuration_Changed method, you need to execute
csrun /update:;
You can retrieve the deployment id from the compute emulator UI.
You can find the instance count in the ServiceConfiguration.cscfg in your Azure project
I have been trying to get IIS Logs from Azure, and I was able to get to get it going once - now, no matter what I try, I can't get it to transfer logs to my Storage account.
I was trying to do this without re-deploying my code, which after reading around seemed possible. And, as I mentioned, I was successful. But this is driving me insane, it just won't do it anymore. Although, it does create the Queue in my storage account when I start a transfer, but that's all it seems to do.
The basic steps I am doing are:
Adding the storage name and key to my config as "DiagnosticsConnectionString"*.
Setting a DiagnosticMonitorConfiguration for one minute, with a DirectoriesBufferConfiguration.
Starting an OnDemand Transfer with a new queue name.
I've done all of the above both programmatically, and through the cmdaplets for PowerShell. As soon as I start a transfer, it just stays with a status of "Not Yet Published (Do not end/cancel)".
I have tried Logs, Directories and even deleted and recreated my storage account. Nothing seems to be working. It appeared to work when I directly added my storage account info to my role config via the azure portal; after it Updated the deployment I saw the logs. But this is not working anymore. Does anyone have some good advice/material? I just want to transfer my IIS logs to my storage account - I've been at it for days.
Update:*This is my config: . My WebRole.cs contained the following, when it worked:
DiagnosticMonitor.Start("DiagnosticsConnectionString");
I've updated it to start transfers:
var diag = new DiagnosticMonitorConfiguration()
{
ConfigurationChangePollInterval = TimeSpan.FromMinutes(1),
Directories = new DirectoriesBufferConfiguration()
{
ScheduledTransferPeriod = TimeSpan.FromMinutes(1)
},
Logs = new BasicLogsBufferConfiguration()
{
ScheduledTransferLogLevelFilter = LogLevel.Verbose,
ScheduledTransferPeriod = TimeSpan.FromMinutes(1)
}
};
DiagnosticMonitor.Start("DiagnosticsConnectionString", diag);
Change one line:
From
var diag = new DiagnosticMonitorConfiguration()
to
var diag = DiagnosticMonitor.GetDefaultInitialConfiguration()
Afterwords, use the existing objects within the diag and not add your own.
This is my OnStart:
var config = DiagnosticMonitor.GetDefaultInitialConfiguration();
config.DiagnosticInfrastructureLogs.ScheduledTransferLogLevelFilter = LogLevel.Information;
config.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
config.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
DiagnosticMonitor.Start("DiagnosticsConnectionString", config);
It could be that the logs are not being generated, rather than a problem at the download time.
There is a progam called AzureLogFetcher that may help, tips on getting logging to work can be found here