How to specify AzureWebJobsStorage in latest azure webjob 3.03 - azure

I updated my old azure webjob code to package to 3.03 and then it is just not working.
I managed to fix all compile-time errors, but when running locally, it throws this error:
Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexingException
HResult=0x80131500
Message=Error indexing method 'MvQueueProcessorV2.ProcessMVRequest'
Source=Microsoft.Azure.WebJobs.Host
StackTrace:
at Microsoft.Azure.WebJobs.Host.RecoverableException.TryRecover(ILogger logger) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Exceptions\RecoverableException.cs:line 81
at FE.Toolkit.MvPaaS.WebJob.Program.<Main>(String[] args)
Inner Exception 1:
InvalidOperationException: Storage account 'Storage' is not configured.
For me, this seems to indicate that it can't find setting AzureWebJobsStorage? However, it sleeps nicely in the app.config file. So I assumed that I should put my connection string to appsettings.json, so this is what I did in my appsettings.json:
{
"ConnectionStrings": {
"AzureWebJobsDashboard": "xxx",
"Storage": "yyy"
}
}
However, it gives me the same error. So how do I set storage for webjob 3.0?
This is my code in program.cs
var builder = new HostBuilder()
.UseEnvironment("Development")
.ConfigureWebJobs(b =>
{
b.AddAzureStorageCoreServices()
.AddAzureStorage()
.AddTimers()
.AddFiles()
.AddDashboardLogging();
})
.ConfigureLogging((context, b) =>
{
b.SetMinimumLevel(LogLevel.Debug);
b.AddConsole();
})
.ConfigureServices(services =>
{
services.AddSingleton<INameResolver, ConfigNameResolver>();
})
.UseConsoleLifetime();

Please add this line of code in your program.cs:
.ConfigureAppConfiguration((context, config) => {
config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
})
I have tested at my side, and works fine.
code in Program.cs:
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
namespace WebJob1template
{
class Program
{
static void Main()
{
var builder = new HostBuilder()
.UseEnvironment("Development")
.ConfigureAppConfiguration((context, config) => {
config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
})
.ConfigureWebJobs(
b =>
{
b.AddAzureStorageCoreServices()
.AddAzureStorage()
.AddTimers()
.AddFiles();
//.AddDashboardLogging();
})
.ConfigureLogging((context, b) =>
{
b.SetMinimumLevel(LogLevel.Debug);
b.AddConsole();
})
.UseConsoleLifetime();
var host = builder.Build();
using (host)
{
host.Run();
}
}
}
}
appsettings.json(note that set it's property "Copy to Output Directory" as Copy always):
{
"ConnectionStrings": {
"AzureWebJobsDashboard": "xxxx",
"AzureWebJobsStorage": "xxxx"
}
}
Function.cs:
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
namespace WebJob1template
{
public class Functions
{
public static void ProcessQueueMessage([QueueTrigger("queue")] string message, ILogger log)
{
//log.WriteLine(message);
log.LogInformation(message);
}
}
}
The test result:

I ran into the same problem and here is my solution.
Below are the Azure assemblies that I am referring to
With above setup, there is no need to invoke c.AddJsonFile("appsettings.json", ... ) if name of config file is 'appSettings.json'
By default, framework will look for the file named 'appsettings.json'
However, if name of file is something else the we need to tell HostBuilder what is name of our config file.
HostBuilder builder = new HostBuilder();
//Below piece of code is not required if name of json file is 'appsettings.json'
builder.ConfigureAppConfiguration(c =>
{
c.AddJsonFile("Myappsettings.json", optional: false, reloadOnChange: true);
});
Simple and important step that cause problem while debugging on my local machine was 'Copy to Output Directory'property of my 'appsettings.json' file. #Ivan Yang has already mentioned this in his answer.
Here is the git hub link of the source code.
Note: I have followed this documentation to implement the code base.

Related

AspNetCoreRateLimit on Azure

I am struggling with getting the package AspNetCoreRateLimit version 4.0.2 to work when hosting a Blazor WebAsssembly project in Azure.
The solution has been developed in Visual studio 2022 and is made up of 5 individual projects, where one of them is a standard API project. It is based on the net6.0 framework. I have a startup.cs configuration and a RateLimitingMiddleware class used to setup the rate limits and the configuration.
Startup.cs:
public void ConfigureServices(IServiceCollection services)
{
services.AddRateLimiting();
services.AddHsts(options =>
{
options.Preload = true;
options.IncludeSubDomains = true;
options.MaxAge = TimeSpan.FromDays(365);
});
services.AddDbContext<ApplicationDbContext>(options =>
{
options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection"),
sqlServerOptionsAction: sqlOptions =>
{
sqlOptions.EnableRetryOnFailure(
maxRetryCount: 10,
maxRetryDelay: TimeSpan.FromSeconds(30),
errorNumbersToAdd: null);
});
}, ServiceLifetime.Transient, ServiceLifetime.Singleton);
services.AddAutoMapper(typeof(Startup));
....
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.UseRateLimiting();
if (env.IsDevelopment())
{
app.UseWebAssemblyDebugging();
app.UseSwagger();
}
else
{
app.UseExceptionHandler("/Error");
}
app.UseHsts();
...
}
RateLimitingMiddleware.cs
internal static class RateLimitingMiddleware
{
internal static IServiceCollection AddRateLimiting(this IServiceCollection services)
{
// Used to store rate limit counters and ip rules
services.AddOptions();
services.AddMemoryCache();
services.Configure<ClientRateLimitOptions>(options =>
{
options.EnableEndpointRateLimiting = true;
//options.RealIpHeader = "X-Real-IP";
options.ClientIdHeader = "authorization";
//options.EndpointWhitelist = new List<string> { "get:/_framework/*", "get:/_content/*", "*:/lib/*", "*:/css/*", "*:/js/", "*:/appsettings.json", "*:/images/" };
options.GeneralRules = new List<RateLimitRule>
{
new RateLimitRule
{
Endpoint="*:/api/*",
Period = "15m",
Limit=30
},
new RateLimitRule
{
Endpoint="*:/api/*",
Period = "12h",
Limit=20000
},
new RateLimitRule
{
Endpoint="*:/api/*",
Period = "7d",
Limit=1000000
}
};
});
services.AddInMemoryRateLimiting();
// Inject Counter and Store Rules
services.AddSingleton<IClientPolicyStore, MemoryCacheClientPolicyStore>();
services.AddSingleton<IRateLimitCounterStore, MemoryCacheRateLimitCounterStore>();
services.AddSingleton<IClientPolicyStore, DistributedCacheClientPolicyStore>();
services.AddSingleton<IRateLimitCounterStore, DistributedCacheRateLimitCounterStore>();
services.AddSingleton<IRateLimitConfiguration, RateLimitConfiguration>();
services.AddSingleton<IProcessingStrategy, AsyncKeyLockProcessingStrategy>();
//services.AddSingleton<IRateLimitConfiguration, CustomRateLimitConfiguration>();
//
// Return the services
return services;
}
internal static IApplicationBuilder UseRateLimiting(this IApplicationBuilder app)
{
var ipPolicyStore = app.ApplicationServices.GetRequiredService<IIpPolicyStore>();
ipPolicyStore.SeedAsync().GetAwaiter().GetResult();
var clientPolicyStore = app.ApplicationServices.GetRequiredService<IClientPolicyStore>();
clientPolicyStore.SeedAsync().GetAwaiter().GetResult();
app.UseClientRateLimiting();
app.UseIpRateLimiting();
return app;
}
}
With the above configuration the api calls are being limited when testing localhost and using Postman. However when the api is uploaded to our Azure environment the rules are not being implemented correctly.
As stated in the rule defined in the middelware I would like to use the authorization token to limit the number of count the number of requests but cannot get it to work.
I hope that someone has dealt with the same issue and can see where I am going wrong?
Thanks,
Problem solved. Wrong ratelimitheader. Should be CLIENT-IP.

Asp.net core: How to use appSettings for a particular environment

In .NET 6.0 WebApi application
I have the following two files:
appSettings.json
appSettings.Linux.json
I want the application to use appSettings.Linux.json when running in linux environment. The following is the code that I have tried so far but unfortunately it is not working. It is still accessing appSettings.json file while running in linux environment.
if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
IConfigurationBuilder builder = new ConfigurationBuilder()
.AddJsonFile("appSettings.Linux.json" , optional: true, reloadOnChange: true);
builder.Build();
}
I am adding a new builder as follows
in .NET 6, you can do like this.
if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
builder.Environment.EnvironmentName = "Linux";
builder.Configuration.AddJsonFile("appSettings.Linux.json");
}
and then it will override all the configuration values.
appsettings.Linux.json
accessing value from Configuration
Updated answer based on feedback.
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration(config =>
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
config.AddJsonFile("appSettings.Linux.json", optional: true, reloadOnChange: true);
}
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});

Azure Function not binding to appsettings when deployed

I have an Azure Function app written in C# with Visual Studio using version 3.0.9 of Functions SDK. In the same directory as the csproj file and the host.json, I have an appsettings.json file with the following content:
{
"test-queue": "test,
"myOptions": {
"batchSize": 5000
}
}
The function works fine when I run it locally but for some reason it doesn't seem to bind to the appsettings file when it is deployed to Azure. I use the following startup class:
public class Startup : FunctionsStartup
{
public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder)
{
builder.ConfigurationBuilder
.SetBasePath(Environment.CurrentDirectory)
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddEnvironmentVariables();
var builtConfig = builder.ConfigurationBuilder.Build();
var keyVaultName = builtConfig["AzureKeyVaultName"];
if (!string.IsNullOrEmpty(keyVaultName))
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
builder.ConfigurationBuilder.AddAzureKeyVault($"https://{keyVaultName}.vault.azure.net/");
}
builder.ConfigurationBuilder
.AddJsonFile("local.settings.json", true)
.AddEnvironmentVariables()
.Build();
}
public override void Configure(IFunctionsHostBuilder builder)
{
FunctionsHostBuilderContext context = builder.GetContext();
builder.Services.AddOptions<MyOptions>()
.Configure<IConfiguration>((settings, configuration) =>
{
configuration.GetSection("myOptions").Bind(settings);
});
}
}
I have a function class like this:
public class MyFunction
{
private readonly MyOptions options;
public MyFunction(IOptions<MyOptions> options)
{
this.options = options.Value;
}
[FunctionName("Test")]
public async Task Run(
[QueueTrigger("%test-queue%")] MyParameters parameters,
ILogger log)
{
log.LogInformation($"Batch size: {options.BatchSize}");
}
}
The trouble is, the function doesn't seem to be using the appsettings file. I get an InvalidOperationException on startup saying '%test-queue%' does not resolve to a value." I can get rid of the queue binding and hard code the queue name. That makes the funtion run Okay but then the log file says the batch size is 0 instead of 5000.
Again, it works when running locally but not when it is deployed. The optional flag is false when registering the appsettings file in the startup so it must be getting found.
I've got a partial solution by replacing
builder.ConfigurationBuilder
.SetBasePath(Environment.CurrentDirectory)
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddEnvironmentVariables();
with
FunctionsHostBuilderContext context = builder.GetContext();
builder.ConfigurationBuilder
.AddJsonFile(Path.Combine(context.ApplicationRootPath, "appsettings.json"), optional: false, reloadOnChange: false)
.AddEnvironmentVariables();
It is still making me hard code the queue names but batchSize variable is getting set at least.

How do I need to log to see my logs in App Service logs

I have enabled logging to Application logging to both FileSystem and Blob. I log messages using ILogger<T>.LogInformation() and Trace.WriteLine(). None of those are visible in blob. I also cannot find them on filesystem. I also can't see them when I enable log streaming.
Do I need to configure something in my ASP.NET Core application?
The [Microsoft.Extensions.Logging.AzureAppServices][1] provider package writes logs to text files in an Azure App Service app's file system and to blob storage in an Azure Storage account.
The provider package isn't included in the shared framework. To use the provider, add the provider package to the project.
To configure provider settings, use AzureFileLoggerOptions and AzureBlobLoggerOptions, as shown in the following example:
public static void Main(string[] args)
{
var host = CreateHostBuilder(args).Build();
var todoRepository = host.Services.GetRequiredService<ITodoRepository>();
todoRepository.Add(new Core.Model.TodoItem() { Name = "Feed the dog" });
todoRepository.Add(new Core.Model.TodoItem() { Name = "Walk the dog" });
var logger = host.Services.GetRequiredService<ILogger<Program>>();
logger.LogInformation("Seeded the database.");
host.Run();
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureLogging(logging => logging.AddAzureWebAppDiagnostics())
.ConfigureServices(serviceCollection => serviceCollection
.Configure<AzureFileLoggerOptions>(options =>
{
options.FileName = "azure-diagnostics-";
options.FileSizeLimit = 50 * 1024;
options.RetainedFileCountLimit = 5;
}).Configure<AzureBlobLoggerOptions>(options =>
{
options.BlobName = "log.txt";
})
)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
You can refer below link for additional reference:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-3.1#azure-app-service-provider
Hope it helps.
Here is a simple way(the referred article is here):
I tested it with .NET core 2.2 mvc project.
The necessary nuget package:
Microsoft.Extensions.Logging.AzureAppServices, version 2.2.0
1.In Startup.cs -> ConfigureServices method, add this line of code:
services.AddSingleton<ILoggerFactory>(new LoggerFactory());
In Startup.cs -> Configure method, change it looks like below:
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler("/Home/Error");
// The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
app.UseHsts();
}
//add the following code here
loggerFactory.AddConsole(Configuration.GetSection("Logging"));
loggerFactory.AddDebug();
loggerFactory.AddAzureWebAppDiagnostics(
new AzureAppServicesDiagnosticsSettings
{
OutputTemplate = "{Timestamp:yyyy-MM-dd HH:mm:ss zzz} [{Level}] {RequestId}-{SourceContext}: {Message}{NewLine}{Exception}"
}
);
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseCookiePolicy();
app.UseMvc(routes =>
{
routes.MapRoute(
name: "default",
template: "{controller=Home}/{action=Index}/{id?}");
});
}
then add following code in the HomeController.cs:
private readonly ILogger _logger;
public HomeController(ILoggerFactory loggerFactory)
{
_logger = loggerFactory.CreateLogger<HomeController>();
}
public IActionResult Index()
{
_logger.LogInformation("This is a Log information!!!");
_logger.LogError("This is a Logger error!!!");
return View();
}
2.After publish to azure, in azure portal -> the web app -> App service logs, set the blob storage. The screenshot as below:
3.Run the web, and check the logs in the specified blob storage:

Serilog: azure webjob logging doesn't seem to work when hosted in azure?

I have azure webjob sdk (v3.0.3) app which has been configured to use serilog for logging.
The log seems to work when I run the app locally in my system. Below is the configuration:
static void Main(string[] args)
{
try
{
var builder = new HostBuilder()
.ConfigureAppConfiguration(SetupConfiguration)
.ConfigureLogging(SetupLogging)
.ConfigureServices(SetupServices)
.ConfigureWebJobs(webJobConfiguration =>
{
webJobConfiguration.AddTimers();
webJobConfiguration.AddAzureStorageCoreServices(); //this is to store logs in azure storage
})
.UseSerilog()
.Build();
builder.Run();
}
}
The code for SetupConfiguration is below:
private static void SetupConfiguration(HostBuilderContext hostingContext, IConfigurationBuilder builder)
{
var env = hostingContext.HostingEnvironment;
_configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: false, reloadOnChange: true)
.AddEnvironmentVariables()
.Build();
}
The code for setting up services:
private static void SetupServices(HostBuilderContext hostingContext, IServiceCollection serviceCollection)
{
serviceCollection.AddSingleton<IConfiguration>(_configuration);
Log.Logger = new LoggerConfiguration()
.ReadFrom.Configuration(_configuration)
.CreateLogger();
_logger = serviceCollection.BuildServiceProvider().GetRequiredService<ILoggerFactory>().CreateLogger("test");
}
The logging is setup as following:
private static void SetupLogging(HostBuilderContext hostingContext, ILoggingBuilder loggingBuilder)
{
loggingBuilder.SetMinimumLevel(LogLevel.Information);
loggingBuilder.AddConsole();
loggingBuilder.AddDebug();
loggingBuilder.AddSerilog(dispose: true);
}
In my TimerTrigger method I'm using the logger:
[Singleton]
public async static Task Trigger([TimerTrigger("%Job%")]TimerInfo myTimer)
{
_logger.LogInformation($"From Trigger {DateTime.UtcNow.ToString()}");
}
In appSettings.json, serilog is configured as follows:
"Serilog": {
"MinimumLevel": "Information",
"WriteTo": [
{
"Name": "RollingFile",
"Args": {
"pathFormat": ".\\Log\\log-{Date}.txt",
"retainedFileCountLimit": 7,
"fileSizeLimitBytes": 5000000,
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss} {EventId} [{Level}] [{Properties}] {Message}{NewLine}{Exception}"
}
}
]
}
the folder "Log" and the log files get created when i run the app locally. But when I publish the webjob, the "Log" folder or the log file is not created in the "app_data" folder of webjob. Can anyone help me figureout how to configure serilog to make it work with webjobs?
Following are the nuget packages used:
If you want to use serilog in WebJob , you need to install this package Serilog.Extensions.WebJobs. Then after configuring the serilog, you would be able to use it.
You must inject the ILogger rather than using the global Log.Logger otherwise the log messages will not be written to the Microsoft Azure WebJobs dashboard.
About the detailed description about how to configure and use serilog, you could refer to this doc.
Hope this could help you, if you still have other questions, please let me know.

Resources