Only run code on first Azure WebApp instance - azure

I have a webapp runnin on Azure. The web site is built in asp net core 3, and is running in a Docker container.
There is a background worker doing a few things such as database cleanup and sending emails built into the application.
My question is how I should best handle this if I need to scale out the application. That is if I create multiple instances of it, whats the best way to make sure the background worker is only running on one of the instances.. And if another instance is removed that another takes over the job.
I realize one solution to this is to break the application apart and run the backgroundworker separately as an Azure function. But I would prefer to avoid this for cost (it's a hobby project) and complexity reasons.
So I'm intrested if there are more ways of solving this which keeps things in one docker container.
Is there for example an environment variable that I can query to get the current instance name and a list of all instances (then I can just say that the first instance in alphabetical order is the "primary" instance). And check this every so often to know if the current instance is the primary instance.

Sidenote: Azure Functions don't cost extra if you reuse the App Service Plan of your website. And complexitywise they are probably less complex than what you are currently thinking about. But if your main goal is to run everything in a single container, you can achieve that as well:
You can use the WebJobs SDK to basically run the "event handler side" of Azure Functions, including the coordination of the required work. Use the singleton attribute if you need additional limitation of concurrency. Infrastructure-wise, WebJobs require a storage account how they manage scale.
You can run WebJobs in the same process as the rest of your ASP.NET Core Application. Some code to get you started if you want to go that route:
var builder = Host.CreateDefaultBuilder(args);
builder.ConfigureWebHostDefaults(webBuilder =>
{
// webBuilder.UseStartup<Startup>();
webBuilder.Configure(app =>
{
app.UseRouting();
app.UseEndpoints(endpoints =>
{
endpoints.MapGet("/", async context =>
{
await context.Response.WriteAsync("Hello World!");
});
});
});
});
builder.ConfigureWebJobs(b =>
{
b.AddAzureStorageCoreServices();
b.AddAzureStorage();
b.AddTimers();
});
IHost host = builder.Build();
using (host)
{
await host.RunAsync();
}

Related

Writing all your functions in one cloud function

What if I put multiple function inside a single cloud function so that its instance lives at max and that I will have to deal with cold start once?
Why is this a bad idea?
export const shop = functions.https.onCall(async (data, context) => {
switch (data.type) {
case "get_fruits":
return await getFruits();
case "place_order":
return await placeOrder();
case "add_to_cart":
return await addToCart();
default:
return;
}
});
It will work but, IMO, it's not a good thing to do. There are many principles and patterns that exist today and that you do not enforce your solution.
Microservice
One of them is the split in microservices. There is no problem to build a monolith, but when I'm seeing your example (get_fruit, place_order, add_to_cart), I'm seeing different roles and responsibilities. I love the separation of concern: 1 service does 1 thing.
Routing
But, maybe your service is only a service for the routing and call functions deployed independently (and you enforce the microservice principle). If so, your service can become a bottleneck, if there are a lot of entries and a lot of queries.
In addition, there are services dedicated for routing: load balancers. They use the URL path of the requests and reach the correct microservices to serve them
Developer usage
Yes a URL, not a field in the body of your message to route the traffic. Today, the developers are familiar with the REST API. To get the fruit, they perform a GET request to the /fruit URL and they know they will get the fruits. If they want to add to the cart, they perform a POST request to the /cart URL and it works!
You USE URL, standard REST definition, load balancers and microservices.
You can imagine other benefits:
Each microservice can scale independently (you can have more get_fruit request than place_order, the service scale differently)
The security is easier to control (no security to get the catalog (fruits)), but you have to be authenticated to place an order
Evolution velocity can be decoupled between the services
...

Azure Blob Trigger function app : running same instance for multiple blobs upload

I have created a Blob triggered function app in Python. My requirement is to run a separate instance for each blob upload (for parallel processing), but it's not happening. Even I have modified the host.json as below as per the below link:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob
{
"version": "2.0",
"extensions": {
"blobs": {
"maxDegreeOfParallelism": "4"
}
}
}
Still, the same instance is running and processing files one by one. Am I missing something here?
I'm afraid we can't implement this requirement. As far as I know, we can just set the function app to scale out to maximum n(in your case is 4) instances, but we can't scale out instances manaually.
When you modify the configuration to allow the function app to scale out for multiple instances, it can just scale out automatically when lots of requests coming. If there are only 4 request, only one instance will be started in most cases.
Here is another post I did research in the past which is similar problem with this case for your reference.

Is it possible to run a Change Feed Processor host as an Azure Web Job?

I'm looking to use the Change Feed Processor SDK to monitor for changes to an Azure Cosmos DB collection, however, I have not seen clear documentation about whether the host can be run as an Azure Web Job. Can it? And if yes, are there any known issues or limitations versus running it as a Console App?
There are a good number of blog posts about using the CFP SDK, however, most of them vaguely mention running the host on a VM, and none of them or any examples running the host as an azure web job.
Even if it's possible, as a side question is, if such a host is deployed as a continuous web job and I select the "Scale" setting of the web job to Multi Instance, what are the approaches or recommendations to make the extra instances run with a different instance name, which the CFP SDK requires?
According to my research,Cosmos db trigger could be implemented in the WebJob SDK.
static async Task Main()
{
var builder = new HostBuilder();
builder.ConfigureWebJobs(b =>
{
b.AddAzureStorageCoreServices();
b.AddCosmosDB(a =>
{
a.ConnectionMode = ConnectionMode.Gateway;
a.Protocol = Protocol.Https;
a.LeaseOptions.LeasePrefix = "prefix1";
});
});
var host = builder.Build();
using (host)
{
await host.RunAsync();
}
}
But it seems only Nuget for c# sdk could be used,no clues for other languages.So,you could refer to the Compare Functions and WebJobs to balance your needs and cost.
The Cosmos DB Trigger for Azure Functions it's actually, a WebJobs extension: https://github.com/Azure/azure-webjobs-sdk-extensions/tree/dev/src/WebJobs.Extensions.CosmosDB
And it uses the Change Feed Processor.
Functions run over WebJob technology. So to answer the question, yes, you can run Change Feed Processor on WebJobs, just make sure that:
Your App Service is set to Always On
If you plan to use multiple instances, make sure to set the InstanceName accordingly and not a static/fixed value. Probably something that identifies the WebJob instance.

autofac and multithreading

when doing parallel/multithreading, if there are dependencies that are not thread safe, what kind of instance method can be used with autofac to get an instance per thread? from what I know, autofac is the kind of DI container for certain framework like asp.net/mvc but for the rest of the app type like windows service, it does not have any support. in my scenario, i am doing multithreading for a windows service that also hosting a web api service. what kind of registration can be used so that it will work for web api instanceperhttprequest and instanceperlifetimescope. two separate container?
EDIt:
using this parallel extension method here:
public static Task ForEachAsync<T>(this IEnumerable<T> source, int dop, Func<T, Task> body)
{
return Task.WhenAll(
from partition in Partitioner.Create(source).GetPartitions(dop)
select Task.Run(async delegate
{
using (partition)
{
while (partition.MoveNext())
{
await body(partition.Current);
}
}
}));
}
so the body will be use to do the work. DI will need to be inside of the body func.
It doesn't matter what kind of application you run; the pattern is always the same. You should resolve one object graph per request. In a web application, a request means a web request, in a windows service, a request is usually a timer pulse.
So in a windows service, each 'pulse' you start a new lifetime scope, and within this scope you resolve your root object and call it.
If however, you process items in parallel within a single request, you should see each processed item as a request of its own. So that means that on each thread you should start a new lifetime scope and resolve a sub object graph from that scope and execute that. Prevent passing services that are resolved from your container, from thread to thread. This scatters the knowledge of what is thread-safe, and what isn't throughout the application, instead of keeping that knowledge centralized in the startup path of your application where you compose your object graphs (the composition root).
Take a look at this article about working with dependency injection in multi-threaded applications. It's written for a different DI library, but you'll find most of the advice generically applicable to all DI libraries.

Concurrency in the Task-based API in Azure SDK for .NET

I currently have a couple of concurrency issues with the Task-based asynchronous API in the Azure SDK for .Net version 3.0.2-prerelease.
I have a list of web site names
var webSites = new [] { "website1", "website2" };
and from these, I'm using the task based API to create or delete the WebSites. Both occasionally fail:
await Task.WhenAll(webSites.Select(x => webSiteClient.WebSites.CreateAsync(
"westeuropewebspace",
new WebSiteCreateParameters
{
SiteMode = WebSiteMode.Limited,
ComputeMode = WebSiteComputeMode.Shared,
Name = x
WebSpaceName = "something"
}
)));
Seldom, I get an exception complaining that the Server Farm "Default1" already exists. I get that this server farm is implicitly created for Free web sites, but there is currently no way to create this Server Farm through the API before creating the WebSites (only the "DefaultServerFarm" can be).
When deleting, something similar happens:
await Task.WhenAll(webSites.Select(x => webSiteClient.WebSites.DeleteAsync(
"westeuropewebspace",
x,
new WebSiteDeleteParameters
{
DeleteAllSlots = true,
DeleteEmptyServerFarm = true,
DeleteMetrics = true,
}
)));
Often (about every second time), I get an Exception that "website2" could not be found, although it definitely existed. The WebSite is deleted, though.
Update:
I have serialized this second Task.WaitAll into a foreach-loop and I still get the exception. The only difference now is that when deleting "website1" fails, "website2" still exists in the cloud (because the second delete request is not sent) and I have to delete it manually through the portal.
You are right - the create site api also tries to create a server farm implicitly and if called concurrently that may cause conflicts. A safer way is to create a server farm explicitly using API and then use that server farm when creating web sites. That way you explicitly control the placement of sites to server farms and there are no implicit server farm creations.
The Azure SDK API contain a method to create server farm explicitly.
https://github.com/Azure/azure-sdk-for-net/blob/master/src/WebSiteManagement/Generated/ServerFarmOperations.cs

Resources