I need to to deploy 2 different worker role instances but each needs it's own configuration data (ID code, password, SenderCompID, etc.) to connect to a trading server. I can't share the credentials across the instances.
Each instance for any role (Worker or Web) are identical in terms of application based configuration. This is because all the instances are created from the same application will read exact same application configuration data.
If you write your application in a way that when application starts it reads data outside the machine (from azure storage, azure table or anything else outside the VM, mostly available on some server) and then configure itself then you could achieve your objective. You also need to provide instance specific data on server so each instance gets its own data. If I choose this option, i might use Azure table name i.e. Instance_ID# so each instance gets its own configuration and configured itself. This way I can modify the data any time on Azure Table and restart the role to load updated configuration. Other may have some other way to make it happen.
The other option is to have two role (worker or web) in same Azure application and while application code could be same in between two or more worker roles however you sure can configure them separately. Each of above options have its own pros and con.
Related
This is a bit descriptive so please bear with me. :)
In the application that I'm trying to build, there are distinct functionalities of product. Users can choose to opt-in for functionality A, B, D but not C. The way I'm building this, is that each of the distinct functionality is a Service (stateless, I'm thinking of storing the data in Azure SQL DBs and exposing REST APIs from each service). Bundled all services together is an ApplicationType. For each customer tenant (consider this as an shared account of a group of users) that is created, I'm thinking of creating a new concrete instance of registered ApplicationType using a TenantManagementService and calling client.ApplicationManager.CreateApplicationAsync() on a FabricClient instance so that I can have a dedicated application instance running on my nodes for that tenant. However, as I mentioned, a tenant can choose to opt-in only for specific functionality which is mapped to a subset of services. If a tenant chooses only service A of my Application, rest of the service instances corresponding to features B, C, D shouldn't be idly running on the nodes.
I thought of creating actors for each service, but the services I'm creating are stateless and I'd like to have multiple instances of them actively running on multiple nodes to load balance rather than having idle replicas of stateful services.
Similar to what I'm doing with application types, i.e., spawning application types as a new tenant registers, can I spawn/delete services as and when a tenant wants to opt-in/out of product features?
Here's what I've tried:
I tried setting InstanceCount 0 for the services at when packaging my application. In my ApplicationParameters XML files:
<Parameters>
<Parameter Name="FeatureAService_InstanceCount" Value="0" />
<Parameter Name="FeatureBService_InstanceCount" Value="0" />
</Parameters>
However, Service Fabric Explorer cribs when instantiating the application out of such application type. The error is this:
But on the other hand, when a service is deployed on the fabric, it gives me an option to delete it specifically, so this scenario should be valid.
Any suggestions are welcome!
EDIT: My requirement is similar to the approach mentioned by anderso in here - https://stackoverflow.com/a/35248349/1842699, However, the problem that I'm specifically trying to solve is to upload create an application instance with one or more packaged services having zero instance count!
#uplnCloud
I hope I understand everything right.
Your situation is the following:
Each customer should have separate Application (created from the same ApplicationType).
Each customer should have only subset of Services (defined in ApplicationType).
If I get it right then this is supported out of the box.
First of all you should remove <DefaultServices /> section from the ApplicationManifest.xml. This will instruct Service Fabric to don't create services automatically with the application.
Now the algorithm is the following:
Create application using FabricClient.ApplicationManager.CreateApplicationAsync()
For each required feature create a new corresponding Service using FabricClient.ServiceManager.CreateServiceAsync() (you need to specify the Application name of newly created Application)
Also note that CreateServiceAsync() accepts ServiceDescriptor that you can configure all service related parameters - starting from partitioning schema and ending up with instance count.
Unfortunately you can't have 0 instance services, Service Fabric has the idea that a named service always exists(running). In this case, when you define a service (give a name to a serviceType instance), it will have at least 1 instance running, otherwise, you shouldn't even have the definition of this service on your application if it does not need to be running.
But what you can have is the ServiceType definition, that means, you have the binaries but you will create it when required.
I assume you are being limited by the default services, where you declare the application and services structure upfront(before deployment of any application instance), instead, you should use dynamic service creation via FabricClient like you described, or via Powershell using New-ServiceFabricApplication and New-ServiceFabricService .
This link you guide you how to do it using FabricClient
I'll just add this as a new answer instead of commenting on another answer.
As other have mentioed, remove DefaultServices from your ApplicationManifest. That way, every new instance of the ApplicationType you create will come online without services, and you'll have to create those manually depending on what functionallity your customer has selected.
Also, going with the "services per customer" approach, make sure you got enough nodes to handle the load when you get customers online. You'll end up with a lot of processes (since Application Instances) runs their own processes of the services, and if you have few nodes with a lot of these, reboots to your cluster nodes can take a bit to stabilise since it can potentionally have many services it needs to relocate. Altough running Stateless Services will aleviate a good part of this.
My understanding of the VMs involved in Azure Cloud Services is that at least some parts of it are not meant to persist throughout the lifetime of the service (unlike regular VMs that you can create through Azure).
This is why you must use Startup Tasks in your ServiceDefinition.csdef file in order to configure certain things.
However, after playing around with it for a while, I can't figure out what does and does not persist.
For instance, I installed an ISAPI filter into IIS by logging into remote desktop. That seems to have persisted across deployments and even a reimaging.
Is there a list somewhere of what does and does not persist and when that persistence will end (what triggers the clearing of it)?
See http://blogs.msdn.com/b/kwill/archive/2012/10/05/windows-azure-disk-partition-preservation.aspx for information about what is preserved on an Azure PaaS VM in different scenarios.
In short, the only things that will truly persist are things packaged in your cscfg/cspkg (ie. startup tasks). Anything else done at runtime or via RDP will eventually be removed.
See - How to: Update a cloud service role or deployment - in most cases, an UPDATE to an existing deployment will preserve local data while updating the application code for your cloud service.
Be aware that if you change the size of a role (that is, the size of a virtual machine that hosts a role instance) or the number of roles, each role instance (virtual machine) must be re-imaged, and any local data will be lost.
Also if you use the standard deployment practice of creating a new deployment in the staging slot and then swapping the VIP, you will also lose all local data (these are new VMs).
We have a VM say SampleVM depoyed & running on Azure Environment and on the same we have created 2 instances One is WebRole & other is WorkerRole running onto Slot staging.
My prob is I am able to Start/Stop SampleVM through powershell command but How can I Start/Stop a single instance(WebRole or WorkerRole) running on a SampleVM.
However when I stop SampleVM then both the Instance also stopped but I want to stop only one Instance/VM i.e. WebRole Or WorkerRole.
Please provide some powershell command with the argument pass to Stop/Start a single instance
Good answer by Gaurav, but I wanted to add a bit more detail, as I think there may be a bit of confusion around web and worker roles. Each role is a definition for a group of virtual machines that do the same thing, as constructed by you (you don't deal with the OS - you just kickstart your app, and Azure takes care of the VM itself).
When a Cloud Service is running, there will be a minimum of one instance of each role type. So in your case, running both a web role and worker role, you'll have at least two VMs running.
If you choose to scale out your web role to, say, 3 instances, and then decide to dial it back to 2 instances, you cannot choose which one to shut down; this is taken care of by Azure's fabric. Remember that each instance of a role is running identical code, and Azure load-balances traffic to your role instances (through external endpoints that you define). The only thing you need to worry about is shutdown. You have approx. 5 minutes to clean up any running processes (and you can easily take a particular instance out of the load balancer during shutdown, since you receive a Stopping() event).
You cannot shut down an entire role (e.g. all instances of a role) within a cloud service (so... you can't take down your worker role instances while leaving your web role instances running). If that's a requirement, then you can always consider running your web role in one Cloud Service and worker role in another Cloud Service. If they use queues to pass data, everything will still work as before. If the web role instances need direct access to worker role instances, you could put both Cloud Services into a Virtual Network.
One more thing to consider: You don't have to have separate roles. If cost is a factor, you can run all your code in your web role. Takes little work to spin up additional processes / threads in your web role, during OnStart() - remember that role instances are full Windows Server virtual machines; run whatever you want. With a single role definition, scaling is a bit coarse: Everything is scaled together. With separate roles, you can fine-tune your scaling (much more important when building larger systems).
David's broader points about how to model your PaaS service are all correct. But to add to it, there is a new Service Management API that has just been released called Delete Role Instance which will let you delete a specific instance of a role - http://msdn.microsoft.com/en-us/library/windowsazure/dn469418.aspx. This functionality is brand new to allow you to choose which instance to delete instead of being subject to the default fabric behavior that David describes (always deleting the last instance).
Simple answer is that as of today, you can't stop a single instance of your web/worker role. When you stop a role, all instances are stopped. You could remove instances from your role but again you can't specify which particular instance you want to remove. See #kwill's answer below.
You may also find following links useful for deleting specific role instances:
http://gauravmantri.com/2013/10/16/a-new-version-of-windows-azure-service-management-api-is-available-with-delete-specific-role-instances-and-more-goodies/
https://github.com/richorama/Two10.Azure.SelfDestruct
The company I work for is looking to develop a few apps against the cloud.
An ASP.NET Web Api application hosted in an Azure web role.
A Windows Server type application hosted in an Azure worker role.
We are completely new to web or cloud development and would like to know the following:
When being served to the consumer, is the same instance of the application being served to all, is it one per request or are multiple roles being created and served to consumers?
When being served to the consumer, is the same instance of the application being served to all?
That depends on how many instances you've asked Azure to run your application on. If you've only deployed to 1 instance, then it will of course be the same instance that responds to all requests. But if you deploy to multiple instances, requests will be load-balanced, which means you have no guarantee that multiple requests from the same user will be handled by the same instance.
When you're asking this question, it could be because you might be tempted to store local data on the machine running the instance. However, this is not a good idea. Windows Azure can at any time tear down your instance and start your application on a completely different machine. They call this "healing", because it usually happens because Windows Azure tries to be helpful and avoid any potential problem that could mean downtime for your instance. But it also happens if your machine for some reason locks up or something else bad happens. This process of healing means that anything that's not part of your deployment package will be lost. So for example, if you're logging to a file on the disk, this log will be lost if Azure "heals" your instance.
is it one per request or are multiple roles being created and served to consumers?
I'm not completely sure what you mean here, so I'll take a guess and risk interpreting your question wrongly. My guess is that you're asking if there will be one instance per user request. No, there will only be the number of instances that you have decided. Remember that you have to pay per instance that's running, so it's only fair that the number of instances running is dictated by you.
When you have your application packaged and ready to be deployed to Windows Azure, you can decide how many instances of each role you want to have running. You set this number in the deployment package, so that when your package is deployed, Azure will automatically start the requested number of instances. However, you can change the number of running instances of each role after deployment and on-the-fly. This makes it possible for you to scale with more instances within minutes.
I hope this helps and that I understood your questions correctly. :-)
Azure Web and Worker roles in an Azure Cloud Service are deployed on at least one instance (managed VM). Azure allows you to size (memory, CPU) and scale (number of instances). Azure actually lets you scale dynamically, i.e. add more instances on demand. You pay by the hour for the size & number of instances deployed.
For example, a Cloud service can have a single instance of a worker role (background processing) and multiple instances of the Web role. Multiple instances are handled behind a load balancer and the client is unaware of what instance they are using (all instances are created equal).
An Azure role instance is a VM with some specific payload.
For example, in your service you declare you want three instances of "Frontend" web role and two instances of "Backend" worker role. When Azure deploys your service it starts five VMs and three of them will run "Frontend" payload and have IIS started and two of them will run "Backend" payload and have no IIS started.
Now until you ask Azure to change that configuration it remains persistent no matter what requests come and what load occurs. You have five VMs with 3+2 configurations. To change the configuration you need some action on your part.
There're two way to have the configuration changed. You can use Management Portal or an external program to change the "instance count" in either or both roles. You can also add auto-scaling code that will gather metrics and make Management API requests to alter the "instance count". Either way when "instance count" goes up Azure starts more VMs with the same payload and when it goes down it stops some of the VMs.
We have a WebRole which deals with request coming in off a WCF service, this validates then puts the messages into an Azure queue in Cloud storage. In the same application we have a WorkerRole which reads the information from the queue and makes a call to our persistence layer which processes the request and returns the result. We were wondering why the worker roles didn't pick up any of our configuration settings and hence was not providing the Trace information we were looking for. We realised that the worker role was likely looking for an app.config and couldn't find it.
Is there a way to point the worker role to the web config, or to be able to load our Enterprise Library settings into the ServiceConfiguration.cscfg file which in either case would mean both could read from a common place?
Many thanks in advance
Kindo
As far as I'm aware there is no way for your worker role to get access to the web config of a web role out of the box.
If you move your configuration items to the ServiceConfiguration.csfg file and both the worker and web role are in the same cloud project, the settings will be in the same file. But because the web role and the worker role are different projects within that cloud project, their settings are in different sections of that .csfg file. If you want the settings to be the same for both of them, you will have to duplicate the settings.
Putting your settings in this file gives you the advantage that you can change the settings while the roles are running and have the roles respond however you like e.g. you might want certain settings to restart the roles, for others you may just want to update a static variable. In order to update a web.config or app.config you need to redeploy that role.
You do need to be aware though that the ServiceConfiguration file is not a replacement for a webconfig. If you're using tools that look for their settings in a web or app config, unless they're particularly smart and aware of the Azure environment, they won't go looking for settings in the ServiceConfiguration file.
I know you didn't ask this question, but if you're expecting your worker role to be providing an almost synchronous response to your web role by using a request queue and a response queue, you'll probably find that this won't scale very well. If you want to do synchronous calls to a worker role in Azure you're best off using WCF (this isn't highlighted anywhere in the guides). As you said that all of this is just a WCF service anyway, the answer to your problem may be to just do away with the worker role.
You cannot share web.config's OR .cscfg's across Roles in Azure, as there is no guarantee a role is in the same host, cluster, or even datacenter as another role.
If you are simply trying to share items like connection Strings, app-specific variables, etc., I would simply create your own "config manager" class that would obtain some XML and parse it into a collection of settings. This way, you could store that config on Azure Blob Storage and changes would be as simply as updating that blob and signaling your apps to reload. (very easy using the Service Management API).