I don't understand why some Azure resources can be aligned with an Affinity Group, but others can't.
Example: I just tried creating a Service Bus namespace. I'd like to align it with an existing affinity group, but I'm only able to select a region. I had the same issue trying to create a SQL Database server.
This is most likely a misunderstanding of exactly how Affinity Groups work on my part, but I don't understand the inconsistency.
What am I missing something here? Can someone please explain the reasoning behind not being able to align a Service Bus namespace with an Affinity Group?
It's not exactly a technical reason. It has to do with how Microsoft was internally aligned back when the platform called Windows Azure started. You have to remember that different teams created the Service Bus (used to be Biztalk Services), SQL Database (used to be SQL Azure, SQL Server Data Services, etc.), and Windows Azure (used to be Red Dog). Each service was created independently and in fact didn't even rely on the other (in the beginning at least).
I can't even recall now the myriad of different brandings and platforms that were there - it doesn't matter anymore. The reality is that there are still different teams that build those products and they don't always align at first. SQL Database used to be deployed into different clusters (not sure if that is true anymore) so affinity groups were not relevant. Same with Service Bus. For now, just remember that Affinity Groups only apply to Windows Azure compute and storage. Over time, I would expect a more cohesive experience (like the portal). However, some concepts are still rooted to where the team that built them resides.
Related
I've deployed web sites and services to the cloud before but it was a while ago and I wanted to revisit my approach to inventorize my skills. During the research, I've been told to use a worker role but I'm not sure in what constellation to apply it.
The image presents my choices. I'll be setting up two things (preferably on the same base URL).
1. A web site (ASP.NET, most likely MVC powered by Razor)
2. A service layer (guessingly WCF, as there's not much else to pick from today)
So, in my naive ignorance, I added ASP.NET Web Role for the former and WCF Service Web Role for the latter. Then, according to the hint, I also added Worker Role. And this is where I got humble and started to suspect that my ignorance was rather an arrogance...
Do I need all the three of them? Or is it perhaps so that Worker Role covers the others? Or are the others sufficient and I need to Worker Role? Or am I totally confusing the concepts here?
I've tried to google those but I realize that I haven't reached the threshold of learning by doing in this area yet. I get more confused and headacheish the more I read. Admittedly, my problem might lie in the wrong choice of search words and/or linguistic misconception. If so, my apologies...
The answer is, it depends...
A web role is essentially a Worker role with IIS installed + configured. You could host a WebApi/MVC, WCF AND process events all from the same web role if you really wanted to, reducing costs.
Remember that each role is a separate VM that you have to pay for, so adding extras roles to keep everything separate may not always be the best idea.
In one of our projects for example, we use a web role to host a WebApi. A Worker role to process internal events, and a worker role to host WCF services (you can also use a web role for this). We split them because they take very different workloads and perform separate functions, so being able to scale them independently made sense.
HTH
There's no right answer to how many roles to use in a cloud service. But it's important to understand exactly what those roles are.
Adding a bit to #Peter's answer: Each role is a definition of a VM (its contents) - think of it as a VM template. And for each role (template), you must have a minimum of one instance (VM) running. If you have one role, your minimum footprint will be one VM (of whichever size you specify for that role). If you have three roles, you'll have minimum 3 VMs running.
Whether you have one role or many depends on how you want to scale your application. Each role defines not only what goes in it, but also the size of the VMs uses by the role instances. By having different roles for different parts of your architecture, you can choose to scale those parts differently. For example, you might only need low-resource instances to handle your web tier, but maybe more CPU power for your service tier. And maybe your web tier scales dynamically based on user traffic, but you're able to handle, say, your service tier with just one or two instances. Of course, you can put everything in one role definition, and scale everything together. It's totally up to you.
Iam relatively new to Cloud Computing and azure. I was wondering whether you can have more than one web and worker role in an Azure application. If so what advantages can I get using multiple roles and where do they apply?
Yes, you can have more than 1 web or worker role in an Azure Cloud Service. You can have up to 25 different roles per deployment I believe in any mix of Web and Worker roles. See the Azure Subscription and Service Limits, Quotas and Constraints link for more information.
The advantage of having the roles within the same cloud service is simply that within that cloud service they can see all the other roles and instances easily (unless you configure them otherwise). They will all be relatively close to each other within a data center because a cloud service is assigned to a stamp of machines and controlled by a Fabric Controller assigned to that stamp. You can watch this video by Mark Russinovich which sheds more light on the inner workings of Azure and talks a bit about stamps I think. A cloud service is a security boundary as well, so you get some benefits from that encapsulation if you need to do a lot of inter machine communication that ISN'T going across a queue for some reason.
The disadvantage of batching a whole bunch of roles together is that they are tied pretty closely together at that point. You can certainly scale them separately, and you can do updates that target only a single role at a time. However, if you want to deploy changes to multiple roles you may end up having to do a full deployment to all roles (even those that haven't changed) or do updates to single roles one at a time until all the ones you need updated are, which can take some time. Of course, it could be argued that having them in separate cloud services would still have you doing updates concurrently depending on your architecture and/or dependencies.
My suggestion is to group only roles that REALLY belong together in the same solution. These are role that have workloads that are interrelated. Even then, there's nothing stopping you from separating these as well into separate deployments (though you may benefit from the security boundaries that being within the same cloud service). Think about how each role will be updated, and if they would generally be updated together or not. There are many factors in thinking about how to package roles together.
What is established best practice in porting a Windows Service to Azure? Should it be changed into a Worker Role or moved into a VM Role? Are there other options? Assume that my services write to external persistence sources (MSMQ, databases, WCF) rather than to the file system directly.
You are far better off converting your Windows Services to Worker-Roles than VM roles. VM roles are meant to house applications that require complex un-automatable installation procedures. They are also a bigger pain to manage and you want to stay away from VM roles as much as possible. If you can find a way to automate deployment of your existing Windows Services via Worker-Roles, it is definitely the way to go.
You can also looking into HPC roles and depending on the on-prem/off-prem and load/compute requirements, adding Azure machines to your HPC cluster maybe of benefit.
All types of Roles (Web/Worker/VM/HPC) are stateless and require to be able to spin-up or tear-down from scratch on demand. All types of Roles are meant to run more than one VM instance at a time.
HTH
I wrote a blog post about this a while back. It is here:
http://blogs.msdn.com/b/golive/archive/2011/02/11/installing-a-windows-service-in-a-worker-role.aspx
Note that a Windows Service won't communicate directly with the fabric controller, so you need to ping it periodically to check health, then take remediative actions as needed.
Putting a Windows Service into a worker or web role is accepted practice. The main reason to go with VM Role is if there is significant (>10 minutes) setup required. My blog post details how to install your service.
Of course, if you want to move the code into a worker role, that's also fine. In this case you don't need any special steps to ensure the fabric controller is aware of its health.
If cost is an issue, combining functions into web/worker is also accepted practice. And you can save by not working over your code to get it into a web/worker.
Azure has a special type of Web Role called "WCF Service Web Role" which corresponds to a Windows WCF Service. This is a good point for migrating existing services.
Ideally the migration should be followed by taking advantage of Azure specific features, for instance using queues and work roles to maximise perfromance and scalability.
We are experiencing a very serious unscheduled downtime of our Azure application today for what is now coming up to 9 hours. We reported to Azure support and the ops team is actively trying to fix the problem and I do not doubt that. We managed to get our application running on another "test" hosted service that we have and redirected our CNAME to point at the instance so our customers are happy, but the "main" hosted service is still unavailable.
My own "finger in the air" instinct is that the issue is network related within our data center (west europe), and indeed, later on in the day the service dash board has gone red for that region with a message to that effect. (Our application is showing as "Healthy" in the portal, but is unreachable via our cloudapp.net URL. Additionally threads within our application are logging sql connection exceptions into our storage account as it cannot contact the DB)
What is very strange, though, is that the "test" instance I referred to above is also in the same data centre and has no issues contacting the DB and it's external endpoint is fully available.
I would like to ask the community if there is anything that I could have done better to avoid this downtime? I obeyed the guidance with respect to having at least 2 roles instances per role, yet I still got burned. Should I move to a more reliable data centre? Should I deploy my application to multiple data centres? How would I manage the fact that my SQL-Azure DB is in the same datacentre?
Any constructive guidance would be appreciated - being a techie, I've never had a more frustrating day being able to do nothing to help fix the issue.
There was an outage in the European data center today with respect to SQL Azure. Some of our clients got hit and had to move to another data center.
If you are running mission critical applications that cannot be down, I would deploy the application into multiple regions. DNS resolution is obviously a weak link right now in Azure, but can be worked around (if you only run a website it can be done very simply using Response.Redirects or similar)
Now, there is a data synchronization service from Microsoft that will sync up multiple SQL Azure databases. Check here. This way, you can have mirror sites up in different regions and have them be in sync with SQL Azure perspective
Also, be a good idea to employ a 3rd party monitoring service that would detect problems with your deployed instances externally. AzureWatch can notify or even deploy new nodes if you choose to, when some of the instances turn "Unresponsive"
Hope this helps
I can offer some guidance based on our experience:
Host your application in multiple data centers, complete with Sql Azure databases. You can connect each application to its data center specific Sql Server. You can also cache any external assets (images/JS/CSS) on the data center specific Windows Azure machine or leverage Azure Blog Storage. Note: Extra costs will be incurred.
Setup one-way SQL replication between your primary Sql Azure DB and the instance in the other data center. If you want to do bi-rectional replication, take a look at the MSDN site for guidance.
Leverage Azure Traffic Manager to route traffic to the data center closest to the user. It has geo-detection capabilities which will also improve the latency of your application. So you can redirect map http://myapp.com to the internal url of your data center and a user in Europe should automatically get redirected to the European data center and vice versa for USA. Note: At the time of writing this post, there is not a way to automatically detect and failover to a data center. Manual steps will be involved, once a failover is detected and failover is a complete set (i.e. you will failover both the Windows Azure AND Sql Azure instances). If you want micro-level failover, then I suggest putting all your config the in the service config file and encrypt the values so you can edit the connection string to connect instance X to DB Y.
You are all set now. I would create or install a local application to detect the availability of the site. A better solution would be to create a page to check for the availability of application specific components by writing a diagnostic page or web service and then poll it from a local computer.
HTH
As you're deploying to Azure you don't have much control about how SQL server is setup. MS have already set it up so that it is highly available.
Having said that, it seems that MS has been having some issues with SQL Azure over the last few days. We've been told that it only affected "a small number of users". At one point the service dashboard had 5 data centres affected by a problem. I had 3 databases in one of those data centres down twice for about an hour each time, but one database in another affected data centre that had no interruption.
If having a database connection is critical to your app, then the only way in the Azure environment to ensure against problems that MS haven't prepared against (this latest technical problem, earthquakes, meteor strikes) would be to co-locate your sql data in another data centre. At the moment the most practical way to do this is to use the synch framework. There is an ability to copy SQL Azure databases, but this only works within a data centre. With your data located elsewhere you could then point your app at the new database if the main one becomes unavailable.
While this looks good on paper though, this may not have helped you with the latest problem as it did affect multiple data centres. If you'd just been making database copies on a regular basis, that might have been enough to get you through. Or not.
(I would have posted this answer on server fault, but I couldn't find the question)
This is just about a programming/architecture issue, but you amy also want to ask the question on webmasters.stackexchange.com
You need to find out the root cause before drawing any conclusions.
However. my guess one of two things was the problem
The ISP connectivity differs for the test system and your production system. Either they use different ISPs, or different lines from the same ISP. When I worked in a hosting company we made sure that ou IP connectivity went through at least two different ISPS who did not share fibre to our premises (and where we could, they had different physical routes to the building - the homing ability of backhoes when there's a critical piece of fibre to dig up is well proven
Your datacentre had an issue with some shared production infrastructure. These might be edge routers, firewalls, load balancers, intrusion detection systems, traffic shapers etc. These typically are also often only installed on production systems. Defences here involve understanding the architecture and making sure the provider has a (tested!) DR plan for restoring SOME service when things go pair shaped. Neatest hack I saw here was persuading an IPS (intrusion prevention system) that its own management servers were malicious. And so you couldn't reconfigure it at all.
Just a thought - your DC doesn't host any of the Wikileaks mirrors, or Paypal/Mastercard/Amazon (who are getting DDOS'd by wikileaks supporters at the moment)?
Is there a way do do dynamic elasticity in Windows Azure? If my workers begin to get overloaded, or queues start to get too full, or too many workers have no work to do, is there a way to dynamically add or remove workers through code or is that just done manually (requires human intervention) right now? Does anyone know of any plans to add that if its not currently available?
Microsoft shipped the Autoscaling Application Block (Wasabi) to provide dynamic scaling. Some of the supported scenarios:
Autoscaling both web and worker roles in Windows Azure by dynamically changing instance counts or performing application throttling.
Autoscaling Windows Azure roles based on timetables.
Autoscaling Windows Azure roles based on metrics collected from the application and/or Windows Azure but constrained by upper and lower bounds on the instance count per role.
Preventing fast oscillations in the number of role instances with the stabilizer. The stabilizer can also help to optimize costs by limiting scaling up operations to the beginning of the hour and scaling down operations to the end of the hour.
Monitoring and logging autoscaling activity.
Sending notifications to preview any scaling operations before they take place.
Encrypting the rules and other configuration in Windows Azure blob storage or in local file storage.
Managing the autoscaler configuration by using Windows PowerShell.
A comprehensie sample application (Tailspin Surveys) showcasing all these features is provided (installation instructions are available here). Also, check out the Developer's Guide and the Channel9 video walkthrough.
The block is available as standalone download of binaries, source or via NuGet.
Here are a couple of talks/demos showing Wasabi in action:
CloudCover Episode on autoscaling
p&p symposium talk "Windows Azure app scaling to need"
There's a Service Management API, and you can use that to scale your application (from code running in Windows Azure or from code running outside of Windows Azure).
http://msdn.microsoft.com/en-us/library/ee460799.aspx and http://code.msdn.microsoft.com/Release/ProjectReleases.aspx?ProjectName=windowsazuresamples&ReleaseId=3233.
Windows Azure has just added the autoscaling feature built into the platform. Now it's trivially easy to configure your autoscaling rules right in the management portal:
See the announcement and the demo. I've also written a post comparing Windows Azure Autoscale to Wasabi and outlining the path forward.
Create a queue named autoscale.[your_role_name].instance_count
In the Management Portal, set the autoscale to Queue.
Set the Target Count field to 1.
Now you can use standard enqueue and dequeue operations on that queue to control the number of worker role instances. You've got 7 days to process a message before it expires, so you might want to create a worker role that can ensure that the number of messages in the queue is tracking your target instance count.
If you're after dynamic elasticity, you've probably already got a worker-role-based controller in mind already, so that's probably not a problem.
Lokad.Cloud open source project for Windows Azure contains distributed executor framework. Among other things it provides auto-scaling with VM provisioning feature.