I am designing the architecture for an Azure application, and I have a few questions on how to proceed. I am familiar with the basics of Azure, but have never built and deployed an Azure application before. I have extensive experience with conventional non-cloud, web-hosted applications, though.
My application will be the usual database-centric business system with a web user interface. We want to start very small and grow it slowly as we gain user base. I am planning to use an SQL Azure database for relational storage as well as blob storage for documents and the like. These will be accessed by a Data Access Layer, which in turn will be operated by a Business Layer. The web user interface will be built using ASP.NET and will rest on the Business Layer.
All this is very traditional, but I wonder how well it fits with Azure. I have some specific and inter-related questions:
I see the Data Layer and Business Layer as part of an Application Tier that can be deployed on a worker role, whereas the web user interface can be deployed as a Front-End Tier on a web role. Is separating the business and presentation logic like this a wise decision on Azure?
Having said the above, having two separate roles wouldn't make sense while the user base is very small, so I would rather deploy everything together on a single web role until we get bigger. What do I need to do to make sure that these two tiers can be easily reconfigured to work as either one or two roles with any recoding?
The communication between the web user interface and the Business Layer must be fast; I am concerned that it won't be very fast especially when these two are deployed as separate tiers on different web/worker roles. What is the best communications mechanism in Azure that I should use? I have considered queue storage, service bus and virtual network, but I am not sure how to make a decision here.
I have been reading some best practices posts and documents online, but they seem to address advanced issues. I would rather like to have answers to these quite basic concerns in the form of pointers to best practices articles or the like. Thank you.
Related
I have been assigned to think of a layered microservices architecture for Azure Service Fabric. But my experience mostly been on monolithic architectures I can't come up with a specific solution.
What I have thought as of now is like...
Data Layer - This is where all the Code First entities resides along with DBContext.
Business Layer - This is where all the Service Managers would be performing and enforcing the Business Logic i.e. UserManager (IUserManager), OrderManager (IOrderManager), InvoiceManager (IInvoiceManager) etc.
WebAPI (Self Hoted Inside Service Fabric) - Although this WebAPI is inside Service Fabric but does nothing except to receive the request and call respectic Services under Service Fabric. WebAPI Layer would also do any Authentication and Authorization (ASP.NET Identity) before passing on the call to other services.
Service Fabric Services - UserService, OrderService, InvoiceService. These services are invoked from WebAPI Layer and DI the Business Layer (IUserManager, IOrderManager, IInvoiceManager) to perform it's operation.
Do you think this is okay to proceed with?
One theoretical issue though, while reading up for several microservices architecture resources, I found that, all of them suggests to have Business Logic inside the service so that the specific service can be scaled independently. So I believe, I'm violating the basic aspect of microservices.
I'm doing this because, the customer requirement is to use this Business Layer across several projects, such as Batch Jobs (Azure Web Jobs), Backend Dashboard for Internal Employees (ASP.NET MVC) etc. So If I don't keep the Business Layer same, I have to write the same Business Logic again for Web Jobs and Backend Dashboard which I feel is not a good idea. As a simple change in Business Logic would require change in code at several places then.
One more concern is, in that case, I have to go with Service to Service communication for ACID transactions. Such as, while creating an Order, a Order and Invoice both must be created. So in that case, I thought of using Event Driven programming i.e. Order Service will emit an event which the Invoice Service can subscribe to, to create Invoice on creation of Order. But the complications are if the Invoice Service fails to create invoice, it can either keep trying do that infinitely (which is a bad idea I think), or emit another event to Order Service to subscribe and roll back the order. There can be lots of confusion with this.
Also, I must mention that, we are using a Single Database as of now.
So my questions are...
What issue do you see with my approach? Is it okay?
If not, please suggest me a better approach. You can guide me to some resources for implementation details or conceptual details too.
NOTE : The requirement of client is, they can scale specific module in need. Such as, UserService might not be used much as there won't be many signups daily or change in User Profile, but OrderService can be scaled along as there can be lots of Orders coming in daily.
I'll be glad to learn. As this is my first chance of getting my hands on designing a microservices architecture.
First of all, why does the customer want to use Service Fabric and a microservices archtecture when it at the same time sounds like there are other parts of the solution (webjobs etc) that will not be a part of thar architecture but rather live in it's own ecosystem (yet share logic)? I think it would be good for you to first understand the underlying requirements that should guide the architecture. What is most imortant?
Scalability? Flexibility?
Development and deployment? Maintinability?
Modularity in ability to compose new solutions based on autonomous microservices?
The list could go on. Until you figure this out there is really no point in designing further as you don't know what you are designing for...
As for sharing business logic with webjobs, there is nothing preventing you from sharing code packages containing the same BL, it doesn't have to be a shared service and it doesn't mean that it has to be packaged the same way in relation to its interface or persistance. Another thing to consider is, why do you wan't to run webjobs when you can build similar functionality in SF services?
I am using Azure Table storage to store the data of my application. I need to build an API to retrieve the data from Azure table storage. There are like a million records in the table. In the initial stage, my API would be getting approximately 100-1000 hits per day. What would be the best choice to develop that API on, API apps or Service Fabric?
It depends on a lot of aspects:
How do you want to maintain it?
What performance requirements do you have? Do all those 100-1000 hits per day happen at the same time or are they distributed across the day?
How do you want to handle scaling?
As described by #feranto the main difference is the architectural pattern. The decision will have a lot of implications for you, not only when it comes to development, but you will have to consider how to maintain your application.
Only based on your short description, it sounds like a relatively straightforward solution and not very complex nor very high performance/scale requirements. Given that, it might be easier for you to go for a more basic API App. Service Fabric is an excellent framework and architecture that allows us to build very flexible, highly scalable solutions, but it should be noted that that also comes with a lot more complexity when it comes to handling, monitoring and maintaining both the underlying service cluster and it's applications.
Setting up an API App, connecting to an Azure Storage table and control that with for instance Application Insights is relatively straight forward and you can find a lot of documentation and samples for that. As a contrast, Service Fabric is fairly nascent in documentation and samples, and you would have to read up on a lot to get a basic stable solution running if you are new to it.
The main difference between Azure API and Azure Service Fabric is the architecture pattern each one follows:
Azure API APPs supports a monolithic architecture. You have all your code running as one single package. You can develop this using Java, PHP, nodejs, python or .net.
Service Fabric follows a microservice architecture. Every service is deployed on a separate node, which can scale independently, and you can monitor the health of every service in every node. You can develop this using Java or C# mainly.
Here you can see advantages and disadvantages between microservices and a monolithic architecture.
Background
Having run through the Getting started with API Apps and ASP.NET in Azure App Service tutorial (https://azure.microsoft.com/en-gb/documentation/articles/app-service-api-dotnet-get-started/), we had an architecture question arise today around the design decisions made to split out the To Do List Application API layers into a Middle tier API app and Data tier API app.
When approaching build of an application using a distributed architecture, what considerations should take place to understand when this type of separation should occur in your API layers?
Another way of asking this question is what are the pros and cons of having a separate middle tier API layer and data tier API app when building your application?
Other Questions
I had a read of Web apps architecture: 1 or n API question (see link that follows) which while being insightful, was slightly different to the question we are asking. We are talking a single domain which has separate API layers for middle tier (logic) and the data tier.
Web apps architecture: 1 or n API
This of course, depends. Deciding whether to build out what I call "infrastructure services" is very strongly dependent on your needs and your application(s).
Infrastructure tier services generally get much more re-use than business logic tier services. They are very easy to recompose into new applications. The most common instance of this is building an admin interface as a separate application.
If you have already build several applications in your organization, and have found reuse to be occurring regularly, then I would seriously contemplate infrastructure services. If your organization is writing it's first application, and you don't see this fanning out to additional interfaces, then maybe just isolate your data access in a DAO pattern, it's fairly straightforward to refactor it out to a stand-alone service later.
I think the example design is somewhat confusing. In real world, I have not seen such design yet because design looks like having every function to be http/rpc call?
My experience would be the SPA uses a Public API (or Gateway API) which then calls your Internal API / Microservices to aggregate results. It is your Microservices that may have DAOs and, most importantly, the business logic
I have an ASP.NET MVC 2 Azure application that I am trying to switch from being single tenant to multi-tenant. I have been reviewing many blogs and posts and questions here on Stack Overflow, but am still trying to wrap my head around the specifics of what's right for this particular app.
Currently the application stores some information in a SQL Azure database, as well as some other info in an Azure Storage Account. I'm considering writing the tenant provisioning code to simply create a new database for a new tenant, along with a new azure storage account. This brings me to the following question:
How will I go about testing this approach locally? As far as I can tell, the local Azure Storage Emulator only has 1 storage account. I'm not sure if I'm able to create others locally. How will I be able to test this locally? Or will it be possible?
There are many aspects to consider with multitenancy, one of which is data architecture. You also have billing, performance, security and so forth.
Regarding data architecture, let's first explore SQL storage. You have the following options available to you: add a CustomerID (or other identifyer) that your code will use to filter records, use different schema containers for different customers (each customer has its own copy of all the database objects owned by a dedicated schema in a database), linear sharding (in which each customer has its own database) and Federation (a feature of SQL Azure that offers progressive sharding based on performance and scalability needs). All these options are valid, but have different implications on performance, scalability, security, maintenance (such as backups), cost and of course database design. I couldn't tell you which one to choose based on the information you provided; some models are easier to implement than others if you already have a code base. Generally speaking a linear shard is the simplest model and provides strong customer isolation, but perhaps the most expensive of all. A schema-based separation is not too hard, but requires a good handle on security requirements and can introduce cross-customer performance issues because this approach is not shared-nothing (for customers on the same database). Finally Federations requires the use of a customer identifyer and has a few limitations; however this technology gives you more control over performance distribution and long-term scalability (because like a linear shard, Federation uses a shared-nothing architecture).
Regarding storage accounts, using different storage accounts per customer is definitively the way to go. The primary issue you will face if you don't use separate storage accounts is performance limitations, such as the maximum number of transactions per second that can be executed using a single storage account. As you are pointing out however, testing locally may be a problem; however consider this: the local emulator does not offer 100% parity with an Azure Storage Account (some functions are not supported in the emulator). So I would only use the local emulator for initial development and troubleshooting. Any serious testing, including multitenant testing, should be done using real storage accounts. This is the only way you can fully test an application.
You should consider not creating separate databases, but instead creating different object namespaces within a single SQL database. Each tenant can have their own set of tables.
Depending on how you are using storage, you can create separate storage containers or message queues per client.
Given these constraints you should be able to test locally with the storage emulator and local SQL instance.
Please let me know if you need further explanation.
I am having a hard time understanding Windows Azure service bus and access control concepts. In layman's terms, what are they? What are they used for?
The Service Bus component of Windows Azure is meant to handle the problems arising from services that are living in multiple networks. Basically, a service bus just makes it appear as if your code is running on a single machine, while in reality it could be running anywhere within the Azure datacenters.
Access Control lets you use "federated authentication for your service based on a claim-based RESTful model. (Sorry, copy&Paste from an O'Reilly book about Azure!)
Basically, when you create an Azure site, application or service, it could be running on any of the thousands of systems within the datacenter. And each of those systems has it's own IP address, it's own network, memory, processor and whatever more. To let them collaborate and to appear as a single system, these two services have been created.
If you want to learn more about Azure, this would be a good moment to buy a book! :-)
Azure is quite complex and service buses and access control are a bit more advanced topics.
Service Bus is a solution for the integration between multiple applications whether they are hosted on the same infrastructure or even spread along multiple infrastructure or/and Cloud Computing provider. If you search more in the internet you might find a lot about EAI (Enterprise application integration) here is my blog post about this topic:
http://hhaggan.wordpress.com/2013/03/07/introduction-to-enterprise-application-integration-eai/
and here another that I hope that helps you understand better what is the service bus:
http://hhaggan.wordpress.com/2013/03/09/introducing-service-bus/
in another words, it is a messaging platform that helps you communicate with multiple applications, softwares or services no matter what programming language they are written with or on which os or platform they are hosted on. you will feel its effect specially when you work on connecting multiple nodes together, I don't mean 5 or 6 nodes but 10 and above.
Certainly there are several types of service bus, whether they are based on relayed messaging service or brokered messaging service, each one of them has several uses, its purpose and way of working.
For the Access control, this is so easy, it is a way of authentication and authorization for your application using third parties, It is a claim based identity that you can do the required authentication through the third party database. you wont need to build everything from scratch in your database. this helps a lot during development and I believe that this can help a lot in social media marketing and branding because of the use of facebook, twitter during the authentication.