When publishing an Azure ML Web Service and preloading data in our R model we see inconsistent performance. First calls are slow but following calls are fast, waiting a bit (couple of minutes) for the next call ends up showing longer response times.
The way Azure ML Web Services work in the background means that instances hosting the models are provisioned and moved in a very dynamic multi-tenant environment. Caching data (warming up) can be helpful but this doesn't mean all subsequent calls will land on the same instance with the same data available in the cache.
For models that need a lot of in-memory data there is a limit to what the Azure ML Web Services hosting layer can offer at this point. Microsoft R server could be an alternative to host these big ML workloads and looking at Service Fabric to scale
Related
Please help me understand why we say azure functions is a serverless compute service. It does require cloud to host it and run. Cloud is also a server still why we are saying it is serverless?
Serverless computing does not mean that servers are out of the picture. Servers are very much required, just like they have been for all these years, or else, where will your code run. The reason why the phrase was coined is that as a developer, you do not need to worry about what server your code runs on. In fact, you do not know which server it eventually runs on. Once your code is deployed, Azure assigns the responsibility of executing the code to the next available server. What Azure ensures, and what is ultimately important for you, is that your code will execute whenever required.
Ref: Serverless Computing with Azure Functions
Hope it makes sense :)
To get a better idea this is how we evolved. Cloud providers are making sure we should only worry about the business logic but nothing else.
IaaS (Infrastructure as a service)
You get a running VM somewhere in the data centre but you are required to maintain everything. From Deployment to patching your VMs or anything running on the VM.
PaaS(Platform as a service)
You are not longer required to maintain platForm but you are still responsible to manage your server in terms of load balancing etc.
FaaS(Function as a service)
Servers are abstracted from you . You are only required to maintain your code without worrying about what's under the hood or how to load balance your servers. It's then cloud provider responsibility to package your code and run it for you. But servers are still there.
Going by the official documentation of Azure Serverless computing service, Azure Functions can be defined as;
Azure Functions is a serverless compute service that enables you to run code on-demand without having to explicitly provision or manage infrastructure. Use Azure Functions to run a script or piece of code in response to a variety of events.
Azure Functions is an event driven, compute-on-demand experience that
extends the existing Azure application platform with capabilities to
implement code triggered by events occurring in virtually any Azure or
3rd party service as well as on-premises systems. Azure Functions
allows developers to take action by connecting to data sources or
messaging solutions, thus making it easy to process and react to
events. Azure Functions scale based on demand and you pay only for the
resources you consume.
Here the serverless compute service is like a metaphor which implies, the end user doesn't need to manage the servers or infrastructure to run the applications over the Azure and can spend time to focus on managing and improving the business logic.
Few more points to consider,
Serverless in Azure builds on an open-source foundation, the core of
which is Azure Functions, an event-driven compute experience and open
source project. Community contributions include support for new
languages, integrations and deployment targets.
Azure Functions can be used on-premises, in hybrid environments such as Azure Stack, on IoT Edge devices and deployed on top of orchestrators such as Kubernetes – as well as in other clouds.
They enable faster time to market with lower infrastructure and operating costs.
There are heaps of definition of serverless which you can easily google. But I will share my understanding anyways.
1. It does require cloud to host it and run.
You are correct with this. But anything on Cloud requires Cloud, doesn't it? Azure being one of the cloud providers consists of hundreds of services to cater to different needs people are after from using Cloud.
2. Cloud is also a server still why we are saying is serverless
This is not quite right. Cloud is different from a server. Server is a physical box sitting somewhere. With hundreds of thousands of servers all over the world, Cloud hosts all sorts of different services on these servers.
The reason we say Functions are serverless is that the infrastructure of hosting a Function is abstracted away from devs. It is still deployed to some servers but Azure is responsible for all the resource managing, configuration, load balancing, scaling and networking etc. It allows developers to focus primarily on their code, not having to worry about servers.
We have a scenario for on prem to upload a bunch of pdf/tiffs then service on the cloud to process them. In the same time a queue is populated with metadata on processing instructions.
We are trying to decide if whether the worker role or the Azure Batch is the right choice for this. Our primary goal are,
need to scalable base on queue size
scale ramp up time need to be quick
of course cost is another factor
You should not use PaaS cloud services for new workloads. There are various options depending on the complexity of the processing you need to do. The following are likely better choices than Azure Batch: Azure Web Jobs, Virtual Machine Scale Sets. The first is easier to use, and is based on the Web Apps technology, while the second is the (preview) way to provide scale-out VMs.
PaaS cloud services are classic now, you can use webjobs with appservices. Another interesting option is AZURE STACK which is really good for hybrid cloud. refer this link
https://azure.microsoft.com/en-us/overview/azure-stack AZURE STACK is really good for hybrid cloud, As if now it is in Technical preview.
I'm trying to figure out a solution for recurring data aggregation of several thousand remote XML and JSON data files, by using Azure queues and WebJobs to fetch the data.
Basically, an input endpoint URL of some sort would be called (with a data URL as parameter) on an Azure website/app. It should trigger a WebJobs background job (or can it continuously running and checking the queue periodically for new work), fetch the data URL and then callback an external endpoint URL on completion.
Now the main concern is the volume and its performance/scaling/pricing overhead. There will be around 10,000 URLs to be fetched every 10-60 minutes (most URLs will be fetched once every 60 minutes). With regards to this scenario of recurring high-volume background jobs, I have a couple of questions:
Is Azure WebJobs (or Workers?) the right option for background processing at this volume, and be able to scale accordingly?
For this sort of volume, which Azure website tier will be most suitable (comparison at http://azure.microsoft.com/en-us/pricing/details/app-service/)? Or would only a Cloud or VM(s) work at this scale?
Any suggestions or tips are appreciated.
Yes, Azure WebJobs is an ideal solution to this. Azure WebJobs will scale with your Web App (formerly Websites). So, if you increase your web app instances, you will also increase your web job instances. There are ways to prevent this but that's the default behavior. You could also setup autoscale to automatically scale your web app based on CPU or other performance rules you specify.
It is also possible to scale your web job independently of your web front end (WFE) by deploying the web job to a web app separate from the web app where your WFE is deployed. This has the benefit of not taking up machine resources (CPU, RAM) that your WFE is using while giving you flexibility to scale your web job instances to the appropriate level. Not saying this is what you should do. You will have to do some load testing to determine if this strategy is right (or necessary) for your situation.
You should consider at least the Basic tier for your web app. That would allow you to scale out to 3 instances if you needed to and also removes the CPU and Network I/O limits that the Free and Shared plans have.
As for the queue, I would definitely suggest using the WebJobs SDK and let the JobHost (from the SDK) invoke your web job function for you instead of polling the queue. This is a really slick solution and frees you from having to write the infrastructure code to retrieve messages from the queue, manage message visibility, delete the message, etc. For a working example of this and a quick start on building your web job like this, take a look at the sample code the Azure WebJobs SDK Queues template punches out for you.
Azure Cloud Services have auto-scale based on CPU / Queue. We have a set of machines running API for uploading and processing files. Although we moved the processing part on Worker Role that scale depending on the queue size, the servers but also take care of the upload while responding to other operations like downloading.
Right now we're using more machines for the just in case scenario, but we want to build a way to scale and to be cost-efficient while having a great upload experience for our users.
What would your approach be for creating a way to detect the network usage across all machines from the same Cloud Service and auto-scale if necessary?
I would:
1) Create metrics that calculate the amount of time it takes to download/upload a file.
2) Aggregate the metrics in some persistence layer (we have plenty in Azure)
3) Create a service that looks those metrics
4) Check the thresholds
5) Use the Management Libraries for .NET to trigger scaling on the Cloud Service(s) affected.
This approach also scales with your solution. You can eventually separate the scaling part from the checking part and have them as two different services, communicate asynchronously.
We also have an old, open source now project that does some of that for you, so you don't have to reinvent the wheel. It's called WASABi. Be careful though as this is not maintained anymore but as I said, you can use it as inspiration.
I am recently evaluating Windows Azure. One of the problems I found is that Azure start charging as soon as the app is deployed even if it is in the testing stage.
I want to ask existing Azures how much of your tests are done locally and how much are done after it is deployed? Does Azure provide any means of testing web services locally?
Many thanks.
Yes, Azure provides an emulation framework that largely (but not completely) mimics the Azure deployment environment. This is usually sufficient for testing.
Costs of test deployments can be controlled somewhat, however:
It's possible to deploy "extra-small" instances that are significantly less expensive than larger instances, at the expense of throughput - which unless you're doing load testing isn't usually an issue
You won't generally need to have multiple instances of a role deployed, just one will usually do, unless you have major concurrency issues under load
Some of the cost of Azure is in data traffic, which will obviously be less expensive for test instances
It's not necessary to have test instances permanently available. They can be torn down or re-deployed at will; if your environment becomes sophisticated this can be done programmatically by a continuous integration engine.
In practice we're finding that the cost of test instances is relatively insignificant compared to the cost of our developers and the alternative, which would be to provision and maintain our own data centre.
In particular, being able to quickly spin up a test environment that is a direct mimic of production in a few minutes is a very powerful feature.
Windows azure already provide option to do testing locally.
The Microsoft Azure storage emulator provides a local environment that emulates the Azure Blob, Queue, and Table services for development purposes. Using the storage emulator, you can test your application against the storage services locally, without creating an Azure subscription or incurring any costs. When you're satisfied with how your application is working in the emulator, you can switch to using an Azure storage account in the cloud.
To get complete detail please check link below.
https://azure.microsoft.com/en-in/documentation/articles/storage-use-emulator/