I am relatively new to cloud, so please guide me through complete process.
I have an application that will be hosted on containers in cloud environment. I want some temporary storage on the container or the cloud environment, and access it via my web application (written in C#), meaning I will generate a file and keep it there. First of all, is it possible without costing me extra? Secondly, if it is possible, how can I access the area with C# code? And even if it costs me extra, will I have any access issues?? Also, please let me know the limitations of that free space, in terms of storage, accessibility and cost.
Using an App Service you can store temporal files inside %TMP% folder which is maped to %SYSTEMDRIVE%\local\Temp with no extra cost.
Depending on your App Service Plan you will have from 10GB to 140GB free space to store files. Beware, your files will dissapear if App service is restarted.
Refer to this link:
https://github.com/projectkudu/kudu/wiki/Understanding-the-Azure-App-Service-file-system
Related
SUMMARY: Can I use an Azure Storage Plan as additional storage that is available to an Azure Web App?
DETAILS: We have a very simple Web App using a low amount of CPU and large amount of storage (all it does is list and allow you to download files). I need at least 50GB of storage for video and audio files for this app and to get that amount of storage on a plan pushes me up to unreasonable CPU and other resources that I don't need with the consequences of a much higher price.
My plan (hope) is that I could create a Web App and remap the root directory of the Web App to a Storage Plan that is 50GB. Two challenges that I have spent the last day researching but at this point, I have not been able to find an answer.
I have created the Web App. I see that /wwwroot is on the D: drive of the Web App. I also created a storage plan and have created a sharable area of 50GB on the storage plan.
So... Can anyone give me some insight into the following:
How do I tell the Web App to use the 50GB of storage which is now available to it?
How can I map a drive letter to the area in the storage plan?
How do I tell the Web App that it should use the 50GB area on the storage plan as the root drive instead of using D:/Webroot
Many Thanks!
Refer this feedback link on a similar request: https://feedback.azure.com/forums/169385-web-apps/suggestions/13536996-the-ability-to-store-iis-logs-in-azure-file-storag - it has been denied.
By default, on Azure WebApps, all files are stored in the file system with the application, including the media files. You may wish to know about the main types of files (https://github.com/projectkudu/kudu/wiki/Azure-runtime-environment) that are dealt on Azure WebApp (Persisted files, Temporary files and Machine level read-only files)
Refer to the article (https://github.com/projectkudu/kudu/wiki/File-structure-on-azure) File structure on azure to know the sets of files & dirs on Azure WebApp, and check the directories which include the possible increasing files, such as LogFiles, site/repository, site/deployments(for deployment slots) and your directory for uploading files.
To verify you can go to your scm site's debug console
(https://{sitename}.scm.azurewebsites.net/DebugConsole) and query for the free space on d:\local. The “Disk Space” depends on the App Service plan you're using. It's 1 GB for Free, 10 GB for Basic, 50 GB for Standard and 250 GB for Premium, refer the document for more details on these limits: https://azure.microsoft.com/en-us/pricing/details/app-service/.
If your requirement fits you may use ASE - Azure App Service Environment is an Azure App Service feature that provides a fully isolated and dedicated environment for securely running App Service apps at high scale.
https://learn.microsoft.com/en-us/azure/app-service/environment/app-service-web-how-to-create-a-web-app-in-an-ase
According to Microsoft Azure Support:
"... since the Product Group confirmed that it is not possible to mount
additional storage to the web app, you can integrate Azure storage
with the Azure SDK or rest API. But you can't mount the drive and use
it as storage.
Another option that you have would be to replicate the scenario on a
Virtual Machine where you can choose its capabilities (Number of
cores, RAM, and Storage Memory)."
So there you have it. It appears that WebApps are pretty fixed configurations which means that when you scale up a Web App, you get more CPU resources AND more Disk storage. It's a packaged deal most likely designed for ease of deployment. Nothing, it appears, you can do about that.
The best alternative, it seems, is to spin up a VM with your chosen OS and then add additional disk storage as needed. It's a "do-it-yourself" approach but the best solution that seems to be available.
In an Azure Web App I need to efficiently query the MaxMind GeoIP2 City Database (due to the volume of queries and the latency requirements we cannot use the MaxMind's rest API).
I'm wondering what's the best approach for storing the db (binary MMDB format, accessed via the official .NET api) so that it's easy to update with minimal downtime (we are going to subscribe Monthly updates) and still cost effective as to what regards Azure storage and transactions.
Apparently block blobs are the way to go, but I'm not sure about the monthly updates and the fact that the GeoIP2 api load in memory the whole db (I do not know if this would be a problem for the Web App, if I need a web worker to keep it up or I need something else), but actually I do not know yet how large the file is.
What's the most cost effective solution that preserve low latency over a huge volume?
According to the API docs you must have the database available in a file system (the API doesn't know anything about Azure storage and related REST API). So, regardless where you permanently store it, you'll need to have it on a disk somewhere.
I have no idea how large the database footprint is, but Web Apps, Cloud Services (web/worker roles) and Virtual Machines (whether Linux or Windows) all have local disks. And you have read/write access to these disks. So, you'd need to copy the database binary file (or csv) to local disk from somewhere. At this point, when you initialize the SDK, you'd create a DatabaseReader and point it to your locally-downloaded copy of the database file.
You mentioned storing the database in blob storage. There's nothing stopping you from doing so and simply downloading a copy to local disk. And there's nothing stopping you from storing multiple versions in multiple blobs. Note: You may also take advantage of Azure File storage (an SMB share). Which you choose is up to you.
As far as most cost effective solution: You'll need to do the pricing workup yourself to see what's most effective. You'd also need to evaluate how much RAM is available for the given size VM/role instance/Web App you choose. You mentioned Web Apps in your question: Web App instances scale from 0.5GB to 14GB, depending on the tier you choose (again, you'll need to evaluate this).
I am trying to a deploy an app which has a frontend app and a backend worker. The worker runs a CPU intensive process. Now my requirements are to run the web app in a Azure A0 instance while the CPU intensive process runs in a D2 instance. Now both the instance must be able to share the files. I have read at places where they spoke of SBS.
I tried creating the linux VMs in same cloud services but couldnt figure out how to ssh into them separately since they use the same cloud service url. i followed this http://azure.microsoft.com/en-us/documentation/articles/cloud-services-connect-virtual-machine/
to create the 2nd vm.
Can anyone suggest me as how to achieve this setup? Also if possible how do i check if the disks are available to both the instances?
Azure docs aren't as helpful as aws. :(
If the two VMs just want to share files and you don't want to go to the extra effort of coding for blob storage then consider Azure Files which exposes an SMB share against a blob storage back end. This allows you to do standard file IO operations instead of custom blob storage code. See http://blogs.msdn.com/b/windowsazurestorage/archive/2014/05/12/introducing-microsoft-azure-file-service.aspx which shows how to create the file share in Windows and Linux VMs.
[Probably easier to give an answer here]
BlobStorage is a universal storage container that can effectively act as the common drive you are looking for. Access to the blob storage container is made over HTTP / HTTPS either through a BlobStorage Client or over REST, where you will have functions to upload, download, list objects, etc.
For Python, you'll hopefully find this article sufficient although I've no experience with Python on Azure to comment, or if choosing REST and http requests - that should work fine.
HTH
We plan to migrate the existing website to Windows azure, and i have been told that we need to store files to blob storage.
My questions is:
If we want to use blob storage, that means i need to re-write the file storage function(we use file system for now), call blob service api to store files, that's very strange for me just because we want to use windows azure, how about in the future we want to use Amazon EC2 or other cloud platform, they might have there own way to store file, then may be i need to re-write the file storage function again, in my opinion , the implementation of a project should not depends on the cloud platform(or cloud server)! Can any body correct me, thanks!
I won't address the commentary about whether an app should have a dependency on a particular cloud environment (or specific ways to deal with that particular issue), as that's subjective and it's a nice debate to have somewhere else. What I will address is the actual storage in Azure, as your info is a bit out-of-date.
One reason to use blob storage directly (and possibly the reason you were told to use blob storage) is that it provides access from multiple instances of your app. Also, blob storage provides 500TB of storage per storage account, and it's triple-replicated within the deployed region (and optionally geo-replicated). With attached storage (either with local disk or blob-backed Azure Disk), the access is specific to a particular instance of your app. Shifting from file system access to blob storage access does require app modification.
If you choose not to modify your app's file I/O operations, then you can also consider the new Azure File Service, which provides SMB access to storage (backed by blob storage). Using File Service, your app would (hopefully) not need to be modified, although you might need to change your root path.
More information on Azure File Service may be found here.
Why does it seem strange? You need to store your files somewhere and the cloud is a good a place as any IF it suits your needs. The obvious advantages are redundancy and geo replication, sharing files across multiple projects and servers, The list goes on. It's difficult to advise on whether it would be a good idea or not without hearing some specifics.
You could use windows azure storage with amazon in the future if you wanted to (you'd just need to set up the access for it), obviously with slighter longer delay. Then again that slight performance drop may be significant and you may end up re-writing it.
Most importantly, swapping over from one cloud provider to another is not trivial depending on just how much you use it or how much data you've got in it, so I would strongly suggest looking at the advantages / disadvantages of each platform closely before putting your lot in with either one and then fully learn that platform.
Personally, I went for Azure cloud services + storage etc even though it was slightly more expensive at the time, because i'm a Microsoft Person (not that I didn't do my research). It was annoying in the early days when key features were missing, but it's really matured now and I like the pace that it's improving.
It's cheap to test, why not try both and see which one suits you? A small price to pay when you have big decisions to make.
Disclaimer: I don't know the current state of Amazon web services.
Nice question. We are in the middle of a migration of an old PHP/MySQL/LocalShare to WebRole/SQLAzure/AzureStorage ERP application. We faced the same problem and decision. Let me write some thoughts about the issue :
It is a good option to just be able to switch the storage provider but is it reasonable? You can always build the abstraction but do you plan how to do the actual change of storage provider - migration/sync while in production? What kind of argument will exactly drive the transition to another storage provider? How much users and data do you have? Do you plan to shard-rebalance the storage in the future? How reliable must be this system during this storage provider switch? Do you want to totally move the data when you want to switch or you just want to shard it so that you start using this different provider? Does the cost development of these (reliable) storage layers and the cost of development of reliable transitions (or bi-directional syncs) outweighs the money difference between any two storage providers?
Just switching storage mechanism from Azure Blob to Amazon will incur heavy latency penalty if your other services are on Azure - When you create Storage and Services on Azure you set affinity groups by region so that you minimize the network latency.
These are only a few of the questions to answer before doing all the weightlifting. We have abstracted the file repository (blob) because we planned to move from local NFS to Blob transparently and gradually and it answers our needs.
I would like to create a Metro application that allows a group of people to interact. One person would create data and serve as the owner, and multiple others would be invited in and be allow to modify that data. I heard from Build talks that each Metro application will get per-user Azure storage, but will it be possible to share that data between multiple users? Does anyone have a link they could share where I could research this?
I think that you are confusing SkyDrive with Azure Blob Storage.
SkyDrive
Personal to a Live ID
Not really meant as a base for collaborative work
Azure Blob Storage
You can have public files that anyone can view and update
You can have a lease on file that only allows certain people to edit it
Since you own the Azure account you also control the content
You can learn the basics here
If you want to share private app data between users, the best way to do so would be via a shared server of some sort. You should have a server (running on Azure, Amazon EC2, or anything really) that exposes a REST-ful web service which each application connects to. The shared state then lives on that server.
This is better than trying to use skydrive or some file-based system for storing shared data. With a file on skydrive and multiple users trying to access it, you would run into concurrency issues when more than 1 person tries to write to it.
You don't get Azure with Metro.
With Live you get a free SkyDrive that is a personal cloud storage. Like 10 GB. Can share files but it is via sending an email link. It is not file storage that would readily support a server type application to manage that sharing.
Azure is a cloud platform for file and data sharing. Azure is not free but storage cost is only $0.125 / GB per month. 10 GB = $1.25 / month. Using SkyDrive as shared storage you are giving up a lot of developer and hosting tools that come with Azure to save $1.25 / month.
It looks like there is a more formal definition of this with the updated help now available. They were referring to roaming application data. I found the following links that provide guidance:
http://msdn.microsoft.com/en-us/library/windows/apps/hh464917.aspx
http://msdn.microsoft.com/en-us/library/windows/apps/hh465094.aspx
The general is that a small amount of temporary application data is provided on a per-app, per-user basis. The actual size you get is not detailed, but the guidance is pretty clear - app settings only, no large data sets, and don't use it for instant synchronization. Given this guidance, my plan is not a good one and will change.