My Azure Function will require resources to run properly (ranging from 5 to 30 Mb). I'm thinking of putting them on the Blob Storage and loading from the Storage each time function runs, but since it will be quite often (a couple times per day) I'm thinking of simply putting them as resources in the function project.
The question is, how much space can I allocate for the function itself? How big (in terms of disk space) a Azure Function can be?
It depends on which host plan your Function app is on.
For dedicate App service plan(Pricing tier Basic, Standard and so on), we can see the size of disk storage allocated for this plan.
Just like #Slava Utesinov has mentioned, we can also check it from Platform features -> App Service plan -> File system storage as well.
For Consumption plan, File system storage shows 0KB usage among 1GB available. Because Function apps on Consumption plan are stored in Storage File Share, which is specified by WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE Application settings on Azure portal, check doc. As for space, max size of a File share is 5TB, see doc.
Related
i have isolated i2 service plan the app service runs windows .net web apps .
the local\temp keeps filling up.(I have a process to delete files but it sometimes can't keep up)
should I just attach a azure disk drive "E"?
Or is there a way to increase temp size?
It seems that you can attach a azure disk drive, but according to this post, the disk usage amount seems to be based on SKU. In other words, the higher the SKU level, the larger the disk size that can be used, and the size cannot be specified manually. If possible, you can try to scale to I3.
Unfortunately, you cannot increase the temp folder size inside a App Service. If you wish to increase temp space you can do it by scaling up the app service plan to higher tiers.
The “Disk Space” depends on the App Service plan you're using. It's 1 GB for Free, 10 GB for Basic, 50 GB for Standard and 250 GB for Premium, refer the document for more details on these limits: https://azure.microsoft.com/en-us/pricing/details/app-service/.
App Service also limits the amount of temporary local storage available to each app to prevent individual apps from consuming excessive amounts of local file storage.
Here is the information where you can know about the files that are dealt on Azure WebApp (Persisted files, Temporary files and Machine level read-only files) and there limits as well
Refer the below blog which will help you in understanding the disk space availability
Azure App Service - What is taking my disk space for my Azure Web App
We are using an Azure Storage account to store some files that shall be downloaded by our app on the users demand.
Even though there should be no write operations (at least none I could think of), we are exceeding the included write operations just some days into the billing period (see image).
Regarding the price it's still within limits, but I'd still like to know whether this is normal and how I can analyze the matter. Besides the storage we are using
Functions and
App Service (mobile app)
but none of them should cause that many write operations. I've checked the logs of our functions and none of those that access the queues or the blobs have been active lately. There are are some functions that run every now and then, but only once every few minutes and those do not access the storage at all.
I don't know if this is related, but there is a kind of periodic ingress on our blob storage (see the image below). The period is roundabout 1 h, but there is a baseline of 100 kB per 5 min.
Analyzing the metrics of the storage account further, I found that there is a constant stream of 1.90k transactions per hour for blobs and 1.3k transactions per hour for queues, which seems quite exceptional to me. (Please not that the resolution of this graph is 1 h, while the former has a resolution of 5 minutes)
Is there anything else I can do to analyze where the write operations come from? It kind of bothers me, since it does not seem as if it's supposed to be like that.
I 've had the exact same problem; after enabling Storage Analytics and inspecting the $logs container I found many log entries that indicate that upon every request towards my Azure Functions, these write operations occur against the following container object:
https://[function-name].blob.core.windows.net:443/azure-webjobs-hosts/locks/linkfunctions/host?comp=lease
In my Azure Functions code I do not explicitly write in any of container or file as such but I have the following two Application Settings configured:
AzureWebJobsDashboard
AzureWebJobsStorage
So I filled a support ticker in Azure with the following questions:
Are the write operation triggered by these application settings? I
believe so but could you please confirm.
Will the write operation stop if I delete these application settings?
Could you please describe, in high level, in what context these operations occur (e.g. logging? resource locking, other?)
and I got the following answers from Azure support team, respectively:
Yes, you are right. According to the logs information, we can see “https://[function-name].blob.core.windows.net:443/azure-webjobs-hosts/locks/linkfunctions/host?comp=lease”.
This azure-webjobs-hosts folder is associated with function app and it’s created by default as well as creating function app. When function app is running, it will record these logs in the storage account which is configured with AzureWebJobsStorage.
You can’t stop the write operations because these operations record necessary logs to storage account used by Azure Functions runtime. Please do not remove application setting AzureWebJobsStorage. The Azure Functions runtime uses this storage account connection string for all functions except for HTTP triggered functions. Removing this Application Settings will cause your function app unable to start. By the way, you can remove AzureWebJobsDashboard and it will stop Monitor rather than the operation above.
These operations is to record runtime logs of function app. These operations will occur when our backend allocates instance for running the function app.
Best place to find information about storage usage is to make use of Storage Analytics especially Storage Analytics Logging.
There's a special blob container called $logs in the same storage account which will have detailed information about every operation performed against that storage account. You can view the blobs in that blob container and find the information.
If you don't see this blob container in your storage account, then you will need to enable storage analytics on your storage account. However considering you can see the metrics data, my guess is that it is already enabled.
Regarding the source of these write operations, have you enabled diagnostics for your Functions and App Service? These write diagnostics logs to blob storage. Also, storage analytics is also writing to the same account and that will also cause these write operations.
For my case, I have a Azure App Insight which took 10K transactions on its storage per mintues for functions and app services, even thought there are only few https requests among them. I'm not sure what triggers them, but once I removed app insights, everything becomes normal.
We have created a blob triggered azure function to process files placed in blob storage. Load on this blob will not be consistent.
For example, for some hours there will be hundreds or even thousands of file will be placed in that blob every minutes. On the other hand there will be some hours during which we will not find even a single file.
Some files will be processed in very few seconds and some can take more than 10-15 minutes.
So my question is: In this type of unpredictable scenario which plan will be better for us? App service plan or Consumption plan?
If you can optimize your code so that the maximum processing time is 10 minutes, so Consumption Plan is your best option from cost perspective considering your fluctuating workload
As #Peter Bons, mentioned in the comments, this is your best reference
Edit
According to the above document,
if your function app is on the Consumption plan, there can be up to a
10-minute delay in processing new blobs if a function app has gone
idle.
If you want to avoid that delay and still use consumption plan to benefit from its cost effectiveness, you can replace Blob Trigger with Event Grid Trigger but it is not fully supported by Azure Functions nowadays
SUMMARY: Can I use an Azure Storage Plan as additional storage that is available to an Azure Web App?
DETAILS: We have a very simple Web App using a low amount of CPU and large amount of storage (all it does is list and allow you to download files). I need at least 50GB of storage for video and audio files for this app and to get that amount of storage on a plan pushes me up to unreasonable CPU and other resources that I don't need with the consequences of a much higher price.
My plan (hope) is that I could create a Web App and remap the root directory of the Web App to a Storage Plan that is 50GB. Two challenges that I have spent the last day researching but at this point, I have not been able to find an answer.
I have created the Web App. I see that /wwwroot is on the D: drive of the Web App. I also created a storage plan and have created a sharable area of 50GB on the storage plan.
So... Can anyone give me some insight into the following:
How do I tell the Web App to use the 50GB of storage which is now available to it?
How can I map a drive letter to the area in the storage plan?
How do I tell the Web App that it should use the 50GB area on the storage plan as the root drive instead of using D:/Webroot
Many Thanks!
Refer this feedback link on a similar request: https://feedback.azure.com/forums/169385-web-apps/suggestions/13536996-the-ability-to-store-iis-logs-in-azure-file-storag - it has been denied.
By default, on Azure WebApps, all files are stored in the file system with the application, including the media files. You may wish to know about the main types of files (https://github.com/projectkudu/kudu/wiki/Azure-runtime-environment) that are dealt on Azure WebApp (Persisted files, Temporary files and Machine level read-only files)
Refer to the article (https://github.com/projectkudu/kudu/wiki/File-structure-on-azure) File structure on azure to know the sets of files & dirs on Azure WebApp, and check the directories which include the possible increasing files, such as LogFiles, site/repository, site/deployments(for deployment slots) and your directory for uploading files.
To verify you can go to your scm site's debug console
(https://{sitename}.scm.azurewebsites.net/DebugConsole) and query for the free space on d:\local. The “Disk Space” depends on the App Service plan you're using. It's 1 GB for Free, 10 GB for Basic, 50 GB for Standard and 250 GB for Premium, refer the document for more details on these limits: https://azure.microsoft.com/en-us/pricing/details/app-service/.
If your requirement fits you may use ASE - Azure App Service Environment is an Azure App Service feature that provides a fully isolated and dedicated environment for securely running App Service apps at high scale.
https://learn.microsoft.com/en-us/azure/app-service/environment/app-service-web-how-to-create-a-web-app-in-an-ase
According to Microsoft Azure Support:
"... since the Product Group confirmed that it is not possible to mount
additional storage to the web app, you can integrate Azure storage
with the Azure SDK or rest API. But you can't mount the drive and use
it as storage.
Another option that you have would be to replicate the scenario on a
Virtual Machine where you can choose its capabilities (Number of
cores, RAM, and Storage Memory)."
So there you have it. It appears that WebApps are pretty fixed configurations which means that when you scale up a Web App, you get more CPU resources AND more Disk storage. It's a packaged deal most likely designed for ease of deployment. Nothing, it appears, you can do about that.
The best alternative, it seems, is to spin up a VM with your chosen OS and then add additional disk storage as needed. It's a "do-it-yourself" approach but the best solution that seems to be available.
I would like to create containers for each virtual machine, then deploy them.
As I known deploy is not free.
Is it free to create container in Azure?
For Azure Storage, you only pay for consumption, measured in GB/month (e.g. less than a nickel/month per GB). Containers are essentially free, since they take up virtually zero data.
The pricing of Azure storage is based on consumption or how much you use. More information of the pricing can be found on the Azure website.
Pricing also varies by region and by type of storage you provision.