I use a storage account to host 3 simple Azure Functions, which perform read, write and delete operations on a database. Surprisingly the Used Capacity value under Metrics is constantly increasing (please see screenshot below). Why? Those functions don't write anything to this storage. Since one pays for capacity used - am I going to pay more and more, if it continues to increase like that? I am a "Pay-as-you-go" customer by the way.
Edit#1: If I check the folder size of File Shares it says 2MB (see below). No clue where the values in Metrics are coming from...
Edit#2: Below are the Application Settings. Just the default values + Link to the MongoDB Atlas cluster. Could it be related to AzureWebJobsStorage?
Every Azure Function requires a storage account.
Storage account requirements
When creating a function app in App Service, you must create or link
to a general-purpose Azure Storage account that supports Blob, Queue,
and Table storage. Internally, Functions uses Storage for operations
such as managing triggers and logging function executions.
Note
When using the Consumption hosting plan, your function code and
binding configuration files are stored in Azure File storage in the
main storage account. When you delete the main storage account, this
content is deleted and cannot be recovered.
Related
I have some e-mail attachments being saved to Azure Blob.
I am now trying to write a Azure Functions App that would connect to that blob storage, run some scripts and re-save the file.
However, when selecting a storage account for the function, I couldn't select my blob storage account.
I went on the website and it said this:
When creating a function app, you must create or link to a general-purpose Azure Storage account that supports Blob, Queue, and Table storage. Some storage accounts don't support queues and tables. These accounts include blob-only storage accounts and Azure Premium Storage.
I'm wondering, is there any workaround this? and if not, perhaps any other suggestions? I'm becoming a little lost in all the options, and which one to actually choose.
Thanks!
EDIT: Might I add I writing the function Python
I think you are overlooking the fact that you can have multiple storage accounts. In order for an Azure Function to work you need a storage account. That storage account is used to store runtime information of the Azure Function for internal purposes like state management. This storage account is subject to restrictions as you already found out. There is no workaround for that.
However, if the function you are writing needs to access another storage account it is free to do so. You just have to provide details to connect to that specific storage account. In that case you also have a clear seperation between the storage account that is used by the azure function for its internal operations and the storage account your application needs to connect and which you have total control about withouth having to worry that you break things by deleting internal used blobs/tables/queues.
You can have a blob triggered function that gets triggered when changes occur on your specific blob storage. That doesn't need to be the storage account that the azure function internally uses, which is created/selected when creating the azure function.
Here is a sample that shows how to add a blob triggered azure function in Python. MyStorageAccountAppSetting refers to an app setting that holds the connection string to the storage account that you use for storage.
The snippet from the website you are quoting is for storing the function app code itself and any related modules. It does not pertain to what your function can access when the code of your function executes.
When your function executes it will need to use the Azure Blob Storage SDK/modules to connect to your blob storage account and read the email attachments. Here's a quickstart guide for using Azure Storage with Python: Quickstart with Azure Storage Blobs SDK for Python
General-purpose v2 storage accounts support the latest Azure Storage features and incorporate all of the functionality of general-purpose v1 and Blob storage accounts here
There are more integration options with GPv2 accounts including Azure Function Triggers. See: Azure Blob storage bindings for Azure Functions
Further refer: Types of storage accounts
If Blob, based on your need, you can choose an access tier based on the frequency of access for the data (e-mail attachments)Access tiers for Azure Blob Storage - hot, cool, and archive. If General purpose storage account, its standard performance tier.
Can I use azure function app storage account for other purposes like storing files in blob storage? If yes will it according to Microsoft guidelines and also will it cause any performance issue? Specially when size of blob storage get increased to GBs?
I am near to production, so please come up with any suggestions, best practices, solutions as soon as possible.
Can I use azure function app storage account for other purposes like
storing files in blob storage?
Yes, you can.
If yes will it according to Microsoft guidelines and also will it
cause any performance issue? Specially when size of blob storage get
increased to GBs?
It depends. Each Azure Storage account has some pre-defined throughput limits. As long as you stay within those limits, you should be fine.
Having said this, ideally you should have a separate storage account. Considering creation of storage account doesn't cost you anything till the time you do some transactions in it, you may be better off creating a separate account to store data required by your application.
I've searched the web and contacted technical support yet no one seems to be able to give me a straight answer on whether items in Azure Blob Storage are backed up or not.
What I mean is, do I need to create a twin storage account as a "backup" and program copies of all content from one storage to another, or are the contents of a client's Blob Storage automatically redundantly backed up by Microsoft?
I know with AWS, storage is redundantly backed up via onsite drives as well as across other nodes in the cluster.
do I need to create a twin storage account as a "backup" and program
copies of all content from one storage to another, or are the contents
of a client's Blob Storage automatically redundantly backed up by
Microsoft?
Yes, you will need to do backup manually. Azure Storage does not back up the contents of your storage account automatically.
Azure Storage does provide geo-redundant replication (provided you configure the redundancy level for your storage account as GRS or RA-GRS) but that is not back up. Once you delete content from your primary account (location, it will automatically be removed from secondary account (geo-redundant location).
Both AWS (EBS) and Azure(Blob Storage) options provides durability by replicating the data across different data centers. This is for the high availability and durability of the data to provide the guarantee by the cloud provider.
In order to ensure that your data is durable, Azure Storage has the
ability to keep (and manage) multiple copies of your data. This is
called replication, or sometimes redundancy. When you set up your
storage account, you select a replication type. In most cases, this
setting can be modified after the storage account is set up.
For more details refer the replication section in documentation.
If you need to capture changes to the storage and allow restore to previous versions (e.g In situations like data corruption or application feature requirements like restore points, backups), you need to take a SnapShot manually. This is common for both AWS and Azure.
For more details on creating a Snapshot of Blob in Azure refer the documentation.
I have two webapps on separate plans each has multiple instances of large size (P3) and it says I get 250GB of storage on P3.
I also have azure storage to store photos.
I want to know, how is Azure storage related to the webapp plans... meaning, what if I reduce the webapp to S3 where it's only 50GB, how will that affect storage?
Also, do I get 50GB for each instances or for the entire plan?
Thank you
Azure App Service plans represent the collection of physical resources used to host your apps.
App Service plans define:
Region (West US, East US, etc.)
Scale count (one, two, three instances, etc.)
Instance size (Small, Medium, Large) SKU (Free, Shared, Basic, Standard, Premium)
If you scale down your App Service plan to S3, yes you will get 50GB storage.
This storage includes/stores all of the resources and deployment files, logs etc.
You can only store data/files up to the available Storage according to the pricing tier that you choose. To increase the storage you can scale up your pricing tier.
Also, note that increase/decreasing instances is nothing but increase/decrease the number of VM instances that run your app. You get only one Storage account for all the instances not individual Storage.
Before scaling based on instance count, you should consider that scaling is affected by Pricing tier in addition to instance count. Different pricing tiers can have different numbers cores and memory, and so they will have better performance for the same number of instances (which is Scale up or Scale down).
For more details, you may refer the Azure App Service plans in-depth overview and App Service pricing.
Hope this answers your questions.
App Service storage is completely different than Azure Storage (blobs/tables/queues).
App Service Storage
For a given tier size (e.g. S1), you get a specific amount of durable storage, shared across all instances of your web app. So, if you get 50GB for a given tier, and you have 5 instances, all 5 instances share that 50GB storage (and all see and use the same directories/files).
All files in your Web App's allocated storage are manipulated via standard file I/O operations.
App Service Storage is durable (meaning there's no single disk to fail, and you won't lose any info stored), until you delete your web app. Then all resources (including the allocated storage, in this example 50GB) are removed.
Azure Storage
Azure Storage, such as blobs, is managed completely independently of web apps. You must access each item in storage (a table, a queue, a blob / container) via REST or a language-specific SDK. A single blob can be as large as 4.75TB, far larger than the largest App Service plan's storage limit.
Unlike App Service / Web App storage, you cannot work with a blob with normal file I/O operations. As I mentioned already, you need to work via API/SDK. If, say, you needed to perform an operation on a blob (e.g. opening/manipulating a zip file), you would typically copy that blob down to working storage in your Web App instance (or VM, etc.), manipulate the file there, then upload the updated file back to blob storage.
Azure Storage is durable (triple-replicated within a region), but has additional options for replication to secondary regions, and even further, allowing for read-only access to the secondary region. Azure Storage also supports additional features such as snapshots, public access to private blobs (through Shared Access Policies & Signatures), and global caching via CDN. Azure Storage will remain in place even if you delete your Web App.
Note: There is also Azure File Storage (backed by Azure Storage), which provides a 5TB file share, and acts similarly to the file share provided by Web Apps. However: You cannot mount an Azure File Storage share with a Web App (though you can access it via API/SDK).
I am trying to implement azure storage metrics code in my role but i am checking if there is easy way to get azure storage metric data about my files usage. my code is stable and i do not want to change code again.
Actually if you already have Windows Azure role running, then you don't need to make any changes to your code and you still can get Windows Azure Blob storage Metrics data.
I have written a blog about it last time as Collecting Windows Azure Storage REST API level metrics data without a single line of programming, just by using tools
Please try above and see if this works for you.
Storage analytics is disabled by default, so any operations against your storage up til now has not been logged for analysis.
You may choose to enable analytics at any time, for both logging (detailed access information for every single object) and metrics (hourly rollups). Further, you may choose which specific storage service to track (blobs, tables, queues) and which operations to track (read, write, delete). Once analytics are enabled, you may access the resulting analytics data from any app (as long as you have the storage account name + key).
Persistent Systems just published a blog post on enabling storage analytics for Java apps. The same principles may be applied to a .net app (and the sdk's are very similar).
Additionally, Full Scale 180 published a sample app encapsulating storage analytics (based on REST API, as it was written before SDK v1.6 came out).