Azure function connection to Azure Blob storage behind Vnet issue - azure

We are currently migrating to a new Azure Subscription and are having issues executing Azure Functions that worked as expected in our old Azure Subscription. The man difference between our old Subscription and our new Subscription is that we have set up a Virtual Network with Subnets and have deployed our Resources behind the Subnets.
We have also had to migrate from an Azure App Service in the old Subscription to a Azure App Environment in the new Subscription.
Our Azure environment consist of:
App Service Environment
App Service Plan I1
The Azure App Environment and Storage Containers are on the same Virtual Network but different Sub Nets. The Function is using a Managed Identity which has Owner Role on Storage Account.
The code listed below worked just fine in our old environment which did not contain the Virtual Network, but fails in our new environment.
Any guidance would be greatly appreciated.
The Azure function which connects to Azure Storage works when run locally from Visual Studio 2019, but fails when run from Azure portal.
Code Snippet below:
This section works just fine:
string storageConnectionString = XXXXConn.ConnectionETLFileContainer();//Get Storage connection string
var myDirectory = "XXXX/Uploads"; ///XXXX-etl-file-ingest/ABSS/Uploads/ CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageConnectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();// Create a CloudBlobClient object for credentialed access to Azure Blob. CloudBlobContainer blobContainer = blobClient.GetContainerReference("XXXX-etl-blobfile-ingest");// Get a reference to the Blob Container we created previously. CloudBlobDirectory blobDirectory = blobContainer.GetDirectoryReference(myDirectory);// Get a reference to the Blob Directory.
var blobs = blobDirectory.ListBlobs(useFlatBlobListing: true); //set useFlatBlobListing as true
This statement fails: Failure occurs when trying to iterate through the Blob files and get specific file info.
foreach (var myblob in blobs)

In the azure portal open storage account blade under that go to configuration blade , you will be able to see the list of networks for which your storage account has allowed access to.Once you have the allowed network list kindly check if the function app is on one of those networks if not then you need to get the network on which your function app is hosted added to the list.

Update 2:
The simplest explanation/cause that I found is when an App Service or Function App has the setting WEBSITE_VNET_ROUTE_ALL set to 1, all traffic to public endpoints is blocked. So if your Storage Account has no private endpoint configured, requests to it will fail.
Docs: "To block traffic to public addresses, you must have the application setting WEBSITE_VNET_ROUTE_ALL set to 1."
https://learn.microsoft.com/en-us/azure/app-service/web-sites-integrate-with-vnet#network-security-groups
Update 1:
My answer below was only a workaround for my problem. Turns out I did not link the Private DNS Zone (this is created for you when you create a new Private Endpoint) to my VNET.
To do this, go to your Private DNS Zone in the Azure Portal and click on Virtual network links in the left menu bar. There add a new link to the VNET your Function is integrated in.
This may not have been relevant for the OP, but hopefully it will help others.
Original answer:
In my case this was solved by enabling the Microsoft.Storage Service Endpoint on the App Service's subnet (dedicated subnet).

Related

Access denied to storage account from Azure Data Factory

My goal is to run an exe file stored in a private Azure Blob container.
The exe is simple : it creates a text file, write the current datetime in it, and then push it to the private Azure Blob container.
This has to be sent from Azure Data Factory. To do this, here is my environment :
Azure Data Factory running with the simple pipeline :
https://i.stack.imgur.com/txQ9r.png
Private storage account with the following configuration :
https://i.stack.imgur.com/SJrGX.png
A linked service connected to the storage account :
https://i.stack.imgur.com/8xW5l.png
A private managed virtual network approved :
https://i.stack.imgur.com/G2DH3.png
A linked service connected to an Azure Batch :
https://i.stack.imgur.com/Yaq6C.png
A batch account linked to the right storage account
A pool running on this batch account
Two things that I need to add in context :
When I set the storage account to public, it works and I find the text file in my blob storage. So the process works well, but there is a security issue somewhere I can't find.
All the resources (ADF, Blob storage, Batch account) used have a role has contributor/owner of the blob with a managed identity.
Here is the error I get when I set the storage account to private :
{
"errorCategory":0,
"code":"BlobAccessDenied",
"message":"Access for one of the specified Azure Blob(s) is denied",
"details":[
{
"Name":"BlobSource",
"Value":"https://XXXXXXXXXXXXXXXXX/testv2.exe?sv=2018-03-28&sr=b&sig=XXXXXXXXXXXXXXXXXX&sp=r"
},
{
"Name":"FilePath",
"Value":"D:\\batch\\tasks\\workitems\\XXXXXXXXXXX\\job-1\\XXXXXXXXXXXXXXXXXXXXXXXX\\testv2.exe"
}
]
}
Thank you for your help!
Solution found Azure community support :
Check Subnet information under Network Configuration from the Azure portal > Batch Account > Pool > Properties. Take note and write the information down.
Navigate to the storage account, and select Networking. In the Firewalls and virtual networks setting, select Enable from selected virtual networks and IP addresses for Public network access. Add the Batch pool's subnet in the firewall allowlist.
If the subnet doesn't enable the service endpoint, when you select it, a notification will be displayed as follows:
The following networks don't have service endpoints enabled for 'Microsoft.Storage'. Enabling access will take up to 15 minutes to complete. After starting this operation, it is safe to leave and return later if you don't wish to wait.
Therefore, before you add the subnet, check it in the Batch virtual network to see if the service endpoint for the storage account is enabled.
After you complete the configurations above, the Batch nodes in the pool can access the storage account successfully.

For VM diagnostics agent, we create SA, if the SA is not publicly allowed, do we need to create Private end point for both Blob and Table for this SA?

As per this Microsoft documentation the VM uses both blob and table for storing the diagnostic logs.
As described here that the StorageType can be Table, Blob, or TableAndBlob. And let say we are using TableAndBlob.
The storage account that we are creating for this purpose will only be accessible via private endpoints. So do, I need to create private endpoint for both blob and table with private DNS zones privatelink.blob.core.windows.net and privatelink.table.core.windows.net?
Also, if I choose StorageType as Table, I just need the table(privatelink.table.core.windows.net) private endpoint?
I tried to reproduce the same in my environment public access does not allow in the traffic. we can able to create a private endpoint for both blob and table as shown below:
In your storage account -> networking under Security + networking ->private endpoint connection -> create
Make sure in Firewalls and virtual networks try to enable Allow Azure services on the trusted services list to access this storage account. as below
while creating Private endpoint connection in a target sub resource you can able to create a blob and table as below.
And DNS also automatically created in Private endpoint connection as privatelink.blob.core.windows.net and i have created for both blob and table.
You can check on your DNS configuration under setting once you click on your table or blob.
Additionally, you can connect in azure account through storage explore to get access storage account -> right click connect -> next
You can make use of Azure Private Link for storage Accounts through virtual network

Connecting Blob Storage to a Synapse Workspace with Public Network Workspace Access Disabled

I'm trying to connect blob storage from my RG storage account to the Data tab in my synapse workspace, but I get the following error: "The public network interface on this Workspace is not accessible. To connect to this Workspace, use the Private Endpoint from inside your virtual network or enable public network access for this workspace."
Public network access to my workspace must be disabled for company reasons. I made private endpoint connections on my synapse resource to Dev, Sql, and Sql-On-Demand, but I'm not sure where to go from there.
Thanks!
Go to Azure Synapse -> Manage -> Managed private endpoints -> +New and add private endpoints.
Accessing blob storage : If you already created linked service the follow below image. If not, please created and follow this Ms Doc for creating linked service.
Fastest way to access Azure blob storage
For for information follow this reference by Dennes Torres.

Accessing Azure Storage services from a different subscription

We are looking to deploy our Azure cloud services to multiple subscriptions but want to be able to be able to access the same Storage accounts for storing blobs and tables. Wanted to know if it is possible to access storage accounts from across different subscriptions using just the storage account name and key?
Our data connection takes the form
Trying to use the above and it always try to find end point for given accountname within the current subscription
If i understood your question...
able to access the same Storage accounts
Via Azure Panel (Management Portal) : you can access the storage account only in the subscription.
Via Visual Studio: you can attach storage account outside your current login account in visual studio <-> azure with account name and key (and manage it)
Via Code: You can access storage account (blob, queue, table) from all your apps with storage connection strings (don't put it in code)
If you want, you can restrict blob access with CORS settings. Something like this :
private static void InitializeCors()
{
ServiceProperties blobServiceProperties = blobClient.GetServiceProperties();
//Attiva e Configura CORS
ConfigureCors(blobServiceProperties);
//Setta
blobClient.SetServiceProperties(blobServiceProperties);
}
private static void ConfigureCors(ServiceProperties prop)
{
var cors = new CorsRule();
cors.AllowedOrigins.Add("www.domain1.net, www.domain2.it");
prop.Cors.CorsRules.Add(cors);
}

deploying Azure webrole to the cloud, but dont understand dataconnection string (for queues)

I have written and successfully deployed a test app to the azure cloud, but I am lost now that I have added a queue to the application.
Currently I using a configuration string:
Setting name="DataConnectionString" value="UseDevelopmentStorage=true"
then create/open the queue with the following code:
var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
var queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference("messagequeue");
queue.CreateIfNotExist();
This works fine in local mode, however,
I do not undertsand how to change the DataConnectionString to use the cloud!
I have tried:
Setting name="DataConnectionString" value="DefaultEndpointsProtocol=http;AccountName=*XXXXX*;AccountKey=*YYYYY*"
but this does not work - it wont run locally.
Help is certainly appreciated!
Thanks
You'll need to make sure you've created a hosted azure storage service via the Windows Azure portal. When creating the storage service, you provide the account name and the system will assign two keys. Use these two values in your connection string settings. You can either manually edit the string in the service configuration, or my preferred approach is to set it via the role's property settings. Simply right click on the role in the cloud service project in visual studio, then select properties. You'll be able to access the role's settings via one of the tabs. Use the provided dialog box to modify the connection string by inputing the account name and connection string for your storage service.

Resources