We have time triggered Azure Function deployed on portal to perform some iterative task at specific time. Our azure function uses the database deployed on Azure VM via connection string provided at AppSettings. The function throws following error on running:
MySql.Data: Authentication to host 'xxx' for user 'xxx using method 'mysql_native_password' failed with message:
Client with IP address 'x.x.x.x' is not allowed to connect to this MySQL server. MySql.Data: Client with IP address 'x.x.x.x' is not allowed to connect to this MySQL server*
When we white-list the IP mentioned in error message, the function runs successfully. But since the azure function has no determined work-station or PC of same IP that handles the execution, whenever the function runs from new IP, it throws the error back. Therefore, we require a mechanism by which we can white-list all IPs of PCs that will be running our function app OR some better mechanism to authenticate and allow azure function to access our database hosted on Azure VM.
What we tried?
We white-listed the Virtual IP address of the function app. But it doesn't work every time.
We tried to white-list the IP ranges obtained from Microsoft Datacenter of the region in which our function app is deployed. But this method also didn't work.
Azure application can't access database on Azure VM?
Thus, is there any way by which azure function can access our database deployed on Virtual machine securely?
I have opened up the issue on github but no reply yet from there.
Finally, after thorough researching, found the solution.
One need to white-list all Outbound IPs of Function App in Virtual Machine where DB is deployed. The outbound IP address can be found from resources.azure.com. On searching your resource (in my case Function app's name), there will be a long json output from where you have to pick possibleOutboundIpAddresses parameter as shown in the image. Whitelist all Ips and then your azure function app can access your database deployed on virtual machine.
Also, I searched whether these IPs tend to change on regular basis or not. While I didn't find any official word from it, but from various internet sources I came to know that even if the IPs gets planned to be changed ever, everyone will get enough notifications before that to prevent any problems.
You need to set up a Virtual Network (VNet) where both your App Service Plan hosting the Azure Function and the VM participate.
Then from Azure Functions, view All Properties > Networking and you should see the virtual network to connect.
This method doesn't require you to whitelist IP addresses for your VM and secures your VM by allowing only internal network traffice.
Note that your Azure Function must be set up on an App Service Plan, rather than on a Consumption plan.
Related
In my current situation, we have several Azure Function apps that talk to an Azure SQL database (using Entity Framework, should it matter), using functions that trigger on an Azure ServiceBus trigger.
In the last weeks we have been improving security by using a VNET and subnets to only allow access to the Azure Database server by only the Function apps that need to use it. However, I have run into a strange issue. It seems that now the database server is set to disallow traffic apart from the defined subnets in which my fuction apps run, the function apps start giving intermittent SQLExceptions when connecting to the database with the message that some specific IP is not allowed by the database Firewall rules. The weird thing is that this error is not consistent. I would expect either for the function app to be declined at the firewall for it's IP, or be allowed all the time, but not randomly as is currently the case.
Question
Is there something that I am missing with my setup? Or, how do I force my function apps in subnets to use their internal IP that is allowed by the database server firewall rules, and not some other outbound IP address that is not in the database firewall rules?
Alternatively: What can possibly explain that access to the database sometimes succeed (indicating a proper internal IP used by the funcation app), and sometimes fail on the firewall (with an unknown IP address), seemingly at random.
Hopefully somebody can help!
Detailed Description of situation
The Function App has a function that is triggered by a Service Bus trigger. The Function App is running with a P1v2 Premium service plan with Vnet integration on.
The app is running inside a Subnet in our environment with a defined IP adress range with a /26 subnet mask. If I check the environment variables of the funciton app in Kudu I can see the PRIVATE_IP_ADDRESS setting is in the subnet range. The database firewall is set up to disallow all traffic, apart from the subnet of my function app as follows:
Triggering the function app which will write some stuff into the database works sometimes (Indicating that the access to the database is working at least when the IP address of the function is the correct one) however, there are also a lot of SQLExceptions with the following error:
Cannot open server 'database-server-name' requested by the login. Client with IP address 'XXX.XXX.XXX.XXX' is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may take up to five minutes for this change to take effect.
The IP address mentioned in the error is NOT the internal IP defined on the Vnet or subnet IP range. It is also not even one of the IP's that show up in the Possible Outbound Ip addresses of the function app,
These happen more or less randomly. Sometimes they don't appear for x triggers, sometimes it fails for hundres of calls in a row.
Enabling the Datbaase server setting "Allow Azure services and resources to access this database server" stops the error from occuring, but of course that is counter to configuring the firewall to allow certain subnets
What I have tried
Setting the configuration settings WEBSITE_VNET_ROUTE_ALL and WEBSITE_DNS_SERVER on the function app to force traffic to use the VNET route as metioned here: Unable to connect Azure Function with Azure SQL using private endpoint
Adding Storage as service endpoint to the subnet as mentioned here: Unable to connect to Azure Function App after integrating into VNET
Restarting and stopping/starting the Function app
Changing the Function app Scaling to force a change in outbound IP addresses as mentioned here: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#outbound-ip-address-changes
Your Azure Function will only use a private outbound IP address picked from your VNet when you have VNet integration enabled. The VNet integration option is only available in the Function Premium Plan.
Additionally, the environment variables WEBSITE_VNET_ROUTE_ALL and WEBSITE_DNS_SERVER you already mentioned should be set as well as mentioned here (assuming your SQL server is in the same VNet).
Of course a new day and fresh perspective brings the probable answer.
While I havent changed anything about the setup I mentioned in my question, I was reading this post again: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#outbound-ip-address-changes
It mentioned switching the service plan temporary force an IP change. I initially misinterpreted this as switching between the DEV and premium plans, which cannot happen because the DEV plans don't support VNET integration. So I switched plans between the P1v2 and P1v3 plans. This however did not work.
What was meant here was to switch beween S1 and P1v2 plans. The Standard plans are hidden behind this link:
It also shows this small message net to the Apply button.
After switching between the S1 and P1v2 plans for a moment and resubmitting deadlettered servicebus messages to the function app, everything started working again.
I assume the IP switch was necessary, but switching between P-plans is not what triggers it. It has to be between standard and premium.
I have been using azure timer based function in my project.
Since I started using the function, I was getting error 403 related to access forbidden from Azure storage account.
I tried adding AzureWebJobsStorage Key in function's configuration
I tried adding outbound IPs of Allowed ip ranges of azure storage accounts
I tried checking the option of allowing azure managed services to the storage account.
However, I'm still getting the error in that particular timer function of my function app. The other functions run properly.
Although, if I allow all the networks in storage account firewall and VN settings, my function runs proper.
Both the function and storage account are in same region.
I want to enable it somehow that I do not need to choose the allow all networks. What can be done?
In case anyone else is searching for this... I had a similar issue. I had a function app that I had created a private endpoint and regional VNet integration back with the VNet interacting with a Storage Account that also had a private endpoint with the same VNet. The Storage Account's network/firewall settings only allowed connections from the VNet (no external traffic allowed). Both the storage account and function app reside in the same region.
Attempt at fix #1 (not ideal):
I added code to determine what IP the function app was running from. That led me to add all of the IP's in the portal under function app --> Properties --> Additional Outbound IP Addresses. This is exposed by Terraform if using that.
Attempt at fix #2 (better):
The resolution is to ensure you have the proper function app settings set.
See: Microsoft documentation
Setting
Suggested value
Description
WEBSITE_CONTENTOVERVNET
1
Create this app setting. A value of 1 enables your function app to scale when your storage account is restricted to a virtual network.
WEBSITE_DNS_SERVER
168.63.129.16
Create this app setting. When your app integrates with a virtual network, it will use the same DNS server as the virtual network. Your function app needs this setting so it can work with Azure DNS private zones. It's required when you use private endpoints. This setting and WEBSITE_VNET_ROUTE_ALL will send all outbound calls from your app into your virtual network.
WEBSITE_VNET_ROUTE_ALL
1
Create this app setting. When your app integrates with a virtual network, it uses the same DNS server as the virtual network. Your function app needs this setting so it can work with Azure DNS private zones. It's required when you use private endpoints. This setting and WEBSITE_DNS_SERVER will send all outbound calls from your app into your virtual network.
Note: The 168.63.129.16 is a static value for Azure DNS.
After setting all of these, my function app was able to connect to the storage account through the VNet as expected.
There're already some answers about this issue, you can see here and here.
In short, if the function and storage account are in same region, they communicate in an internal way without going through outboundIpAddresses.
The workaround is that create them in different regions.
I am currently working with a client that requires access to all Azure resource locking down as much as possible and I am having problems with the Storage Account that is utilised by our Azure Functions.
With the Firewalls and Virtual Networks blade in portal set to "All Networks" I am able to deploy to the Function App and it runs without issue.
However once I enable the access restriction by checking "Selected Networks"no matter what virtual network subnets I enter or IP Addresses I can not get the communication to work
I have entered the Outbound IP Addresses of our Consumption based Function App and also check that the additional IP Addresses from the Powershell and all have been added to the whitelist. I have also added all the CIDR IP ranges of the local Azure datacenter but again it does not work.
The problem we have is that once the access restrictions have been put into place we are unable to deploy to the Function App and the app no longer runs. Is this scenario supported and what is the mechanism for tying down access to the Storage Account so that only the Function App can utilise it.
As far as I know, you have two options to restrict access to your storage account from your function app or web app.
Whitelist the outboundIpAddresses and possibleOutboundIpAddresses of the function app in the firewall of the storage account. However, it does not work if the Azure function app and Azure storage located in the same region refer to Sam's answer.
when you hit the storage account from your function, because they are
in the same region as each other, all the traffic goes over the
internal Azure network on internal IP's, not the public IPs listed in
the web app, and so is not allowed over the firewall.
If your resources were in different regions, you could use the network section of function app to allow function app to access resources in a VNet, then enable service endpoint for Microsoft.Storage in this app integration subnet. But you need Azure Functions Premium plan referring to this tutorial: integrate Functions with an Azure virtual network.
Sometimes, the deployment order for networking is important. In this case, you will deploy the followings:
Firstly, you could deploy new VNet integration with an unused subnet. After the VNet Integration is completed and the function app is restarted, you could enable service endpoint for this subnet. In the end, you could add the subnet in the firewall of the storage account.
Note that the new version is in Preview, currently. You could also check these characteristics and get more references from this thread.
While configuring an Azure managed Postgres service, I am trying to configure connecting from local machines through VPN.
I can connect to the DB when white-listing IPs in Connection Security.
I have added the subnet the VPN-gateway is connected to to the VNET Rules – this doesn't seem to make a difference.
I can connect to VMs through the VPN from my local machine.
However to make that work, I added the VMs' (private IP, Azure URL)-pairs to my local machines hosts-file.
I can't find any IP for the DB-service (which seems to make sense for a managed service), so I can't make the same trick.
The error I'm getting, when trying to connect to the DB, is similar to the ones I got before adding hosts mappings.
This all leads me to believe I need some way of having Azure resolve the URL (which might also preempt the need for hosts-mappings in general).
From this article, I tried setting my DNS for 168.63.129.16, but that doesn't work at all (nothing at all is resolved).
Is there a way (and if so, how) to connect from a local machine to an Azure Database for Postgresql service through a VPN gateway?
I don't think there is a way to do this as your desired. You want to map an Azure database logical server private IP to your local hosts file, then access it via VPN gateway.
You only know the public IP for the Azure database server. The public IP addresses of Azure services change periodically. You could find an IP address list by filtering your region. It does not recommend to use such a dynamic IP address. Refer to this blog.
Since Azure database is a fully managed Platform as a Service (PaaS) Database Engine not IaaS like Azure virtual machines, It's public and does not expose the database server private IP address. We only could access the database via Azure database server name over the Internet.
Furthermore, if you want to restrict its access only from a private network with virtual Network service endpoints. However, this works to allow resources like Azure VM in the authorized subnet to access directly your Azure database in a private network, could not guarantee that if you could access the Azure database from your local machine via VPN. This seems no on-premise route to your Azure database.
I have an Azure Cloud Service (Worker Role) that needs to connect to my Azure SQL database and also connect to an external database.
In development the external database was on the public Internet and connectivity was not a problem.
However, the solution now needs to be deployed in a production environment and access to the external database is to be restricted by setting up a Virtual Network.
The Cloud Service, when deployed in the Virtual Network, gets an IP from the subnet, but seems to become inaccessible to the outside world, and is not connecting to the Azure SQL database (I also cannot RDP to it).
This seems to be beyond my level of understanding of networking, but I don't see why it should lose access to its neighbours in the Azure environment.
What am I missing? Do I need to get involved with Endpoints? Is the Virtual Network misconfigured?
Thanks in advance.
Your question is quite vague, in terms that it does describe the whole picture in the best possible way. Let me put my answer based on my understanding about your issue.
First of all - Azure virtual Network is Virtual Network. It is designed to enable secure cross-premisses connectivity with Windows Azure Data Center.
When you deploy a proper PaaS Cloud Service (Worker Role / Web Role) in a Virtual Network, the role instances get IP Addresses allocated from the defined DHCP pool (the VNet Definition).
When you deploy any service in an Azure Virtual Network you have to take care of Name Resolution! Meaning that, if you do not provide a proper DNS Server, your cloud instances will not be able to resolve any address. That includes Azure SQL Database servers. More on Name Resolution can be read here.
Next, but not less important - Azure SQL Database servers are not part of, and, as of March'2013, cannot be added to Azure Virtual Network!
The last statement means that in order for your Worker Role to access Azre SQL Database server, you need to provide a proper DNS server in your Azure Virtual Network.
And lastly, when you deploy a PaaS service into a Virtual Network, in order to access it via Remote Desktop need to:
* Properly enable and configure RDP extension. it will anyway create Input Endpoint. But this is the only way to enable RDP on PaaS right now;
* You could probably enable RDP via PowerShell startup script and access RDP via the VPN tunnel for the Virtual Network - say you configured a Site-to-Site or Point-to-site VPN for your Azure VNet.
Check the building cross-premises Virtual Network guide here.