I have whitelisted all the IPs under my functionapp to access a KeyVault with Managed Identity. I know that the MI works, because when I turn off the IP filtering, I can access the secrets. Using the IP filtering has worked in my other environments. I checked the logs of the KV to check the last IP addresses that had tried to access my KV, and saw a new IP address I hadn't seen before. Adding that IP address fixed the problem. However, this IP doesn't show up under my Functionapp properties. So is filtering IPs based on the function app properties not a viable solution anymore?
Azure Functions are subject to Outbound IP changes depending on the consumption plan you use - see official documentation - for scaling purposes.
You might have to whitelist the whole Outbound IP range (which is not the most secure way of doing of course.., attackers can come from Azure as well!) or use VNet NAT gateway mechanism or App Service Environments.
Related
In my current situation, we have several Azure Function apps that talk to an Azure SQL database (using Entity Framework, should it matter), using functions that trigger on an Azure ServiceBus trigger.
In the last weeks we have been improving security by using a VNET and subnets to only allow access to the Azure Database server by only the Function apps that need to use it. However, I have run into a strange issue. It seems that now the database server is set to disallow traffic apart from the defined subnets in which my fuction apps run, the function apps start giving intermittent SQLExceptions when connecting to the database with the message that some specific IP is not allowed by the database Firewall rules. The weird thing is that this error is not consistent. I would expect either for the function app to be declined at the firewall for it's IP, or be allowed all the time, but not randomly as is currently the case.
Question
Is there something that I am missing with my setup? Or, how do I force my function apps in subnets to use their internal IP that is allowed by the database server firewall rules, and not some other outbound IP address that is not in the database firewall rules?
Alternatively: What can possibly explain that access to the database sometimes succeed (indicating a proper internal IP used by the funcation app), and sometimes fail on the firewall (with an unknown IP address), seemingly at random.
Hopefully somebody can help!
Detailed Description of situation
The Function App has a function that is triggered by a Service Bus trigger. The Function App is running with a P1v2 Premium service plan with Vnet integration on.
The app is running inside a Subnet in our environment with a defined IP adress range with a /26 subnet mask. If I check the environment variables of the funciton app in Kudu I can see the PRIVATE_IP_ADDRESS setting is in the subnet range. The database firewall is set up to disallow all traffic, apart from the subnet of my function app as follows:
Triggering the function app which will write some stuff into the database works sometimes (Indicating that the access to the database is working at least when the IP address of the function is the correct one) however, there are also a lot of SQLExceptions with the following error:
Cannot open server 'database-server-name' requested by the login. Client with IP address 'XXX.XXX.XXX.XXX' is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may take up to five minutes for this change to take effect.
The IP address mentioned in the error is NOT the internal IP defined on the Vnet or subnet IP range. It is also not even one of the IP's that show up in the Possible Outbound Ip addresses of the function app,
These happen more or less randomly. Sometimes they don't appear for x triggers, sometimes it fails for hundres of calls in a row.
Enabling the Datbaase server setting "Allow Azure services and resources to access this database server" stops the error from occuring, but of course that is counter to configuring the firewall to allow certain subnets
What I have tried
Setting the configuration settings WEBSITE_VNET_ROUTE_ALL and WEBSITE_DNS_SERVER on the function app to force traffic to use the VNET route as metioned here: Unable to connect Azure Function with Azure SQL using private endpoint
Adding Storage as service endpoint to the subnet as mentioned here: Unable to connect to Azure Function App after integrating into VNET
Restarting and stopping/starting the Function app
Changing the Function app Scaling to force a change in outbound IP addresses as mentioned here: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#outbound-ip-address-changes
Your Azure Function will only use a private outbound IP address picked from your VNet when you have VNet integration enabled. The VNet integration option is only available in the Function Premium Plan.
Additionally, the environment variables WEBSITE_VNET_ROUTE_ALL and WEBSITE_DNS_SERVER you already mentioned should be set as well as mentioned here (assuming your SQL server is in the same VNet).
Of course a new day and fresh perspective brings the probable answer.
While I havent changed anything about the setup I mentioned in my question, I was reading this post again: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#outbound-ip-address-changes
It mentioned switching the service plan temporary force an IP change. I initially misinterpreted this as switching between the DEV and premium plans, which cannot happen because the DEV plans don't support VNET integration. So I switched plans between the P1v2 and P1v3 plans. This however did not work.
What was meant here was to switch beween S1 and P1v2 plans. The Standard plans are hidden behind this link:
It also shows this small message net to the Apply button.
After switching between the S1 and P1v2 plans for a moment and resubmitting deadlettered servicebus messages to the function app, everything started working again.
I assume the IP switch was necessary, but switching between P-plans is not what triggers it. It has to be between standard and premium.
I have one azure function which call clients api.
Client want a azure function ip address to whitelist them in there api network so that my azure function can access there api.
I tried to give them all Outbound Ip Address I can get from azure function properties but i can see everytime im getting all new ip address.
How to get all possible ip address which i can give client to whitelist pls advice.
For production im using App Service Plan (S1: 1)
You can do this. There are different possibilities depending on the pricing tier. From not possible at all, to possible but may change, to possible and guaranteed that IP address will not change.
The bad news is that, possible and guaranteed that IP address will not change, requires using the isolated tier.
See: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#dedicated-ip-addresses
I suggest you look into using service tags: https://learn.microsoft.com/en-us/azure/virtual-network/service-tags-overview#available-service-tags
/MMT
I have an Azure Scheduler Job which calls a REST endpoint over https.
The REST API is locked down by IP address so it's currently (and understandably) failing. So I need to add the Scheduler Job IP address to the API firewall.
The problem I have is I can't seem to find the IP address anywhere in the Azure Scheduler web interface.
Am I missing something or is it not possible to get the IP address in same way you can for other Azure resources (web apps, worker roles etc)?
Unfortunately Azure Scheduler does not reserve static IP, so the IP will change from deployment to deployment as we ship new changes. You may consider Logic Apps, which also supports calling REST endpoint over HTTPs, and does have static IP. See this link for more details: https://learn.microsoft.com/en-us/azure/app-service-logic/app-service-logic-limits-and-config?toc=%2fazure%2flogic-apps%2ftoc.json#configuration
Very late to the show but incase anyone else stumbles across this again, the IP addresses from the Azure Resource Explorer should get you what you need. See here for details on getting the IPs. You can also add the IPs for the relevant Azure Data Centers to your allow list if necessary.
I'm using Azure App Services that calls an external API that uses white-listing of IP addresses for defense-in-depth protection.
I'm aware I can find my Outbound IP addresses of my App Services under the WebApp -> Settings -> Properties -> Outbound IP addresses (showing a list of 4 comma separated IP addresses) which can be supplied to the external API whitelist. I understand Microsoft publishes a regularly updated list of Azure datacenter's IP addresses for outbound traffic that I can whitelist: https://www.microsoft.com/en-us/download/details.aspx?id=41653
The issue is the external API can only handle a number of IP addresses and not the full list of Azure datacenter IP's. Would it be safe to just provide the 4 comma separated IP addresses? Is there clear Microsoft documentation on how often or when the IP address can dynamically change?
I have tried to look for the answer and found two external sites that suggested it only changes when moving Azure regions [Ref 2] or if you scale up/down (but scale out/in is apparently fine) [Ref 1]. Is this correct information?
Is the Azure App Services Environment the only other viable alternative in my situation?
Late to the party on this one but just to flesh this out:
Unless you use an Application Service Environment (ASE) you cannot guarantee that the inbound or outbound IPs will change, however there some differences between inbound and outbound IP addresses.
When does an Inbound IP change?
Inbound IP will change when:
Delete an app and recreate it in a different resource group.
Delete the last app in a resource group and region combination and recreate it.
Delete an existing SSL binding, such as during certificate renewal
However, this can be countered by getting a static IP for inbound connections configure an IP-based SSL binding - you can even use a self-signed cert if you're not using any SSL functionailty.
Outbound IP addresses
This is where you sit and it's the space that is a little more volitile, this is from the docs directly:
The set of outbound IP addresses for your app changes when you scale your app between the lower tiers (Basic, Standard, and Premium) and the Premium V2 tier.
So in a nutshell if you will never scale up or down then you should be fine.
Edit: Re-reading the above and with a little test it looks like it's saying that the IPs will not change unless you ramp right up to the Premium V2 tier from a lower tier (or vice versa). A cursory test backs this up so:
Running under F1 free tier and scaling up to P1v2 gave me the following IP addresses:
Scaling back down to any of the lower tiers reverted the IP addresses to the same as the F1 tier.
Possible remediation
A very solution specific example but in-case it helps - where I've solved this before is monitoring the outbound IP addresses for the webapp, if it notices a change then it could send those IP addresses to a whitelist for update via an API call - but this could also just cause a high priority ticket to be raised.
I should note that we could do this because it wouldn't be the end of the world for a little bit of outage and we weren't expecting to scale up and down very often.
Otherwise, as you've noted, an Application Service Environment is your only belt and braces option.
References
When do inbound IPs change?
When do outbound IPs change?
Get a static inbound IP addresses
In addition to above resources about changes the other thing is that additional IP's might be added to accommodate additional capacity for outbound n/w calls.
Specific comment to look out for is "With the recent upgrades the Websites service has a stable set of outbound IP addresses allocated to each of its scale units. We continue to monitor network utilization and we might add (though never remove) additional IP addresses." by Stefan_MS
https://social.msdn.microsoft.com/Forums/azure/en-US/fd53afb7-14b8-41ca-bfcb-305bdeea413e/maintenance-notice-upcoming-changes-to-increase-capacity-for-outbound-network-calls?forum=windowsazurewebsitespreview
I have a WebJob on an Azure Website that needs to connect to a VM Endpoint to make REST calls.
My Endpoint is configured to deny all except my company's IP range. Now what rule would I need to add or url should I use so my webjob can connect to the endpoint?
I have tried the following without success:
Allow my website virtual IP address in the ACL
Connect to the endpoint using the internal IP instead of the DNS without changing
the ACL
Connect to the endpoint using the public virtual IP instead
of the DNS without changing the ACL
This works but is not what I am looking for:
Remove the current ACL and allow all
Keep the ACL but add a /16 rule with my website IP
Thank you for your help, and let me know if you need precision!
I need the same thing but it seems as though is not possible right now. Looking at this answer on a related question:
Azure Web Sites do not have dedicated outbound IP addresses for each
deployment. This precludes you from using ACLs or Virtual Networks to
connect to your Redis / Solr virtual machines.
So even though you can have a (reasonably) fixed incoming IP address on Azure Websites, the outgoing address is highly unpredictable and as far as I can see, the only exclusion that you could make was to restrict it to the entire range of IP addresses for that data centre which is far from ideal.
A solution moving forward will be to connect your Azure Website and the VM on the same Virtual Network. As of my writing this it is still in Preview so it still is not ready for production use just yet.
Here is more information on it: http://azure.microsoft.com/blog/2014/09/15/azure-websites-virtual-network-integration/