I have an Elasticsearch cluster that I need to access using an Azure Functions App. The current way to access it is via an IP whitelist. I've created my consumption plan Azure Functions and noticed that the IP address kept changing and after further research found that I'd need to whitelist the whole Azure Cloud which is not a secure option in our case. So I've upgraded Azure Functions to Premium plan and thus the VNet Integration appeared in the app settings allowing me to create a Virtual Network and a static Outbound IP Address.
Now it is working.
But since I'm using the Premium plan and the other mentioned services, the cost went up.
I've experimented with Azure Data Factory in the past and there was an option to install a local Integration Runtime so when ADF was accessing Elasticsearch via the Local Integration Runtime, it worked. Is there something similar I can use with Azure Functions? Like a Data Gateway I can install on a local server that is already whitelisted on Elasticsearch and all I'd need to do is connect from Azure Functions through it?
I guess the ideal scenario I'm looking for is to be able to use a Consumption Plan on Azure Functions.
Kind Regards,
Andrei
Just to close this one, in the end, I've used a Standard AppSevicePlan (S1) (costs $74|£51/month or about $2.5|£1.7/day). I've had to create a static Outbound IP, a NAT Gateway (which costs about another $30|£23/month or $1|£.77/day) and a VNET.
The tutorial can be found here: https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-use-nat-gateway
Another option that I've considered to reduce cost was to use a Hybrid Connection which would install a sort of gateway to a local server to which Azure Functions would connect first before reaching the ES endpoint...that would eliminate the need for the Outbound IP, NAT Gateway and VNET (so $30 less per month) but would still require at least an S1...so no option on consumption-based, and I don't know if there is any charge associated with this. However, we didn't go with this option as we want to eliminate any local dependencies (e.g. if the server where the gateway is running would shut down).
To sum up, use an S1, follow the tutorial mentioned above and should cost about $104|£74/month
Related
In my current situation, we have several Azure Function apps that talk to an Azure SQL database (using Entity Framework, should it matter), using functions that trigger on an Azure ServiceBus trigger.
In the last weeks we have been improving security by using a VNET and subnets to only allow access to the Azure Database server by only the Function apps that need to use it. However, I have run into a strange issue. It seems that now the database server is set to disallow traffic apart from the defined subnets in which my fuction apps run, the function apps start giving intermittent SQLExceptions when connecting to the database with the message that some specific IP is not allowed by the database Firewall rules. The weird thing is that this error is not consistent. I would expect either for the function app to be declined at the firewall for it's IP, or be allowed all the time, but not randomly as is currently the case.
Question
Is there something that I am missing with my setup? Or, how do I force my function apps in subnets to use their internal IP that is allowed by the database server firewall rules, and not some other outbound IP address that is not in the database firewall rules?
Alternatively: What can possibly explain that access to the database sometimes succeed (indicating a proper internal IP used by the funcation app), and sometimes fail on the firewall (with an unknown IP address), seemingly at random.
Hopefully somebody can help!
Detailed Description of situation
The Function App has a function that is triggered by a Service Bus trigger. The Function App is running with a P1v2 Premium service plan with Vnet integration on.
The app is running inside a Subnet in our environment with a defined IP adress range with a /26 subnet mask. If I check the environment variables of the funciton app in Kudu I can see the PRIVATE_IP_ADDRESS setting is in the subnet range. The database firewall is set up to disallow all traffic, apart from the subnet of my function app as follows:
Triggering the function app which will write some stuff into the database works sometimes (Indicating that the access to the database is working at least when the IP address of the function is the correct one) however, there are also a lot of SQLExceptions with the following error:
Cannot open server 'database-server-name' requested by the login. Client with IP address 'XXX.XXX.XXX.XXX' is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may take up to five minutes for this change to take effect.
The IP address mentioned in the error is NOT the internal IP defined on the Vnet or subnet IP range. It is also not even one of the IP's that show up in the Possible Outbound Ip addresses of the function app,
These happen more or less randomly. Sometimes they don't appear for x triggers, sometimes it fails for hundres of calls in a row.
Enabling the Datbaase server setting "Allow Azure services and resources to access this database server" stops the error from occuring, but of course that is counter to configuring the firewall to allow certain subnets
What I have tried
Setting the configuration settings WEBSITE_VNET_ROUTE_ALL and WEBSITE_DNS_SERVER on the function app to force traffic to use the VNET route as metioned here: Unable to connect Azure Function with Azure SQL using private endpoint
Adding Storage as service endpoint to the subnet as mentioned here: Unable to connect to Azure Function App after integrating into VNET
Restarting and stopping/starting the Function app
Changing the Function app Scaling to force a change in outbound IP addresses as mentioned here: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#outbound-ip-address-changes
Your Azure Function will only use a private outbound IP address picked from your VNet when you have VNet integration enabled. The VNet integration option is only available in the Function Premium Plan.
Additionally, the environment variables WEBSITE_VNET_ROUTE_ALL and WEBSITE_DNS_SERVER you already mentioned should be set as well as mentioned here (assuming your SQL server is in the same VNet).
Of course a new day and fresh perspective brings the probable answer.
While I havent changed anything about the setup I mentioned in my question, I was reading this post again: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#outbound-ip-address-changes
It mentioned switching the service plan temporary force an IP change. I initially misinterpreted this as switching between the DEV and premium plans, which cannot happen because the DEV plans don't support VNET integration. So I switched plans between the P1v2 and P1v3 plans. This however did not work.
What was meant here was to switch beween S1 and P1v2 plans. The Standard plans are hidden behind this link:
It also shows this small message net to the Apply button.
After switching between the S1 and P1v2 plans for a moment and resubmitting deadlettered servicebus messages to the function app, everything started working again.
I assume the IP switch was necessary, but switching between P-plans is not what triggers it. It has to be between standard and premium.
I'm handling my VPN Gateway via Azure Automation script, totally executed on Azure and nothing on premises. The project is to activate the VPN gateway by launching an Azure PS Runbook (which creates and configure the object I need for my VPN) when I need it, and then remove the VPN Gateway with another Azure PS Runbook, to limit the cost of the VPN Gateway provisioning (I need it only in business hours/days, not 24/7).
So far so good, I can create my VPN Gatway when I want it, and I can download the client configuration to connect to my VNet.
The problem is that every time I destroy a VPN Gateway, the previous installed configuration on client it's not valid anymore for the next VPN Gateway: it can't connect anymore. I've tried by using a Standard static public IP address but it doesn't solve the problem. I see that the address used as server name on client changes every time (it's a string like 'azuregateway-9a430069....843cc.vpn.azure.com'). This is annoying since it means that I need to update the client configuration each time the VPN is destroyed, which means to log into Azure portal to download the configuration (very uncomfortable if I have more than one or two client).
Do Azure have a way to avoid this? I'm open to every solution, the best is to avoid completely the configuration update on client.
Maybe something like obtain the new VPN Gateway parameter and display it as result of the PS Runbook, or the worst case to obtain the *.zip file with the new configuration and store it somewhere I want (like a OneDrive).
I've contacted the Microsoft Tech Support and they told me that I can't do as I described in my question: once the Virtual Network Gateway resource is disposed, the link for that istance is deleted and the next allocation won't use the same link, because of datacenter allocation policies.
I'm running an azure function which gets data from an API and stores it in a blob. Everything worked fine and stopped working out of nowhere. We then got in contact with our provider and they told us they made some changes in their API. After we made the necessary changes in our code,started getting an IP denied error from their part. I then searched and found the possible outbound IP addresses for the Azure Function. They whitelisted the whole list and still
They aren't getting any requests from those IP's,
We are not able to access that data for the same reason our IP is denied.
We've been running the code in a local machine and it works completely fine, but this is just a temporary fix and we want to keep everything in the cloud.
I've been stuck with this for about 3 weeks. I've looked into different solutions and I found about Azure Logic Apps and Azure Service Fabric.
Is there something missing in my Azure Function that isn't allowing me to make requests to the API? Am I using the wrong outbound IP? Also, if I use any of the other two services, will I encounter with the same problem? I did some research on them and I think they both also use multiple outbound IP addresses, so I'm worried I'll get the same problem.
Using NAT gateway you can specify a static IP address for outbound traffic, your function app need to be attached on a subnet which is not available for consumption plan.
Here is where you should be getting the Azure IP address ranges from. Azure Functions originate from the App Service Plan ranges. Note that this is updated weekly, things change, but not too often. Your provider will need to open all the relevant ranges and keep up to date with any changes. If your solution is not mission critical with a high SLA, then having your service provider open the relevant ranges and deal with failures, updating the ranges on an ad-hoc basis should be fine.
Secondly, if you have a good relationship with the provider, ask them to check the firewall, they will be able to give you an indication of the IP's getting blocked by checking the firewall logs. This will help you find the right range.
The only guaranteed way to solve this in a mission critical solution is to run your Azure Functions from an dedicated app service plan with a dedicated IP address. This is an expensive option but will be the most robust.
Additional helpful information here on how App Services works with IPs can be found here.
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.
We have a client who wants to connect their premises to Azure. Their main hindrance at this point is determining the best way to connect to Azure given their current connectivity configuration. They have two redundant ISP connections going to the head office for internet access. They want to be able to configure a VPN connection to Azure that would operate in a similar way i.e. if ISP A went down it would seamlessly use ISP B and vice versa. The normal multi-site VPN configuration does not fit this since there is one local network behind which means the network behind separate VPNs over each ISP would have overlapping IP address ranges which is not supported. Is such a configuration possible? (See diagram below)
Either that or is there a way to abstract the two ISP connections onto one VPN connection to Azure.
They’re currently considering using a Cisco ASA device to help with this. I’m not familiar with the features of this device so I cannot verify if it will solve their issue. I know there is also a Cisco ASAv appliance in the Azure marketplace don't know if that could also be a part of a possible solution if they went with such a device.
required vpn configuration
The Site-to-Site VPN capability in Azure does not allow for automatic failover between ISPs.
What you could do are the following
- Have automation task created that would re-create the local network and gateway connection upon failover. Manual and would take some RTO to get it up and running
- Use the Cisco CSRs to create a DMVPN mesh. You should be able to achieve the configuration you want using that option. You would use UDRs in Azure to ensure proper routing
I havent done it in Azure, but here is what you do in AWS (And I am sure there would be parallel in Azure)
Configure a "detached VGW" (virtual Private gateway) in aws. Use DMVPN cloud to connect CSRs to multi-site on-prem.
Also, for failover between ISPs you could have a look at DNS load balancing via a parallel to AWS's Route 53 in Azure.
Reference thread :
https://serverfault.com/questions/872700/vpc-transit-difference-between-detached-vgw-and-direct-ipsec-connection-csr100