I am hosting our app on Azure and the Insights logs shows that all requests coming into the application are logging 0.0.0.0 as the client_IP.
For example, this kusto query:
requests
| where client_IP != '0.0.0.0'
...returns no results.
As the app uses the AspNetCoreRateLimit package to rate limit user requests for one particular endpoint, the net effect appears to be that all users are rate limited, rather than on a per user basis.
My understanding is that if Insights can't get the client IP address, then our code (and any NuGets it uses) will not be able to, either.
Is it a fair assumption that it is Azure which is preventing both Insights and our code from acquiring the client IP address? If that is a possibility, how could I confirm that?
Is it a fair assumption that it is Azure which is preventing both Insights and our code from acquiring the client IP address? If that is a possibility, how could I confirm that?
App Insights doesn't store the ip by design:
By default IP addresses are temporarily collected, but not stored in Application Insights.
...
This behavior is by design to help avoid unnecessary collection of personal data. Whenever possible, we recommend avoiding the collection of personal data.
...
To enable IP collection and storage, the DisableIpMasking property of the Application Insights component must be set to true. This property can be set through Azure Resource Manager templates or by calling the REST API.
So your code should still be able to collect ip addresses.
Related
I'm following the guide here to setup Application Insight telemetry on a frontend web form. I wish to use the snippet-based setup. I notice, however, that it requires me to embed the connection url in the html page. Is that a security issue?
There would be nothing to stop a malicious user from using browser dev tools to grab that url and then send any API calls to that url. Should I be concerned about this? If so, what is the recommended approach for securing this connection url.
... what is the recommended approach for securing this connection url.
There is none. For now you have to accept it is visible somehow. See also this open issue regarding the topic
Should I be concerned about this?
Not so much. The instrumentation key cannot be used to read any telemetry. However, it could be used to send bogus telemetry to your application insigths resource. This could lead to higher costs depending on the amount of data ingested and it could clutter your logs, possible masking possibly relevant telemetry.
Unless the application is hosted on a vnet integrated resource you cannot restrict access to application insights resource. If it is, then you can set application insights to deny queries or ingestion from external sources in the network isolation setting.
https://learn.microsoft.com/en-us/azure/azure-monitor/logs/private-link-security
So even if someone gets the url they cannot access.
Hello i was wondering if someone could answer this question for me:
Is there a way for me to view logs of incoming requests and their IP Addresses.
Here is the scenario:
We have multiple host machines that every 5 minutes submit data into our .NET Web Application via a simple MVC controller. One of the machine's configuration is pointing to a correct domain, but the wrong controller name.
So every 5 minutes this generates a 404 error on Azure Portal. I would like to identify which machine is configured wrongly by identifying the IP Address of the incoming request that is causing this issue. We are running .NET web application with 12 VM Instances and I have checked the ApplicationInsights/Logs section, but can not find any references to the IP Address.
Any way to track it via Azure Portal site ?
Thanks in advance.
As long as the Application Insights .NET or .NET Core SDK is installed and configured on the server to log requests, you can create/update an Application Insights resource on Azure that shows the client's IP address.
You may currently be seeing the IP 0.0.0.0 in logs, which is the default:
This behavior is by design to help avoid unnecessary collection of personal data. Whenever possible, we recommend avoiding the collection of personal data.
From the same article you can see the setting to configure as follows (shortened for brevity).
{
// ...
"properties": {
// ...
"DisableIpMasking": true
}
}
After this setting is configured, logs will begin showing with the client ip addresses when queried in Application Insights.
We are looking at using webhooks from various vendors outside our network. They would publish the event to us. We would be the webhook listener/receiver, not pushing the events. We have done proof of concept of creating an Azure Function to receive the event. From the research we have done most have the security of passing a sha1/sha256/sha512 hash for us to verify they are who we want to receive the events. This all worked as expected with the POC Azure Function.
From a enterprise network security standpoint is there anything else available? The process above puts the security in the function. I'm sure our Network Security group would not want us to have 10 functions, one for each vendor to worry about the security. I've read about whitelisting of IP's that would be sending the events but most of our vendors are Cloud based so I'm not sure how readily that would be available. Maybe one function to validate all events that come in then let pass through? Would that be an acceptable solution? Azure API Gateway or API Management able to address somehow? Any other network type of product that handles webhook security specifically?
Any insight or link to information most appreciated.
Thanks.
Wow, that's really really so open conversation.
You can use Azure Front Door with the Web Application Firewall attached to it. So any SQL injection, DDoS or similar attacks can be prevented by AFD and WAF.
However, I would say the securest way is to put IP restriction as well. So you need to force your vendor to get their IP address. That can be multiple maybe hundreds. But that doesn't matter. You can implement CIDR IP address format so you can cover all network. And you can easily set these IP address restriction during the CI/CD pipeline with Azure PowerShell script.
You can also useAPI Management in front of Azure Functions and you can create access restriction policies. You can either restrict IP based or JWT based. APIM might be a little bit pricey tho.
https://learn.microsoft.com/en-us/azure/api-management/api-management-access-restriction-policies
You can also create advanced policies with APIM
https://learn.microsoft.com/en-us/azure/api-management/api-management-advanced-policies
Apart from that, the AFD & WAF and IP restriction are on the network layer. But you can also implement token-based authentication on your code side.
https://learn.microsoft.com/en-us/azure/app-service/overview-authentication-authorization
You can either you Azure Active Directory, IdentityServer or JWT for this.
Good luck!
Is there any way to secure an HTTP endpoint of an Azure Logic App ?
For example if I'm using my Logic App's HTTP request endpoint to be triggered as a webhook from a payment gateway, I'd want to restrict only certain static IP Addresses to access it and enable HTTPS.
I didn't find any firewall options like those present in Azure SQL for IP base restrictions.
You can put the manual trigger endpoint behind Azure API Management, using its "Restrict caller IPs" policy should help you accomplish what you need.
See https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-securing-a-logic-app
Restrict incoming IP addresses
In addition to the Shared Access Signature, you may wish to restrict calling a logic app only from specific clients. For example, if you manage your endpoint through Azure API Management, you can restrict the logic app to only accept the request when the request comes from the API Management instance IP address.
This setting can be configured within the logic app settings:
In the Azure portal, open the logic app you want to add IP address restrictions
Click the Access control configuration menu item under Settings
Specify the list of IP address ranges to be accepted by the trigger
A valid IP range takes the format 192.168.1.1/255. If you want the logic app to only fire as a nested logic app, select the Only other logic apps option. This option writes an empty array to the resource, meaning only calls from the service itself (parent logic apps) fire successfully.
I could be wrong but all App Services (Web Apps, Logic Apps, API Apps) are, by default, publically accessible and, by default, do not enable IP Filtering via Azure resource configuration (meaning, a setting on the Logic App). Options I can think of enabling this would be:
If you have access to a Web.config, use an element to restrict traffic to only a specific set of address (see MSDN)
Consider putting the Logic App behind an API Management resource and enforce a IP restriction policy (see Azure API Management Documentation); I'm guessing this may not apply for Logic Apps but including it anyway
Scale to a Premium App Service Plan, deploy Logic App to an App Service Environment which gives you the ability to specify whether there is a Public IP (VIP) used for load balancing requests or internal load balancer (which would be used for cross-premise connectivity between your LAN and Azure); you can also deploy a virtual Web Application Firewall (WAF) which would allow you to place explicit IP filters as well
Not knowing your requirements, I'd lean more towards option 3 simply because there is documentation supporting that scenario (at least for Web, Mobile and API Apps -- Logic Apps are not mentioned). Otherwise, if you're not able to modify the web.config to include the filters and cannot put an API Management instance in front of the logic app, I'm not sure what other options you may have.
Hope this helps and if you find a solution, I'd be interested in learning it as well.
For the last couple of weeks, we have been seeing an increasing amount of entries in the web logs of our Azure website whose originating IP address (in the c-ip column of the log) appears to be in the range 100.90.X.X. It has now reached more than half of all the traffic being logged, and is interfering with our ability to perform analytics and threat detection.
According to the Wikipedia entry on reserved IP addresses, this block is part of one "Used for communications between a service provider and its subscribers when using a Carrier-grade NAT, as specified by RFC 6598", so could this be a problem in Azure?
Looking at the logs, the traffic comes from many different user agents (both normal users and the common legitimate bots) and is requesting a broad range of resources, so does not immediately appear suspicious other than the IPs. It looks more like legitimate traffic is being given an incorrect (internal) IP.
It seems to be only affecting static content (e.g. images and XML files), but not ALL static content.
We are using a single Small Standard instance in Western Europe, with a single web app running on it. We are not using any scaling features. There is a linked SQL database, and the website runs primarily over HTTPs. 95%+ of our traffic comes from UK sources. We have not made any changes to logging, which is handled by Azure.
Is there any way that we can return to seeing the actual IPs here, or is this malicious traffic?
It’s possible to alter the logging, but not on an app. The app diagnostic setting is pretty rudimentary — just a switch “to log or not to log?”
What you’ll be interested in is this comparison between apps (it was called “sites” then), roles (available through Cloud Service)1 and virtual machines. The article mentions that there is more control over logging in the roles environment, which I would assume means that you can set up custom logs. This article details how to set up logging for the headers you choose in IIS. Now, you can fiddle with your IIS in a virtual machine, but there is a chance a cut-down version of this would work in a web role, for example. This article discusses how to enable diagnostics logging in your cloud service hosted application.
Moving to cloud service from app environment is not trivial, since you have many more things you must set up. Possibly you’re looking at changing your solution’s structure, maybe altering the architecture of your app. So I wouldn’t consider doing it just so I could see a client’s IP.
The simplest thing you can do is try attaching analytics. There used to be a solution straight from Azure, but I can't find it in the portal. Google analytics is my go-to solution for traffic analysis. It may get you the information you want.
It’s really annoying how Microsoft rebrands an azure service every few months.