I am hosting Azure static web site using Azure Storage, CDN Profile and End Point. I want to see the outgoing traffic. Currently I can only see that for the storage account but I can't see where I can actually count unique site visitors. Please, help. I am OK if it can only be done through powershell script.
There is no built in feature available to get the user visits count, However there are two ways to achieve that:
By Enabling Application Insights: Once you enable you will be able to view the user device,name,count etc
View Storage Analytics Logs: This is like a backend view, You can view the storage analytics logs to view the operations performed on blobs
Related
There's an awful lot of disjointed documentation on monitoring network/resources in Azure. What I'm looking for is which pieces are needed to get information from VMs, NVA firewalls, azure load balancers, and other network resources and network connectivity into a single pain of glass in Azure. Only concerned about Azure, not on-prem for now.
I've come across azure monitor, log analytics work spaces, event hub, vm extensions, network watcher, insights, etc...but I'm not sure which are required and which are not. One doc leads to the next and I end up with 30 tabs open. I'll also need to be able to push logs to other security devices such as a SIEM.
Does anyone know of a deployment guide that wraps this all up in a more logical fashion? Does anyone have any feedback on which pieces from azure (not 3rd parties) are required at a minimum to accomplish a single pane of glass to view my Azure environment holistically?
General overview of observability in Azure
Likely, the thing you're looking for is Azure Monitor. It's an umbrella term for everything observability related inside Azure.
To store Metrics and Logs you need Log Analytics: it can query data with kusto query language, visualize results, define Alerts on queries.
Alerts is quite a complex beast, as it is spread across the entire cloud. Two types that I use the most:
log-analytics alert (which I mentioned above)
Alerts tab, which is available at every Azure component view. for example, open resource group, and scroll down to Monitoring section
Each component also has a subset of built-in metrics. Likely, you noticed that many azure components on the Overview view display some charts. For example, Azure Storage Account displays Total egress, Total ingress, and other line-charts. When you click on these charts you can customize them. These metrics and charts are free to use.
Microsoft also has all-in-one observability solution for Azure Functions and Web Apps: Application Insights
Dashboards allows to join multiple charts into a single view and share it with others.
If you care about security, Azure proposes Azure Security Center
Deployment/management strategy
I suggest to start with:
Create Log Analytics Workspace, which is the storage for metrics and logs. The azure docs article explains how to design it: how many instances to use, how to rate-limit ingestion (it might be expensive if goes out of control), how to access it and so on.
To get Azure components logs, look for Diagnostic Settings tab at a component page at Azure portal, but not all components has it (sic!). I suggest
sending the most critical data to Log Analytics workspace to store them in a queryable format for 30 days (it's in free tier). This is needed for investigating current issues with your infrastructure
if you might need logs later than 30 days - send them to Storage Account
you mentioned SIEM integration - route required events to Event Hub and then process the stream according to your requirements
So, if you need long-term storage - you need to create Azure Storage Account.
If you need real-time analysis - you need to build a pipeline based on Azure Event Hub.
If you have Azure Functions and Web Apps - add Application Insights. According to my experience, I would suggest starting with a separate instance per each Azure Function resource or Service.
Create Alerts for each component separately. If you do it through UI - open component page at the portal and look for Alerts tab there. If you're automating the process (please do so as soon as possible), do not expect easy trip: I used ARM templates and terraform - in both cases, there are dozens of barely documented features.
Join related components core-metrics into Dashboards and share it with the team. This guide is a good starting point. Note, when you share the dashboard, it's also persisted as an azure resource in the subscription.
I've got a static web site hosted in Azure Blob Storage with Cloudflare as my CDN. It's such a small site (not even 1Mb and only 1 blog post), but I'm getting 1.1-1.2Gb of requests each month for the past 6 months or so with no explanation. Is there a way to find out what is being requested? In Azure, I can only find information about the performance, up-time, etc, but nothing about url's and I need to pay to get this info from Cloudflare (I believe). Has anyone else experience such strange requests?
I suggest you open Diagnostic settings and download Azure Storage Explorer to view the logs.
When you finished settings, u can check logs by tools. You can see request urls and http status info.
The previous data should not be visible, but you can monitor and analyze future requests.
When I did a lookup on those two IP's, they were both registered to Cloudflare, one makes sense given I'm using their service, but to have what appears to be their bot hit my site with this frequency doesn't... Wonder if there's a setting
I can configure the Azure CDN against a single storage account presently. What I'm wondering is in the event of a disaster, where that particular region becomes unavailable (outages etc..). If I need to refresh the cache at that point I don't have any regional fallbacks. What is the correct way of supporting multiple storage accounts with the CDN?
One way that I can see it is the Traffic Manager. Traffic Manager receives the request and sends it to one of the X CDNs configured for X Storage Accounts based on performance. That way if one of the regions become unavailable, Traffic Manager should fallback to another one. This is an expensive solution though, so I'm looking for something where I can get one CDN and X Storage Accounts ideally and the CDN should handle the world-wide performance, along with a fallback region.
Here are the steps to configure AFD:
Create AFD from Portal.
Click on Front Door Designer. You will have 3 sections. First is Frontend which will be already configured. Then Baclkend Pools and Routing rules.
Click on Backend Pools and add a new backend pool. Select Storage as Host type and then pick your Primary Storage blob page and provide priority as 1.
Once that is done configure the Health probes. Then add your second Storage blob page and then provide priority as 2.
Configure Routing rules and make sure you have /* as matching pattern. Also you can enable caching in the rule and you can cache based on the query string. Moreover if have a dynamic page, then you can enable dynamic compression.
Once that is done, try accessing AFD URL and check how it works.
Here is the Public Documentation for your reference: https://learn.microsoft.com/en-us/azure/frontdoor/front-door-routing-methods
You try using Azure FrontDoor. It is a combination of CDN and L7 load balancer. You can try implementing your ask with Azure FrontDoor.
Let me know if you face any difficulties.
We've logs(W3CIISLogs) on Log analytics workspace for websites hosted on VMs. Similarly we have app insights enabled for websites hosted on App service. Now we want to access telemetry data of both type of websites thru single interface, either via app insights or via Log analytics. Just wondering if it's possible and what's the best way.
With Azure Monitor you can now query not only across multiple Log Analytics workspaces, but also data from a specific Application Insights app in the same resource group, another resource group, or another subscription. This provides you with a system-wide view of your data. You can only perform these types of queries in Log Analytics.
Querying across Log Analytics workspaces and from Application Insights - reference another workspace in your query, use the workspace identifier and for an app from Application Insights, use the app identifier.
Cross-resource query limits:
The number of Application Insights resources that you can include in
a single query is limited to 100.
Cross-resource query is not supported in View Designer. You can Author a query in Log
Analytics and pin it to Azure dashboard and visualize a log search.
Cross-resource query in log alerts is supported in the new
scheduledQueryRules API. By default, Azure Monitor uses the legacy
Log Analytics Alert API for creating new log alert rules from Azure
portal, unless you switch from legacy Log Alerts API. After the
switch, the new API becomes the default for new alert rules in Azure
portal and it lets you create cross-resource query log alerts rules.
You can create cross-resource query log alert rules without making
the switch by using the ARM template for scheduledQueryRules API –
but this alert rule is manageable though scheduledQueryRules API and
not from Azure portal.
Documentation Reference - Cross-Resource Log queries in Azure Monitor
Hope the above information helps.
I have an Azure Storage account where I have blobs stored in containers.
I would like to limit the access to this storage account to specific Azure resources and prevent internet connections.
I currently have access limited to IPs from our office locations. This allows us to support the process and use Azure Storage Explorer.
I tried adding the Outgoing IP Addresses from the Logic App but that did not allow access.
Then in the Logic App designer, I get the following Error.
I would like to additionally allow access from an Azure Logic app that would work with data stored there.
Is the IP you allowed known in the list of Logic Apps IPs? If not then I think you will need to whitelist the one on the list.
This is the list of Logic App IP's per country & connector:
Logic App IPs
I am having the same issue. Apparently this configuration is not supported. Quoted from an Azure ticket yesterday:
"Yea we have had couple (sic) customers reporting this issue. Unfortunately this feature is not supported as of now. The azure networking team was working on adding this support for logic apps. As of last month there was no ETA given."
Also, in my storage account logs the failed logic app requests are coming from 10.157.x.x, which I cannot whitelist in the storage account firewall. I even tried "fooling" the firewall by creating a vnet containing that subnet and allowing that. No dice.
Have you used the blob storage connector in your logic app ? Once you add the credential connection details, you'd be able to connect from the logic app.
The full documentation can be found here