I have a self-developed dashboard on which I can see certain errors from my apps. I also have Application Insights.
Now I'd like to have links in the graphs of my custom dashboard to the search section of AI with the right filters to find the log events that correspond there.
The URLs in the Azure Portal are horrible though.
Is there any way to do that at all?
There are couple ways of pinning filtered views:
Pin Analytics query (as a chart). Click through will navigate to Analytics with this query opened
Apply needed filters on Failure / Performance blades and pin them. Click through will navigate to these blades and will preserve all configured filters
Related
There's an awful lot of disjointed documentation on monitoring network/resources in Azure. What I'm looking for is which pieces are needed to get information from VMs, NVA firewalls, azure load balancers, and other network resources and network connectivity into a single pain of glass in Azure. Only concerned about Azure, not on-prem for now.
I've come across azure monitor, log analytics work spaces, event hub, vm extensions, network watcher, insights, etc...but I'm not sure which are required and which are not. One doc leads to the next and I end up with 30 tabs open. I'll also need to be able to push logs to other security devices such as a SIEM.
Does anyone know of a deployment guide that wraps this all up in a more logical fashion? Does anyone have any feedback on which pieces from azure (not 3rd parties) are required at a minimum to accomplish a single pane of glass to view my Azure environment holistically?
General overview of observability in Azure
Likely, the thing you're looking for is Azure Monitor. It's an umbrella term for everything observability related inside Azure.
To store Metrics and Logs you need Log Analytics: it can query data with kusto query language, visualize results, define Alerts on queries.
Alerts is quite a complex beast, as it is spread across the entire cloud. Two types that I use the most:
log-analytics alert (which I mentioned above)
Alerts tab, which is available at every Azure component view. for example, open resource group, and scroll down to Monitoring section
Each component also has a subset of built-in metrics. Likely, you noticed that many azure components on the Overview view display some charts. For example, Azure Storage Account displays Total egress, Total ingress, and other line-charts. When you click on these charts you can customize them. These metrics and charts are free to use.
Microsoft also has all-in-one observability solution for Azure Functions and Web Apps: Application Insights
Dashboards allows to join multiple charts into a single view and share it with others.
If you care about security, Azure proposes Azure Security Center
Deployment/management strategy
I suggest to start with:
Create Log Analytics Workspace, which is the storage for metrics and logs. The azure docs article explains how to design it: how many instances to use, how to rate-limit ingestion (it might be expensive if goes out of control), how to access it and so on.
To get Azure components logs, look for Diagnostic Settings tab at a component page at Azure portal, but not all components has it (sic!). I suggest
sending the most critical data to Log Analytics workspace to store them in a queryable format for 30 days (it's in free tier). This is needed for investigating current issues with your infrastructure
if you might need logs later than 30 days - send them to Storage Account
you mentioned SIEM integration - route required events to Event Hub and then process the stream according to your requirements
So, if you need long-term storage - you need to create Azure Storage Account.
If you need real-time analysis - you need to build a pipeline based on Azure Event Hub.
If you have Azure Functions and Web Apps - add Application Insights. According to my experience, I would suggest starting with a separate instance per each Azure Function resource or Service.
Create Alerts for each component separately. If you do it through UI - open component page at the portal and look for Alerts tab there. If you're automating the process (please do so as soon as possible), do not expect easy trip: I used ARM templates and terraform - in both cases, there are dozens of barely documented features.
Join related components core-metrics into Dashboards and share it with the team. This guide is a good starting point. Note, when you share the dashboard, it's also persisted as an azure resource in the subscription.
We are looking into building-out some reporting dashboards for use by product owners/etc that might not be familiar with Application Insights or Azure Monitor. We would like to display a list of historical alerts by type as well as some basic aggregations. We see where we can display this information (somewhat) from within Azure Monitor, but we do not see how we can actually surface this information on a dashboard within Azure.
I have found this API call, which seems to retrieve the data, but we are not looking to develop a custom solution.
I hope Serverless360 will be of great fit to your requirements where you will be able to display the historical alerts by type as well as with some advanced aggregations. You can easily set up the powerful dashboards and which can be easily handled by product owners (doesn't necessarily to be familiar with Azure Monitor) to get a high level of insights.
Below is a sample of how a dashboard would look like
I'm trying to pin a query result to AppInsights dashboard but the data is getting grouped in the background.
When I run the query by itself, the graph is showing up as expected but when I click "Pin to dashboard" and view the same in dashboard, it only shows completely different data.
After pinned to dashboard
Before pinning to dashboard
Realized it was because in Azure app insights dashboard, pinned charts are limited to 30 days of data, regardless of what is specified in the query or the tile configuration
Update:
Op is correct. For application insights dashboard, it will only display the data within 31 days. The official link is here.
This may due to you're not selecting a proper start-time and end-time for the dashboard.
Try to select a proper start-time and end-time in the dashboard, a screenshot as below:
And if the issue still occurs, please share us your query code, and other dashboard settings.
We have a large web application. We need to store user analytics about which ads has users seen. What accordions has been clicked etc. As far as I can see log analytics and application insights is more targeted towards logging and instrumentation (correct me if I am wrong). Does Azure provide any services for this kind of functionality, or would you have to store it your self to a SQL server/Table storage/CosmosDB and make a job that calculates the different statistics needed?
If "which ads has users seen" is actually a page view / request operation, then you can choose application insights.
If you're using app insights SDK, then you can track the events for these ads by using methods like TrackEvent. This event data is useful for analytics in application insights.
For analytics, you can use log queries which is available for azure monitor / application insights(to use log query, nav to azure portal -> your application insights -> in the left panel, select "Logs(Analytics)" under Monitoring). With log query, you can build your own query for any different statistics, or generate some charts.
Please let me know if you have more concerns.
is it possible to show the Podstats of AKS on a shared Dashboard?
Why not, you would just need to pull that data from the OMS and create a custom dashboard from those queries.
if you click on the individual entry it will navigate to the OMS instance and show you the query needed to get that data.
Danny, Unfortunately there is no simple "click to pin this chart" functionality available currently in Container Insights (more modern name for that thing is Azure Monitor for Containers). We're looking to add it within a couple of months.
The chart does a query to Log Analytics store and goes into a bit of custom processing on the data received to render the chart. You can go to your cluster, "Metrics" on the menu on the left and chart the same thing there. You can pin from the metrics. Let me know if you need help with that I can provide more detailed instructions...