I have an Azure Stream Analytics job outputting to PowerBI - outputs are present and the data set is present in PowerBI - however, the graph is not live as I have to manually refresh the page to receive the updated data.
How do I go about having a live feed of data?
Cheers!
After pinning the graph to a dashboard it should refresh in real time. There are some limitations about the throughput that a PowerBI dashboard can handle and sending a large volume of events may result in latency in the view. Details in the Limitations and Best Practices topic here: https://azure.microsoft.com/en-us/documentation/articles/stream-analytics-power-bi-dashboard/
Related
I want to show the insights(page view, comments, posts, etc) data of my Facebook page in a Power BI dashboard, that is embedded into my web-page, I am able to show the dummy report by downloading the Facebook page CSV report. If I have to real-time update the dashboard what is the procedure for that.
there are at least two options:
Connector
API
Connector
At this very moment there might be an issue connecting Power BI to Facebook but in general you should be able to use the built-in connector.
Please note that with the connector, once your report is published in Power BI Service it will not refresh data in real-time, instead in semi-real time. For example, you can define to refresh the data source up to 8 times a day (using Pro license) or even more if you have Enterprise Premium Capacity.
API
Instead, if you really need to deal with real-time data the alternative is using the API offered in the real-time streaming service of Power BI.
I need to build a dashboard which will visuallize the usage and cost of many azure subscriptions. accounts, departments.
My plan was:
Send the data that is 'behind' the Azure Cost Analysis view, to the log analytics workspace.
In the log analytics workspace, perform custom aggregations / filters.
Display those aggregations as charts in Azure Metrics or directly in Azure Dashboard.
Problem is with step 1, I dont know how to send the data that is 'behind' the Azure Cost Analysis view, to the log analytics workspace.
I thought of two solutions:
Fetching the data from azure cost & billing API.
Schedule Export cost analysis data to a storage account, and then somehow moving the data from the storage account to the log analytics workspace.
Both solutions seems to me a bit 'overkilling' - is there a more direct approach to send the cost analysis data to log analytics workspace?
If there is no option such as that, I would be happy to know how would you suggest moving the exported data from the storage account to the log analytics, or do you have some other idea?
Thank you!
The only native solution is, to schedule from the Costs-Blade an export of the Costs as CSV into a StorageAccount. If you want to load the Data into a Log-Analytics-Workspace, Azure Automation and a scheduled Script would work.
I believe a direct approach is currently not available but I see this feature request raised in UserVoice / feedback forum for the same requirement. If interested, you may upvote it because in general the responsible Azure product / feature team would triage / start checking feasibility and prioritizing a received feedback based on various factors like number of votes a feedback receives, feasibility, open prioritized backlog items, etc.
I would suggest you to fetch the data from azure cost & billing API and send that data to Log Analytics from a REST API client by using the HTTP Data Collector API. For more information and illustration with examples, refer this Azure document. Or else if you want to fetch the data from azure cost & billing API and store it in a machine then you may go with custom logs. For more information w.r.t it, refer this Azure document.
Other related references:
Use cost alerts to monitor usage and spending
Supported metrics with Azure Monitor
We have legacy applications that currently write out various run time metrics (SQL calls run time, api / http request run times etc) to local SQL DB.
format:( source, event, data, executionduration)
We are moving away from storing those in local SQL DB, and are now publishing those same metrics to azure event hub.
Looking for a good place to store those metrics for the purpose of monitoring the health of the application. Simple solution would be to store in some DB and build custom application to visualize the data in custom ways.
We are also considering using Azure Monitor for this purpose via data collector API (https://learn.microsoft.com/en-us/azure/azure-monitor/platform/data-collector-api)
QUESTION: Are there any issues with azure monitor that would prevent us from achieving this type of health monitoring?
Details
each event is small (few hundred characters)
expecting ~ 10 million events per day
retention of 1-2 days is enough
ability to aggregate old events per source per event is important (to have historical run time information)
Thank you
You can do some simple graphs and with the Log Analytics query language, you can do just about any form of data analytics you need.
Here's a pretty good article on Monitor Visualizations.
learn.microsoft.com/en-us/azure/azure-monitor/log-query/charts
In the context of Azure IoT hub, when would one use Stream Analytics over Time Series Insights?
The product pages and documentation for both indicates they are heavily geared for IoT/data applications. However, I'm not clear on the differences.
The use case I have is both real time monitoring as well as ETL analysis. Could (or even should?) the two be used together?
One immediate difference I can see is that Time Series Insights stores the data whereas Stream Analytics (I think) would need the developer to integrate storage.
In short, stream analytics is about transforming, filtering and aggregation of data and time series insight is about visualising (stored) data.
Data passed through stream analytics is typically forwarded to resources like power bi (for realtime monitoring) or storage like a database for later analysis or processing.
One immediate difference I can see is that Time Series Insights stores the data whereas Stream Analytics (I think) would need the developer to integrate storage.
This is a correct statement. TSI is a data store, but its purpose is to create an environment to (visually) analyze that data. ASA cannot be used to analyze data on its own.
You could use ASA to transform the data and have the data send to Event Hub. That same Event Hub can then be used as a data source for TSI.
since a couple of weeks I'm working with Microsoft Azure and I wonder if there is a possibility to create realtime charts in my Web App for external customers.
I know Microsoft provides two different services called 'Power BI', which supports realtime charts and 'Power BI embedded'. But my problem is that, as far as I know, Power BI is only intended for internal users and Power BI embedded, which is inteded for charts e.g. in Web Apps for external customers, only provides reports which are not realtime.
Am I missing something or is it currently not possible to provide realtime charts inside web apps with the given services of Azure? If yes, what would be alternatives to achieve my goal?
Thank you very much in advance.
Kind regards,
Felix
I would look at Power BI Embedded, with the data source using a Direct Query connection to Azure SQL Database or Azure SQL Datawarehouse. Every user action in the report (filtering, drilling etc) will generate a query against the database.
That Power BI Embedded architecture is explained on this page:
https://learn.microsoft.com/en-us/azure/power-bi-embedded/power-bi-embedded-what-is-power-bi-embedded
Direct Query is explained on this page:
https://powerbi.microsoft.com/en-us/documentation/powerbi-azure-sql-database-with-direct-connect/
1) Consider that Real Time is like in an IOT scenario where you see your graphics on your dashboard moving in Real Time and not after a refresh. So in this context you should consider using Azure Stream Analytics Jobs. It's get an input from a blob storage, an event Hub, ..., and then in output you can use your power BI account to write in real time events ingested from Azure Streaming Analytics. Very powerfull! you use SQL for querying the input, the only thing to be aware is the thumbling time window that is somehow new to the SQL language.
2) Letting your customer access to the dashboard I would suggest you to pubblicate your dashboard for free access, and then secure your dashboard inside a web app on which you can apply a security pattern. You can also invite people outside of your organisation via email. Which is faster than the previous solution, but people accessing to your report must have a power BI Pro license. You can use the free trial for 60 days.
Hope that helps!
Cheers!