I am able to find enough blog/posts on how to use Power BI to analyze Azure cognitive search data. But what I am looking is exactly opposite of this.
Can we use Azure cognitive search inside/for power BI reports to query on power BI data? Basically a search bar in power BI or anywhere in app that can do the following : -
Query on Power BI data sources and return visualization.
for example :- top 5 hotels by people visited result in relevant visualization.
smart enough like Azure cognitive search to understand human search errors.
FYI I have already explored QnA visualization of power BI, but I still want to explore the possibility of leverage Azure cognitive search instead of that.
I have my data stored in sql server in Azure VM and Azure blob storage.
is it possible? if yes can someone please share relevant links/videos to help me accomplish that.
Azure Cognitive Search can send operation logs and service metrics to an Azure Storage account, which you can then visualize in Power BI.
Refer this document for detail description.
Power BI provides access to a set of functions from Azure Cognitive Services to enrich your data in the self-service data prep for Dataflows. The services that are supported today are Sentiment Analysis, Key Phrase Extraction, Language Detection, and Image Tagging.
These options are running based on the power of Azure Cognitive Services. These Azure services are leveraged within Power BI by a few simple clicks.
For more details refer this document.
You can build the Power BI solution with Azure Cognitive Services, without using the Power BI Premium for more details refer this document
Related
I have to create chart and metrices in Azure Monitor overview dashboard based on Azure SQL database data. How can we do that? I don't want to show database performance. I want to display database data in dashboard.
Azure Monitor helps you maximize the availability and performance of
your applications and services like CPU performance, memory usage,
storage usage, etc. It can't be used to visualize the stored data in
Azure SQL database or any other service.
To visualize the data, you should either use tools like Azure Data Explorer or Power BI.
Please check Visualize data with Azure Data Explorer dashboards.
I'm no MS expert - recently hopped onto the Azure train and apologies in advance if I get some information wrong.
Basically need some input in Azure's architecture utilising Azure Data Factory (as the ETL/ELT tool) and Azure SQL database (as the storage), to a BI output - Power BI. My situation is this;
I have on-premise data sources such as Oracle DB, Oracle Cloud SSAS, MS SQL server db
I'd like to have a MS cloud infrastructure solution for reporting purposes.
No data migration needed - merely pumping on-prem data onto cloud and producing a BI reporting solution
Based on my limited knowledge and Google research, Azure Data Factory caters for all my on-prem sources, as well as the future cloud Azure SQL database. If future analysis is needed, Azure Storage and Azure Databricks can be added in to this architecture. I have sketched out the architecture of my proposed solution.
Just confirming my understanding
Without Azure Storage & Databricks (the 2 pink boxes), the 2 Azure component (DF & SQL database) is sufficient to take data from on-premise sources, process on cloud & output into Power BI.
With Azure Storage & Databricks (the 2 pink boxes), processing will be more efficient as their summarised function is to store training data models & act as an analytics processing engine.
Azure SQL database is more suitable, as compared to Azure SQL datawarehouse as my data sources does not exceed 1TB; cost-wise is cheaper AND one of my data sources contain data from call centers, hence OLTP is more suitable. Plus I have Azure Databricks to support the analytical bit that SQL datawarehouse does (OLAP).
Any other comments to help me understand this whole architecture will be great!
I am a new learner of Azure. I was wondering if we have #Query (value="...") kind or any equivalence for DocumentDb (CosmosDB). Because, the documentDB does not take #Query. I am looking to convert the sql query (From jpa to cosmosDB).
Taking data from on-prem or IaaS sources like SQL on a VM, Oracle etc, requires a Self-Hosted Integration Runtime (SHIR).
Please review the Modern Data Warehouse pattern which sounds similar to what you are proposing.
I am new to Azure. I would like to learn the architecture deployed in my company which i shown below on diagram. Can anyone point me to some video example or something that could reflect that from diagram below. I also have access to Azure portal that i have some money credit so if it is possible i could create some test environment based on that diagram.
P.S Is it possible to use Visual Studio for any kind of work based on that diagram or everything have to be created and develop from Azure portal?
Datasource Oracle DB --> on prem gateway --> ADF--> Azure DB --> AAS --> PowerBI
SQL EDP --------------------------------------^
You've got a fairly straightforward BI architecture there with the following logical components:
raw / source data
integration
data mart / dimensional model
semantic
visualisation
The physical components look a bit like this:
The physical components can be described like this:
Oracle database - former market leader database product. I would guess your employers have rejected OBIEE for some reason
Self-hosted Integration Runtime (SHIR)On-premises data gateway - the SHIR gateway enables the movement of data from on-prem data sources to the cloud. This must be used when moving data from on-prem to Azure SQL DB using Data Factory. Use the SHIR with Data Factory and the Gateway with Power BI and Azure Analysis Services.
Data Factory - Azure ELT tool for moving data from place to place. ETL feature Data Flow currently in preview.
Azure SQL DB - PaaS SQL database, scalable via service tiers. If your data in Oracle is not already in a data mart / dimensional format, then it can be made so here
Azure Analysis Services (AAS) - PaaS OLAP in-memory engine, scalable for fast slice-and-dice, drill down and semantic modelling. Tabular only.
Power BI - increasingly powerful visualisation tool. Run dashboard in DirectQuery / LiveConnection mode to avoid entirely duplicating the tabular model from AAS in Power BI.
In answer to some of your questions: you can have one Azure Data Factory with many pipelines. The Visual Studio Azure Data Factory project type is now defunct.
As to "why" for certain technologies:
why Oracle - Who knows.
why SHIR - SHIR is compulsory when moving data from on-prem to cloud with ADF
why Azure SQL DB - lightweight and powerful PaaS DB requiring no infra and low TCO; scalable. Might be location for restructuring of data from raw / relational structure to dimensional in readiness for semantic layer if your data is not already in that format in Oracle
why AAS - fast, in-memory slice-and-dice; scalable, can pause, can be interrogated by Excel, Power BI Desktop, SSMS, VS, other clients etc. Optionally has row-level security (RLS)
Power BI - online service Power BI.com offers easy sharing within organisation, even externally.
why all the components together - you could (in theory) go straight from Oracle to Power BI with a Power BI gateway (I think) BUT you would then have to do all the modelling in Power BI and your model is then only really accessible from Power BI. In this model, users with SQL skills can query the data mart, users with DAX (or Excel, or Power BI Desktop) skills can query the AAS tabular model, AAS is very scalable component, etc
These opinions are strictly my own personal ones and the value of them may go down, as well as up.
HTH
Azure Data Factory has a 1:M capability with various data sources. One instance of Azure Data Factory will support multiple data movement capabilities: Data movement activities
Information about On-Premise Gateway:
The on-premises data gateway acts as a bridge, providing secure data transfer between on-premises data sources and your Azure Analysis Services servers in the cloud. In addition to working with multiple Azure Analysis Services servers in the same region, the latest version of the gateway also works with Azure Logic Apps, Power BI, Power Apps, and Microsoft Flow. You can associate multiple services in the same subscription and same region with a single gateway.
Connecting to on-premises data sources with Azure On-premises Data Gateway
I would like to use our azure search index as data source for Power BI dashboard/report. Is there any azure search service connector available for power bi like document DB connector ?.
I don't know where the data is stored in the Azure Search? Please suggest me a way for connecting the data with power BI.
Note: Not asking about the Azure Search Traffic Analytics.https://powerbi.microsoft.com/en-us/blog/visualizing-azure-search-data-with-power-bi/
Thanks in Advance!
I understand you are not interested in traffic analytics. I am also assuming that you are not interested in displaying index statistics.
The actual structure of the "inverted index" created using Azure Search is not exposed, so there is no way for you to visualize it in PowerBI.
Luis Cabrera
Program Manager
Cloud AI Platform - Microsoft
According to this: https://azure.microsoft.com/en-us/blog/unleashing-insights-from-data-in-documentdb-with-power-bi/ DocumentDB connector is available for both Power Bi online and Power Bi Desktop.
But when I go to Get Data -> Databases in Power Bi online, I do not see any way to connect to Document DB. I can see the connector only in Power Bi Desktop.
Is Document DB supported as data source in Power Bi online? if no, is there any workaround to make it work? I have a requirement to feed schema-less data (located in Azure) to Power Bi online, I'm searching for any way to do it, whether this is Document DB or some other NoSQL database
please allow me to clarify that connectivity to a DocumentDB account via PowerBI.com is actually there. Currently, the "Get Data" in PowerBI.com only offers data sources with DirectQuery capability. Please see this article on DirectQuery. We are working to support DirectQuery in future. For now, there are two ways to push DocumentDB data to PowerBI.com. The first method is what Denny has suggested: build the report via PowerBI Desktop and publish the report to PowerBI.com. The second method is to leverage PowerBI API to connect and push data from a DocumentDB account to PowerBI.com.
First of all, I'd like to apologize for my initial answer as I completely missed your key issue about utilizing Power BI Online. Allow me to try to make amends by attempting a more complete and proper answer.
In the case of Power BI Online, because the Power BI-to-DocumentDB connector is currently in beta, it is not available as a data source within Power BI Online under the Get Data section as you noted. Saying this, you can create a connection to DocumentDB using the Power BI desktop, publish it to Power BI Online, and then refresh it within Power BI Online. Following the "Power BI tutorial for DocumentDB: Visualize data using the Power BI connector" tutorial, here is a screenshot of running the data refresh on PowerBI.com using the credentials to your DocumentDB database.
Note, there are is already a request on the Power BI Ideas forum to have DocumentDB available directly within Power BI Online - you can read more and vote for it (if appropriate) at: https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/7160908-documentdb.
Some additional references that may be helpful:
Data Refresh in Power BI: Here is some additional information about how Data Refresh works in Power BI.
Power BI Content Packs: For more information on connectors between Power BI and other data sources, please reference Content Packs.