I am using Snowflake as my backend database and created & published a dataset in Power BI with direct query. As a next step I am trying to analyze the data (to get Pivot experience) in excel.
I am observing the performance from excel is not good compare to Power BI desktop or PBI Service. For example one query is getting completed in ~4 sec in Snowflake, PBI desktop and PBI service both are returning within in like ~10 sec; but the same thing when I am trying it is taking over a minute or so.
Is there any setting in excel level we can apply to handle this?
Please let me know if there is any other solution to handle this.
I am working on a case where, we have a large volume of data. All the data is located in different Excel files which are having more than 500000 rows in each file. The files will gets added monthly.
I have created Power BI Report and each Month when the new file gets loaded, it takes a lot of time to refresh.
To make it simpler, we are trying to use SSIS where the data will be imported from the Excel files in to a DB and then the DB will be used as Base Data to the Power BI File.
In this case, we are downloading the data locally from cloud (SharePoint Online) and then again Uploading it to Cloud (to Power BI Premium Workspace.).
Guys, I am not sure whether this is the right approach and efficient way to optimize the Refresh Time. Can any please suggest if is there any better solution available. Perhaps we may have a solution which will consolidate all the data on the cloud only and so no need to download and transfer it it using SSIS.
Regards,
i am using a Web server service where i am able to get real-time data using RestAPI calls. Now i want to be able to collect the data - store them somehow and then visualise them in a nice way (produce graphs basically). My approach would be to store them in a database and then use the PowerBI's internal feature "Get Data" from an "SQL Server Database". No idea if this the correct approach. Can anyone advise here ?
Hello and welcome to Stack Overflow!
I agree with Andrey's comment above. But if you want to know about all the Data sources that PowerBI supports connecting to, please check the following resources:
Data sources in Power BI Desktop
Power BI Data Source Prerequisites
Connect to a web page from Power BI Desktop
Real-time streaming in Power BI
Additionally, you may also go through Microsoft Power BI Guided Learning to understand the next steps for visualization.
Hope this helps!
There's another approach that is to build a custom data connector for that API to Power BI.
That allows you to fetch the data inside Power BI, and build the visuals. You can store it in excel files or sql (you can use python scripts for this) and you can schedule refreshes on the service.
I want to create Webapps with PowerBI Embedded from the german central datacenter. Unfortnuatly this service is not available and i don't know when it will become available.
Therefore my idea is to migrate PowerBi Embedded later and start with all other services located in german central. Is this possible or strongly recommended to have the PowerBi Embedded service and the azure SQL Datawarehouse in the same place?
When you place your data source (SQL Data warehouse) and your BI tool (Power BI) in different datacenters there are two things that should be mindful of:
Latency and network speed between the data centers may affect the performance of your BI solution significantly (in a negative way), especially if you are manipulating and analysing large amounts of data. It depends a little on how you set up your Power BI embedded. If you use DirectQuery then you will be hit with the latency penalty every time the query runs (whenever you look at your report), if not then you will only be hit with the latency when you refresh your imported data. However, without DirectQuery you may have to import more data in order to aggregate etc from the imported dataset.
Egress, you pay for traffic going out of data centers. If you continuously send large amounts of data between two data centers then the egress cost can be a factor for you. In a normal setup the traffic charges are almost negliable, but if your BI solution streams a lot of data on every refresh then it may build up to a lot of money.
What the differences between "Cognos TM1" and "Cognos 10 BI"?
Which one is consider as BI Tools by IBM?
There are huge differences between "Cognos TM1" & "Cognos 10 BI"!
Cognos TM1 is a (OLAP) multidimensional database which can be queried from Excel and the Web through "TM1 Web", "TM1 Executive Viewer" and "Cognos 10 BI". Within this database, you'll be able to create almost anykind of OLAP / Decisional application.
Cognos 10 BI is a web based reporting application. Users, depending on their rights (licence) will be able to run reports, schedule reports and/or create ad hoc analysis against relational and/or multidimensional databases. In order to do so, a logical layer has to be built using "Framework Manager". This logical layer is used to hide database complexity and to provide to the end users relevant information.
In simplest terms:
TM1 is a database engine and a collection of applications for accessing and managing its databases.
Cognos BI is a collection of web applications that provide pretty interfaces for viewing and doing stuff with data.
More often than not, Cognos BI uses TM1 databases as its data source, but it does not have to. If Cognos BI is using TM1, most of the same user functions that are possible in TM1 applications are also available through Cognos BI, except they are available in a more user-friendly manner. Cognos BI also adds functionality not in TM1 applications to allow additional data management. IBM's marketing is confusing, but generally Cognos BI and Cognos TM1 collectively are considered to be the BI Tools package that they offer.
Now to be a little more technical about TM1, it is not just a plain old database. As others here have mentioned, it is a multidimensional OLAP database. It is able to handle numeric and string data, but it does not have a concept of NULL values. Numeric data can be summarized using consolidated elements. It has attributes to store metadata in. It has a built-in rules engine to handle business logic and custom calculations. It has processes for ETL and database maintenance tasks. It has chores for scheduling processes at various intervals. It links to Excel workbooks. Lastly, in addition to these features and more that are provided out of the box, TM1 exposes an API for programming against using 3GLs such as C++, C#, Java, and even VBA.
The key difference is Cognos TM1 is meant to work with excel easily wheras Cognos 10 BI is mainly browser based.
Have a look here for gory details.
http://www.tm1forum.com/viewtopic.php?f=5&t=1442
Cognos TM1 is also referred to by IBM as Financial Performance Management FPM. It's an in-memory MOLAP cube application with real-time rules calculation. As it provides end-user writeback it's often used by the Office of Finance for budgeting and modelling - therefore the MS Excel interface is frequently used. However it also comes with a zero footprint web "front end" as well as Cognos Insight, which permits distributed or disconnected access to the TM1 cubes (located server side).
TM1 may be integrated into Cognos BI, so that Cognos BI reports from TM1 cubes; or TM1 may be accessed from a portlet within a Cognos BI dashboard.
difference between cognos10 and cognos tm1 :-BI is the reporting tool and cognos tm1 is analysis and reporting tool , in tm1 bulid the CUBES and BI generatead the reports