I am using a GCP Cloud Function to read from a BigQuery Table and output the results to a CSV file located on a network drive (all the infrastructure parts necessary to communicate with on-prem are in place). I was wondering whether there is a way to write data out to this location using Pandas and PYSMB?
I have done a fair bit of reading on the topic and couldn't find a way, but thought someone with more experience may have an idea.
Thank you very much for your help.
Regards,
Scott
Can any help me in fetching data from power BI endpoint without the need of using Power Shell, as want to know a way of directly fetching in Linux only?
I know a power shell can be installed in Linux , but is there any way I can skip and directly fetch the data?
reference - https://learn.microsoft.com/en-us/power-bi/admin/service-premium-connect-tools
Your Power BI XMLA endpoint is accessible through your Azure Analysis Services (AAS) instance tied to the given datasource/workspace, which means that you should be able to connect to that AAS instance and work with the data there via the web. I am not aware of any currently available Linux compatible tools that allow this. I did a bit of research and was surprised to find that there was not a VS Code extension that allowed this (might have to get to work on that ;)).
That being said, Microsoft has several different client libraries (for both AMO and ADOMD.NET) built within their .NET Core framework that would theoretically be able to used by a client application that could be built for supported Linux OS (Microsoft doc here). In other words, (again, theoretically) it should be relatively painless to build a simple tool for a supported Linux OS that takes in XMLA commands and executes them on a provided connection.
EDIT: Another good option to consider might be Microsoft's Power BI REST API (documentation here). If the functionality you are looking for is available within their REST API, you should be able to write a client tool (using one of many different options, but .NET Core could still be the in there) targeting Linux that makes use of the API for your Power BI instance in place of directly using the XMLA endpoint. I would consider this the better alternative. This is going is a less 'Microsoft-y' way of doing this, and is going to be much easier to maintain and develop over time. I would start by confirming that the functionality you want is not available in this API first.
EDIT: After reading further in above linked document regarding AMO and ADOMD.NET client libraries:
TCP based connectivity is supported for Windows computers only.
Interactive login with Azure Active Directory is supported for Windows computers only. The .NET Core Desktop runtime is required.
So it looks like there are currently some limitations to these libraries regarding a Linux runtime. I am not positive that you could use something other than TCP based connectivity to accomplish this, but if I find a way (or someone is able to suggest something), then I will update.
I am trying to find a way to connect powerBI, but coudn't found any resources to do same. e.g we can connect to mssql,postgress etc
thanks.
You can run Python scripts directly within Power BI and this allows access to the data sets as inputs and outputs back in to the data model.
Here is a link to the official documentation for the feature
You can also use the Python visual. This doesn't allow you to feed in to the data model but instead uses it to generate a visualisation which is then directly displayed.
Documentation on the Python visualisation is here
I have been asked by my boss to find a tool or design one for testing rest services hosted in cloud environment. He also asked me if I can read the input data from excel sheet so that other junior members can write tests in excel. I have created keyword driven framework using Apache poi so I know how to read data from excel in Java program. I have also worked on httpClient, so I can tie these together. But I am hearing a lot about rest assured and want to know can I use rest assured where input can be read from excel. Is it worth spending time on? Also with cloud infrastructure which approach is best? Thanks.
You can safely proceed with your boss's request, since there is no link between Excel Inputs & REST assured framework, as both of them are separate jars.
You can read test inputs/outputs from excel and use them in REST assured. REST assured provides an easy to implement REST testing DSL with BDD flavor.
The following link gives you a quick getting started tutorial.
http://www.hascode.com/2011/10/testing-restful-web-services-made-easy-using-the-rest-assured-framework/
I'm studying an introductory course in databases and one of the exercises is to work with MS-Access. However I'm using Linux at home and although I can use the computer classes at the university it is far from convenient (limited open time - my studying time is mostly nights).
So how can I use an Access file (*.mdb) in Linux? By use I mean changing tables, writing queries and so on.
Are there tools to convert it to another database format (mysql, postgresql or even gadfly)?
Also what problems may I encounter?
Although a bit dated, I've had good success with mdbtools which is a set of command line tools for accessing and converting Access databases to other formats. I've used it for importing databases into PostgreSQL.
If you're running an Ubuntu variant you can install it with:
sudo apt-get install mdbtools
or you can download it from here.
You're out of luck. Access has no real equivalent on Linux and while Kexi is an interesting alternative that can import Access files and aims to provide similar functionality, it doesn't actually uses Access files once the data is imported.
If your assignment is to develop an Access application with forms etc as opposed to just using and mdb database as a store, then you can try a recent release of Wine with a compatible Access version (see compatibility list) or, even better, find a Windows machine where you're sure it's going to work.
Not to be forgotten, the use of a Virtual Machine loaded with Windows would help you achieve the same thing on your Linux box.
I am currently trying Access with Wine on Ubuntu and I seem to be getting there. I have found that I need to copy various dlls manually, but that could easily be lack of reading up on the subject.
From the documentation: Connecting To Microsoft Access. However, this seems to indicate that you need access running in a windows host and connect via ODBC... See also Known Problems.
I recently discover https://dbeaver.io/ which is a software (in java) to manage different database types (MySQL, PostGreSQL…), a bit like phpmyadmin (but as a host based soft, no server require) and it can manage MS Access excep if version is too old (it is probably my case)
You can work with Access through a connection (ODBC or OLEDB), as long as you only need to manage the "database" dimension of the file (tables and views, which are called "queries" in Access).
Once the connection is open (see here for connection strings), you can send SQL commands to your mdb database, such as (where cn is here a connection object):
cn.execute "CREATE TABLE myTableName (myTable_id autoNumber, myTable_code Text, ...)"
Please note that MsAccess uses a specific DDL that looks like the standard T-SQL but is not really it. Check the syntax in MsAccess help.
Depending on your database (and its constraints, default values, primary keys used, relations, data validation rules, aso), transfering Access can be easy and straight or might not even be possible. You will encounter a problem each time your database implement an access-specific/non-standard SQL rule.
If you really need to convert your access data to something else, I'd adise you to (1) export it under MS-SQL (the free version will be ok, an upsizing wizard is available in Access or on this site), (2)use an additional tool like this one to generate a "CREATE DATABASE" SQL Script, including or not data inserts, (3) use this script to try to create the database and its data on another database server.
If you've got an assignment to work with Access, then frigging find a Windows computer and do your exercise on the native platform for Access. It's completely senseless to do anything else, as you won't be learning anything useful about Access.
If the assignment is to use a Jet data store, then that's something of a different story. And if it is, then you should have worded your question differently. I wouldn't recommend using Jet on anything but a native Windows file system. Certainly if the project is to actually read/write data to a Jet data file then you're not really fulfilling the assignment if you're not using Windows at least as the ODBC host.