The SHOW TABLES IN mydb query lists tables and views, while SHOW VIEWS IN mydb only lists views. Is there any way to list only the tables of a given database ?
Related
I have a databricks database and I want to subset all tables under this database for a specific set of values. Is there an easy way to do it instead of querying each table separately ?
I have the list of five tables name. I need to delete rest of all the tables in data bricks, which is not in the list. I don't know what command or method to be used to solve this.
Please help me on this.
Regards,
Manoranjini Muthuraj
#pyspark code
#list of tables to keep
keep_tables = ['table_1', 'table_2', 'table_3', 'table_4', 'table_5']
#get list of all tables from my_database
df = spark.sql('show tables in my_database')
#loop thru the tables and if table not in keep_tables then do the operation on each table (drop/delete/count etc).
#**Careful** the code will drop tables not in the keep list
for t in df.collect():
if t not in keep_tables:
#do the table operation (drop/delete/count etc)
print('operate on table {}'.format(t.tableName))
spark.sql('drop table my_database.{}'.format(t.tableName)))
I am trying to get a list of tables and columns in a database, so I could find which tables have a particular column, the best I could find is use separate queries like one to show all tables , and then one to show all columns in one table, e.g. SHOW TABLES FROM database_name, SHOW COLUMNS FROM databasename.tablename. It will not be ideal when you have many tables to go through. Any solution out there at all?
Unfortunately, there is no way to fetch all metadata in one call. You can only do show databases, show tables in ..., describe table .... There is also spark.catalog.listTables, etc., but they could be slower than corresponding SQL queries.
I answered to related question yesterday - you can find code there.
Is there any way to search or filter for particular Table from Dataset using Table name while calling List operation? I understand that documentation mentions use of Labels to filter Tables but in my case this will not suffice as there is no restriction on number of Tables that can be created under a Dataset with or without Label . I am using Node library for my operations.
The prefered way to search or filter for particular Table (or any other metadata object) is to query INFORMATION_SCHEMA. There are multiple INFORMATION_SCHEMA tables which could be used - INFORMATION_SCHEMA.TABLES, INFORMATION_SCHEMA.TABLE_OPTIONS, INFORMATION_SCHEMA.COLUMNS etc.
More info at https://cloud.google.com/bigquery/docs/information-schema-tables
I would like to list all empty tables in my database Athena.
I tried :
select table_schema, table_name from information_schema.tables
where table_schema = 'database'
But like this I list only table name with database name.
Thanks for your help.
I do not think it is possible within a single query. Your query gives you a list of tables. Having that I think you could now iterate over that from the external tool.