Tablespace name = SYSCATSPACE
Tablespace ID = 0
Tablespace Type = Database managed space
Tablespace Content Type = All permanent data. Regular table space.
Tablespace Page size (bytes) = 16384
Tablespace Extent size (pages) = 4
Automatic Prefetch size enabled = Yes
Buffer pool ID currently in use = 1
Buffer pool ID next startup = 1
Using automatic storage = Yes
Auto-resize enabled = Yes
How can I select the tablespace name if the auto resize tag is set to yes? (I need to do this to x amounts to tablespaces that are displayed with additonal information).
What db2 command are you using to get this output ?
Instead of trying to parse the output, you should select the tablespaces from db2 catalog sytem tables, and filter your rows per your need..
SELECT MEMBER, TBSP_ID, TBSP_NAME FROM TABLE(MON_GET_TABLESPACE('',-2)) AS t
WHERE TBSP_AUTO_RESIZE_ENABLED = 1
easier.
PS. I don't know what you mean by doing it a "pythonic" way. python has no magic way, under the covers, it has to ask DB2 for the information. I am giving you the best way to retrieve the information you want, from db2 perspective.. regardless the language being used.
Related
I am trying to automate the process of creating PNGs (screenshots) of a Tableau Dashboard with different filter values specified.
Ex.
stock_list = ["Microsoft","Apple","Google"]
for i in stock_list:
param_dict = {"stock_filter": f"vf_Stock={i}"}
png = conn.query_view_image(view_id=id, parameter_dict=param_dict)
with open(f"{i}.png","wb") as file:
file.write(png.content)
In this example, conn is a Tableau connection I've already established (seems to work). The dashboard filter (Stock) has no values with special characters - therefore no translation (e.g., parse.quote()) required, and the workbook/dashboard id is found earlier in the script using querying.get_views_dataframe.
The script produces an output, but it is printing one view (e.g., Stock filter = Microsoft) to all the files. Why am I not able to retrieve the other filtered views? Am I missing something?
hello I pushed some rows into a bigquery table as follows:
errors = client.insert_rows("course-big-query-python.api_data_set_course_33.my_table_aut33",[string_tuple], selected_fields = schema2)
assert errors == []
however when I verify the result at the visual interface I see that the actual table size is 0,
I verify the Streaming buffer statistics there is the table successfully inserted:
I also excecuted a query to the table and the result is appearing stored in a temporal table as follows:
So I would like to appreciate support to insert the table in the corresponding place rather than a temporary table
To load data in BigQuery, you can either stream or batch it in.
If you choose streaming, data will go straight into a temporal space until it gets consolidated into the table.
You can find a longer description of how a streaming insert works here:
https://cloud.google.com/blog/products/gcp/life-of-a-bigquery-streaming-insert
If you want to batch instead of stream, use jobs.load instead of insert_row.
I have a problem with MySQL Cluster.
I created a table in order to be stored in the disk but i think it's always stored in memory.
I have have created a table with the query below:
CREATE TABLE IF NOT EXISTS user (
`FIELD_KEY` varchar(255) NOT NULL,
`FIELD0` text,
`FIELD1` text,
`FIELD2` text,
`FIELD3` text,
`FIELD4` text,
`FIELD5` text,
`FIELD6` text,
`FIELD7` text,
`FIELD8` text,
`FIELD9` text,
PRIMARY KEY (FIELD_KEY)
) ENGINE=ndbcluster DEFAULT CHARSET=utf8 max_rows=40000000
TABLESPACE ts_1 STORAGE DISK;
ALTER TABLE usertable4M PARTITION BY KEY() PARTITIONS 4
My Environment (3 nodes in the same VM):
MySQL Cluster 7.5
management nodes: 1
data node: 1
sql nodes: 1
VM Config : 16GB RAM, 100GB HDD
MySQL Cluster settings (config.ini) are set to:
[ndbd default]
noofreplicas=1
DataMemory=2048M
# Memory to allocate for index storage
IndexMemory=1024M
# To support multi-thread processors
MaxNoOfExecutionThreads=2
FragmentLogFileSize=256M
NoOfFragmentLogFiles=12
#-------
RedoBuffer=32M
SharedGlobalMemory=256M
DiskPageBufferMemory=4096M
NoOfFragmentLogParts = 4
#-------
[ndb_mgmd]
HostName=192.168.1.10
DataDir=/var/lib/mysql-cluster
[ndbd]
HostName=192.168.1.10
NodeId=2
DataDir=/usr/local/mysql/data
[mysqld]
HostName=192.168.1.10
In order to store data in Disk i created a LOGFILE and TABLESPACE with the queries below:
CREATE LOGFILE GROUP lg ADD UNDOFILE 'undo1.dat'
INITIAL_SIZE=12288M
UNDO_BUFFER_SIZE=32M
ENGINE=NDB;
CREATE TABLESPACE ts_1
ADD DATAFILE 'data1.dat'
USE LOGFILE GROUP lg
INITIAL_SIZE=256M
ENGINE=NDB;
ALTER TABLESPACE ts_1 ADD DATAFILE 'data2.dat' INITIAL_SIZE=256M ENGINE=NDB;
.
.
.
ALTER TABLESPACE ts_1 ADD DATAFILE 'data24.dat' INITIAL_SIZE=256M ENGINE=NDB;
MemoryUsage report
$ sudo ndb_mgm -e 'all report MemoryUsage'
Connected to Management Server at: 192.168.1.10:1186
Node 2: Data usage is 94%(62259 32K pages of total 65536)
Node 2: Index usage is 2%(2801 8K pages of total 131104)
When I ran a query to insert 4000000 records (4GB) to my table I get the error (The table 'user' is full) and can not insert data any more to the table and only 1611787 records has been inserted . I don't understand why it is so.
Could somebody explain me what is the situation. And how can I resolve the problem.
"In a Disk Data table, the first 256 bytes of a TEXT or BLOB column are stored in memory; only the remainder is stored on disk."
You can check this in detail.
A couple days ago I've been working with an sqlite3 database and noticed that after deleting a lot of data (an entire table (DROP) to be exact) the file size did not change (the table used 5MB) at all. Not the database I'm asking about, but maybe this is relevant to my question.
I'm having a bunch of databases that I was looking through using grep -a for a specific message, I successfully found the database in question having 50MB in size, seems perfect! However, opening the database in an sqlite3 database viewer (or the sqlite3 command line tool) shows only a little over 800 entries in the messages table. The current auto increment index is above 18.000, which is the amount of entries the table should have. Going through the .db file with grep, everything I want seems to be there, but I'm guessing it's somehow "hidden"(?).
How can I retrieve ALL the data from an sqlite database? Note that .dump <table> does also not include the messages I'm looking for. For a quick explanation on this and why data doesn't actually get deleted/filesize won't shrink when I delete a 5MB table I'd be thankful too.
sqlite> .dbinfo
database page size: 4096
write format: 1
read format: 1
reserved bytes: 0
file change counter: 366
database page count: 11405
freelist page count: 10372
schema cookie: 2
schema format: 4
default cache size: 0
autovacuum top root: 0
incremental vacuum: 0
text encoding: 1 (utf8)
user version: 7
application id: 0
software version: 3008005
number of tables: 3
number of indexes: 1
number of triggers: 0
number of views: 0
schema size: 737
(Note that I did not create nor ever edited this database, it's Instagram's direct messages database.)
Deleted data is not immediately removed from the database file; see change sqlite file size after "DELETE FROM table".
As shown by the freelist page count, that file is mostly empty, and running VACUUM is likely to throw away the remains of the data you're searching for.
There is no easy way to recover deleted data (because some important management information is missing). If you are interested in only a small amount of data, try a hex editor.
I have data generated for region table is below format.
0|AFRICA|lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to |
1|AMERICA|hs use ironic, even requests. s |
2|ASIA|ges. thinly even pinto beans ca |
3|EUROPE|ly final courts cajole furiously final excuse |
4|MIDDLE EAST|uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl |
I tried several versions of load command to load above data in region table
CREATE REFERENCE TABLE REGION (" +
"R_REGIONKEY INTEGER NOT NULL PRIMARY KEY," +
"R_NAME CHAR(25) NOT NULL," +
"R_COMMENT VARCHAR(152))
1. stmt.execute(s"load data infile \'$filename\' into table REGION COLUMNS TERMINATED BY \'|\'");
2. stmt.execute(s"load data infile \'$filename\' into table REGION FIELDS TERMINATED BY \'|\'");
3. stmt.execute(s"load data infile \'$filename\' into table REGION COLUMNS TERMINATED BY \'|\' LINES TERMINATED BY \'\n\'");
4.stmt.execute(s"load data infile \'$filename\' into table REGION FIELDS TERMINATED BY \'|\' LINES TERMINATED BY \'|\'");
But no success. Help me to format proper load command to load data in given format. Also let me know how can I achieve same using Memsql loader utility.
The default output format from dbgen should be loaded with fields terminated by '|' lines terminated by '|\n'