I have got a web service which is looking for the last create table
[name_YYYYMMddHHmmss]
I have a persister job that creates and loads a table (insert or bulk)
Is there something that hides a table until it is fully loaded ?
First, I have created a technical table, it works but I will need one by keyspace (using cassandraAuth). I don’t like this.
I was thinking about tags, but it doesn’t seem to exist.
- create a table with tag and modify or remove it when the table is loaded.
There is also the table comment option.
Any ideas?
Table comment is a good option. We use it for some service information about the table, e.g. table versions tracking.
Related
Does anyone know a way to force a child table to update from the source table? it would be a one off command to run when the child table is created, then we have an auto update policy in place.
Googling suggests to try this, however it produces a syntax error
.update policy childTable with (sourceTable)
Thanks!:)
Update policy is a mechanism that works during the ingestion and does not support backfill.
You can consider using materialized view with backfill property (if your transformation logic falls under the limitations) or create the child table based on a query, using the .set command.
If your source table is huge you might need to split it to multiple commands.
We had to use this:
.append childTable <| updateFunction
could you please tell me if this command could create problems with overwriting all tables in the DB:
df.write.option(“path”, “path_to_the_db/hive/”).mode(overwrite).saveAsTable("result_data")
table_name is a new table in the DB, it hasn't existed.
After these commands, all tables disappeared.
I was using Spark3 and tried to solve an error:
Can not create the managed table('result_data').
The associated location('dbfs:/user/hive/warehouse/result_data') already exists.
I expected that a new table will be created without any issues if it doesn’t exist.
If path_to_the_db/hive contains other tables, then you overwrite into that folder, it seems possible that the whole directory would be emptied first, yes. Perhaps you should instead use path_to_the_db/hive/result_data
According to the error, though, your table does already exist.
You can use Spark to register a temporary table in SQL code, then run INSERT OVERWRITE query for existing tables.
All over the place you read a managed table is a table that is created in the default location (/user/hive/warehouse/)
But Databricks_own_examples_in_documentation creates a managed table in /user/blabla/bla
So what TRULY constitutes a managed table?
It certainly isn't simple anything created on the default database.
The remark, "A managed table is just something we create without the 'LOCATION' keyword" is ... not exactly correct.
Is it anything written in /user?
Is it any table that reads a source on /user?
.. If this is true then when I read a file from /mnt/mymount/fil.csv to create tmpView then
create table myTable as select * from tmpView
to re-write it to the default managed location, why is myTable still an external table?
What REALLY defines a managed table?
There it is.
If_I_create_a_DATABASEwith_a_LOCATION_value
then every table in this database I create without a LOCATION values is a managed table.
But the table will be a subdirectory of a database's location regardless of the cluster's default location for the user Hive warehouse.
Certainly a small but useful nuance.
My question is pretty simple. I am working on Kentico 9 with its SQL Server database which contains several tables which had been added directly from the SQL Management Studio by an external contractor. The fact is that those tables are being used to store custom content which will be displayed for a site, but, in the code they don't have the code for making queries. I mean, they don't have Info and Provider classes.
https://docs.kentico.com/display/K82/Retrieving+database+data+using+ObjectQuery+API
According with this, all tables into the Kentico database can be accessed by invoking methods on these classes, but I don't have it this time.
Something like this, it will not work if I use my table name:
var user = UserInfoProvider.GetUserInfo("administrator");
var items = CustomTableItemProvider.GetItems("MyTable")
.TopN(10)
.WhereEquals("ItemCreatedBy", user.UserID)
.OrderBy("ItemCreatedWhen");
My question is:
can I query any table by its name?
One last thing:
I cannot declared those table as "custom table" because it seems to be a bug in the CMS.
Or you can pull data using your own SQL query:
var ds = ConnectionHelper.ExecuteQuery("select ....", null, QueryTypeEnum.SQLQuery);
Nevertheless I would recommend to create a custom class inside a custom module (much more robust than custom tables) instead and use the generated Info and InfoProvider classes to get and manipulate data.
I think an object has to be registered within the system (created through Kentico UI or API) in order to be pulled from DB with object query.
So I'd choose one of the following options:
Use Entity Framework or something similar to work with that data
Create appropriate custom tables or even custom module and push data there. Not sure why you can't create a custom table... What is an error you're getting?
If you need to present data on the UI only (without processing on the back end) - use just custom queries
Hope this helps.
If you are accessing in code then you could do it the good old fashioned way. If you want to pull data from the database to display on the website you could also do so by creating a custom query and using a transformation to display the fields, then use a repeater on the page to display the transformed data. Alternatively you can use a SQL datasource with a basic repeater, but you still have to create a transformation to display the data. Both methods allow you to access the data in the tables from within the CMS UI, no need to touch any code behind.
If your objective is to read data from these database tables to transform on webpage e.g. using CMS Repeater webpart, you can simply create custom query(s) in Kentico itself and load data using it. You can find the detail here on how to create custom custom queries and load data using it.
On the other hand you can also write your custom classes and define the custom methods where you can pull data using your own SQL query like this:
var ds = ConnectionHelper.ExecuteQuery("select ....", null, QueryTypeEnum.SQLQuery);
Lastly I don't think there should be any issue to create custom table instead of those direct DB tables, only thing we have to ensure code name of custom table should be unique means don't try to use exact same name because it'll cause exception due to same table name already exist in DB. You can please share exception you getting while creating custom table so that I can help you out further.
I use Hibernate in J2ee, when the datebase has't table the framework can create table and when fields changed it can update datebase, Can orm2 be done?
As this wiki pages explains, orm2 can create the database tables if they do not exist. Update of an database table is possible by another library, however i haven't tried this: https://github.com/dresende/node-orm2/issues/103