Following the inputs from the below forum, system properties were specified and customized names for DATABASECHANGELOG & DATABASECHANGELOGLOCK tables were used & setup (on liquibase update execution).
http://forum.liquibase.org/topic/configurable-databasechangelog-table-name
Liquibase version-3.5.1, Database-Oracle 12c, OS-Redhat Linux
But, on subsequent attempts to execute future liquibase updates (against same database schema), the execution fails as its trying to recreate the customized DATABASECHANGELOG table again - failing with Object Name already in use. This does not happen when trying to use the standard liquibase control tables names (i.e. DATABASECHANGELOG & DATABASECHANGELOGLOCK)
Is there an option to skip recreation of customized liquibase control tables OR another fix for this this issue?
Why setting these as system properties?
You can invoke liquibase like the following (or these arguments defined in the properties file):
liquibase <regular arguments > --liquibaseSchemaName=YOUR_SCHEMA \
--databaseChangeLogTableName=YOUR_DBCHANGELOG \
--databaseChangeLogLockTableName=YOUR_DBCHANGELOGLOCK ....
It works fine for us (liquibase 3.5.1, Oracle 12c).
Thanks
The issue was due to the case-sensitivity of the custom table names when executing against oracle databases. Weird error, but its resolved when specifying the value in upper case (via system properties/command line)
Related
I'm using EXEC CICS SYNCPOINT and EXEC CICS SYNCPOINT ROLLBACK to commit/backout updates to VSAM and DB2 tables when abend happens. However, only updates to DB2 tables are backed out not on VSAM. Am I missing something? CICS parameter RLS is set to RLS=NO.
It will depend on the type of files that you are using. If you are using RLS files then you have to define the files correctly using idcams using the LOG parameter see:
https://www.ibm.com/docs/en/zos/2.2.0?topic=cics-recoverable-nonrecoverable-data-sets
If you are using non-RLS files then you need to set the attributes correctly on your FILE definition.
See the following page within the CICS documentation that describes about file recovery:
https://www.ibm.com/docs/en/cics-ts/5.6?topic=resources-recovery-files
I have (several) database project in a solution. In one I have a referesnce to a dacpac (this is ACTUALLY a copy of one of the main databases as we take a SQL snapshot at end of day and some code needs to reference this (DBANME_Daily) rather that DBNAME).
now this builds correctly, the code with SELECT * FROM DBNAME_DAILY.schema.table all compiles and builds with no error.
ON deployment however I get the unresolved reference to DBNAME_DAILY.schema.table
You want to add a dacpac for that database as a reference, using variables for the database. Should be different database, different names. You'd then use that variable in your code and you'd pass in the variable name for your build/publish tasks depending on the environment.
This is a little older article, but still pretty accurate:
http://schottsql.com/2012/10/31/ssdt-external-database-references/
You can tweak that a little bit using a variable for the database name. When I wrote this, it was mostly just a different database with the same name across environments. For your case, you'd just use the DB variable. Then replace your "DBName.schema." with "[$DBNameVariable].schema." (or something similar)
Sorted, my mistake was the dacpac reference would only allow the project to BUILD. For deployment the database DBNAME_DAILY MUST EXIST.
Lesson Learnt.
I have the following problem in Azure Databricks. Sometimes when I try to save a DataFrame as a managed table:
SomeData_df.write.mode('overwrite').saveAsTable("SomeData")
I get the following error:
"Can not create the managed table('SomeData'). The associated
location('dbfs:/user/hive/warehouse/somedata') already exists.;"
I used to fix this problem by running a %fs rm command to remove that location but now I'm using a cluster that is managed by a different user and I can no longer run rm on that location.
For now the only fix I can think of is using a different table name.
What makes things even more peculiar is the fact that the table does not exist. When I run:
%sql
SELECT * FROM SomeData
I get the error:
Error in SQL statement: AnalysisException: Table or view not found:
SomeData;
How can I fix it?
Seems there are a few others with the same issue.
A temporary workaround is to use
dbutils.fs.rm("dbfs:/user/hive/warehouse/SomeData/", true)
to remove the table before re-creating it.
This generally happens when a cluster is shutdown while writing a table. The recomended solution from Databricks documentation:
This flag deletes the _STARTED directory and returns the process to the original state. For example, you can set it in the notebook
%py
spark.conf.set("spark.sql.legacy.allowCreatingManagedTableUsingNonemptyLocation","true")
All of the other recommended solutions here are either workarounds or do not work. The mode is specified as overwrite, meaning you should not need to delete or remove the db or use legacy options.
Instead, try specifying the fully qualified path in the options when writing the table:
df.write \
.option("path", "hdfs://cluster_name/path/to/my_db") \
.mode("overwrite") \
.saveAsTable("my_db.my_table")
For a more context-free answer, run this in your notebook:
dbutils.fs.rm("dbfs:/user/hive/warehouse/SomeData", recurse=True)
Per Databricks's documentation, this will work in a Python or Scala notebook, but you'll have to use the magic command %python at the beginning of the cell if you're using an R or SQL notebook.
I have the same issue, I am using
create table if not exists USING delta
If I first delete the files lie suggested, it creates it once, but second time the problem repeats, It seems the create table not exists does not recognize the table and tries to create it anyway
I don't want to delete the table every time, I'm actually trying to use MERGE on keep the table.
Well, this happens because you're trying to write data to the default location (without specifying the 'path' option) with the mode 'overwrite'.
Like said Mike you can set "spark.sql.legacy.allowCreatingManagedTableUsingNonemptyLocation" to "true", but this option was removed in Spark 3.0.0.
If you try to set this option in Spark 3.0.0 you will get the following exception:
Caused by: org.apache.spark.sql.AnalysisException: The SQL config 'spark.sql.legacy.allowCreatingManagedTableUsingNonemptyLocation' was removed in the version 3.0.0. It was removed to prevent loosing of users data for non-default value.;
To avoid this problem you can explicitly specify the path where you're going to save with the 'overwrite' mode.
Does MemSQL support user variables in the load data command, similar to MySQL (see MySQL load NULL values from CSV data for examples)? The MemSQL documentation (https://docs.memsql.com/docs/load-data) doesn't give a clue, and my attempts at using user variables have failed.
No, variables in LOAD DATA are not currently supported in general (as of MemSQL 5.5). This is a feature we are tracking for a future release.
We only support the following syntax to skip the contents of a column in the file using a dummy variable (briefly mentioned in the docs https://docs.memsql.com/docs/load-data):
load data infile 'foo.tsv' into table foo (bar, #, #, baz);
We have several databases, say DB1, DB2, DB3 etc.
They have to have identical code base, so we use a DB project in Visual Studio 2012 and generate a SQL script for deployment based on comparison between the project and UAT/Prod DB1. Then this script is applied to DB1-DBn.
For the very first time in the history of this DB project I had to create a function that contained a hardcoded database name, example:
inner join DB1.schema.table1 as t1 on
And now the project cannot be built or comparison cannot be updated or script generated (Update and Generate Script buttons disabled) due to a number of errors pertaining to that database reference, as VS seems to believe that DB1 does not exist.
I tried to add a project level SQLCMD variable $(DB) and set it to DB1 default value and use it as
inner join [$(DB1)].schema.table1 as t1 on
to work around the errors, but it did not seem to make any difference.
Edit:
A suggestion was made to add a circular project reference to itself and assign to it the same variable I was trying to add manually, not sure how to accomplish that.
As per this article the reference should be added to a manually extracted .dacpac file as follows:
Extracted .dacpac file from the targed DB with the following command:
"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\SQLDB\DAC\120\sqlpackage.exe" /SourcePassword:p /SourceUser:u /Action:Extract /ssn:192.168.2.1 /sdn:DB1 /tf:DB1.dacpac
Included that as a database reference. It automatically assigned the correct SQLCMD variable name and the error disappeared.
From the source control point, even though when adding a database reference to a .dacpac file automatically creates a SQLCMD variable, it does not add the file to the project. The .dacpac file used still has to be added to the project as an existing item, which is kind of lame. Doing that in the solution explorer I encountered an error and had to do that through the team explorer instead, where that worked.