Is it possible to have a folder or database with a database in Azure Databricks? I know you can use the "create database if not exists xxx" to get a database, but I want to have folders within that database where I can put tables.
Thanks.
If I correctly understand your question you want to have a database stored in specific location? Then you can use CREATE DATABASE with explicit LOCATION parameter (see docs), like this:
CREATE DATABASE IF NOT EXISTS <name>
LOCATION 'path to folder';
And then all data for tables in this database will be stored under given location.
P.S. You can also specify path for table explicitly, when writing, using the option path:
df.write.format("delta").option("path", "path_to_table")\
.saveAsTable("database.table")
Related
I am using nodejs, Mongoose and mongodb database.
Sometimes I have seen that if I drop a database that included some collections inside it, when I run my node which makes some collections inside the database ( with the same name ), then the information that I want to save as a new entity does not show correctly.
I need to also mention when I drop the database in mongo db shell I also manually delete all the catch files inside c:/data/db ( default mongo db folder).
I checked the mongo documentation, sometimes it could make a problem if you make a database with the same name of the deleted database. Located at https://docs.mongodb.com/manual/reference/method/db.dropDatabase/#db.dropDatabase
You answered your own question. The document you have linked describe the solution:
Warning: If you drop a database and create a new database with the same name, you must either restart all mongos instances, or use the flushRouterConfig command on all mongos instances before reading or writing to that database. This action ensures that the mongos instances refresh their metadata cache, including the location of the primary shard for the new database. Otherwise, the mongos may miss data on reads and may write data to a wrong shard.
I have created a spotfire visualization and saved it . However when it is tried accessing through web link I am getting below error
Some part of the data table "XXXXxxxxx" could not be loaded. A data source may be missing or has been changed.
I have done below settings :
1) The data is loaded as "Linked to source".
2) Data connections Properties -->Data connections Settings -->data Source--> --> Credentials--> credentials is given (Profile credentials has been shared)
3) I have used an Stored procedure and it is been created under the database which has spotfire access (including schema).
Please help me to solve the issue.
You mentioned that you accessed the DXP via a web link, which suggests to me that you are using WebPlayer. WebPlayer will not have access to files that are stored locally. WebPlayer will load data remotely using database connections and Information Links, however.
I assume that the data load properly when you load the analysis using the desktop client?
A workaround would be to save a copy of the file for access in WebPlayer and select "Stored Data" (i.e. embedded) in the "Change all applicable data sources to:" dropdown.
This issue has been resolved . This is due to the temp tables used in the query, it acts weirdly when the temp table used in the stored procedure (as per my observation)
I'm confused how to explain myself well here is my scenario.
I have a database named "database1" and i have used typed dataset in visual studio and have added more than 200 stored procedures in the table adapters.
it is a desktop based application. now i want to deployee the same software in other school same database but i have changed the database name
when i generate new database from query and write all the stored procedures in the database and change the database name in the connection string it doesn't work.
I'd recommend you don't change the database name, or if it's something specific to the first client (like ParisTechnicalCollegeDatabase), change it now to something generic (SchoolManager) so you never have to change it again. There's no specific problem I can think of regards reusing a typed dataSet on a different database: I do it daily on database that aren't clones of each other. Ensure your second server is set up with a user and default schema that is specified in the connection string. The problem will eithe per be faulty connection string or incorrect database setup, not the fault of the dataSet
For more targeted help, post up the error messages that appear when you try to run your app on the new database
I am trying to export a bacpac from Azure and restore it locally on SQLEXPRESS 2016. When I try to restore it though I get the following errors from the Import Data-tier Application wizard in SSMS:
Could not import package.
Warning SQL72012: The object
[TestBacPacDB_Data] exists in the target, but it will not be dropped
even though you selected the 'Generate drop statements for objects
that are in the target database but that are not in the source' check
box.
Warning SQL72012: The object [TestBacPacDB_Log] exists in the
target, but it will not be dropped even though you selected the
'Generate drop statements for objects that are in the target database
but that are not in the source'
Error SQL72014: .Net
SqlClient Data Provider: Msg 33161, Level 15, State 1, Line 1 Database
master keys without password are not supported in this version of SQL
Server. Error SQL72045: Script execution error. The executed script:
CREATE MASTER KEY;
After some digging I found that a credential and master key have been added to the database. The credential name references a blob storage container, so I'm thinking maybe auditing was set up at some point with the container as an external resource or something similar.
I would like to delete this credential so I can restore the database locally, but the database throws an error stating that it is in use. I've tried disabling the logging in Azure, but the credential still can't be deleted.
I know sometimes it takes time for Azure to shut down resources, so maybe that's the cause, but I was wondering if anyone else has had a similar problem.
I'm trying to avoid having to set a password for the master key, since I don't care about the credential locally as in this question: SSMS 2016 Error Importing Azure SQL v12 bacpac: master keys without password not supported
Ultimately, we ended up creating a master key. In order to restore our databases locally in this way, we create the database by hand first in SSMS, then add a master key to it. This allows the data import to work correctly.
I had exactly the same problem, and tried a myriad of potential fixes found all over the place. Most were relating to rekeying the system, making a copy first, etc... and absolutely nothing worked.
As insane as this is, the only way I could finally get around it was manually editing the internal structure:
Take the bacpac from original source or copy, anywhere
Rename to .zip, and uncompress the folder structure
Edit "model.xml", search for anything to do with "master key" and / or "shared access signature" and delete the corresponding nodes
Calculate the Sha-256 checksum for the now modified model.xml
Replace the checksum at the bottom of "Origin.xml"
Rezip all the files and rename back to xxx.bacpac
Import onto a local system as you normally would
I finished setup for the making Azure hub and installing Client Agent and Database .
Then define dataset.
That time whatever database i chose and click get latest schema, got the error.
Error is
The get schema request is either taking a long time or has failed.
When check log ,it said like below :
Getting schema information for the database failed with the exception "There is already an open DataReader associated with this Command which must be closed first.
For more information, provide
tracing id ‘xxxx’ to customer support.
Any idea for this?
the current release has maximum of 500 tables in sync group. also, the drop down for the tables list is restricted to this same limit.
here's a quick workaround:
script the tables you want to sync
create a new temporary database and run the script to create the tables you want to sync
register and add the new temporary database as a member of the sync group
use the new temporary database to pick the tables you want to sync
add all other databases that you want to sync with (on-premise databases and hub database)
once the provisioning is done, remove the temporary database from the sync group.