I'm trying to use DynamoDB locally and am using the container https://hub.docker.com/r/amazon/dynamodb-local combined with AWS Workbench (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.html).
I have successfully created a table on my local container but now I wish to delete it. I'm assuming AWS Workbench has this functionality(?) and I'm just being dumb... can anybody point me at the button I need to press?
Many thanks.
In case anybody else is looking, at time of writing aws workbench does not support the functionality to delete a table. Got my answer straight from the DynamoDb team.
Came across this question while trying to update data from NoSql Workbenck into my local DDB table.
My issue was now knowing how to re-commit/update the data after my first commit to my local docker ddb server as I was getting this error
Error
Could not commit to DynamoDB as table creation failed: ResourceInUseException: Cannot create preexisting table
What worked for me was to:
stop my local instance (ctrl + c)
restart my local ddb server (docker run -p 8000:8000 amazon/dynamodb-local)
and commiting my changes to my local DDB again from NoSql Workbench
Just in case anyone else is trying to solve the same problem and if you haven't tried this yet.
You now can use PartiQL with NoSQL Workbench to query, insert, update, and delete table data in Amazon DynamoDB
Posted On: Dec 21, 2020
However, you cannot still delete the table from dynamodb.
Related
I'm having some issues when trying to export binlog information and mysql dump with --master-data=1 from my Aurora MySQL instance. The error I'm receiving is
"mysqldump: Couldn't execute 'FLUSH TABLES WITH READ LOCK': Access denied for user 'user'#'%' (using password: YES) (1045)"
After some digging I found out that one way to do it is to create a read replica from the master, stop replication then perform the dump.
Sadly this does not work as I expected. In all AWS guides I've found they say to create a read replica from the "Actions" button, but I have no such option, doesn't even appear in the dropdown.
One option does appear, "Add a reader", which I did and after connecting to it, it seems like it's not a replica but more like a master with read only permissions, even if in the AWS console the "replica latency" column for that instance has a value attached to it.
It's a replica but it's not really a replica?
My main question here is how could I perform a dump of an Aurora MySQL in order to start replication on another instance?
I tried following most of the guides that are available from aws regarding mysql replication as well as lots of other stackoverflow questions.
There is an unfortunate overloading of "replica" here.
read replica = some non-Aurora MySQL server (RDS or on-premises) that you replicate to.
Aurora replica = the 2nd, 3rd, and so on DB instance in an Aurora cluster. Read-only. Gets notified of data changes by an entirely different mechanism than binlog replication. Thus this term is phased out in favor of "reader instance", but you will still see it lurking in documentation that compares and contrasts different ways to do replication.
Aurora read replica = a read-only Aurora cluster that gets data replicated into it via binlog from some other source.
If you select an RDS MySQL instance in the console, that has an option to create a read replica, because that's the only kind of replication it can do. Aurora MySQL clusters only have "add reader"* because that is the most common / fastest / most efficient way for Aurora. The instructions here cover all the different permutations:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Replication.MySQL.html
That page recommends using a snapshot as the initial "dump" from the Aurora cluster.
There is also an option "Create cross-Region read replica" for Aurora clusters. But for that capability, it's preferable to do "Add AWS Region" instead - that uses an Aurora-specific mechanism (Aurora global database) to do low-latency replication between AWS Regions.
I've got an existing Cassandra application I want to try in Datastax Astra. I've created my Astra database and am able to connect to it from NodeJS successfully. The next task I have is to apply my existing schema.cql to the keyspace.
It should be easy enough right? But never-the-less I can't see any obvious way to do this. Can someone walk me through it please. I can then I think use the dsbulk tool to upload a dataset.
Thanks in advance Astra experts.
Rod
The easiest way to do it is to cut-and-paste the CQL statements one-by-one into the CQL console and run them one at a time. This is what I would recommend since you have guarantees that each DDL statement has completed successfully before you can execute the next one.
Alternatively, you can download the Standalone CQL Shell to your laptop/desktop and run the cqlsh commands from there. You will need to configure it with your app token + secure connect bundle to be able to connect to your database.
For more info, see the instructions for Installing standalone cqlsh. Cheers!
Converting comment to answer:
Unauthorized('Error from server: code=2100 [Unauthorized]
message="No SELECT permission on <table system_virtual_schema.keyspaces>
What is the role you have created your token for in Astra DB? Otherwise, create a new one as "Database Administrator" and that should be able to SELECT FROM system_virtual_schema.keyspaces.
While creating use-specific roles is a good idea, only privileged roles can run data definition language (DDL) commands.
Pretty new to Databricks.
I've got a requirement to access data in the Lakehouse using a JDBC driver. This works fine.
I now want to stub the Lakehouse using a docker image for some tests I want to write. Is it possible to get a Databricks / spark docker image with a database in it? I would also want to bootstrap the database on startup to create a bunch of tables.
No - Databricks is not a database but a hosted service (PaaS). You can theoretically you can use OSS Spark with Thriftserver started on it, but the connections strings and other functionality would be very different, so it makes no sense to spend time on it (imho). Real solution would depend on the type of tests that you want to do.
Regarding bootstrapping database & create a bunch of tables - just issue these commands, like, create database if not exists or create table if not exists when you application starts up (see documentation for an exact syntax)
I connected via SSH to Dev Endpoint in Glue.
There is Spark 2.4.1 running.
I want to run a simple query select * from pg_namespace;
Also after that, want to move data from S3 to Redshift using COPY command.
How to write that in a Spark console?
Thanks.
Am not sure if you can use COPY command directly, and i haven't tried it.
For moving data from S3 to Redshift, you can use AWS Glue APIs. Please check here for sample codes from AWS? Behind the scenes, I think AWS Glue uses COPY / UNLOAD commands for moving data between S3 and REDSHIFT.
You can use aws cli and psql from your ssh terminal.
For psql check https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-from-psql.html
Then u can run select and copy command from it.
But I will not recommend as AWS Glue is serverless service so your cluster will be different everytime.
We have an aurora database(aws) that we use for production. We would like to have a clone database that will be updated on a daily basis and will be used for qa(one way sync from the production to qa db). What is the best way to do it?
Thanks
There's an open source Python Library that can do this for you, or you could take a look at that approach and do the same:
https://github.com/blacklocus/aurora-echo
You can launch following script daily:
Convert production automatic snapshot to manual: aws rds copy-db-cluster-snapshot
Now you can share your manual snapshot with test account: aws rds modify-db-snapshot-attributes --attribute-name restore --values-to-add dev-account-id
Restore your snapshot to cluster with aws rds restore-db-cluster-from-snapshot
Add instance
Rename db cluster to (it is about 10 seconds)
Rename db cluster to (it is about 10 seconds)
If new cluster works, you can delete cluster with instances.