I am running cadence with cassandra externally running using docker run -e CASSANDRA_SEEDS=10.x.x.x e ubercadence/server:. and its running sucessfully.
Azure cosmos says, any system running on Cassandra can use Azure cosmos using provided cosmos cassandra APi, by modifying the client connection creation code, for example : GO app sample code :
func GetSession(cosmosCassandraContactPoint, cosmosCassandraPort, cosmosCassandraUser, cosmosCassandraPassword string) *gocql.Session {
clusterConfig := gocql.NewCluster(cosmosCassandraContactPoint)
port, err := strconv.Atoi(cosmosCassandraPort)
clusterConfig.Authenticator = gocql.PasswordAuthenticator{Username: cosmosCassandraUser, Password: cosmosCassandraPassword}
clusterConfig.Port = port
clusterConfig.SslOpts = &gocql.SslOptions{Config: &tls.Config{MinVersion: tls.VersionTLS12}}
clusterConfig.ProtoVersion = 4
session, err := clusterConfig.CreateSession()
...
return session
}
From my end, I can connect external cassandra's cqlsh(which cadence is using for persisting) to azure cosmos and can create KeySpace, table in azure cosmo db.
However, when I run Cadence server, all new tables are still created on local cassandra itself(instead of Axure cosmos) might be, cadence is connected to cassandra only.
So there are basically 2 question shared below :
1.Since cadence is written in GO, can we modify the source code to establish connection to AzureCosmoDb.
or
or can we pass the cosmocassandra's host, port, username, password, while running the cassandra and cadence separately (docker run -e CASSANDRA_SEEDS=10.x.x.x e ubercadence/server:)
cosmosCassandraContactPoint : xyz.cassandra.cosmos.azure.com
cosmosCassandraPort : 10350
cosmosCassandraUser : xyz
cosmosCassandraPassword : xyz
I am actively working on supporting other NoSQL DBs: https://github.com/uber/cadence/issues/3514 it will be easier to use Azure cosmos/AWS Keyspace after that's done.
Basically we will just need to customize a small part from the existing Cassandra model.
Related
I'm attempting to connect to a new Aurora PostgreSQL instance with Babelfish enabled.
NOTE: I am able to connect to the instance using the pg library through the normal port 5432 (the Postgres TDAS endpoint).
However, for this test, I am attempting to connect through the Babelfish TDS endpoint (1433) using the standard mssql package.
If I specify a database name (it is correct), I receive the error 'database "postgres" does not exist':
var config = {
server: 'xxx.us-east-1.rds.amazonaws.com',
database: 'postgres',
user: 'xxx',
password: 'xxx'
};
and the connection closes since the connection fails.
if I omit the database property in the config, like:
var config = {
server: 'xxx.us-east-1.rds.amazonaws.com',
user: 'xxx',
password: 'xxx'
};
It will connect. Also, I can use that connection to query basic things like SELECT CURRENT_TIMESTAMP and it works!
However, I can't access any tables.
If I run:
SELECT COUNT(1) FROM PERSON
I receive an error 'relation "person" does not exist'.
If I dot-notate it:
SELECT COUNT(1) FROM postgres.dbo."PERSON"
I receive an error "Cross DB query is not supported".
So, I can't connect to the specific database directly and if I connect without specifying a database, I can't cross-query to the table.
Any one done this yet?
Or, if not, any ideas on helping me figure out what to try next? I'm out of ideas.
Babelfish databases (that you connect to on port 1433) have nothing to do with PostgreSQL databases (port 5432). Essentially, all of Babelfish lives within a single PostgreSQL database (parameter babelfishpg_tsql.database_name).
You seem to have a single-db setup, because Cross DB query is not supported. With such a setup, you can only have a single database via port 1433 (apart from master and tempdb). You have to use CREATE DATABASE to create that single database (if it isn't already created; ask sys.databases).
I can't tell if it is supported to create a table in PostgreSQL (port 5432) and use it on port 1433 (the other way around is fine), but if so, you have to create it in a schema that you created with CREATE SCHEMA while connected on port 1433.
The answer was that I should be connecting to database "master".
Even though there is no database titled master in the instance, you still do connect to it.
Once connected, running the following:
select current_database();
This will indicate you are connected to database "babelfish_db".
I don't know how that works or why a database would have an undocumented alias.
The bigger answer here is that cross-DB object references are not currently supported in Babelfish, outside your current SQL Server database.
This is currently being worked on. Stay tuned.
I made an test account in datastax (https://astra.datastax.com/) and want to test cassandra.
In there homepage is an cqlsh console. If I select datas is goes very fast maybe 1ms.
If I use it with nodejs and cassandra driver it takes 2-3 seconds. And I have only ONE row.
Why it takes time? Its my code fault?
const { Client } = require("cassandra-driver");
async function run() {
const client = new Client({
cloud: {
secureConnectBundle: "secure-connect-weinf.zip",
},
keyspace: 'wf_db',
credentials: {
username: "admin",
password: "password",
},
});
await client.connect();
// Execute a query
const rs = await client.execute("SELECT * FROM employ_by_id;");
console.log(`${rs}`);
await client.shutdown();
}
// Run the async function
run();
Unfortunately, it's not an apples-for-apples comparison.
Every time your app connects to a Cassandra cluster (Astra or otherwise), the driver executes these high-level steps:
Unpack the secure bundle to get cluster info
Open a TCP connection over the internet
Create a control connection to one of the nodes in the cluster
Obtain schema from the cluster using the control connection
Discover the topology of the cluster using the control connection
Open connections to the nodes in the cluster
Compute query plan (list of hosts to connect to based on load-balancing policy)
And finally, run the query
In contrast when you access the CQL Console on the Astra dashboard, the UI automatically connects + authenticates to the cluster and when you type a CQL statement it goes through the following steps:
Skipped (you're already authenticated to the cluster)
Skipped (it's already connected to a node within the same local VPC)
Skipped (already connected to cluster)
Skipped (already connected to cluster)
Skipped (already connected to cluster)
Skipped (already connected to cluster)
Skipped (already connected to cluster)
And finally, run the query
As you can see, the CQL Console does not have the same overhead as running an app repeatedly which only has 1 CQL statement in it.
In reality, your app will be reusing the same cluster session to execute queries throughout the life of the app so it doesn't have the same overhead as just re-running the app you have above. The initialisation phase (steps 1 to 6 above) are only done when the app is started. Once it's already running, it only has to do steps 7 and 8. Cheers!
I am trying to use a VM machine with R on Azure. I want to connect it to a mariaDB on Azure.
I use the RmySQL package, and I use the following connection string:
require(RMySQL)
con <- dbConnect(RMySQL::MySQL(),
dbname="energidb",
host="energidb.mariadb.database.azure.com",
port=3306,
user="USER",
password="MY_PWD")
However, No luck I have in trying this, as R says. "Error in .local(drv, ...) :
Failed to connect to database: Error: Unknown database 'energidb'"
On azure, I promise and guarentee with my life, the name of the mariaDB is "energidb"
What am I doing wrong?
If you just create a Maria DB resource on Azure portal, then you should only have a database server.
You can connect to the Maria DB without specifying a database name. And then list all the existing databases as following:
con <- dbConnect(RMySQL::MySQL(),
host="your host address, jackmariadb.mariadb.database.azure.com",
port=3306,
user="user name, jack#jackmariadb",
password="password, d*******5")
rs = dbSendQuery(con, "show databases")
fetch(rs, n=-1)
In my case, it will just show the 3 system databases:
It turns out that on Azure, I created a mariaDB server (without any database). The database reference i made was to the server, which ofcourse is meaningless.
The solution is first to create a database (which is done by pointing at the server). From there I can point on the database.
I'm trying to connect to the OpenEdge database so that I can perform queries on it. However, I'm having trouble figuring out how to connect to it to make these queries.
So far I've tried making a config file, but I'm unsure how to establish a connection to the server using it:
"HMMv10": {
"dbConfig": {
"connectionString": "DRIVER={Progress OpenEdge 11.3 Driver}",
"UID": "SYSPROGRESS",
"pwd": "***",
"host": "host.local",
"port": "18210",
"db": "hmm10"
}
}
I don't know what that config file goes with or how it works but before trying to setup 3rd party software to connect to an OpenEdge database it is a really good idea to use the provided OE tools to verify that you have a properly configured connection available to connect to.
Progress provides a command line tool called "sqlexp" that you can use to test connections. The easiest way to access that tool is via a "proenv" command shell. On Windows just open "proenv" (it is installed with the other Progress commands in the "Progress" program group). If you have a Linux or UNIX install you run $DLC/bin/proenv.
Then start sqlexp like this (I used my local dbname and port number):
proenv> sqlexp -db s2k -H localhost -S 9500 -user sysprogress -password sysprogress
OpenEdge Release 11.7.2 as of Tue Oct 24 18:20:59 EDT 2017
Connecting user "sysprogress" to URL "jdbc:datadirect:openedge://localhost:9500;databaseName=s2k"... (8920)
SQLExplorer>
If there are no errors and you get the SQLExplorer> prompt that is sufficient to show that there is a working connection available.
If you would like to go the extra mile and prove that you have been granted permission to fetch data try a simple select statement:
SQLExplorer> select count(*) from pub.customer;
count(*)
--------------------
1117
SQLExplorer> quit;
proenv>
If this does not work then the local DBA has not configured SQL access, or you have incorrect configuration/credentials and no amount of working with your other tool will fix that.
OpenEdge SQL notes:
If you are not already aware - all OpenEdge data is variable length. Many SQL tools expect that the width of a field is known use metaschema data to get a default value. If the local DBA has not been maintaining that data via "dbtool" then you are likely to have errors due to data being wider than expected.
sqlexp expects commands to end with ";".
The "pub" schema is the schema where OpenEdge application tables live. Table names need to have "pub." pre-pended.
Table and field names that contain "-" will need to be quoted.
I'm putting up what I found to be the answer just in case anyone else is looking for this. Using a REST API in NodeJS I used npm-odbc and the OpenEdge odbc drivers to connect to the database. Even though the npm-odbc package only mentions Unix it does in fact work on windows.
I am using HAProxy to for AWS RDS (MySQL) load balancing for my app, that is written using Flask.
The HAProxy.cfg file has following configuration for the DB
listen mysql
bind 127.0.0.1:3306
mode tcp
balance roundrobin
option mysql-check user haproxy_check
option log-health-checks
server db01 MASTER_DATABSE_ENDPOINT.rds.amazonaws.com
server db02 READ_REPLICA_ENDPOINT.rds.amazonaws.com
I am using SQLALCHEMY and it's URI is:
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://USER:PASSWORD#127.0.0.1:3306/DATABASE'
but when I am running an API in my test environment, the APIs that are just reading stuff from DB are executing just fine but the APIs that are writing something to DB are giving me errors mostly that:
(pymysql.err.InternalError) (1290, 'The MySQL server is running with the --read-only option so it cannot execute this statement')
I think I need to use 2 URLs now in this scenario, one for read-only operation and one for writes.
How does this work with Flask and SQLALCHEMY with HAProxy?
How do I tell my APP to use one URL for write operations and other HAProxy URL to read-only operations?
I didn't find any help from the documentation of SQLAlchemy.
Binds
Flask-SQLAlchemy can easily connect to multiple databases. To achieve
that it preconfigures SQLAlchemy to support multiple “binds”.
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://USER:PASSWORD#DEFAULT:3306/DATABASE'
SQLALCHEMY_BINDS = {
'master': 'mysql+pymysql://USER:PASSWORD#MASTER_DATABSE_ENDPOINT:3306/DATABASE',
'read': 'mysql+pymysql://USER:PASSWORD#READ_REPLICA_ENDPOINT:3306/DATABASE'
}
Referring to Binds:
db.create_all(bind='read') # from read only
db.create_all(bind='master') # from master