We have events data stored in clickhouse DB, for querying and visualization we are using superset and trino being used for query processing and combining data sources.
Beside clickhouse we have postgresql which is working fine, however in clickhouse datasource we are seeing string fields as base64 encoded, when query clickhouse directly using datagrip it works fine but when trying to do via superset -> trino it is encoded.
Any ideas how this could be fixed?
Superset is using trino (sqlalchemy)
and this is how its defined in trino
connector.name=clickhouse
connection-url=jdbc:clickhouse://10.x.x.x:8123/
connection-user=default
connection-password=
Trino has the fields by default considered as varbinary by setting the connection property
clickhouse.map-string-as-varchar=true
Fixed the mapping issue to use String and FixedStrings
https://trino.io/docs/current/connector/clickhouse.html
Related
hello I would like to know if it is possible to store data in a database with GraphQL using python without going through mongodb or sql?
Yes. GraphQL doesn't care about the underlying database. You can connect to a csv file, a JSON file, a REST interface whatever.
By default Elasticsearch seems to query its own database in the indexes defined during the search.
Is it possible that Elasticsearch is not querying its database but mine in PostgresSql?
No.
Elasticsearch is a database on its own rights, it's not an interface/middleman for other backends.
If you want to conditionally query different databases, you need to implement that logic at application level.
I know it is better to convert xml to json to store it in CosmosDb especially to query documents using SQL API. But will it be OK to flatten xml data to store it inside of a document then query them using SQL API? Not even sure if SQL API support xml query or not.
The simple answer is NO :(
The CosmosDB is stores JSON documents:
https://learn.microsoft.com/en-us/azure/cosmos-db/introduction
Depending the choosen API, you can handle these JSON documents in graph model (Gramlin), or in document collection (MongoDb, DocumentDB). Or SQL etc... But the result is always an JSON document.
But there are a lot of tools to convert XML to JSON and convert back. Here is a discussion about it:
How to convert JSON to XML or XML to JSON?
I hope it helps.
Regards
gy
Per my knowledge, it is not possible. More information for your reference: Query Azure Cosmos DB data with SQL queries
I'm interacting with MongoDB with several drivers including PyMongo and Node.js Native MongoDB driver.
What I noticed is that when I do fs.put for GridFS storage from PyMongo it only sets few fields including UploadDate, but when I do fs.put for GridFS storage from Node.JS it also inserts additional fields to storage that I don't need including ContentType.
That may be good practice, but I dont want this field, how do I configure MongoDB insert to ignore insertion of this field when adding file to GridFS from Node.JS? According to docs you can either set it or it will use default ContentType (content_type).
How to not insert this field it all?
Well, seems like there's no support for this feature. So I just commented stuff for fields contentType and aliases in MongoDB Driver library for Node.JS. Its bad practice, but unfortunately there's no another option for that right now.
I am new to couchDB but have a good experience working with relational databases. Can anyone tell how to connect to couchDB database and retrieve the data stored in it. I am giving an example in relational database and i need help regarding how to do similar task in couchDB.In mysql we use a connector to get connected to the database and the for example we give "select username from tablename where password="abc" ".
CouchDB talks HTTP and JSON, then you can use any HTTP client and JSON parser/generator. You can find a nice introduction in The Definitive Guide.
Try this URL: http://localhost:5984/_utils/, it will open FUTON editor.
CouchDB is a NOSQL database. So it works using HTTP requests (url based). Data that is stored in couchDB is in the form of JSON documents, so there is no concept of tables. In short, database in SQL represent database in couchDB and the rows in a table of SQL represent Documents in couchDB.
Coming back to your question, to retrieve data from couchDB, there is a concept called views which uses Map and Reduce functions (which are JavaScript functions). Using these views couchDB indexes your search function spanning through the complete database (includes all documents), so you need to write a Map function specifying the condition to be used to search. Here's an example -
function(doc) {
if (doc.password) {
emit(doc.username, doc);
}
}
The above example is a simple Map function. Search for the documents of the database where there is a password and return the usernames from all the documents in the database. Password input value (in this case "abc") should be specified in the query string that you will be sending out to couchDB URL. Now, you might ask where is the database specified to search for? I said that we have to create views in order to search. These views are stored in that particular database where you want to search. So, if you want to search a database with name "User_Credentials", then create a view in the "User_Credentials" with the above Map function. More details on how it can be done can be found here: CouchDB Guide to Views