I would like to know where, if it is possible, I can configure default catalog and server values to use when executing the presto CLI.
Presto CLI info:
ls -lthr /opt/presto-server-0.169/presto
/opt/presto-server-0.169/presto -> presto-cli-0.169-executable.jar
And instead of executing:
/opt/presto-server-0.169/presto --server localhost:6666 --schema abc --catalog catalog-1
I would like to execute:
/opt/presto-server-0.169/presto
with it picking up localhost:6666 as my server and catalog-1 as my catalog. I would like to specify the schema once I make the connection.
Any help will be appreciated!
Thanks.
There is no such option to set host in console lazily. The server needs to be defined upfront by default localhost:8080 is used.
If you cannot pass proper arguments to the presto-cli and cannot use the default server host, you can change default values in presto-cli source code and compile your version.
You need to checkout project at github.
Change default values in ClientOptions.
Package jar for presto cli: cd presto-cli && mvn package
You can find a jar in target/presto-cli-0.201-SNAPSHOT.jar
For schema/catalog, you can define it in the console itself with USE command. The syntax as follows: USE [<catalog>.]<schema>.
Please note that with each version of presto you need also compile and maintain your own version of presto-cli, which might become a burden quite soon.
Related
I am trying to load a csv into a database in DataStax Astra using the DSBulk tool.
Here is the command I ran minus the sensitive details:
dsbulk load -url D:\\App\\data.csv -k data -t data -b D:\\App\\secure-connect-myapp -u username -p password
Here is the error I get back:
Operation LOAD_20221206-004421-512000 failed: Invalid bundle: missing file config.json.
Here is the full log:
2022-12-06 00:44:21 INFO Username and password provided but auth provider not specified, inferring PlainTextAuthProvider
2022-12-06 00:44:21 INFO A cloud secure connect bundle was provided: ignoring all explicit contact points.
2022-12-06 00:44:21 INFO A cloud secure connect bundle was provided and selected operation performs writes: changing default consistency level to LOCAL_QUORUM.
2022-12-06 00:44:21 INFO Operation directory: C:\Program Files\dsbulk-1.10.0\bin\logs\LOAD_20221206-004421-512000
2022-12-06 00:44:21 ERROR Operation LOAD_20221206-004421-512000 failed: Invalid bundle: missing file config.json.
java.lang.IllegalStateException: Invalid bundle: missing file config.json
at com.datastax.oss.driver.internal.core.config.cloud.CloudConfigFactory.createCloudConfig(CloudConfigFactory.java:114)
at com.datastax.oss.driver.api.core.session.SessionBuilder.buildDefaultSessionAsync(SessionBuilder.java:876)
at com.datastax.oss.driver.api.core.session.SessionBuilder.buildAsync(SessionBuilder.java:817)
at com.datastax.oss.driver.api.core.session.SessionBuilder.build(SessionBuilder.java:835)
at com.datastax.oss.dsbulk.workflow.commons.settings.DriverSettings.newSession(DriverSettings.java:560)
at com.datastax.oss.dsbulk.workflow.load.LoadWorkflow.init(LoadWorkflow.java:145)
at com.datastax.oss.dsbulk.runner.WorkflowThread.run(WorkflowThread.java:52)
The error says that config.json is missing, but it isn't. So I'm stuck. Unless it's looking somewhere other than in the bundle I specified, but the bundle definitely has the config.json file.
This error:
...
java.lang.IllegalStateException: Invalid bundle: missing file config.json
at com.datastax.oss.driver.internal.core.config.cloud.CloudConfigFactory.createCloudConfig(CloudConfigFactory.java:114)
...
indicates that the Java driver bundled with DSBulk is unable to connect to your Astra DB because it couldn't get the configuration details from the secure connect bundle.
Please make sure that the valid secure bundle ZIP is accessible to DSBulk. You need to provide the path to the ZIP file, not just the directory. For example:
$ dsbulk ... -b /path/to/secure-connect-db.zip ...
Please check the path in your command then try again. Cheers!
👉 Please support the Apache Cassandra community by hovering over the cassandra tag above and click on Watch tag. 🙏 Thanks!
In order for you to leverage DataStax Bulk Loader (aka DSBulk, in short), you would need to pass in the secure connect bundle (SCB) correctly. What I mean when I say is that you need either the fully qualified path or the relative path to the SCB file.
The correct command in your case would look like:
./dsbulk load -url 'D:\\App\\data.csv' -k data -t data -b 'D:\\App\\secure-connect-myapp.zip' -u username -p password
Note that -b option takes in the full SCB filename along with .zip file extension.
Other Resources:
Load data using DSBulk into DataStax Astra DB
-b command-line option reference
BONUS TIP: One could easily configure everything within a configuration file and leverage that. See documentation for additional details.
Under Ubuntu environment, NodeJS Google Vision complains:
Error: Unable to detect a Project Id in the current environment.
Even though I already put json credential through
$ export GOOGLE_APPLICATION_CREDENTIALS=/var/credential_google.json"
Please help.
As a quick hack you can try this :
$ GOOGLE_APPLICATION_CREDENTIALS="/var/credential_google.json" node app.js
It's not recommended to use a .json config file locally. I've seen these leak on production servers causing whole platforms to be deleted + the introduce environmental switching and security issues.
Setup Google Cloud CLI.
Now the server will 'look' at the local environment and use that.
If you get the error "Unable to detect a Project Id in the current environment.", it means the auth library cannot find the project default id.
You need to have a base project in Google Cloud set, regardless of environmental variables and project you're running.
Run
gcloud config set project [some-project-id]
Now if you run (node example)
"dev": "NODE_ENV=dev GCP_PROJECT=some-project-id nodemon index.ts",
It will load the project environment. This also allows you to deploy easier with:
"deploy:dev": "y | gcloud app deploy --project some-dev-project app.yaml",
"deploy:prod": "y | gcloud app deploy --project some-prod-project app.yaml"
App engine has security setup automatically with standard environments. With flex you can use one of the manage images Google Provides.
If you are usually a windows user and trying out Ubuntu (like me), the problem is likely with the assumptions that the export command exports variable to all terminal sessions and that you need to open a new terminal to get it to use (as expected in a windows terminal for an environment variable).
The export command doesn't export the variable to another terminal session. So if you export it in a terminal, you use it on the same terminal.
If you would like to export it permanently, then you can try the solution listed here
You can put the path to the JSON credentials directly when instantiating the client, by passing it as an argument.
For example:
const client = new speech.SpeechClient( {keyFilename: "credential_google.json"});
Also, for me setting it in the terminal didn't work.
If I start the Cassandra service everything is ok, but when I try to start Cassandra on the foreground using "cassandra -f" I get the following error:
Error: Could not find or load main class
Files\DataStax-DDC\apache-cassandra.logs.gc.log
Do I need to configure anything in particular to run Cassandra in the foreground?
Looks like the space in your "Program Files" directory is not escaped in your CASSANDRA_HOME environment variable. It gets set in your cassandra-env.ps1 (in conf/) config file, you could manually set it.
Deploy the presto on single node . when running Presto-cli ,i got following errors:
presto:default> select * from test1;
Query 20131116_233859_00005_5a2yh failed: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe org.apache.hadoop.hive.contrib.serde2.RegexSerDe does not exist)
hive is Operating normally. why prestodb was failed?
my profile:
export JAVA_HOME=/usr/java
export JRE_HOME=/usr/java/jre
export HADOOP_HOME=/usr/hadoop
export HIVE_HOME=/usr/hive
export PRESTO_HOME=/usr/presto
export CLASSPATH=:.:$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME:`find /usr/hadoop -name '*.jar' | grep -v 'test' | grep -v 'example' | perl -e '#jars=<STDIN>;chomp #jars; print join(":",#jars);'`:$PRESTO_HOME/lib:$HADOOP/lib:$HIVE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PRESTO_HOME/bin:$HIVE_HOME/bin
export HADOOP_HEAPSIZE=4096
I got it,when the table of hive was created by RegEx, the presto-cli running failed.
I had no idea. somebody help me please!
On Master node:
Place your RegexSerDe.jar into hive connectors plugin directory related to your hadoop distribution. (Ex: for hadoop2 distribution you may place JAR file to ../presto/plugin/hive-hadoop2/)
Make sure the RegexSerDe.jar file has correct ownership as other jar files present in this directory.
Restart the presto-server process. (sudo service presto-server restart). If this do not work, you may need to restart with launcher sudo /usr/lib/presto/bin/launcher restart
Repeat this on all slave nodes !
Followed the same process as you mentioned above getting same error any suggestions please.
Query failed: org/apache/hadoop/hive/serde2/SerDe
Using hive-hadoop2 version placed the serde jar in the following path
presto/plugin/hive-hadoop2/hive-serde-1.0.0.jar and restarted presto.
You have to put the SerDe jar inside the plugin directory (plugin/hive-cdh4).
I didn't use RegexSerDe but it worked for CSVSerDe.
I am trying to startup the ArangoDB server with:
→ bin/arangod --server.endpoint tcp://127.0.0.1:8529 --database.directory tmp
However I get a:
2013-05-07T12:30:30Z [83160] INFO ArangoDB 1.3.devel -- ICU 49.1.2, V8 version 3.16.14.1, SSL engine OpenSSL 0.9.8r 8 Feb 2011
2013-05-07T12:30:30Z [83160] INFO using default language 'de_DE'
2013-05-07T12:30:30Z [83160] INFO using endpoint 'tcp://127.0.0.1:8529' for http non-encrypted requests
2013-05-07T12:30:30Z [83160] FATAL you must specify an admin directory, giving up!
What is wrong? And what can I do to make the error go away?
If you are not going to "make install" ArangoDB in its final destination, but instead want to start it from the source directory, please use
bin/arangod -c etc/relative/arangod.conf --server.endpoint tcp://127.0.0.1:8529 --database.directory tmp
The configuration files in the "etc/relative" directory hold all the defines to start arangod and arangosh without installing it.