Is there an easy way to export a table out of Presto using command line? I need it in csv format.
You can use below command to set the output in CSV format. Make sure to set PRESTO_PASSWORD before you run it.
./presto --server {host}:{port_number} --catalog {catalog_name}
--schema {schema_name} --user {user_name} --password --execute "SELECT * FROM table_name" --output-format CSV > output_filename.csv
Related
Yugabyte,
is there a way I can create the YB DB and it's schema by running one script file with all the commands mentioned here (https://docs.yugabyte.com/latest/quick-start/explore/ysql/#docker)
You can execute queries with ysqlsh from the command line using -c, example we use that to create the database:
./bin/ysqlsh -c 'create database ybdemo;'
You can also execute sql scripts with ysqlsh using -f (https://docs.yugabyte.com/latest/admin/ysqlsh/#f-filename-file-filename), example:
./bin/ysqlsh -d ybdemo -f share/schema.sql
This will execute the schema.sql script into ybdemo database. And repeat that for every .sql file.
I have a shell script that runs a command:
psql -h $DBHOST -U $DBUSERNAME -c "\copy sometable FROM '$PWD/sometable.csv' WITH DELIMITER ',' CSV HEADER"
which works fine.
Now, as I have some requirements to implement more advanced logic, I am migrating some of these commands to nodejs code.
Is it possible to run this \copy command with postgres-node?
If not, I see an alternative to run this command as it is as a shell command from nodejs with require('child_process').spawn.
you are looking for https://github.com/brianc/node-pg-copy-streams I suppose. It's same authors "extension" to node-pg
I'm planning on running a .sh script that will run periodically through cron on linux. I'm running postgres 8.4 on centos.
My script will have something like this in it:
psql -U username -d db_name -c "COPY orders TO stdout DELIMITER ',' CSV HEADER;" > orders.csv
I know there are other ways to dump tables into csv files but this is the only one I could use without admin rights.
My problem is naming the files. I want to specifically name the file something along the lines of:
yyyymmdd-hhmm-orders.csv
I'm not the best scripting guru out there (as you can tell) so how can I get the dumps to dynamically do this?
Thanks
`date '+%Y%m%d-%H%M'`-orders.csv
I personally also add the seconds %S to the file name
man date
Will show the other formatting options
Use below code and its worked fine
Assigned date format with one variable and used the same
Code:
I=`date +%Y%m%d-%H%M%S -d`
psql -U username -d db_name -c "COPY orders TO stdout DELIMITER ',' CSV HEADER;" > $i-orders.csv
I've been using the psql Postgres terminal to import CSV files into tables using the following
COPY tbname FROM
'/tmp/the_file.csv'
delimiter '|' csv;
which works fine except that I have to be logged into the psql terminal to run it.
I would like to know if anyone knows of a way to do a command similar to this from the Linux shell command line similar to how Postgres allows a shell command like bellow
/opt/postgresql/bin/pg_dump dbname > /tmp/dbname.sql
This allows the dumping of a database from the Linux shell without being logged into psql terminal.
The solution in the accepted answer will only work on the server and when the user executing the query will have permissions to read the file as explained in this SO answer.
Otherwise, a more flexible approach is to replace the SQL's COPY command with the psql's "meta-command" called \copy which which takes all the same options as the "real" COPY, but is run inside the client (with no need for ; at the end):
psql -c "\copy tbname FROM '/tmp/the_file.csv' delimiter '|' csv"
As per docs, the \copy command:
Performs a frontend (client) copy. This is an operation that runs an SQL COPY command, but instead of the server reading or writing the specified file, psql reads or writes the file and routes the data between the server and the local file system. This means that file accessibility and privileges are those of the local user, not the server, and no SQL superuser privileges are required.
In addition, if the the_file.csv contains the header in the first line, it can be recognized by adding header at the end of the above command:
psql -c "\copy tbname FROM '/tmp/the_file.csv' delimiter '|' csv header"
As stated in The PostgreSQL Documentation (II. PostgreSQL Client Applications - psql) you can pass a command to psql (PostgreSQL interactive terminal) with the switch -c. Your options are:
1, Client-side CSV: \copy meta-command
perform the SQL COPY command but the file is read on the client and the content routed to the server.
psql -c "\copy tbname FROM '/tmp/the_file.csv' delimiter '|' csv"
(client-side option originally mentioned in this answer)
2. Server-side CSV: SQL COPY command
reads the file on the server (current user needs to have the necessary permissions):
psql -c "COPY tbname FROM '/tmp/the_file.csv' delimiter '|' csv;"
the DB roles needed for reading the file on the server:
COPY naming a file or command is only allowed to database superusers
or users who are granted one of the default roles
pg_read_server_files, pg_write_server_files, or
pg_execute_server_program
also the PostgreSQL server process needs to have access to the file.
To complete the previous answer, I would suggest:
psql -d your_dbname --user=db_username -c "COPY tbname FROM '/tmp/the_file.csv' delimiter '|' csv;"
The most flexible way is to use a shell HERE document, which allows you to use shell variables inside your query, even inside (double or single) quotes:
#!/bin/sh
THE_USER=moi
THE_DB=stuff
THE_TABLE=personnel
PSQL=/opt/postgresql/bin/psql
THE_DIR=/tmp
THE_FILE=the_file.csv
${PSQL} -U ${THE_USER} ${THE_DB} <<OMG
COPY ${THE_TABLE} FROM '${THE_DIR}/${THE_FILE}' delimiter '|' csv;
OMG
The problem that I am having is that I want to run the following command (and I can't):
cqlsh < cql_directory/cql_create_stuff.cql
Because I have not logged in to cqlsh.
So I logged in:
cqlsh -u 'my_username' -p 'my_super_secret_password'
and now I tried doing the command in cqlsh shell but It just responds with a syntax error.
Basically, how do I login into cqlsh and run an external CQL script in my file system?
Use the SOURCE
http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/source_r.html
You can use -f option as well to execute commands from file
http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/cqlsh.html
Assuming that the path of the file with the CQL commands is /mydir/myfile.cql, there are two ways:
If you are not logged in to cqlsh:
cqlsh -u 'my_username' -p 'my_password' -f /mydir/myfile.cql
If you are logged in to cqlsh:
SOURCE '/mydir/myfile.cql'
Notice the single quotation marks. The shorthand notation for $HOME (for example, '~/mydir/myfile.cql') is also supported.
Both ways also work with relative paths (to the current directory).
Assuming your filename is "tables.cql" and it is placed as: /files/tables.cql;
A - Locally
cqlsh -f /files/tables.cql
B - Connecting To A Docker Container Running Cassandra
Assuming the name of the Docker container that which running Cassandra is "cas" (keep in mind that you can also use the hash id of the docker container if there is no name assigned to it);
docker exec -it cas cqlsh -f /files/tables.cql
As stated on other answers, -u and -p options can be added in order to use the username/password combinations.
This is for Window system
suppose you cassandra dir is
C:\Program Files\DataStax-DDC\apache-cassandra\bin
Suppose directory where your .cql file OR cql query file is
D:\ril\s\developement\new one\excel after parse\Women catalogue template.cql
Now follow below steps for importing cql file
Go on command prompt (cmd)
Go on the directory where cql file is there (cd "..\ril\sizeguide\developement\new one\excel after parse")
Run below command
"c:\Program Files\DataStax-DDC\apache-cassandra\bin\cqlsh.bat" <"Women catalogue template.cql"
And its Done.
Important Note:
Please make sure column value should not have single quote ' character like ('If you don't find a exact match, go for the next large size') other wise it will fail.
If you want single quote to be inserted, please use it two times like below and Cassandra will treat it as one time
('If you don''t find a exact match, go for the next large size')
All text column should be enclosed by single quote '' like 'Sale category'. For empty value, please use two single quote ''.