I'm using cassandra. I am trying to update the gc_grace value using new bind variable.
ALTER table keyspace.table_name with gc_grace_seconds = ? ;
I got the following error:
no viable alternative at input '?'
How can I solve this?
As I see from the source code (maybe I'm wrong), but ALTER TABLE doesn't support bindings, so you can't use them for this command (and all DDL commands), and need to just use execute with specific value
It looks like you're trying to bind parameters programatically to set GC grace on a table. It isn't possible to do that using the Cassandra drivers.
It will only work through cqlsh. For example:
cqlsh> ALTER TABLE community.maptbl WITH gc_grace_seconds = 3600;
It doesn't make sense to do it in your app and it is not recommended. Cheers!
Related
create table if not exists map_table like position_map_view;
While using this it is giving me operation not allowed error
As pointed in documentation, you need to use CREATE TABLE AS, just use LIMIT 0 in SELECT:
create table map_table as select * from position_map_view limit 0;
I didn't find an easy way of getting CREATE TABLE LIKE to work, but I've got a workaround. On DBR in Databricks you should be able to use SHALLOW CLONE to do something similar:
%sql
CREATE OR REPLACE TABLE $new_database.$new_table
SHALLOW CLONE $original_database.$original_table`;
You'll need to replace $templates manually.
Notes:
This has an added side-effect of preserving the table content in case you need it.
Ironically, creating empty table is much harder and involves manipulating show create table statement with custom code
Spark 2.0 - pyspark
I seen the following 2 properties paired. What is the difference between them?
hive> SET hive.exec.dynamic.partition=true;
hive> SET hive.exec.dynamic.partition.mode=non-strict;
I know what the outcome is when they are used - you can use dynamic partitioning to load/create multiple partitions, but I don't know the difference between these two similar commands.
When I was running this code
input_field_names=['id','code','num']
df \
.select(input_field_names) \
.write \
.mode('append')\
.insertInto('test_insert_into_partition')
I got an error message that says Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict
Using spark.sql("SET hive.exec.dynamic.partition.mode = nonstrict") the code works. It doesn't require me to use the other one.
Why don't I need to set SET hive.exec.dynamic.partition=true; and what else should I know to choose which one to use.
Although there is much to google, here is a short answer.
If you want to insert dynamically into Hive partitions both values need to be set and you can then load many partitions in one go:
SET hive.exec.dynamic.partition=true;
SET hive.exec.dynamic.partition.mode=nonstrict
create table tblename (h string,m string,mv double,country string)partitioned by (starttime string) location '/.../...'
INSERT overwrite table tblename PARTITION(starttime) SELECT h,m,mv,country,starttime from tblename2
Otherwise you need to do like this, setting the partition col val yourself / explicity:
INSERT into table tblename PARTITION(starttime='2017-08-09') SELECT h,m,mv,country from tblname2 where to_date(starttime)='2017-08-09'
The purpose of default value of 'strict' for
hive.exec.dynamic.partition.mode is there to prevent a user from
accidentally overwriting all the partitions, i.e. to avoid data loss.
So, there is not a situation of difference, rather a situation of caution, a but like the safety catch on a firearm, as it were.
$describe = new Cassandra\SimpleStatement(<<<EOD
describe keyspace.tablename
EOD
);
$session->execute($describe);
i used above code but it is not working.
how can i fetch field name and it's data type from Cassandra table ?
Refer to CQL documentation. Describe expects a table/schema/keyspace.
describe table keyspace.tablename
Its also a cqlsh command, not an actual cql command. To get this information query the system tables. try
select * from system.schema_columns;
- or for more recent versions -
select * from system_schema.columns ;
if using php driver may want to check out http://datastax.github.io/php-driver/features/#schema-metadata
Try desc table keyspace.tablename;
I'm trying to query a Cassandra 2.0.2 server using the phpcassa libraray.
I do the following:
<?php
require('phpcassa/lib/autoload.php');
use phpcassa\ColumnFamily;
use phpcassa\ColumnSlice;
use phpcassa\Connection\ConnectionPool;
$pool = new ConnectionPool('Cronnection', array('MY_SERVER_IP:9160'));
$conversations = new ColumnFamily($pool, 'conversations');
var_dump( $conversations->get('2521b0f0-8e36-11e3-a489-8f038e859082') );
When I do this I get an uncaught 'cassandra\NotFoundException' exception.
After reading in the source I see that this is because the column family 'conversations' is not getting loaded into the cloumn families array created by the ConnectionPool::describe_keyspace() method.
When I print the result of "describe_keyspace", using the following code, I can see that only one of my column families, called user_profiles, is getting loaded into my column families array.
<?php
require('phpcassa/lib/autoload.php');
use phpcassa\ColumnFamily;
use phpcassa\ColumnSlice;
use phpcassa\Connection\ConnectionPool;
$pool = new ConnectionPool('Cronnection', array('MY_SERVER_IP:9160'));
echo '<pre>';
var_dump($pool->describe_keyspace());
There are 6 column families defined in that keyspace but phpcassa is only listing "user_profiles"; the only difference between "user_profiles" and the other CFs is that "user_profiles" is using the "WITH COMPACT STORAGE" property.
Any thoughts on how to solve this or why is this happenning?
Thanks!
PS: I created the column families using CQL3 through cqlsh, when I do "describe" using cassandra-cli I can't see my column families, only the "user_profiles" one that I mentioned, could this somehow be related?
Yes, this is in fact related to COMPACT STORAGE and cassandra-cli not showing your other column families. phpcassa and other Thrift-based clients (such as cassandra-cli) can only use tables created through CQL3 if they are created WITH COMPACT STORAGE.
In short, I suggest sticking to COMPACT STORAGE tables if you will be working with phpcassa. If you want to read more, there is a somewhat advanced article that shows explains how Thrift, CQL3 and compact storage fit together.
I write because I've a problem with cassandra; after have imported the data from pentaho as show here
http://wiki.pentaho.com/display/BAD/Write+Data+To+Cassandra
when I try to execute the query
Select * FROM mytable;
cassandre give me an error message
Syntax error at position 7: unexpected "*" for Select * FROM mytable;.
and don't show the results of query.Why? what does it mean that error?
the step that i make are the follow:
start cassandra cli utility;
use keyspace added from pentaho; (use tpc_h);
select to show the data added (Select * FROM mytable;)
The cassandra-cli does not support any CQL version. It has its own syntax which you can find on datastax's website.
Just for clarity, in cql to select everything from a table (aka column-family) called mytable stored in a keyspace called myks you would use:
SELECT * FROM myks.mytable;
The equivalent in cassandra-cli would *roughly be :
USE myks;
LIST mytable;
***** In the cli you are limited to selecting the first 100 rows. If this is a problem you can use the limit clause to specify how many rows you want:
LIST mytable limit 10000;
As for this:
in cassandra i have read that isn't possible make the join such as sql, ther isn't a shortcut to issue this disadvantage
There is a reason why joins don't exist in Cassandra, its for the same reason that C* isn't ACID compliant, it sacrifices that functionality for it's amazing performance and scalability, so it's not a disadvantage, you just need to re-think your model if you need joins. Also take a look at this question / answer.