-> Table :
cassandra#cqlsh:coba> CREATE TABLE data(
... nim int,
... nama text,
... alamat text,
... PRIMARY KEY (nim, alamat)
... );
-> Make Index :
CREATE CUSTOM INDEX cari_alamat ON coba.data (alamat) USING 'org.apache.cassandra.index.sasi.SASIIndex';
-> Error :
ServerError: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.cassandra.index.sasi.SASIIndex
-> I would be very happy if you can help me
-> Thanks You
As initially susspected, I think you are running cassandra version lower than 3.4.
(that's why I asked for the version)
I tried it out and got the same error on 3.0.10:
cqlsh:test> CREATE CUSTOM INDEX cari_alamat ON test.data (alamat) USING 'org.apache.cassandra.index.sasi.SASIIndex';
ConfigurationException: Unable to find custom indexer class 'org.apache.cassandra.index.sasi.SASIIndex'
Theoretically you could implement your own with:
Cassandra Custom Secondary Index
But I guess it's just easier to upgrade.
Also be aware there might be some bugs with Sasi indexes:
SASI Indexes in Cassandra seem to have some bugs
But I guess it's better to search cassandra Jira for this one, this is just as a small warning.
Related
I am unable to upsert a row using the datastax driver.
The data in the Cassandra table is stored like follows:
tag | partition_info
------------+--------------------------------------------------
sometag | {{year: 2018, month: 1}, {year: 2018, month: 2}}
tag is primary key and partition_info is a UDT
CREATE TYPE codingjedi.tag_partitions (
year bigint,
month bigint
);
I want that if a tag doesn't exist then it gets created. If tag exists then the new udt value gets appended to old one. I suppose I cannot use insert as it overrides previous value i.e. this will not work
QueryBuilder.insertInto(tableName).value("tag",model.tag)
.value("partition_info",setAsJavaSet(Set(partitionsInfo)))
I am trying to use update but it isn't working. Datastax driver gives error java.lang.IllegalArgumentException for following query
QueryBuilder.update(tableName).`with`(QueryBuilder.append("partition_info",setAsJavaSet(Set(partitionsInfo))))
.where(QueryBuilder.eq("tag", id.tag))
I tried using add and append for primary key but but got the error PRIMARY KEY part tag found in SET part
QueryBuilder.update(tableName).`with`(QueryBuilder.add("tag",id.tag))
.and(QueryBuilder.append("partition_info",setAsJavaSet(Set(partitionsInfo)))) .where(QueryBuilder.eq("tag", id.tag))
You're using the incorrect operation in your update statement - you're using append, but it's used to append data to columns of list types. You can use instead either add if you're adding a single value (your case, so you wont even need to wrap data into Set explicitly), or addAll if you're adding multiple values.
QueryBuilder.update(tableName)
.`with`(QueryBuilder.add("partition_info", partitionsInfo))
.where(QueryBuilder.eq("tag", id.tag))
I use wso2 dss to insert data into a cassandra table.
for exemple this table :
CREATE TABLE logs.test (id int,code int, PRIMARY KEY (id));
Inside wso2 dss, I defined code column with default value like this : #{NULL}
When I Try the dss service like this without given the code parameter:
<p:test xmlns:p="http://ws.wso2.org/dataservice">
<xs:id xmlns:xs="http://ws.wso2.org/dataservice">1</xs:id>
</p:test>
I get this error :
<axis2ns56:source_data_service>
<axis2ns56:data_service_name>Cassandra</axis2ns56:data_service_name>
<axis2ns56:description>N/A</axis2ns56:description>
<axis2ns56:location>\Cassandra.dbs</axis2ns56:location>
<axis2ns56:default_namespace>http://ws.wso2.org/dataservice</axis2ns56:default_namespace>
</axis2ns56:source_data_service>
<axis2ns56:ds_code>UNKNOWN_ERROR</axis2ns56:ds_code>
<axis2ns56:nested_exception>java.lang.NumberFormatException: null</axis2ns56:nested_exception>
Nested Exception:- java.lang.NumberFormatException: For input string: "null"
Best regards,
Nicolas
Would it be possible to get the source of the dataservice?
Did you try with the following payload
<p:test xmlns:p="http://ws.wso2.org/dataservice">
<p:id>1</p:id>
<p:code>2</p:code>
</p:test>
So I guess your issue is in this part
<param defaultValue="#{NULL}" name="code" sqlType="INTEGER"/>.
I do not know your use case but if I remember well it's not so nice to insert null values in Cassandra because it create tombstones.
You could as well have a second query that simply inserts the id like
insert test (id) values (:id).
The execption sound to be raised by dss not cassandra, looks like it is not able to set a null value for integer field
I find a workaround, I use the jdbc cassandra instead of com.datatasax driver.
And it work well. The only problem is that I just can call only one node for the connection and not the cluster.
I hope the problem will be resolve soon and I will use the Dss Cassandra datasource connection again.
Thks for your help
I basically have the same problem as the following Composite key in Cassandra with Pig. The only difference is I try to query for a part of the composite key within the where_clause of pig.
The data structure is similar to the earlier mentioned issue, I'll copy some code/context to minimize the reading of that issue.
We have a CQL table that looks something like this:
CREATE table data (
occurday text,
seqnumber int,
occurtimems bigint,
unique bigint,
fields map<text, text>,
primary key ((occurday, seqnumber), occurtimems, unique)
)
Instead of querying for both the seqnumber and the occurday (as was the issue in previously mentioned issue) I try to query one of the keys.
If I execute this query as part of a LOAD from within Pig, however, things don't work.
-- Need to URL encode the query
data = LOAD 'cql://ks/data?where_clause=occurday%3D%272013-10-01%27' USING CqlStorage();
gives
java.lang.RuntimeException
at org.apache.cassandra.hadoop.cql3.CqlPagingRecordReader$RowIterator.executeQuery(CqlPagingRecordReader.java:665)
at org.apache.cassandra.hadoop.cql3.CqlPagingRecordReader$RowIterator.<init>(CqlPagingRecordReader.java:301)
at org.apache.cassandra.hadoop.cql3.CqlPagingRecordReader.initialize(CqlPagingRecordReader.java:167)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:181)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:522)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: InvalidRequestException(why:occurday cannot be restricted by more than one relation if it includes an Equal)
at org.apache.cassandra.thrift.Cassandra$prepare_cql3_query_result$prepare_cql3_query_resultStandardScheme.read(Cassandra.java:51017)
at org.apache.cassandra.thrift.Cassandra$prepare_cql3_query_result$prepare_cql3_query_resultStandardScheme.read(Cassandra.java:50994)
at org.apache.cassandra.thrift.Cassandra$prepare_cql3_query_result.read(Cassandra.java:50933)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.cassandra.thrift.Cassandra$Client.recv_prepare_cql3_query(Cassandra.java:1756)
at org.apache.cassandra.thrift.Cassandra$Client.prepare_cql3_query(Cassandra.java:1742)
at org.apache.cassandra.hadoop.cql3.CqlPagingRecordReader$RowIterator.prepareQuery(CqlPagingRecordReader.java:605)
at org.apache.cassandra.hadoop.cql3.CqlPagingRecordReader$RowIterator.executeQuery(CqlPagingRecordReader.java:635)
... 7 more
Basically my question is, what am I doing wrong or what don't I understand?
As I understand from CqlPagingRecorderReader Used when Partition Key Is Explicitly Stated
I should be able to query with just part of the partition key?
Also while reading
Add CqlRecordReader to take advantage of native CQL pagination
I get the impression this should be possible, but I am swimming around with (in my opinion) no clear direction on how to accomplish this.
Any help is very very welcome at this point.
Regards,
Lennart Weijl
PS.
I am running on Cassandra 2.0.9 with Pig 0.13.0
According to CASSANDRA-6311, I believe you need to apply the 6331-v2-2.0-branch.txt patch, recompile pig, and then update your LOAD statement to:
data = LOAD 'cql://ks/data?where_clause=occurday%3D%272013-10-01%27' USING CqlInputFormat();
The key change being USING CqlInputFormat() which triggers the use of the new CqlRecordReader that was released in Cassandra 2.0.7.
Edit: Note that the exception is thrown from CqlPagingRecordReader which means you're still using the old record reader.
Is there an easy way to check if table (column family) is defined in Cassandra using CQL (or API perhaps, using com.datastax.driver)?
Right now I am leaning towards executing SELECT 1 FROM table and checking for exception but maybe there is a better way?
As of 1.1 you should be able to query the system keyspace, schema_columnfamilies column family. If you know which keyspace you want to check, this CQL should list all column families in a keyspace:
SELECT columnfamily_name
FROM schema_columnfamilies WHERE keyspace_name='myKeyspaceName';
The report describing this functionality is here: https://issues.apache.org/jira/browse/CASSANDRA-2477
Although, they do note that some of the system column names have changed between 1.1 and 1.2. So you might have to mess around with it a little to get your desired results.
Edit 20160523 - Cassandra 3.x Update:
Note that for Cassandra 3.0 and up, you'll need to make a few adjustments to the above query:
SELECT table_name
FROM system_schema.tables WHERE keyspace_name='myKeyspaceName';
The Java driver (since you mentioned it in your question) also maintains a local representation of the schema.
Driver 3.x and below:
KeyspaceMetadata ks = cluster.getMetadata().getKeyspace("myKeyspace");
TableMetadata table = ks.getTable("myTable");
boolean tableExists = (table != null);
Driver 4.x and above:
Metadata metadata = session.getMetadata();
boolean tableExists =
metadata.getKeyspace("myKeyspace")
.flatMap(ks -> ks.getTable("myTable"))
.isPresent();
I just needed to manually check for the existence of a table using cqlsh.
Possibly useful general info.
describe keyspace_name.table_name
If it doesn't exist you'll get 'table_name' not found in keyspace 'keyspace'
If it does exist you'll get a description of the table.
For the .NET driver CassandraCSharpDriver version 3.17.1 the following code creates a table if it doesn't exist yet:
var ks = _cassandraSession.Cluster.Metadata.GetKeyspace(keyspaceName);
var tableNames = ks.GetTablesNames();
if(!tableNames.Contains(tableName.ToLowerInvariant()))
{
var stmt = new SimpleStatement($"CREATE TABLE {tableName} (id text PRIMARY KEY, name text, price decimal, volume int, time timestamp)");
_cassandraSession.Execute(stmt);
}
You will need to adapt the list of table columns to your needs. This can also be awaited by using await _cassandraSession.ExecuteAsync(stmt).ConfigureAwait(false) in an async method.
Also, I want to mention that I'm using Cassandra version 4.0.1.
I've followed numerous examples on inserting data into a Cassandra database and every time I get an exception about unconfigured column families.
Exception in thread "main" me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:unconfigured columnfamily TestColumnFamily)
at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:45)
at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:252)
at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97)
at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243)
at me.prettyprint.cassandra.model.MutatorImpl.insert(MutatorImpl.java:69)
at CassandraInterface.main(CassandraInterface.java:101)
Caused by: InvalidRequestException(why:unconfigured columnfamily TestColumnFamily)
at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:19477)
at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:1035)
at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:1009)
at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:246)
at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:243)
at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246)
... 4 more
So I looked up how to configure them and found
BasicColumnFamilyDefinition cfdef = new BasicColumnFamilyDefinition();
cfdef.setKeyspaceName(keyspaceName);
cfdef.setName(columnFamilyName);
cfdef.setKeyValidationClass(ComparatorType.UTF8TYPE.getClassName());
cfdef.setComparatorType(ComparatorType.UTF8TYPE);
That didn't configure the column family.
All of the examples I have found are fragments without any context, so I don't know what to import or set up. In addition, some examples appear to mix the Hector API v2 and the original Hector API, so when I use them, I get "class not found" or "function not found" compiler errors.
Hector CassandraClusterTest.java
#Test
public void testAddDropColumnFamily() throws Exception {
ColumnFamilyDefinition cfDef = HFactory.createColumnFamilyDefinition("Keyspace1", "DynCf");
cassandraCluster.addColumnFamily(cfDef);
String cfid2 = cassandraCluster.dropColumnFamily("Keyspace1", "DynCf");
assertNotNull(cfid2);
// Let's wait for agreement
cassandraCluster.addColumnFamily(cfDef, true);
cfid2 = cassandraCluster.dropColumnFamily("Keyspace1", "DynCf", true);
assertNotNull(cfid2);
}
Long story short, keyspace and column family need to exist before you try and insert data into them. You can either manage this in your code, to check to see if they exist, using the example above as a nice reference -- or modify via the command line interface (cassandra-cli)
Hector Unit Tests
Hopefully you've been able to do this by now but this is how I've done it.
I have a cassandra install (using 1.1.4) and assuming you have all the necessary directories created:
/var/lib/cassandra
/var/lib/casandra/data
/var/lib/cassnadra/commitlogs
/var/lib/cassandra/saved_caches
I start it using:
bin/cassandra -f
I create a simple script called schema_create.txt:
CREATE KEYSPACE TEST
WITH strategy_class = 'org.apache.cassandra.locator.SimpleStrategy'
AND strategy_options:replication_factor='1';
use TEST;
CREATE COLUMNFAMILY TestColumnFamily(
userid varchar,
firstname varchar,
lastname varchar,
PRIMARY KEY (userid));
Then from the command line you can run this script using the new CQL tool that comes with cassandra as follows:
bin/cqlsh --cql3 < schema_createt.txt
This will install a keyspace named test with a column family named testcolumnfamily into cassandra.
Now from within your java application you can simply create a test class that has a main method (i will assume your development environment has all necessary dependencies if using maven):
try{
Mutator mutator = HFactory.createMutator(kweyspace, stringSerializer.get());
mutator.addInsertion("iamauser", "tescolumnfamily", HFactory.createStringColumn("firstname", "John"));
mutator.addInsertion("iamauser", "testcolumnfamily", HFactory.createStringColumn("lastname", "Smith"));
mutator.execute();
}
catch(HectorException Hex){ Hex.printStackTrace(); }
finally{ cluster.getConnectionManger().shutdown(); }
Now go back to the command line and enter into cassandra using:
$bin/cqlsh --cql3
use test;
select * from testcolumnfamily;
This will insert a row of data into your cassandra db with the key iamauser, and name as John Smith and you can verify as shown above using the cqlsh tool.
Hope this helps.