I have written a cassandra UDF using java code to encrypt data. For this I have used the javax.crypto.Cipher and javax.crypto.spec.SecretKeySpec.
I am unable to create the UDF as I am getting the following error:
InvalidRequest: Error from server: code=2200 [Invalid query] message="Java source compilation failed:
Line 1: javax.crypto.spec.SecretKeySpec cannot be resolved to a type
Line 1: javax.crypto.spec.SecretKeySpec cannot be resolved to a type
Line 1: javax.crypto.Cipher cannot be resolved to a type
Line 1: javax.crypto.Cipher cannot be resolved to a type
Line 1: javax.crypto.Cipher.ENCRYPT_MODE cannot be resolved to a type
create function encrypt("val" text)
returns null on null input
returns text
language java as
'
String strData="";
javax.crypto.spec.SecretKeySpec skeyspec=new
javax.crypto.spec.SecretKeySpec("abc".getBytes(),"Blowfish");
javax.crypto.Cipher cipher=javax.crypto.Cipher.getInstance("Blowfish");
cipher.init(javax.crypto.Cipher.ENCRYPT_MODE,skeyspec);
byte[] encrypted=cipher.doFinal(val.getBytes());
strData=new String(encrypted);
return strData;
';
Related
I am trying to write a UDF in Cassandra using Java.
I have two list of integers with one to one mapping i.e., first item in the first list corresponds to the first item in the second list. This is how my data is stored in Cassandra table under two columns of list type.
My UDF
CREATE OR REPLACE FUNCTION mykeyspace.get_thread(list1 List<int>, list2 List<int>)
CALLED ON NULL INPUT
RETURNS List<int>
LANGUAGE java
AS '
Map<Integer, Integer> intermediatemap = new HashMap<Integer, Integer>();
for (int i = 0; i < list1.size(); i++) {
if (!intermediatemap.containsKey(i))
intermediatemap.put(list1.get(i), list2.get(i));
}
List<Integer> commentids = new ArrayList<Integer>();
intermediatemap.entrySet().stream()
.sorted(Map.Entry.<Integer, Integer>comparingByValue().reversed())
.forEachOrdered(x -> commentids.add(x.getKey()));
if (commentids.size() > 8)
return commentids.subList(0, 8);
else
return commentids;
';
This is running all good as code. However, when I execute this on cqlsh I am getting error saying
InvalidRequest: Error from server: code=2200 [Invalid query] message="Java source compilation failed:
Line 8: The type java.util.stream.Stream cannot be resolved. It is indirectly referenced from required .class files
Line 8: The method stream() from the type Collection<Map.Entry<Integer,Integer>> refers to the missing type Stream
Is there any problem with the Cassandra version I am using? I am running the Cassandra on my local MAC machine inside a docker. I have tried with 3.11.2 and the image with latest tag.
Also, UDF function is enabled I am able to run simple UDF functions.
You can use only classes that are explicitly defined in the allowed classes list inside Cassandra, and not in the disallowed list defined. If you look to the source code for Cassandra 3.11.2, you can see that java.utils.stream is in the disallowed list, so you can't use it inside UDF/UDA.
This is my first query on the forum so please correct if my understanding are wrong.
I am executing exercise Application Driver Connection from DS201.
Table is as follows:
cqlsh:killrvideo> SELECT * FROM videos_by_tag ;
tag | added_date | video_id | title
-----------+---------------------------------+--------------------------------------+------------------------------
datastax | 2013-10-16 00:00:00.000000+0000 | 4845ed97-14bd-11e5-8a40-8338255b7e33 | DataStax Studio
Now I want to do one task "Python code to insert a new video into the database" as per lab.
I tried this code and getting error:
>>> session.execute(
... "INSERT INTO videos_by_tag (tag, added_date, video_id, title)" +
... "VALUES ('cassandra', '2013-01-10', uuid(), 'Cassandra Is My Friend')")
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File "cassandra/cluster.py", line 2618, in cassandra.cluster.Session.execute
File "cassandra/cluster.py", line 4877, in cassandra.cluster.ResponseFuture.result
cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Type error: cannot assign result of function system.uuid (type uuid) to video_id (type timeuuid)"
>>>
I tried below but failed:
UUIDs.timeBased()
ERROR:
cassandra.InvalidRequest: Error from server: code=2200 [Invalid query] message="Unknown function uuids.timebased called"
cassandra.util.uuid_from_time
ERROR:
cassandra.protocol.SyntaxException: <Error from server: code=2000 [Syntax error in CQL query] message="line 1:109 no viable alternative at input '.' (...)VALUES ('cassandra', '2013-01-10', [cassandra].util...)">
For time being I have hardcoded the value.
session.execute(
... "INSERT INTO videos_by_tag (tag, added_date, video_id, title)" +
... "VALUES ('cassandra', '2013-01-10', 245e8024-14bd-11e5-9743-8238357b7e32, 'Cassandra Is My Friend')")
Success in DB:
cassandra | 2013-01-10 00:00:00.000000+0000 | 245e8024-14bd-11e5-9743-8238357b7e32 | Cassandra Is My Friend
But I want to know this ?
You are correct in that the call to the uuid() function is your problem:
INSERT INTO videos_by_tag (tag,added_date,video_id,title)
VALUES ('cassandra', '2013-01-10', uuid(), 'Cassandra Is My Friend');
The uuid() function generates a type-4 UUID, whereas the video_id column is defined as a type-1 UUID (aka a TIMEUUID).
Instead, invoke the now() function to get a TIMEUUID:
INSERT INTO videos_by_tag (tag,added_date,video_id,title)
VALUES ('cassandra', '2013-01-10', now(), 'Cassandra Is My Friend');
While reading the Datastax docs for supported syntax of Spark SQL, I noticed you can use INSERT statements like you would normally do:
INSERT INTO hello (someId,name) VALUES (1,"hello")
Testing this out in a Spark 2.0 (Python) environment and a connection to a Mysql database, throws the error:
File "/home/yawn/spark-2.0.0-bin-hadoop2.7/python/lib/pyspark.zip/pyspark/sql/utils.py", line 73, in deco
pyspark.sql.utils.ParseException:
u'\nmismatched input \'someId\' expecting {\'(\', \'SELECT\', \'FROM\', \'VALUES\', \'TABLE\', \'INSERT\', \'MAP\', \'REDUCE\'}(line 1, pos 19)\n\n== SQL ==\nINSERT INTO hello (someId,name) VALUES (1,"hello")\n-------------------^^^\n'
However if I remove the explicit column definition, it works as expected:
INSERT INTO hello VALUES (1,"hello")
Am I missing something?
Spark support hive syntax so if you want to insert row you can do as follows
insert into hello select t.* from (select 1, 'hello') t;
I am not able to return map from User Defined Function in CQL 3.x (Cassandra 2.2.3). Here is my code:
CREATE OR REPLACE FUNCTION issaviktempkeyspace.myfilter1(begindate timestamp,argtimestamp timestamp,columnname text,columnvalue text,companyid int)
CALLED ON NULL INPUT
RETURNS map<text,text> LANGUAGE java AS
'
if(begindate.before(argtimestamp)){
Map<String,String> map= Collections.<String,String>emptyMap();
map.put("columnname",columnname);
map.put("columnvalue",columnvalue);
map.put("companyid",Integer.toString(companyid));
return map;
}
else
return null;
';
When I try to create the function, Cassandra says that,
InvalidQueryException: Could not compile function 'issaviktempkeyspace.myfilter1' from Java source: org.apache.cassandra.exceptions.InvalidRequestException: Java source compilation failed:
Line 3: Map cannot be resolved to a type
Line 3: Collections cannot be resolved
com.datastax.driver.core.exceptions.InvalidQueryException: InvalidQueryException: Could not compile function 'issaviktempkeyspace.myfilter1' from Java source: org.apache.cassandra.exceptions.InvalidRequestException: Java source compilation failed:
Line 3: Map cannot be resolved to a type
Line 3: Collections cannot be resolved
-- Any idea how to do this.
I have one more query. Can I have tuple type to be returned as a part of UDF ?
I'm trying out Cassandra 2.1.0-beta2 for nested user defined types. These are the structures that has been created through cqlsh 5.0.0:
CREATE TYPE channelsource(id text, sources set<text>);
CREATE TYPE primaryfeed(name text, type text, multisource channelsource);
CREATE TABLE somedata (
source text PRIMARY KEY,
unitid text,
dayssincebirth text,
reporttime text,
somefeed primaryfeed
);
This sample insert fails:
INSERT INTO somedata (source, unitid, dayssincebirth, reporttime, somefeed)
VALUES('GFDS8-v1.2.3', 'xxxxxxxx-ABCD-1234-B8F9-11111177F4', '89', '13:02:39',
{'dha', 'foo', {'someid', {'aa', 'cc'}}});
With this error:
code=2200 [Invalid query] message="Invalid set literal for ColumnDefinition
{name=somefeed, type=org.apache.cassandra.db.marshal.UserType
(bispace,7072696d61727966656564,6e616d65:org.apache.cassandra.db.marshal.UTF8Type,
74797065:org.apache.cassandra.db.marshal.UTF8Type,6d756c7469736f75726365:
org.apache.cassandra.db.marshal.UserType(bispace,6368616e6e656c736f75726365,6964
:org.apache.cassandra.db.marshal.UTF8Type,736f7572636573:
org.apache.cassandra.db.marshal.SetType(org.apache.cassandra.db.marshal.UTF8Type))),
kind=REGULAR, componentIndex=0, indexName=null, indexType=null} of type primaryfeed"
Any pointers on what the correct syntax I should be using? I'm assuming that nested user defined types are supported in 2.1 onwards.
You basically need to pass also the name of the fields, i.e.
{name: 'dha', type: 'foo', ...}
More details are in this article http://www.datastax.com/dev/blog/cql-in-2-1
Note: the syntax you used ( {'val', 'val'}) is the syntax for sets and that explains the error you are seeing.