gocqlx Updatebuilder and get wrong result - cassandra

first, my table design are follow:
create table io
(
qa text primary key,
created_at text,
messages list<frozen<message>>,
reply boolean,
resolve boolean,
updated_at text,
user text,
uuid uuid
)
and my message of UDT
create type message
(
time text,
who tinyint,
content text,
images map<text, blob>
);
my struct in golang are follow:
type Questions []Question
type Question struct {
Uuid gocql.UUID `json:"uuid"`
User string `json:"user"`
Reply bool `json:"reply"`
Qa string `json:"qa"`
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
Resolve bool `json:"resolve"`
Messages Messages `json:"messages"`
}
type Messages []Message
type Message struct {
Time string `json:"time"`
Who int `json:"who"`
Content string `json:"content"`
Images map[string]string `json:"images"`
}
I use package of gocqlx to build my query,and insert data seems look like correct
...strat the session
...set meta table
...append message to messages (slice of message)
...finally, I run the this
if err := s.Query(t.UpdateBuilder().Add("messages").Set("updated_at").ToCql()).BindStruct(q).ExecRelease(); err != nil {
log.Println("Send message error: ", err)
}
print this query session.Query(table.UpdateBuilder().Add("messages").Set("updated_at").ToCql()).BindStruct(question)
[query statement="UPDATE io SET messages=messages+?,updated_at=? WHERE qa=? " values=[[{Time:2020-05-23 22:15:10+0800 Who:0 Content:yah yah yah Images:map[]}] 2020-05-23 22:15:10+0800 20200523221510]
that will be right on executed. but when I check data in cassandra.
I look null list value
| qa | created_at | messages
------------------------------------------------------------------------------------------
|20200523221510 |2020-05-23 22:15:10+0800| [{time:NULL,who:NULL,content:NULL,images:{}}]
where's wrong ?

Your struct needs to have a cql tag. Something like:
type Questions []Question
type Question struct {
Uuid gocql.UUID `json:"uuid" cql:"uuid"`
// ...
}

Related

How to parse string to AnnualDate in NodaTime

I cannot find anywhere in the documentation how to parse the annual date string, e.g. 09-01 to AnnualDate.
I'm saving AnnualDate as a string in the database and I need to convert it to AnnualDate when implementing custom conversion in EF.
The only constructor AnnualDate has is AnnualDate(int, int) and I cannot find the parse method anywhere.
It's the same as with all other NodaTime types - you use a pattern, in this case NodaTime.Text.AnnualDatePattern.
So:
var result = AnnualDatePattern.Iso.Parse(text);
if (result.Success)
{
var annualDate = result.Value;
...
}
else
{
// Handle failure
}

IfNotExists doesn't return error when duplicate record is added

My query checks while inserting if a record is duplicate
def insertValues(tableName:String, model:User):Insert = {
QueryBuilder.insertInto(tableName).value("bucket",model.profile.internalProfileDetails.get.bucketId)
....
.ifNotExists();
}
I am saving a duplicate entry and expect that Cassandra will return an error. Instead I am getting the existing record back. Shouldn't Insert return an error?
def save(user:User):Future[Option[User]] = Future {
saveDataToDatabase(user)
}
def saveDataToDatabase(data:M):Option[M] = {
println("inserting in table "+tablename+" with partition key "+partitionKeyColumns +" and values "+data)
val insertQuery = insertValues(tablename,data)
println("insert query is "+insertQuery)
try {
val resultSet = session.execute(insertQuery) //execute can take a Statement. Insert is derived from Statement so I can use Insert.
println("resultset after insert: " + resultSet)
Some(data)
}catch {
case e:Exception => {
println("cassandra exception "+e)
None
}
}
}
The table schema is
users (
bucket int,
email text,
authprovider text,
firstname text,
lastname text,
confirmed boolean,
hasher text,
id uuid,
password text,
salt text,
PRIMARY KEY ((bucket, email), authprovider, firstname, lastname)
In my test case, I expect return value to be None but I am getting Some(user)
"UsersRepository" should {
"not save a new user if the user already exist in the system" in {
val insertUserStatement =
s"""
| INSERT INTO users (bucket,email,firstname,lastname,authprovider,password,confirmed,id,hasher,salt) VALUES
| (1,'${testEnv.email}','fn','ln','${testEnv.loginInfo.providerID}','somePassword',false,${testEnv.mockHelperMethods.getUniqueID()},'someHasher','someSalt')
""".stripMargin
testCassandra.executeScripts(new CqlStatements(insertUserStatement))
val userKeys = UserKeys(1, testEnv.email ,testEnv.loginInfo, "fn", "ln")
val cassandraConnectionService = CassandraConnectionManagementService()
val (cassandraSession,cluster) = cassandraConnectionService.connectWithCassandra()
cassandraConnectionService.initKeySpace(cassandraSession,"mykeyspace")
val userRepository = new UsersRepository(testEnv.mockHelperMethods,cassandraSession,"users")
val resultCheckUser = await[Option[User]](userRepository.findOne(userKeys))(Timeout(Duration(5000,"millis")))
val user = User(UUID.fromString("11111111-1111-1111-1111-111111111111"),
UserProfile(
Some(InternalUserProfile(LoginInfo("credentials","test#test.com"),1,false,Some(PasswordInfo("someHasher","somePassword",None)))),
ExternalUserProfile("test#test.com","fn","ln",None)))
println(s"found initial user result ${resultCheckUser}")
resultCheckUser mustBe Some(user)
println(s"user already exists. Will try to add duplicate ")
println(s"adding user with user ${user}")
val resultAddUser = await[Option[User]](userRepository.save(user))(Timeout(Duration(5000,"millis")))
resultAddUser mustBe None
}
}
Output of test execution
insert query is INSERT INTO users (bucket,email,authprovider,firstname,lastname,confirmed,id,password,hasher,salt) VALUES (1,'test#test.com','credentials','fn','ln',false,11111111-1111-1111-1111-111111111111,'somePassword','someHasher','') IF NOT EXISTS;
[info] c.g.n.e.c.Cassandra - INFO [Native-Transport-Requests-1] 2019-06-07 06:13:57,659 OutboundTcpConnection.java:108 - OutboundTcpConnection using coalescing strategy DISABLED
[info] c.g.n.e.c.Cassandra - INFO [HANDSHAKE-localhost/127.0.0.1] 2019-06-07 06:13:57,683 OutboundTcpConnection.java:560 - Handshaking version with localhost/127.0.0.1
resultset after insert: ResultSet[ exhausted: false, Columns[[applied](boolean), bucket(int), email(varchar), authprovider(varchar), firstname(varchar), lastname(varchar), confirmed(boolean), hasher(varchar), id(uuid), password(varchar), salt(varchar)]]
running afterEach statements
afterEach: cassandra state is STARTED
[debug] c.g.n.e.c.t.TestCassandra - Stop TestCassandra 3.11.1
Some(User(11111111-1111-1111-1111-111111111111,UserProfile(Some(InternalUserProfile(LoginInfo(credentials,test#test.com),1,false,Some(PasswordInfo(someHasher,somePassword,None)))),ExternalUserProfile(test#test.com,fn,ln,None)))) was not equal to None
ScalaTestFailureLocation: UnitSpecs.RepositorySpecs.UsersRepositorySpecs at (UsersRepositorySpecs.scala:362)
Expected :None
Actual :Some(User(11111111-1111-1111-1111-111111111111,UserProfile(Some(InternalUserProfile(LoginInfo(credentials,test#test.com),1,false,Some(PasswordInfo(someHasher,somePassword,None)))),ExternalUserProfile(test#test.com,fn,ln,None))))
executeQuery returns ResultSet which has wasApplied method. This method returns true if the insert operation was done, otherwise it returns false. If a record is duplicate, wasApplied is false.
try {
val resultSet = session.execute(insertQuery) //execute can take a Statement. Insert is derived from Statement so I can use Insert.
println("resultset after insert: " + resultSet)
if(resultSet.wasApplied()){
Some(data)
} else {
None
}
}catch {
case e:Exception => {
println("cassandra exception "+e)
None
}
}

Grails 1.3.7 - multi tenant plugin and multi threading operation - thread cannot access to tenant

As you can understand by the title my situation is:
I'm using a very old version of Grails (I can not change it at the moment)
In this project I'm using multitenant plugin, in detail i've installed
multi-tenant-core:1.0.3
multi-tenant-ehcache:1.0.1
I want split a method (it takes 2-3 minutes to execute it) and parallelize the operations.
Here is my code:
def threadPoolSize = 10
def threadPool = Executors.newFixedThreadPool(threadPoolSize)
def threadClosure = { myWork ->
def partialQuantity = 0
myWork.each { currDetail ->
MyTableDomainClass.findByCode(currDetail.myTableCode)
// Do some stuff
}
return partialQuantity
}
try{
def worksForThreads = new ArrayList<org.codehaus.groovy.grails.web.json.JSONArray>(10)
// Prepare works for thread
Integer x = 0
allWorks.each{ singleOrder ->
if(worksForThreads[x] == null)
worksForThreads[x] = new org.codehaus.groovy.grails.web.json.JSONArray()
worksForThreads[x].add(singleOrder)
x = (x+1) % threadPoolSize
}
List<Future> futures = worksForThreads.collect({ myWork ->
println "\t\tPrepare thread with ${myWork.size()} tasks"
threadPool.submit({ ->
threadClosure myWork
} as Callable )
})
//Start thread and collect result
futures.each{
def threadResult = it.get()
println "\t\tThread result ${threadResult}"
totQuantity = totQuantity+threadResult
}
}catch(Exception e){
println "Error during thread works ${e}"
}finally{
threadPool.shutdown()
}
So, the code should be fine, but during the thread's execution I get these errors:
[pool-9-thread-2] [tenant 0] ERROR util.JDBCExceptionReporter - Table 'MySchema.MyTable' doesn't exist
and
Error during thread works java.util.concurrent.ExecutionException: org.springframework.dao.InvalidDataAccessResourceUsageException: could not execute query;
SQL [select this_.id as id24_0_, this_.code as code24_0_, this_.description as descript3_24_0_, this_.incr as incr24_0_, this_.is_default as is5_24_0_, this_.lang as lang24_0_, this_.last_updated as last7_24_0_, this_.min as min24_0_, this_.name as name24_0_, this_.ordasc as ordasc24_0_, this_.parent_id as parent11_24_0_, this_.parent_mult as parent12_24_0_, this_.prod_code as prod13_24_0_, this_.status as status24_0_, this_.unit_name as unit15_24_0_
from MyTable this_ where this_.code=?]; nested exception is org.hibernate.exception.SQLGrammarException: could not execute query
I think that the problem is with the tenant scope. I thought to use a service for getting all data from DB, but I want know if there is a way to get the right tenant scope in a thread or to pass it.
Thanks to all!
I have find out a solution. Basically the multitenant plugin fetch the tennants by a database table:
So in the Config.groovy I have
tenant {
mode = "singleTenant"
datasourceResolver.type = "db"
}
And the DataSource.groovy provide for each environment the right table where find out the tennants:
CREATE TABLE `data_source_tenant_map` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`version` bigint(20) NOT NULL,
`data_source` varchar(255) NOT NULL COMMENT 'JNDI',
`mapped_tenant_id` int(11) NOT NULL,
`file_source` varchar(255) NOT NULL,
`data_source_secondary` varchar(255) DEFAULT NULL,
`db_url` varchar(1000) DEFAULT NULL COMMENT 'Url db multitenant',
`db_username` varchar(100) DEFAULT NULL COMMENT 'Username db multitenant',
`db_password` varchar(100) DEFAULT NULL COMMENT 'Password db multitenant',
`base_path` varchar(255) DEFAULT NULL COMMENT 'Root of path on fs',
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=280 DEFAULT CHARSET=utf8;
With this plugin every thread that is not joined with the main thread loses the Hibernate session and so the tenant reference. To obtain the right tenant I used these method:
def myTenantKey = my_mapped_tenant_id
TenantUtils.doWithTenant(myTenantKey ) {
// Do something as usual
}
You have to import the TenantUtils class
import grails.plugin.multitenant.core.util.TenantUtils
I hope it can be useful for someone else!
Bye!

transforming strings in a golang struct

I've got a json file of AES encrypted secrets. The structure is:
{
"username": "asdf123ASLdf3",
"password": "elisjdvo4etQW"
}
And a struct to hold these values
type Secrets struct {
Username string `json:"username"`
Password string `json:"password"`
}
It's easy to load the encrypted json values into the struct, but what I really want is a struct with the unencrypted values.
So, for each value, I'd like to run it though a function:
aesDecrypt(key string, value string) string
I'm happy to have this done on the first load, or to move everything over into a new struct.
I would like to avoid repeating the json keys or the field names.
What's the best way to do this?
(Also open to other ways to manage encrypted secrets in Go)
One option is to define a custom JSON Unmarshaler. Another is, as you mention, copy it to another struct.
Implementing the Unmarshaler interface
The key insight is knowing that you can override json.Unmarshal's
behaviour by implementing the Unmarshaler interface. In our
case, that means defining a function func (ss *Secrets)
UnmarshalJSON(bb []byte) error that will do the AES Decryption when
you try to unmarshal any JSON to a Secrets.
package main
import "fmt"
import "encoding/json"
type Secrets struct {
Username string `json:"username"`
Password string `json:"password"`
}
func main() {
jj := []byte(`{
"username": "asdf123ASLdf3",
"password": "elisjdvo4etQW"
}`)
var ss Secrets
json.Unmarshal(jj, &ss)
fmt.Println(ss)
}
func aesDecrypt(key, value string) string {
return fmt.Sprintf("'%s' decrypted with key '%s'", value, key)
}
func (ss *Secrets) UnmarshalJSON(bb []byte) error {
var objmap map[string]*string
err := json.Unmarshal(bb, &objmap)
ss.Username = aesDecrypt("my key", *objmap["password"])
ss.Password = aesDecrypt("my key", *objmap["username"])
return err
}
This outputs a Secrets struct:
{'elisjdvo4etQW' decrypted with key 'my key'
'asdf123ASLdf3' decrypted with key 'my key'}
See it in action at the Go Playground.
Copying to another struct
You could simply make a new Secrets struct every time you need to
decrypt the JSON. This could be tedious if you do it alot, or if you
have no need for the intermediate state.
package main
import "fmt"
import "encoding/json"
type Secrets struct {
Username string `json:"username"`
Password string `json:"password"`
}
func main() {
jj := []byte(`{
"username": "asdf123ASLdf3",
"password": "elisjdvo4etQW"
}`)
var ss Secrets
json.Unmarshal(jj, &ss)
decoded := Secrets{
aesDecrypt(ss.Username, "my key"),
aesDecrypt(ss.Password, "my key")}
fmt.Println(decoded)
}
func aesDecrypt(key, value string) string {
return fmt.Sprintf("'%s' decrypted with key '%s'", value, key)
}
Check it out at Go Playground.
This has the same output as above:
{'elisjdvo4etQW' decrypted with key 'my key'
'asdf123ASLdf3' decrypted with key 'my key'}
Obviously, you would use a different version of aesDecrypt, mine's
just a dummy. And, as always, you should actually be checking the
returned errors in your own code.

Adobe Air: convert sqlite's result [object Object] to String?

I am currently trying to do retrieve text from sqlite. I see that the amount of data requested do come correctly, but content, on the other hand, seems to be in an incorrect format. I've tried some conversion:
var data:Array = sqls.getResult().data;
var stData:String = String(data[0]);
Alert.show(stData); // <--- displays "[object Object]"
String conversion does not seem to do what I want. I simply want the text from the sqlite database. How can I convert the [object Object] to the correct string in this case?
Irrespective of what rows are returned (with(out)) specifying columns, unless the itemClass property of the SQLStatement is defined, it will always return an anonymous object. This is essentially how remoting works with AMF.
There are two things you can do (depending on the complexity of your project):
Specify an SQLStatement.itemClass - this will define & popuplate the results of the return with public accessors (either var or get/set) with the same name as the column name.
If left as anonymous - the column names are attached to the object in which you iterate the object just as you would if it was defined.
Super basic example:
//SQL table schema
CREATE TABLE accounts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
num INTEGER NOT NULL,
name TEXT NOT NULL,
last_update DATE
);
//Define an Account class:
public class Account {
public var id:int;
public var num:int;
public var name:String;
public var last_update:Date;
}
//A statement to execute to get all accounts returned as an array of "Account"
var statement:SQLStatement = new SQLStatement();
statement.sqlConnection = myConn;
statement.itemClass = Account;
statement.text = 'SELECT * FROM accounts';
statement.addEventListener(SQLEvent.RESULT, onResults);
statement.execute();
protected function onResults(event:SQLEvent):void
{
var statement:SQLStatement = SQLStatement(event.currentTarget);
var results:SQLResult = statement.getResult();
for each (var account:Account in results)
{
//do something useful like populate a model, grid, etc...
}
}
//Anonymous objects would iterate much the same way -when an item class isn't defined
protected function onResults(event:SQLEvent):void
{
var statement:SQLStatement = SQLStatement(event.currentTarget);
var results:SQLResult = statement.getResult();
for each (var account:Object in results)
{
//our 'Object' will have properties: id, num, name, last_update
//do something useful like populate a model, grid, etc...
}
}
Please try with Objectutil.toString(); function which converts an object of any kind to a string.

Resources