I have the following data structure (in pseudo code):
class GroupedData
{
String key;
List<Tuple<String, String>> records;
}
I figured the best way to model this was by doing something like
#Table(name="record_by_group)
class DataWithGroup{
#PrimaryKey(name="group_key")
String groupKey;
#PrimaryKey(name="data_key"
String dataKey;
String data;
}
I would then stream the GroupedData into a DataWithGroup using batched operations with inside spring. I am doing this inside a webflux application so i am using the reactive cassandra repository what I have noticed is that there is a vanilla CassandraBatchOperations but there is no ReactiveCassandraBatchOperations. I am wondering if I am missing something or is there a way to insert batch reactively. Alternatively how do I insert a something structured like the GroupedData into cassandra I think I should be using composite columns but I couldn't really get my head round them let alone figure out how to map them in using spring-data.
Related
I am using a dynamic data structure for my project. So instead of a predefined class I am using java.util.LinkedHashMap to store my dynamic data and preserve my insertion order as well.
I am able to convert the map to json and get the map and back from Json using ``.
fasterxml.jackson.databind.ObejctMapper mapper;
LinkedHashMap<String, Object> map =
mapper.readValue(json, new TypeReference<LinkedHashMap<String, Object>>() {});
String json = mapper.writeValueAsString(map);
I am trying to do some XSLT transformation on my map data. So I also need to transform from xml to map and map to xml. As there is no direct method to convert these I wrote my own utility for map to xml.
and to convert from xml to map I used -> org.json.JSONObject. I first convert the xml to json using
org.json.XML.toJSONObject(xmlstring)
and can convert the json to map easily using object mapper.
But the problem here is I am loosing the insertion order which is crucial for my data.
How can I convert my data from xml to json so that the insertion order is preserved.
Thats a superb idea to use LinkedHashMap for dynamic data structure.
However JSONObject internally uses HashMap to create the json. So it looses the insertion order.
public JSONObject() {
// HashMap is used on purpose to ensure that elements are unordered by
// the specification.
// JSON tends to be a portable transfer format to allows the container
// implementations to rearrange their items for a faster element
// retrieval based on associative access.
// Therefore, an implementation mustn't rely on the order of the item.
this.map = new HashMap<String, Object>();
}
So If you can override the JSONObject your problem will be solved.
Enjoy!!!
You don't need to change the jar. You just need to create a new class with the same name of the class inside the jar and also have to create the new classes those are dependent on the class.
You need to copy the code from the jar class to your new class and tweek.
then access that class from you code by changing the import statement.
Using Room and RxJava3, my DAO interfaces always have methods with different return types (Maybe and Flowable) but with the exact same query. This is how I solve an important issue, which is that whenever I use Flowables, I want an empty list emitted immediately when the table is empty (rather than nothing at all).
Duplicating the query string may introduce bugs if I ever get sloppy and forget to update all of them. Now that I found I can still get syntax highlighting in Android Studio when storing the query, I came up with the following.
String query = "SELECT * FROM MyObject"; // public, as DAO is a public java interface
#Query(query)
Maybe<List<MyObject>> maybeMyObjectList();
#Query(query)
Flowable<List<MyObject>> flowableMyObjectList();
This enables me to do things like flowableMyObjectList().startWith(maybeMyObjectList().defaultIfEmpty(Collections.emptyList())).distinctUntilChanged()
Still, having SQL queries stored in a public string feels like a bad idea, security-wise. On the other hand, I don't think the database schema in my app bundle is supposed to be secret anyway. Can anyone with a better knowledge than mine confirm that it is a bad as it sounds, or better propose a workaround?
Instead of an interface you can use an abstract class, thus you can then have methods with bodies and private variables.
You then have to make the methods abstract.
So you could have:-
#Dao
abstract class TheDao {
private static final String query = "SELECT * FROM MyObject";
#Query(query)
abstract Maybe<List<MyObject>> maybeMyObjectList();
#Query(query)
abstract Flowable<List<MyObject>> flowableMyObjectList();
}
Question for those who knows Presto API for plugins.
I implement BigQuery plugin. BigQuery supports struct type, which could be represented as RowType class in Presto.
RowType creates RowBlockBuilder in RowType::createBlockBuilder, which has RowBlockBuilder::appendStructure method, which requires to accept only instance of AbstractSingleRowBlock class.
This means that in my implementation of Presto's RecordCursor BigQueryRecordCursor::getObject method I had to return something that is AbstractSingleRowBlock for field which has type RowType.
But AbstractSingleRowBlock has package private abstract method, which prevents me from implementing this class. The only child SingleRowBlock has package private constructor, and there are no factories or builders that could build an instance for me.
How to implement struct support in BigQueryRecordCursor::getObject?
(reminding: BigQueryRecordCursor is a child of RecordCursor).
You need to assemble the block for the row by calling beginBlockEntry, appending the values for each column via Type.writeXXX with the column's type and then closeEntry. Here's some pseudo-code.
BlockBuilder builder = type.createBlockBuilder(..);
builder = builder.beginBlockEntry();
for each column {
...
columnType.writeXXX(builder, ...);
}
builder.closeEntry();
return (Block) type.getObject(builder, 0);
However, I suggest you use the columnar APIs, instead (i.e., ConnectorPageSource and friends). Take a look at how the Elasticsearch connector implements it:
https://github.com/prestosql/presto/blob/master/presto-elasticsearch/src/main/java/io/prestosql/elasticsearch/ElasticsearchPageSourceProvider.java
https://github.com/prestosql/presto/blob/master/presto-elasticsearch/src/main/java/io/prestosql/elasticsearch/ElasticsearchPageSource.java
Here's how it handles Row types:
https://github.com/prestosql/presto/blob/master/presto-elasticsearch/src/main/java/io/prestosql/elasticsearch/decoders/RowDecoder.java
Also, I suggest you join #dev channel on the Presto Community Slack, where all the Presto developers hang out.
I was going through this ticket and could not understand if Spark support UDT in 2.3+ version in any language (Scala, Python , Java, R) ?
I have class something like this
Class Test{
string name;
int age;
}
And My UDF method is:
public Test UDFMethod(string name, int age){
Test ob = new Test();
ob.name = name;
ob.age = age;
}
Sample Spark query
Select *, UDFMethod(name, age) From SomeTable;
Now UDFMethod(name, age) will return Test object. So will this work in Spark SQL after using SQLUserDefinedType tag and extending UserDefinedType class?
As UserDefinedType class was made private in Spark 2.0. I just want to know if UDT is supported in Spark 2.3+. If yes what is the best to use UserDefinedType or UDTRegisteration. As of now both are private in spark.
As you can check, JIRA ticket you've linked has been delayed to at least Spark 3.0. So it means that there is no such option intended for public usage for now.
It is always possible to get around access limits (by reflection, by putting your own code in Spark namespace), but it is definitely not supported, and you shouldn't expect help, if it fails or breaks in the future.
I need to query a table using FreeTextTable (because I need ranking), with SubSonic. AFAIK, Subsonic doesn't support FullText, so I ended up creating a simple UDF function (Table Function) which takes 2 params (keywords to search and max number of results).
Now, how can I inner join the main table with this FreeTextTable?
InlineQuery is not an option.
Example:
table ARTICLE with fields Id, ArticleName, Author, ArticleStatus.
The search can be done by one of more of the following fields: ArticleName (fulltext), Author (another FullText but with different search keywords), ArticleStatus (an int).
Actually the query is far more complex and has other joins (depending on user choice).
If SubSonic cannot handle this situation, probably the best solution is good old plain sql (so there would be no need to create an UDF, too).
Thanks for your help
ps: will SubSonic 3.0 handle this situation?
3.0 can do this for you but you'd need to make a template for it since we don't handle functions (yet) out of the box. I'll be working on this in the coming weeks - for now I don't think 2.2 will do this for you.
I realize your question is more complex than this, but you can get results from a table valued function via SubSonic 2.2 with a little massaging.
Copy the .cs file from one of your generated views into a safe folder, and then change all the properties to match the columns returned by your UDF.
Then, on your collection, add a constructor method with your parameters and have it execute an InlineQuery.
public partial class UDFSearchCollection
{
public UDFSearchCollection(){}
public UDFSearchCollection(string keyword, int maxResults)
{
UDFSearchCollection coll = new InlineQuery().ExecuteAsCollection<UDFSearchCollection>("select resultID, resultColumn from dbo.udfSearch(#keyword, #maxResults)",keyword,maxResults);
coll.CopyTo(this);
coll = null;
}
}
public partial class UDFSearch : ReadOnlyRecord<UDFSearch>, IReadOnlyRecord
{
//all the methods for read only record go here
...
}
An inner join would be a little more difficult because the table object doesn't have it's own parameters collection. But it could...