So far, this is what I understand of the current Cassandra architecture:
Super columns are not desirable any more due to performance issues.
Composite columns (actually keys) are a good choice for indexing hierarchical keys.
Composite columns store nested components in sorted order. There is no actual index.
I have some questions:
Is everything I stated correct?
Can composite columns efficiently process range queries per component (assuming logical usage)?
Are composite columns suited to extremely large numbers of rows while still yielding rapid query results (considering they are not an index per se)?
Can secondary indexes be created against composite columns. If yes, can range queries be efficiently performed?
Thanks in advance.
Yes
Yes
Yes, because they are sorted on write just like any other column
Yes, secondaries can be created against composites as of 1.2. See this JIRA ticket
Related
I've been doing a lot of reading lately on Cassandra data modelling and best practices.
What escapes me is what the best practice is for choosing a partition key if I want an application to page through results via the token function.
My current problem is that I want to display 100 results per page in my application and be able to move on to the next 100 after.
From this post: https://stackoverflow.com/a/24953331/1224608
I was under the impression a partition key should be selected such that data spreads evenly across each node. That is, a partition key does not necessarily need to be unique.
However, if I'm using the token function to page through results, eg:
SELECT * FROM table WHERE token(partitionKey) > token('someKey') LIMIT 100;
That would mean that the number of results returned from my partition may not necessarily match the number of results I show on my page, since multiple rows may have the same token(partitionKey) value. Or worse, if the number of rows that share the partition key exceeds 100, I will miss results.
The only way I could guarantee 100 results on every page (barring the last page) is if I were to make the partition key unique. I could then read the last value in my page and retrieve the next query with an almost identical query:
SELECT * FROM table WHERE token(partitionKey) > token('lastKeyOfCurrentPage') LIMIT 100;
But I'm not certain if it's good practice to have a unique partition key for a complex table.
Any help is greatly appreciated!
But I'm not certain if it's good practice to have a unique partition key for a complex table.
It depends on requirement and Data Model how you should choose your partition key. If you have one key as partition key it has to be unique otherwise data will be upsert (overridden with new data). If you have wide row (a clustering key), then make your partition key unique (a key that appears once in a table) will not serve the purpose of wide row. In CQL “wide rows” just means that there can be more than one row per partition. But here there will be one row per partition. It would be better if you can provide the schema.
Please follow below link about pagination of Cassandra.
You do not need to use tokens if you are using Cassandra 2.0+.
Cassandra 2.0 has auto paging. Instead of using token function to
create paging, it is now a built-in feature.
Results pagination in Cassandra (CQL)
https://www.datastax.com/dev/blog/client-side-improvements-in-cassandra-2-0
https://docs.datastax.com/en/developer/java-driver/2.1/manual/paging/
Saving and reusing the paging state
You can use pagingState object that represents where you are in the result set when the last page was fetched.
EDITED:
Please check the below link:
Paging Resultsets in Cassandra with compound primary keys - Missing out on rows
I recently did a POC for a similar problem. Maybe adding this here quickly.
First there is a table with two fields. Just for illustration we use only few fields.
1.Say we insert a million rows with this
Along comes the product owner with a (rather strange) requirement that we need to list all the data as pages in the GUI. Assuming that there are hundred entries 10 pages each.
For this we update the table with a column called page_no.
Create a secondary index for this column.
Then do a one time update for this column with page numbers. Page number 10 will mean 10 contiguous rows updated with page_no as value 10.
Since we can query on a secondary index each page can be queried independently.
Code is self explanatory and here - https://github.com/alexcpn/testgo
Note caution on how to use secondary index properly abound. Please check it. In this use case I am hoping that i am using it properly. Have not tested with multiple clusters.
"In practice, this means indexing is most useful for returning tens,
maybe hundreds of results. Bear this in mind when you next consider
using a secondary index." From http://www.wentnet.com/blog/?p=77
I have a table like this
CREATE TABLE my_table(
category text,
name text,
PRIMARY KEY((category), name)
) WITH CLUSTERING ORDER BY (name ASC);
I want to write a query that will sort by name through the entire table, not just each partition.
Is that possible? What would be the "Cassandra way" of writing that query?
I've read other answers in the StackOverflow site and some examples created single partition with one id (bucket) which was the primary key but I don't want that because I want to have my data spread across the nodes by category
Cassandra doesn't support sorting across partitions; it only supports sorting within partitions.
So what you could do is query each category separately and it would return the sorted names for each partition. Then you could do a merge of those sorted results in your client (which is much faster than a full sort).
Another way would be to use Spark to read the table into an RDD and sort it inside Spark.
Always model cassandra tables through the access patterns (relational db / cassandra fill different needs).
Up to Cassandra 2.X, one had to model new column families (tables) for each access pattern. So if your access pattern needs a specific column to be sorted then model a table with that column in the partition/clustering key. So the code will have to insert into both the master table and into the projection table. Note depending on your business logic this may be difficult to synchronise if there's concurrent update, especially if there's update to perform after a read on the projections.
With Cassandra 3.x, there is now materialized views, that will allow you to have a similar feature, but that will be handled internally by Cassandra. Not sure it may fit your problem as I didn't play too much with 3.X but that may be worth investigation.
More on materialized view on their blog.
I have a situation where I have a large partition/row with many cells/values. I need to query this row for all the cells sorted by a value (one of the keys). This sort value is dynamic, and changes of often. You can't update any of the primary keys of cassandra because it changes how the data is stored. So, how do I do this? Does cassandra not support normalized queries that the sort can change at any given moment?
Cassandra does not support normalized queries where the sort can change at any given moment. You can do sort on the client or using additional tools like Spark.
we have a table with 15 million records, and ours is a 10 node cassandra cluster. We have a column which has close to 20 repeatable values. Is it advisable to build secondary index on this column?
Assuming completely uniform distribution on that column, then each column value would map to 750,000 rows. Now while the DataStax doc on When To Use An Index states that...
built-in indexes are best on a table having many rows that contain the indexed value.
750,000 rows certainly qualifies as "many." But even given that, remember that you're also talking about 14,250,000 rows that Cassandra has to ignore when fulfilling your query.
Also, unless you have a RF of 10 (and I doubt that you would with 10 nodes), you are going to incur network time as Cassandra works between all of the different nodes required to fulfill your query. For 750,000 rows, that's probably going to timeout.
The only way I think this could be efficient, would be to first restrict your query by a partition key. Using the secondary index while also restricting with a partition key will help Cassandra find your rows more quickly. Even so, with a dataset that big, I would re-evaluate your data model and try to figure out a different table to fulfill that query without requiring a secondary index.
I've to test different datamodels for Cassandra. I'm thinking about to use a composite key made by key1:key2 for the row key.
With this configuration on Cassandra, for example, I can query to have all the rows having a specific key1 value and any key2 value but It's impossible otherwise (obtain all the rows having a specific key2's value and any key1).
Is it right?
thanks in advance
Cesare
If you use Order Preserving Partitioning (OPP), then yes, the keys will be stored sorted, and then you can get slices over a range of keys e.g. A:A to A:Z -- but not necessarily any:A to any:Z.
But, OPP is not guaranteed to evenly distribute the keys across the nodes and you could end up with "hot spots" of too many or too few keys. You probably want to use Random Partitioning (RP) which distributes the keys by storing by hash across all nodes.
However, since Columns are stored sorted, using Composite values can be pretty powerful for accessing ranges of data.
See this question for details on querying Composite columns using Hector .
If necessary, the column names could then be used as keys to do Multiget queries for additional lookups.
I hope these articles help you :)
http://pkghosh.wordpress.com/2011/03/02/cassandra-secondary-index-patterns/
http://www.datastax.com/docs/0.7/data_model/cfs_as_indexes
http://www.anuff.com/2011/02/indexing-in-cassandra.html
Also checkout this question
Storing a list of values in Cassandra