For a complex real-time Apache Storm topology I need aggregates of my data (stored in CassandraDB) for some computation steps. So far the data is queried when needed with CQL (Cassandra Query Language) and aggregated in a Storm bolt. That is a bit slow, so we want to have the data needed for the aggregation cached. Two option are on the table:
Put the data needed in an indexed Ignite Cache and sliding-window-query it from Storm. In this case we would only need one Cache and use different queries, depending on the aggregation.
Put the data in Cassandras in-memory, off-heap cache.
Argument for Ignite: We only need one indexed cache, while we would need one Cassandra table for each aggregation, for fast access. (Also ACID, but obviously we already live with CAP, so not a strong argument for our architects.)
Argument for Cassandra: We don't need to introduce a new technology.
But: What about speed? How fast would an indexed Ignite cache be compared to an optimized (= own table for each query) in-memory Cassandra?
I believe that in-memory indexed SQL in Ignite would be faster than Cassandra CQL queries. Apache Ignite is ANSI-99 SQL compatible, so you should be able to do all sorts of aggregations, joins, order by, group by, etc.
I will raise a point within the Ignite community to see if Cassandra CQL could be benchmarked against Ignite SQL. When done, will post the results here.
Related
I understand that for Cassandra data is sorted on the cluster key only per partition key.
I am wondering if Cassandra has optimizations on global scans. Lets say that the cluster key is an integer value, if I want to search over all data on a Cassandra cluster to find collections with values < 3. The Cassandra query engine will not need to continue looking at collections in a partition after encountering a number >= 3. Are there APIs (such as CDK) offered by Cassandra which exercise these optimizations?
There isn't a native CQL optimisation available for full table scans -- they will always be bad since Cassandra is optimised for OLTP workloads.
There are however optimisations done by the spark-cassandra-connector for analytics (OLAP) workloads with Spark.
OLTP vs OLAP are worlds apart so you have to use the right tool for the job. Cheers!
Querying by partition key is the best way the query in Cassandra. If you want to use clustering key for querying then you can use "ALLOW FILTERING" option. But it is recommended not to use "ALLOW FILTERING" in production.
For scanning complete table and filtering some data, you can use spark to do your work. Why bother C* for which it is not designed, better to take help of its friends (spark in this case)
I would like to store result of continuous queries running against streaming data in such a manner so that results are persisted into distributed nodes to ensure failover and scalability.
Can Spark SQL experts please shed some light on
- (1) which storage option I should choose so that OLAP queries are faster
- (2) how to ensure data available for query even if one node is down
- (3) internally how does Spark SQL store the resultset ?
Thanks
Kaniska
It depends what kind of latency you can afford.
One way is to persist the result into HDFS/Cassandra using Persist() API. If your data is small then cache() of each RDD should give you a good result.
Store where your spark executors are co-located. For example:
It is also possible to use Memory based storage like tachyon to persist your stream (i.e. each RDD of your stream) and query against it.
If latency is not an issue then persist(MEMORY_OR_DISK_2) should give you what you need. Mind you performance is a hit or miss in that scenario. Also this stores the data in two executors.
In other cases if your clients are more comfortable in OLTP like database where they just need to query the constantly updating result you can use conventional database like postgres or mysql. This is a preferred method among many as query time is consistent and predictable. If the result is not update heavy but partitioned (say by time) then Greenplum like systems are also a choice.
As we read about Cassandra, before, we decided to choose it as our main database.The most important, useful and special feature which encourage us to choose this db, was Scalability, which helps us using Large volumes of data.
But, It seems that, it can not meet our requirements by itself. I asked some questions about our requirement in Stackoverfolw and how we can response them using Cassandra, and the answer was using alternative tools on top of Cassandra as Spark, Solr, DSE Search Tools and etc.
Our case is BIG Data Really, but we will have a large variety of Queries, too.
With these explanations, is it wise to stay on Cassandra?... Or It's better to switch to another db?
Cassandra is not adequate for ad-hoc queries, so I would recommend that you use Hive on Cassandra, mapping your Cassandra tables to Hive tables, usig the connector: cassandra_handler_for_hive, ( and then use hive to do joins and conditions on non partition keys)
I should mention that the performance of queries using Hive with Cassandra is not reasonable, (I have had a case where count(*) on a table with 500M records took 1 hour on 4 nodes). As a work around I used to copy the tables in HDFS after that do the computation using data on HDFS, but this is not good solution if you are seeking the fresh data.
Now for your question: To use Cassandra or not, it depends realy on your needs, Cassandra have a good performance in read/write per second record.
If your needs are met with using Hive/Cassandra to do the queries you need, so why not stay on Cassandra?
We are exploring SPARK for cassandra in order to over come limitations with CQL.
We were initially restricted to CQL but faced few road blocks/hurdles over RDBMS. To name a few as below
For comparing >(Greater than) and < (Less than) on a column, we are restricted to have the columns in Clustering key. Even If I have a column in Clustering, I should still provide the Partition key to do < or > on clustering key.
Can't check for NULL on any column value
In order to query on any column other Partition key, we have to create index on that column
ORDER BY a column which isn't a CLUSTERING KEY
GROUP BY Limitations
Join Tables
I am a newbie with cassandra and end up in revisiting my schema often due to the limitations.
Hence similar to HIVE/PIG for HDFS, What additional benefits does Spark give over CQL ?
CQL is not a replacement for SQL. It is really designed for pulling out values from a few, usually one, partition key, and as you pointed out, does not do any sort of aggregation, grouping, very limited sorting, etc. (though Cassandra 3.0 will have UDFs and UDAs).
Here is what Spark offers over CQL:
General aggregation and querying via DataFrames and SQL, including JOINs, GROUP BY, ORDER BY, and UDFs
Significantly faster queries -- orders of magnitude faster -- if you cache the Cassandra data in memory using sqlContext.cacheTable
Integrated machine learning, statistics, graph processing, and virtually any kind of distributed computation you can imagine, using Scala, Java, Python, and R APIs
Ability to ETL in and out of Cassandra tables from and to many other data sources - including various HDFS formats, Amazon S3, DBMSes, Mongo, and most other databases today
Spark is really a completely different beast from CQL. It offers complex analytics over vast quantities of data, CQL doesn't. However, there are some limitations as well:
Spark is not good at highly concurrent queries. For that, you want to keep queries simple and use CQL to pull out a very small amount of data.
Caching data in Spark is not HA and cannot update as you write new data into C*
If you want very fast analytical queries over Cassandra with support for updates and no need to cache, then check out my project http://github.com/tuplejump/FiloDB.
I would like to hear your thoughts and experiences on the usage of CQL and in-memory query engine Spark/Shark. From what I know, CQL processor is running inside Cassandra JVM on each node. Shark/Spark query processor attached with a Cassandra cluster is running outside in a separated cluster. Also, Datastax has DSE version of Cassandra which allows to deploy Hadoop/Hive. The question is in which use case we would pick a specific solution instead of the other.
I will share a few thoughts based on my experience. But, if possible for you, please let us know about your use-case. It'll help us in answering your queries in a better manner.
1- If you are going to have more writes than reads, Cassandra is obviously a good choice. Having said that, if you are coming from SQL background and planning to use Cassandra then you'll definitely find CQL very helpful. But if you need to perform operations like JOIN and GROUP BY, even though CQL solves primitive GROUP BY use cases through write time and compact time sorts and implements one-to-many relationships, CQL is not the answer.
2- Spark SQL (Formerly Shark) is very fast for the two reasons, in-memory processing and planning data pipelines. In-memory processing makes it ~100x faster than Hive. Like Hive, Spark SQL handles larger than memory data types very well and up to 10x faster thanks to planned pipelines. Situation shifts to Spark SQL benefit when multiple data pipelines like filter and groupBy are present. Go for it when you need ad-hoc real time querying. Not suitable when you need long running jobs over gigantic amounts of data.
3- Hive is basically a warehouse that runs on top of your existing Hadoop cluster and provides you SQL like interface to handle your data. But Hive is not suitable for real-time needs. It is best suited for offline batch processing. Doesn't need any additional infra as it uses underlying HDFS for data storage. Go for it when you have to perform operations like JOIN, GROUP BY etc on large dataset and for OLAP.
Note : Spark SQL emulates Apache Hive behavior on top of Spark, so it supports virtually all Hive features but potentially faster. It supports the existing Hive Query language, Hive data formats (SerDes), user-defined functions (UDFs), and queries that call external scripts.
But I think you will be able to evaluate the pros and cons of all these tools properly only after getting your hands dirty. I could just suggest based on your questions.
Hope this answers some of your queries.
P.S. : The above answer is based on solely my experience. Comments/corrections are welcome.
There is a very good effort for benchmark documented here - https://amplab.cs.berkeley.edu/benchmark/