Cassandra: customer data per keyspace - cassandra

Question

Problem: one of our new customers want the data to be stored in his own country (law regulations). However we use existing customer's data spread across few datacenters in different countries.
Question: how we can separate new customer's data to reside in its own country without much changing existing Cassandra architecture?
Potential Solution #1: to use separate keyspace for this customer. Schemas will be the same between keyspaces what adds the complexity for data migration and so on. DataStax support confirmed that it is possible to configure keyspace per region.
However Spring Data Cassandra we use, doesn't allow to choose keyspace dynamically.
The only way is to use CqlTemplate and to run use keyspace blabla everytime before the call or to add keyspace before the table select * from blabla.mytable but it sounds as a hack for me.
Potential Solution #2 to use separate environment for new client but management rejects to do it.
Any other ways to achieve this goal?

Answer 1

Update 3
Example and explanation below is same as in GitHub
Update 2
The example in GitHub is now working. The most future proof solution seemed to be using repository extensions. Will update the example below soon.
Update
Notice that the solution I originally posted had some flaws that I discovered during JMeter tests. The Datastax Java driver reference advises to avoid setting keyspace through Session object. You have to set keyspace explicitly in every query.
I've updated the GitHub repository and also changed solution's description.
Be very careful though: if the session is shared by multiple threads,
switching the keyspace at runtime could easily cause unexpected query failures.
Generally, the recommended approach is to use a single session with no
keyspace, and prefix all your queries.
Solution Description
I would set-up a separate keyspace for this specific customer and provide support for changing keyspace in the application. We used this approach previously with RDBMS and JPA in production. So, I would say it can work with Cassandra as well. Solution was similar as below.
I will describe briefly how to prepare and set-up Spring Data Cassandra to configure target keyspace on each request.
Step 1: Preparing your services
I would define first how to set the tenant ID on each request. A good example would be in-case-of REST API is to use a specific HTTP header that defines it:
Tenant-Id: ACME
Similarly on every remote protocol you can forward tenant ID on every message. Let's say if you're using AMQP or JMS, you can forward this inside message header or properties.
Step 2: Getting tenant ID in application
Next, you should store the incoming header on each request inside your controllers. You can use ThreadLocal or you can try using a request-scoped bean.
#Component
#Scope(scopeName = "request", proxyMode= ScopedProxyMode.TARGET_CLASS)
public class TenantId {
private String tenantId;
public void set(String id) {
this.tenantId = id;
}
public String get() {
return tenantId;
}
}
#RestController
public class UserController {
#Autowired
private UserRepository userRepo;
#Autowired
private TenantId tenantId;
#RequestMapping(value = "/userByName")
public ResponseEntity<String> getUserByUsername(
#RequestHeader("Tenant-ID") String tenantId,
#RequestParam String username) {
// Setting the tenant ID
this.tenantId.set(tenantId);
// Finding user
User user = userRepo.findOne(username);
return new ResponseEntity<>(user.getUsername(), HttpStatus.OK);
}
}
Step 3: Setting tenant ID in data-access layer
Finally you should extend Repository implementations and set-up keyspace according to the tenant ID
public class KeyspaceAwareCassandraRepository<T, ID extends Serializable>
extends SimpleCassandraRepository<T, ID> {
private final CassandraEntityInformation<T, ID> metadata;
private final CassandraOperations operations;
#Autowired
private TenantId tenantId;
public KeyspaceAwareCassandraRepository(
CassandraEntityInformation<T, ID> metadata,
CassandraOperations operations) {
super(metadata, operations);
this.metadata = metadata;
this.operations = operations;
}
private void injectDependencies() {
SpringBeanAutowiringSupport
.processInjectionBasedOnServletContext(this,
getServletContext());
}
private ServletContext getServletContext() {
return ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes())
.getRequest().getServletContext();
}
#Override
public T findOne(ID id) {
injectDependencies();
CqlIdentifier primaryKey = operations.getConverter()
.getMappingContext()
.getPersistentEntity(metadata.getJavaType())
.getIdProperty().getColumnName();
Select select = QueryBuilder.select().all()
.from(tenantId.get(),
metadata.getTableName().toCql())
.where(QueryBuilder.eq(primaryKey.toString(), id))
.limit(1);
return operations.selectOne(select, metadata.getJavaType());
}
// All other overrides should be similar
}
#SpringBootApplication
#EnableCassandraRepositories(repositoryBaseClass = KeyspaceAwareCassandraRepository.class)
public class DemoApplication {
...
}
Let me know if there are any issues with the code above.
Sample code in GitHub
https://github.com/gitaroktato/spring-boot-cassandra-multitenant-example
References
Example: Using JPA interceptors
Spring #RequestHeader example
Spring request-scoped beans
Datastax Java Driver Reference

Answer 2

After many back and forth, we have decided to not do the dynamic keyspace resolution within the same JVM.
It was taken the decision to have dedicated Jetty/Tomcat per keyspace and on nginx router level to define to what server the request should be redirected to (based on companyId from the request url).
For example, all our endpoints have /companyId/<value> so based on the value, we can redirect the request to proper server which uses correct keyspace.

Answer 3

Advice with 2 keyspaces is correct.
If question is about having just 2 keyspaces why not configure 2 keyspaces.
for Region Dependent client - write to both
for others - write to one (main) keyspace only.
No data migration will be required.
Here is sample how to configure Spring Repositories to hit different keyspaces:
http://valchkou.com/spring-boot-cassandra.html#multikeyspace
the choice of repository can be simple if else
if (org in (1,2,3)) {
repoA.save(entity)
repoB.save(entity)
} else {
repoA.save(entity)
}