I am creating account data in Cassandra. Accounts are most commonly queried based on an account id. However, often the account is queried by a login name. I have created a user table with primary keys (account_id and login_name). Because of this, I have to "ALLOW FILTERING" on the table to query by the login_name.
Is there a better way to create the table that does not have the impact of a filterable table?
A possible approach is to define a new table that has the exact same elements in the primary key but in the reverse order: (login_name, account_id). You can still keep the table you present as the reference table, which stores all the relevant account data, but this new table would allow you more optimized queries based on login_name. I am not sure how much you would win compared with the query "allowing filtering"... But this kind of "data duplication" for query optimization is a normal procedure in NoSQL DBs.
HTH.
Related
I have 6 tables in my database each consisting of approximate 12-15 columns and they have relationship with its id to main_table. I have to migrate my database to cassandra so I have a question should I create one main_table with consisting multiple columns or different table as in my mysql database.
Will creatimg multiple column take more space or multiple table will take more space
Your line of questioning is flawed. It is a common mistake for DBAs who only have a background in traditional relational databases to view data as normalised tables.
When you switch to NoSQL, you are doing it because you are trying to solve a problem that traditional RDBMS can't. A paradigm shift is required since you can't just migrate relational tables the way they are, otherwise you're back to where you started.
The principal philosophy of data modelling in Cassandra is that you need to design a CQL table for each application query. It is a one-to-one mapping between app queries and CQL tables. The crucial point is you need to start with the app queries, not the tables.
Let us say that you have an application that stores information about users which include usernames, email addresses, first/last name, phone numbers, etc. If you have an app query like "get the email address for username X", it means that you need a table of email addresses and the schema would look something like:
CREATE TABLE emails_by_username (
username text,
email text,
firstname text,
lastname text,
...
PRIMARY KEY(username)
)
You would then query this table with:
SELECT email FROM emails_by_username WHERE username = ?
Another example is where you have an app query like "get the first and last names for a user where email address is Y". You need a table of users partitioned by email instead:
CREATE TABLE users_by_email (
email text,
firstname text,
lastname text,
...
PRIMARY KEY(email)
)
You would query the table with:
SELECT firstname, lastname FROM users_by_email WHERE email = ?
Hopefully with these examples you can see that the disk space consumption is completely irrelevant. What is important is that you design your tables so they are optimised for the application queries. Cheers!
can we sync two tables of two different databases in azure without indexes. As, we all know ,if two databases has to be in sync hub database and member data base should have same schema. But is there chance of avoiding indexes.
please help me with this
Each table needs to have a primary key and you cannot avoid that. Please read the following excerpt from Microsoft documentation:
Each table must have a primary key. Don't change the value of the
primary key in any row. If you have to change a primary key value,
delete the row and recreate it with the new primary key value.
Source here.
I am working on a self-bi application where users can upload their own datasets which are stored in Cassandra tables that are created dynamically. The data is extracted from files that the user can upload. So, each dataset is written into its own Cassandra table modeled based on column headers in the uploaded file while indexing the dimensions.
Once the data is uploaded, the users are allowed to build reports, analyze, etc., from within the application. I need a way to allow users to merge/join data from two or more datasets/tables based on matching keys and write the result into a new Cassandra table. Once a dataset/table is created, it will stay immutable and data is only read from it.
user table 1
username
email
employee id
user table 2
employee id
manager
I need to merge data in user table 1 and user table 2 on matching employee id and write to new table that is created dynamically.
new table
username
email
employee id
manager
What would be the best way to do this?
The only option that you have is to do the join in your application code. There are just few details to suggest a proper solution.
Please add details about table keys, usage patterns... in general, in cassandra you model from usage point of view, i.e. starting with queries that you'll execute on data.
In order to merge 2 tables on this pattern, you have to do it into application, creating the third table (target table ) and fill it with data from both tables. You have to make sure that you read the data in pages to not OOM, it really depends on size of the data.
Another alternative is to build the joins into Spark, but maybe is too over-engineering in your case.
You can have merge table with primary key of user so that merged data goes in one row and that should be unique since it is one time action.
Than when user clicks you can go through one table in batches with fetch size (for java you can check query options but that is a way to have a fixed window which will be loaded and when reached move to next fetch size of elements). Lets say you have fetch size of 1000 items, iterate over them from one table and find matches in second table, and after 1000 is reached place batch of 1000 inserts to new table.
If that is time consuming you can as suggested use some other tool like Apache Spark or Spring Batch and do that in background informing user that it will take place.
In azure table storage is it possible to add a unique constraint on a column of a given table?
Azure Table Storage entities don't have columns. Each entity may have properties and these properties don't need to be the same for each entity.
The only unique constraint is the combination of partition key + row key.
There's an excellent article on Azure table storage here:
http://msdn.microsoft.com/en-us/library/windowsazure/dd179338.aspx
Since it only has a Row Key and a Partition Key, and the Row Key is essentially the primary key, it doesn't look like there's any other constraint you can put on it unless you build that sort of thing into your client/server which uses or exposes it.
From the article:
The Table service does not enforce any schema for tables, so two entities in the same table may have different sets of properties. Developers may choose to enforce a schema on the client side. A table may contain any number of entities.
HTH :)
Each user of my system can have contacts. Each contact has details like Name, Address, Email, Phone, etc.
Do you think is a good idea to store this contacts in Azure Tables? I am worried about the following:
How do I search for a specific field (like Email or Phone)?
How do I get only the contacts belonging to a specific user?
How do I sort the contacts by a field?
I think that contacts could be a good candidate for storing in Table Storage - but only if you can partition on the owning person and never really need to search or aggregate across multiple owning users.
One possible design for this is:
store the contacts once with the owning user as partition key and some unique field for row key, but with the fields as columns within each row.
How do I search for a specific field (like Email or Phone)?
You can then ask table storage to search within a partition - it will then do a scan within that partition - which shouldn't be particularly large or slow for any single partition.
How do I get only the contacts belonging to a specific user?
This is just a simple query by partition key only
How do I sort the contacts by a field?
All results from table storage are sorted by (partitionkey, rowkey) so to sort the contacts for a user, you'll need to query for all of them, and then sort them within your web or worker role.
Other designs are, of course, possible -
e.g. you could store each contact in multiple rows in multiple tables - this would then allow you to have pre-formed sort orders within the table storage.
e.g. you could use separate tables instead of separate partitionkeys for each user - this has the advantage that when you delete a user, you can delete the entire table belonging to that user.
Note... while it's possible to use table storage for this... actually I almost always seem to end up back in SQL Azure at the moment - it's just so much more powerful and predictable (IMO). When the team deliver secondary indexing, then I might be tempted to use it for more of my data.