I have a table query below that search by PK, RK and a non-index key.
Will this slow down database search? Will the search by DOB be faster if it performs on web layer via c#?
var name = "John";
var wins = 20;
var dob = DateTimeOffset.Parse("1999-1-1");
string usernameFilter2 =
TableQuery.CombineFilters(
TableQuery.CombineFilters(
TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, name),
TableOperators.And,
TableQuery.GenerateFilterConditionForInt("RowKey", QueryComparisons.Equal, wins)),
TableOperators.And,
TableQuery.GenerateFilterConditionForDate("DateOfBirth", QueryComparisons.GreaterThanOrEqual, dob));
Considering there can be a single entity for a PartitionKey/RowKey combination, I believe your query's 3rd filter criteria is redundant (assuming you've meant to include another attribute there instead of RowKey).
As far as the speed of the query goes, again considering you're searching for an entity with matching PartitionKey and RowKey, this query will be the fastest. If you remove RowKey from your query and search on PartitionKey and any other non-indexed attribute, it will be slower as the query will be doing a partition scan to find a matching entity.
You may find this article helpful: https://learn.microsoft.com/en-us/azure/cosmos-db/table-storage-design-guide.
Related
I am working on Azure storage table. Below is the partitionKey and rowkey
partitionKey = Product Type
rowKey = Region
I have used below filter to get the data (see the code below). If rowkey (region) is null then it should get product Type from all region. However, It get empty data. If I pass valid Partitionkey(ProductType) and RowKey (region) it's get the data.
AsyncPageable<T> result =TableClient.QueryAsync<T>(filter: $"PartitionKey eq '{partitionKey}' and RowKey eq '{rowKey}'");
Note: Partition-key is mandatory
Any advice.
Thanks
If rowkey (region) is null then it should get product Type from all
region.
If this is what you want, you would need to change the code to something like following:
AsyncPageable<T> result = TableClient.QueryAsync<T>(filter: $"PartitionKey eq '{partitionKey}'");
UPDATE
What I would like is If user has provided ProductType and Region then
filter using both. In case user only provide ProductType then include
all regions.
In this case, you will need to use two different queries - one when both PartitionKey and RowKey are present and other when only PartitionKey is present.
Your code would be something like:
var query = string.IsNullOrWhiteSpace(rowKey) ?
$"PartitionKey eq '{partitionKey}'" : $"PartitionKey eq '{partitionKey}' and RowKey eq '{rowKey}'";
AsyncPageable<T> result = TableClient.QueryAsync<T>(filter: query);
I tried to convert an SQL query into Gosu Script ( Guidewire). My script is working only for limited number of records
This is the SQL query
select PolicyNumber,* from pc_policyperiod
where ID in ( Select ownerID from pc_PRActiveWorkflow
where ForeignEntityID in (Select id from pc_workflow where State=3))
This is my script
var workFlowIDQuery = Query.make(Workflow).compare(Workflow#State,Relop.Equals,WorkflowState.TC_COMPLETED).select({QuerySelectColumns.path(Paths.make(entity.Workflow#ID))}).transformQueryRow(\row ->row.getColumn(0)).toTypedArray()
var prActiveWorkFlowQuery = Query.make(PRActiveWorkflow).compareIn(PRActiveWorkflow#ForeignEntity, workFlowIDQuery).select({QuerySelectColumns.path(Paths.make(entity.PRActiveWorkflow#Owner))}).transformQueryRow(\row -> row.getColumn(0)).toTypedArray()
var periodQuery = Query.make(PolicyPeriod).compareIn(PolicyPeriod#ID,prActiveWorkFlowQuery).select()
for(period in periodQuery){
print(period.policynmber)
}
Can anyone find a cause; why the script results in limited records or suggest improvements?
I would suggest you to write a single Gosu Query to select policyPeriod and join 3 entities with a foreign key to other entity.
I am note sure if the PolicyPeriod ID is same as the PRActiveWorkflow ID. Can you elaborate the relation between PolicyPeriod and PRActiveWorkflow entity ?
Meanwhile reading querying tables and entities in Azure Table Storage service on Microsoft docs I found PartitionKey and RowKeys can be filtered in 2 different ways in the myaccount.table.core.windows.net URL as the following:
https://<service-url>/Customers(PartitionKey='MyPartition',RowKey='MyRowKey1')
And using $filter to achieve the same:
https://<service-url>/Customers()?$filter=PartitionKey%20eq%20'MyPartitionKey'%20and%20RowKey%20eq%20'MyRowKey1'
I understand from the documentation PartitionKey and RowKey properties are forming the entity's primary key so the first syntax can be used as well as the Filtering on the PartitionKey and RowKey Properties part states:
Because the PartitionKey and RowKey properties form an entity's primary key, you can use a special syntax to identify the entity.
Questions:
Is there any drawbacks, disadvantages using $filter instead of that special syntax what has been mentioned?
Does it matter which one I use in my solution?
Any clarification is appreciated, thank you!
Interesing question! So I tried to perform both the operations to fetch an entity using Postman and the time it took to get the data was almost identical (between 250ms - 300ms).
So as long as you're using REST API, I don't think it would matter.
When you use the SDK (WindowsAzure.Storage version 9.3.3 for example), there're different methods that uses these features. Take a look at the sample code below:
var account = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
var tableClient = account.CreateCloudTableClient();
var table = tableClient.GetTableReference("TableName");
//This uses https://account.table.core.windows.net/table(PartitionKey='pk', RowKey='rk')
TableOperation op = TableOperation.Retrieve("pk", "rk");
var entity = table.Execute(op).Result as DynamicTableEntity;
//This uses https://account.table.core.windows.net/table?$filter=PartitionKey eq 'pk' and RowKey eq 'rk'
TableQuery query = new TableQuery();
query.FilterString = "PartitionKey eq 'pk' and RowKey eq 'rk'";
var entity = table.ExecuteQuery(query).FirstOrDefault();
In my cassandra table i have a collection of Map also i have indexed the map keys.
CREATE TABLE IF NOT EXISTS test.collection_test(
name text,
year text,
attributeMap map<text,text>,
PRIMARY KEY ((name, year))
);
CREATE INDEX ON collection_test (attributeMap);
The QueryBuilder syntax is as below:
select().all().from("test", "collection_test")
.where(eq("name", name)).and(eq("year", year));
How should i put where condition on attributeMap?
First of all, you will need to create an index on the keys in your map. By default, an index created on a map indexes the values of the map, not the keys. There is special syntax to index the keys:
CREATE INDEX attributeKeyIndex ON collection_test (KEYS(attributeMap));
Next, to SELECT from a map with indexed keys, you'll need the CONTAINS KEY keyword. But currently, there is not a definition for this functionality in the query builder API. However, there is an open ticket to support it: JAVA-677
Currently, to accomplish this with the Java Driver, you'll need to build your own query or use a prepared statement:
PreparedStatement statement = _session.prepare("SELECT * " +
"FROM test.collection_test " +
"WHERE attributeMap CONTAINS KEY ?");
BoundStatement boundStatement = statement.bind(yourKeyValue);
ResultSet results = _session.execute(boundStatement);
Finally, you should read through the DataStax doc on When To Use An Index. Secondary indexes are known to not perform well. I can't imagine that a secondary index on a collection would be any different.
var result = table.ExecuteQuery(new TableQuery<ImageData>()
.Where(TableQuery.CombineFilters(
TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.GreaterThanOrEqual, "12.38"), "AND",
TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.LessThanOrEqual, "12.50"))
)
.Take(10)).ToArray();
Above filter fails with a bad request. It works if i use either of the combined filters, but the combined filter fails.
Instead of "AND", try TableOperators.And (lowercase "and")