I have the following cards container in my cosmos DB:
Card: {
createdAt:"2022-10-17T11:36:53"
modifiedAt:"2022-10-17T11:36:53"
id:"49a9f63c-cca6-4424-8eb1-3f742f4c7fef"
userId:"b66b6801-de31-4ea1-bba7-d77effc2aaaa"
number : "1234567"
}
Each user might have one or more cards. I need a query to select the latest issued card with all details grouped by userId. Is it something possible to achieve with cosmos DB?
Inner queries and self-joins are not supported by Cosmos DB
I tried below SQL query:
SELECT e.userId, max(e.modifiedAt) as modifiedAt
from Card AS e
WHERE ARRAY_CONTAINS([""], e.userId)
group by e.userId
but this gives me only userIds with the latest modified card.
Related
I am using GraphQL for Cassandra database operation. The following search query work perfectly when filtering the column with partition key:
query oneUsers{
users(value: { username:"username" }) {
values {
id
name
username
}
}
}
But getting following errors when trying to search other columns:
graphql: Exception while fetching data (/users) : org.apache.cassandra.stargate.exceptions.InvalidRequestException: Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING
What is the best way to search columns in Cassandra using GraphQL?
You can only retrieve records from Cassandra by specifying (a) the partition key, or (b) the primary key column(s).
If you would like to filter by non-primary key columns, you will need to create an index on the column. For example if you want to filter by id, create an index with:
mutation createIndexes {
users: createIndex(
keyspaceName:"myks",
tableName:"users",
columnName:"id",
indexName:"id_idx"
)
}
You should then be able to query by id.
For details and more examples, see Developing with the Astra DB GraphQL API. Cheers!
I am trying to get the count of all records present in cosmos db in a lookup activity of azure data factory. I need this value to do a comparison with other value activity outputs.
The query I used is SELECT VALUE count(1) from c
When I try to preview the data after inserting this query I get an error saying
One or more errors occurred. Unable to cast object of type
'Newtonsoft.Json.Linq.JValue' to type 'Newtonsoft.Json.Linq.JObject'
as shown in the below image:
snapshot of my azure lookup activity settings
Could someone help me in resolving this error and if this is the limitation of azure data factory how can I get the count of all the rows of the cosmos db document using some other ways inside azure data factory?
I reproduce your issue on my side exactly.
I think the count result can't be mapped as normal JsonObject. As workaround,i think you could just use Azure Function Activity(Inside Azure Function method ,you could use SDK to execute any sql as you want) to output your desired result: {"number":10}.Then bind the Azure Function Activity with other activities in ADF.
Here is contradiction right now:
The query sql outputs a scalar array,not other things like jsonObject,or even jsonstring.
However, ADF Look Up Activity only accepts JObject,not JValue. I can't use any convert built-in function here because the query sql need to be produced with correct syntax anyway. I already submitted a ticket to MS support team,but get no luck with this limitation.
I also tried select count(1) as num from c which works in the cosmos db portal. But it still has limitation because the sql crosses partitions.
So,all i can do here is trying to explain the root cause of issue,but can't change the product behaviours.
2 rough ideas:
1.Try no-partitioned collection to execute above sql to produce json output.
2.If the count is not large,try to query columns from db and loop the result with ForEach Activity.
You can use:
select top 1 column from c order by column desc
I need to use "GROUP BY" clause on Azure Data Explorer but I think it is unsupported.
Someone have any idea to solve or avoid group by?
Best regards,
Finally, Azure Cosmos DB currently supports GROUP BY in .NET SDK 3.3 or later. Support for other language SDK's and the Azure Portal is not currently available but is planned.
<group_by_clause> ::= GROUP BY <scalar_expression_list>
<scalar_expression_list> ::=
<scalar_expression>
| <scalar_expression_list>, <scalar_expression>
There is no group by in Azure Data Explorer as of now. However, there is the summarize operator that can help achieve many of the things that you would use GROUP BY for in SQL.
You can find it at https://learn.microsoft.com/en-us/azure/kusto/query/summarizeoperator
Cosmos DB doesn't support group by feature,you could vote up this if you have urgent need.
Provide a third-party package documentdb-lumenize for your reference here which supports group by feature,it has .net example:
string configString = #"{
cubeConfig: {
groupBy: 'state',
field: 'points',
f: 'sum'
},
filterQuery: 'SELECT * FROM c'
}";
Object config = JsonConvert.DeserializeObject<Object>(configString);
dynamic result = await client.ExecuteStoredProcedureAsync<dynamic>("dbs/db1/colls/coll1/sprocs/cube", config);
Console.WriteLine(result.Response);
You could group by assetId column and get the max timestamp.
Besides,you could refer to my previous case:how to count distinct value in cosmos DB to use stored procedure to implement some aggregation features.
I have two azure databases say DB1 and DB2.
DB1 has got few tables and I want to create stored procedure in DB2 to cross query with joins in the tables in DB1. I have seen examples with Cross Database query but mostly showing only single table. My Stored procedure is like:
select
u.UserID as [UserID],
u.Username as [UserName],
u.LastLoginDate,
ISNULL(au.IsDeleted, 1) as [IsDeleted]
from DB1.[sec].[User] u
join DB1.[sec].[AppUser] au on (u.UserID = au.UserID)
join DB1.[sec].[Application] a on (au.ApplicationID = a.ApplicationID)
where (a.Name = 'name')
Just follow the method of this blog post and set up two tables instead of one. It's currently the only way to do a cross database query in Azure SQL Database.
When i run the following query in the Azure, I am getting no records. and getting the message "Query succeeded: Affected rows: 0."
1) SELECT * FROM sys.resource_usage where database_name='DB_NAME';
When i run this following query in azure, i get this error , "Failed to execute query. Error: Invalid object name 'sys.resource_stats'."
2) SELECT * FROM sys.resource_st where database_name='DB_NAME';
Please help me solve this issue, as im trying to track the daily database usage i.e amount of data stored in DB.
I suppose the issue is with your current db context, what does SELECT DB_NAME() return? Is that a user database? As per BOL 'This view is available to all user roles with permissions to connect to the virtual master database.' you'd have to connect to the master database before query against both sys.resource_usage and sys.resource_stats.
As Lin mentioned ,You have to connect to Master database to see output of sys.resource_stats
For DBspecific DTU usage , you can use sys.dm_db_resource_stats ..this stores data for every 15 seconds
and your question says
as im trying to track the daily database usage i.e amount of data stored in DB.
you can use below DMV to track that
SELECT sys.objects.name, SUM(reserved_page_count) * 8.0 / 1024
FROM sys.dm_db_partition_stats, sys.objects
WHERE sys.dm_db_partition_stats.object_id = sys.objects.object_id
GROUP BY sys.objects.name;
GO