I would like to do some reporting on my CosmosDb
my Query is
Select Max(c.results.score) from c
That works but i want the id of the highest score then i get an exception
Select c.id, Max(c.results.score) from c
'c.id' is invalid in the select list because it is not contained in an
aggregate function
you can execute following query to archive what you're asking (thought it can be not very efficient in RU/execution time terms):
Select TOP 1 c.id, c.results.score from c ORDER BY c.results.score DESC
Group by isn't supported natively in Cosmos DB so there is no out of the box way to execute this query.
To implement this using the out of the box functionality you would need to create a new document type that contains the output of your aggregation e.g.
{
"id" : 1,
"highestScore" : 1000
}
You'd then need a process within your application to keep this up-to-date.
There is also documentdb-lumenize that would allow you to do this using stored procedures. I haven't used it myself but it may be worth looking into as an alternative to the above solution.
Link is:
https://github.com/lmaccherone/documentdb-lumenize
Related
I haven't used logic apps a lot, my boss is having trouble stuffing the results of one query into a variable and then using that variable in another query.
Basically, all he wants to do is get a list of of Id's returned from the first query and use that list in the second.
Here is a picture of what his logic app looks like:
You can see at the end of the second query he wants to check if the id is in the list or not. He's out for the day and I'm not sure if that variable is even receiving the list of id's successfully, but is there anything from the picture that you can tell that needs to be corrected? Or any suggestions that he could try, to achieve what he's trying to achieve?
According to the image, no data is getting stored into the variable AppId. While in the query you can just directly use c.EntityId. Below query to check if c.id is present in c.EntityId.
SELECT c.Vechicle.GrossVechicleWeight as GVW, c.EntityId as ApplicationId FROM c where c.RiskTypeId = 1 and c.Discriminator = 'RiskEntity' and c.EntityTypeId = 4500 and c.id in (c.EntityId)
Consider if you are trying to store c.Entity into AppId variable then you can Query SELECT c.EntityId FROM c and then store the result into the variable using Append to array variable action by extracting only c.EntityId using Parse JSON.
Here is my logic app
RESULT:
I'm trying to do a simple incremental update from an on-prem database as source to Azure SQL database based on a varchar column called "RP" in On-Prem database that contains "date+staticdescription" for example: "20210314MetroFactory"
1- I've created a Lookup activity called Lookup1 using a table created in Azure SQL Database and uses this Query
"Select RP from SubsetwatermarkTable"
2- I've created a Copy data activity where the source settings have this Query
"Select * from SourceDevSubsetTable WHERE RP NOT IN '#{activity('Lookup1').output.value}'"
When debugging -- I'm getting the error:
Failure type: User configuration issue
Details: Failure happened on 'Source' side.
'Type=System.Data.SqlClient.SqlException,Message=Incorrect syntax near
'[{"RP":"20210307_1Plant
1KAO"},{"RP":"20210314MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"2'.,Source=.Net
SqlClient Data
Provider,SqlErrorNumber=102,Class=15,ErrorCode=-2146232060,State=1,Errors=[{Class=15,Number=102,State=1,Message=Incorrect
syntax near
'[{"RP":"20210311MetroFactory"},{"RP":"20210311MetroFactory"},{"RP":"202103140MetroFactory"},{"RP":"20210308MetroFactory"},{"RP":"2'.,},],'
Can anyone tell me what I am doing wrong and how to fix it even if it requires creating more activities.
Note: There is no LastModifiedDate column in the table. Also I haven't yet created the StoredProcedure that will update the Lookup table when it is done with the incremental copy.
Steve is right as to why it is failling and the query you need in the Copy Data.
As he says, you want a comma-separated list of quoted values to use in your IN clause.
You can get this more easily though - from your Lookup directly using this query:-
select stuff(
(
select ','''+rp+''''
from subsetwatermarktable
for xml path('')
)
, 1, 1, ''
) as in_clause
The sub-query gets the comma separated list with quotes around each rp-value, but has a spurious comma at the start - the outer query with stuff removes this.
Now tick the First Row Only box on the Lookup and change your Copy Data source query to:
select *
from SourceDevSubsetTable
where rp not in (#{activity('lookup').output.firstRow.in_clause})
The result of #activity('Lookup1').output.value is an array like your error shows
[{"RP":"20210307_1Plant
1KAO"},{"RP":"20210314MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"20210312MetroFactory"},{"RP":"2'.,Source=.Net
SqlClient Data
Provider,SqlErrorNumber=102,Class=15,ErrorCode=-2146232060,State=1,Errors=[{Class=15,Number=102,State=1,Message=Incorrect
syntax near
'[{"RP":"20210311MetroFactory"},{"RP":"20210311MetroFactory"},{"RP":"202103140MetroFactory"},{"RP":"20210308MetroFactory"},{"RP":"2'.,},]
However, your SQL should be like this:Select * from SourceDevSubsetTable WHERE RP NOT IN ('20210307_1Plant 1KAO','20210314MetroFactory',...).
To achieve this in ADF, you need to do something like this:
create three variables like the following screenshot:
loop your result of #activity('Lookup1').output.value and append 'item().RP' to arrayvalues:
expression:#activity('Lookup1').output.value
expression:#concat(variables('apostrophe'),item().RP,variables('apostrophe'))
3.cast arrayvalues to string and add parentheses by Set variable activity
expression:#concat('(',join(variables('arrayvalues'),','),')')
4.copy to your Azure SQL database
expression:Select * from SourceDevSubsetTable WHERE RP NOT IN #{variables('stringvalues')}
I want to retrieve data from Cosmos DB with the following query:
SELECT * FROM c WHERE c.pi like '09%001'
(This is a SQL query, which I can use in MySQL)
Here, pi is a string value, which can be 09001001 or 09025001.
Is there a way to use a LIKE command in Cosmos DB?
I know that cosmos DB uses CONTAINS, but this cannot be used when you want to match specifically the beginning or end of the string.
UPDATE :
You can now use the LIKE keyword to do text searches in Azure Cosmos DB SQL (core) API!
EXAMPLE:
SELECT *
FROM c
WHERE c.description LIKE "%cereal%"
OLD Answer:
This can be achieved in 2 ways
(i) Currently Azure Cosmosdb supports the CONTAINS, STARTSWITH, and ENDSWITH built-in functions which are equivalent to LIKE.
The keyword for LIKE in Cosmosdb is Contains .
SELECT * FROM c WHERE CONTAINS(c.pi, '09')
So, in your case if you want to match the pattern 09%001, you need to use:
SELECT * FROM c WHERE STARTSWITH(c.pi, '09') AND ENDSWITH(c.pi, '001')
(ii) As 404 mentioned, Use SQL API User Defined Functions which supports regex :
function executeRegex(str, pattern) {
let regex=RegExp(pattern);
return regex.test(str);
}
SELECT udf.EXECUTE_REGEX("foobar", ".*bar")
Another possibility is creating your own User Defined Function. As example here's a regex check:
function matchRegex(str, pattern) {
let regex=RegExp(pattern);
return regex.test(str);
}
Created under the name MATCH_REGEX it can then be used like:
SELECT udf.MATCH_REGEX("09001001", "^09.*001$")
As note: it'll kill any index optimization that for instance STARTSWITH would have. Although allows for much more complex patterns. It can therefor be beneficial to use additional filters that are capable of using the index to narrow down the search. E.g. using StartsWith(c.property1, '09') as addition to the above example in a WHERE clause.
UPDATE:
Cosmos now has a RegexMatch function that can do the same. While the documentation for the MongoApi mentions the $regex can use the index to optimize your query if it adheres to certain rules this does not seem to be the case for the SqlApi (at this moment).
Sorry for being newbie for NodeJs and table query, my question's,
How I could create a query using Nodejs pakcage "azure-storage-node", which selects the sum/addition of two coloumns 'start' and 'period' , if the addition is greater than a threshold it will take the whole raw, my tries which didn't work is something like this,
var query = new azure.TableQuery();
total = query.select(['start']) + query.select(['period']);
query.where('total > ?' , 50000);
or may be something like this,
var query = new azure.TableQuery()
.where('start + period gt 50000');
but it throws an error of '+'.
Thanks
What you're trying to accomplish is not possible with Azure Tables at least as of today as Azure Tables has limited querying support and support for computed columns (if I may say so) is not there.
There are two possible solutions:
Have an attribute called total in your entities that will contain the value i.e. start + period. You calculate this value when you're inserting or updating the entity and store it at that time.
Do this filtering on the client side. For this you will need to download all related entities and then apply this filtering on the client side on the data that you fetched.
I have a hadoop application that source data from two different DocumentDB collection. However, the json schema of documents belonging to these two collections are different. Both has a field showing time, but one is called TimeStamp and the other one is called UpdatedOn. I'd like to know how I can specify a query which is based on this time field and retrive only those json documents satisfying the condition in my query. I specify my query like below
String query = "SELECT * FROM c WHERE c.Timestamp > " + timestamp;
conf.set(ConfigurationUtil.QUERY, query);
This query applies on one of the collection. I need a query like below
"SELECT * FROM collection1 as c1, collection2 as c2 WHERE c1.Timestamp > x1 OR c2.UpdatedOn > x1"
Is this supported in DocumentDB?
This is not supported since it is not documented, your best bet is two execute these two queries and then merge the results using Linq or any other technique to get one result set.
Hope this helps.