I am working with a simple query in azure Synapse, however I think the FOR in the following query is not accepted in synapse :
select stuff((select ','+quotename(C.ClaveReferencia)
from [Landing].[BKPF] as C
for xml path('')), 1, 1, '')
Does anybody know a workaround for that?
Answer:
SELECT ''''+STRING_AGG(CONVERT(NVARCHAR(max), ClaveReferencia), ''',''') AWKEY
FROM [Landing].[BKPF]
Related
Is there a simple way to find out all the tables referenced in a stored procedure in azure analytics data warehouse other than parsing the stored procedure code? I tried few commands like sp_tables, sp_depends but none seems to be working in azure data warehouse.
sys.sql_expression_dependencies is supported in Azure Synapse Analytics, dedicated SQL pools, but only supports tables, views and functions at this time. A simple example:
SELECT * FROM sys.sql_expression_dependencies;
So you are left either parsing sys.sql_modules. Something like this is imperfect (ie doesn't deal with schema name, square brackets, partial matches etc) but could server as a starting point:
SELECT
sm.[definition],
OBJECT_SCHEMA_NAME(t.object_id) schemaName,
OBJECT_NAME(t.object_id) tableName
FROM sys.sql_modules sm
CROSS JOIN sys.tables t
WHERE sm.object_id = OBJECT_ID('dbo.usp_test')
AND sm.[definition] Like '%' + t.name + '%';
I actually use SQL Server Data Tools (SSDT) with dedicated SQL pools so your dependencies can't get out of step and are trackable via the project.
I am trying to get the count of all records present in cosmos db in a lookup activity of azure data factory. I need this value to do a comparison with other value activity outputs.
The query I used is SELECT VALUE count(1) from c
When I try to preview the data after inserting this query I get an error saying
One or more errors occurred. Unable to cast object of type
'Newtonsoft.Json.Linq.JValue' to type 'Newtonsoft.Json.Linq.JObject'
as shown in the below image:
snapshot of my azure lookup activity settings
Could someone help me in resolving this error and if this is the limitation of azure data factory how can I get the count of all the rows of the cosmos db document using some other ways inside azure data factory?
I reproduce your issue on my side exactly.
I think the count result can't be mapped as normal JsonObject. As workaround,i think you could just use Azure Function Activity(Inside Azure Function method ,you could use SDK to execute any sql as you want) to output your desired result: {"number":10}.Then bind the Azure Function Activity with other activities in ADF.
Here is contradiction right now:
The query sql outputs a scalar array,not other things like jsonObject,or even jsonstring.
However, ADF Look Up Activity only accepts JObject,not JValue. I can't use any convert built-in function here because the query sql need to be produced with correct syntax anyway. I already submitted a ticket to MS support team,but get no luck with this limitation.
I also tried select count(1) as num from c which works in the cosmos db portal. But it still has limitation because the sql crosses partitions.
So,all i can do here is trying to explain the root cause of issue,but can't change the product behaviours.
2 rough ideas:
1.Try no-partitioned collection to execute above sql to produce json output.
2.If the count is not large,try to query columns from db and loop the result with ForEach Activity.
You can use:
select top 1 column from c order by column desc
I need to use "GROUP BY" clause on Azure Data Explorer but I think it is unsupported.
Someone have any idea to solve or avoid group by?
Best regards,
Finally, Azure Cosmos DB currently supports GROUP BY in .NET SDK 3.3 or later. Support for other language SDK's and the Azure Portal is not currently available but is planned.
<group_by_clause> ::= GROUP BY <scalar_expression_list>
<scalar_expression_list> ::=
<scalar_expression>
| <scalar_expression_list>, <scalar_expression>
There is no group by in Azure Data Explorer as of now. However, there is the summarize operator that can help achieve many of the things that you would use GROUP BY for in SQL.
You can find it at https://learn.microsoft.com/en-us/azure/kusto/query/summarizeoperator
Cosmos DB doesn't support group by feature,you could vote up this if you have urgent need.
Provide a third-party package documentdb-lumenize for your reference here which supports group by feature,it has .net example:
string configString = #"{
cubeConfig: {
groupBy: 'state',
field: 'points',
f: 'sum'
},
filterQuery: 'SELECT * FROM c'
}";
Object config = JsonConvert.DeserializeObject<Object>(configString);
dynamic result = await client.ExecuteStoredProcedureAsync<dynamic>("dbs/db1/colls/coll1/sprocs/cube", config);
Console.WriteLine(result.Response);
You could group by assetId column and get the max timestamp.
Besides,you could refer to my previous case:how to count distinct value in cosmos DB to use stored procedure to implement some aggregation features.
In Storage Table I have dynamicaly generated PartitionKey in YYYY-MM-DD-HH format. And I need querying this in Data Factory Pipeline
I need something like this:
PartitionKey eq DateTime.Now.ToString("yyyyMMddHH")
It is possible? Thanks ...
You could use formatDateTime dynamic date functions in ADF.
TEST:
configuration:
output:
UPDATE:
Sorry for my uncompletely answer.I figured it out by using below expression:
#concat('PartitionKey eq ''', variables('dateValue'),'''')
Screenshot:
Entire structure:
I got some clues from this link,also provide for you:Azure Data Factory Expression Query for Copy activity.
I'm getting an email regarding a Schema Issue on my SQL Azure Database and it reads something like this:
"Invalid column name 'False'" Error code : 207
However it doesn't appear to have any more information and I wouldn't even know where to start because the schema was created before I started in the project, but I suspect this is coming from a Stored Procedure or View.
This is all being reported in Azure Portal, but I have no idea how to proceed. Is there any other way to get the suggestions on Azure Portal in SSMS? I already tried Tuning Advisor and I got an error that SQL Azure was not supported.
I use this query to to search objects in this scenario. This will find every occurrence of false and should help you find the offending column. This will find every occurrence of whatever you define #searchName as.
I found this on Pinal Dave's blog, way back. This works in Azure SQL.
declare #searchName varchar(50) = 'false'
select #searchName as SearchName,
OBJECT_SCHEMA_NAME(OBJECT_ID) + '.' + OBJECT_NAME(OBJECT_ID) as ObjectName,
[definition]
from sys.sql_modules
where definition LIKE '%' + #searchName + '%'
order by OBJECT_SCHEMA_NAME(OBJECT_ID) + '.' + OBJECT_NAME(OBJECT_ID)