I am getting value cnt=1 from my query in LookUp Activity.
Now I want to check if cnt value is 1 in IF activity then run another activity.
Finally, I found the solution.
Below is the working code for my problem:
#equals(activity('Lookup1').output.firstRow.cnt,1)
Related
I'm working on an ETL pipeline in Azure Synapse.
In the previous version I used an Array set as a parameter of the pipeline and it contained JSON objects. For example:
[{"source":{"table":"Address"},"destination {"filename:"Address.parquet"},"source_system":"SQL","loadtype":"full"}}]
This was later used as the item() and I used a ForEach, switch, ifs and nested pipelines to process all the tables. I just passed down the item parameters to the sub pipelines and it worked fine.
My task is now to create a dedicated SQL pool and a table which stores parameters as columns. The columns are: source_table, destination_file, source_system and loadtype.
Example:
source_table
destination_file
source_system
loadtype
"Address"
"Address.parquet"
"SQL"
"full"
I don't know how to use this table in the ForEach activity and how to process the tables this way since this is not an Array.
What I've done so far:
I created the dedicated SQL pool and the following stored procedures:
create_parameters_table
insert_parameters
get_parameters
The get_parameters is an SQL SELECT statement but I don't know how to convert it in a way that could be used in the ForEach activity.
CREATE PROCEDURE get_parameters AS BEGIN SELECT source_table, destination_filename, source_system, load_type FROM parameters END
All these procedures are called in the pipeline as SQL pool stored procedure. I don't know how to loop through the tables. I need to have every row as one table or one object like in the Array.
Take a lookup activity in Azure data factory/ Synapse pipeline and in source dataset of lookup activity, take the table that has the required parameter details.
Make sure to uncheck the first row only check box.
Then take the for-each activity and connect it with lookup activity.
In settings of for-each activity, click add dynamic content in items and type
#activity('Lookup1').output.value
Then you can add other activities like switch/if inside the for-each activity.
I have to implement one report using Lookup activity and Web activity.
Lookup activity having output from store procedure with multiple records like below
Name ActiveRecords Active
Abc 500 0
XYZ 300 200
Something like the above I have output from the procedure and then I have to use this on web activity.
Also, I am having exciting web activity and I have appended it.
Thanks in advance
Created ADF pipeline as shown below,
In Lookup activity calling sql stored procedure as shown below,
Output of lookup activity as shown below,
The output of Lookup activity is using in web as
#string(activity('Lookup').output)
5. Debug the pipeline and it is taking output of lookup as shown below,
.
I am exec adb notebook in if/else condition from adf
I have a lookup which will check flag condition in delta lake table
SELECT COUNT(*) AS cnt FROM db.check where job_status = 2 and site ='xxx-xxx-xxx'. This will give me a count 2 and I used it in the if part condition #equals(activity('select job status').output.value[0],2) it should call adb notebook else logic app.
Issue After lookup pipline is not going inside if part?
Thank you #anuj for your solution. Posting it as an answer to help other community members.
To refer to lookup activity value of a column in later activities, include the column name in the expression as below.
#equals(activity('select job status').output.value[0].cnt,2)
Is there a way to send the number of times the cycle was executed?
that is, I have a For Each that executes an ExePipeline and it has 6 activities and only to the last activity I need to send it the number of times that the for each was executed.
at the end of For Each it shows how much data "ItemsCount" entered but I couldn't call that value in the last activity of the pipeline.
someone to help me thanks.
It may depend on what you're using for your items in the foreach activity.
But for example, if your foreach activity is looping over the content of a file that you have retrieved by a previous lookup activity - you can get the count of these items by for example:
#activity('Lookup activity name').output.count
I have a requirement where i need to pass the column mapping dynamically from a stored procedure to the copy activity. This copy activity will perform update operation in Dynamics CRM. The source is SQL server (2014) and sink is Dynamics CRM.
I am fetching the column mapping from a stored procedure using look up activity and passing this parameter to copy activity.'
When i directly provide the below mentioned json value as default value to the parameter, the copy activity is updating the mapped fields correctly.
{"type":"TabularTranslator","columnMappings":{"leadid":"leadid","StateCode":"statecode"}}
But when the json value is fetched from the SP , it is not working . I am getting the error ColumnName is read only.
Please suggest if any conversion is required on the output of the loopup activity before passing the parameter to copy activity. Below is the output of the lookup activity.
{\"type\":\"TabularTranslator\",\"columnMappings\":{\"leadid\":\"leadid\",\"StateCode\":\"statecode\"}}
Appreciate a quick turnaround.
Using parameter directly and Using lookup output are different. can you share how did you write the parameter from the output of lookup actvitiy.
you can refer to this doc https://learn.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity