Azure Logic for getting data from SQL to FTP - azure

Question

I've a task of taking data from SQL and uploading the data as a CSV file up to an FTP server.
Now I've done this for a single SQL row just fine. The problem I'm having is looping over all rows (foreach loop) and inserting these rows as the content of the CSV file. I've tried a FTP Create File Task inside a foreach loop, but I can only access a single row at a time to set as the file's content - I need all the rows!
Also to keep in mind is that these files will have 200k+ rows.
I could of course just write a C# console app for this but the ease at which I got this far without writing any code makes it seem like it will be a worthwhile endeavor.

Answer 1

We recently added "Table" primitive for this scenario, support in designer is still work in progress, but you can use it in code view.
In below scenario, I'm getting rows from a table in SQL Azure, producing an CSV with two columns using data from the SQL query (First Name, Last Name), then send it via e-mail.
"Get_rows": {
"inputs": {
"host": {
"api": {
"runtimeUrl": "https://logic-apis-southcentralus.azure-apim.net/apim/sql"
},
"connection": {
"name": "#parameters('$connections')['sql']['connectionId']"
}
},
"method": "get",
"path": "/datasets/default/tables/#{encodeURIComponent(encodeURIComponent('[SalesLT].[Customer]'))}/items",
"queries": {
"$top": 10
}
},
"runAfter": {},
"type": "ApiConnection"
},
"tableCsv0": {
"inputs": {
"columns": [
{
"header": "First Name",
"value": "#item()?['FirstName']"
},
{
"header": "Last Name",
"value": "#item()?['LastName']"
}
],
"format": "csv",
"from": "#body('Get_rows')?['value']"
},
"runAfter": {
"Get_rows": [
"Succeeded"
]
},
"type": "Table"
},
"Send_an_email": {
"inputs": {
"body": {
"Body": "#body('tableCsv0')",
"Subject": "Subject",
"To": "deli#microsoft.com"
},
"host": {
"api": {
"runtimeUrl": "https://logic-apis-southcentralus.azure-apim.net/apim/office365"
},
"connection": {
"name": "#parameters('$connections')['office365']['connectionId']"
}
},
"method": "post",
"path": "/Mail"
},
"runAfter": {
"tableCsv0": [
"Succeeded"
]
},
"type": "ApiConnection"
}

Answer 2

So just following up to show how Derek's answer helped me with my problem to get a large number of rows to up to a file on an FTP server. I ended up using the output body of the Execute Stored Procedure action as the GetRows action was limited to 512 rows.
NOTE: As the Table action is not available in the designer, yet, do everything in the code viewer, opening the designer caused issues and deleted all my code at one point.
"actions": {
"Create_file": {
"inputs": {
"body": "#body('tableCsv0')",
"host": {
"api": {
"runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/ftp"
},
"connection": {
"name": "#parameters('$connections')['ftp']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/files",
"queries": {
"folderPath": "transactions/ready/ecommerce/tickets_test/",
"name": "grma_tickets_#{formatDateTime(utcNow(),'yyyyMMdd_hhmmss')}.csv"
}
},
"runAfter": {
"tableCsv0": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Execute_stored_procedure": {
"inputs": {
"host": {
"api": {
"runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/sql"
},
"connection": {
"name": "#parameters('$connections')['sql']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/procedures/#{encodeURIComponent(encodeURIComponent('[Scheduledjob].[GetBArcodesForGRMA]'))}"
},
"runAfter": {},
"type": "ApiConnection"
},
"tableCsv0": {
"inputs": {
"columns": [
{
"header": "EventDateTime",
"value": "#item()?['EventDateTime']"
},
{
"header": "EventName",
"value": "#item()?['EventName']"
}
],
"format": "csv",
"from": "#body('Execute_stored_procedure')['ResultSets']['Table1']"
},
"runAfter": {
"Execute_stored_procedure": [
"Succeeded"
]
},
"type": "Table"
}