Azure DataFactory DelimitedText dataset with parametrized schema - azure

Question

I'm trying to create a generic CSV dataset with parametrized filename and schema to be able to use it in foreach loops with file lists and I'm having some trouble on publishing, and I don't know if I'm doing something wrong or if the framework docs are not correct.
According to documentation the schema description is:
Columns that define the physical type schema of the dataset. Type: array (or Expression with resultType array), itemType: DatasetSchemaDataElement.
I have a dataset with a parameter named Schema of type Array and the "schema" set to an expression that returns this parameter:
{
"name": "GenericCSVFile",
"properties": {
"linkedServiceName": {
"referenceName": "LinkedServiceReferenceName",
"type": "LinkedServiceReference"
},
"parameters": {
"Schema": {
"type": "array"
},
"TableName": {
"type": "string"
},
"TableSchema": {
"type": "string"
}
},
"folder": {
"name": "Folder"
},
"type": "DelimitedText",
"typeProperties": {
"location": {
"type": "AzureDataLakeStoreLocation",
"fileName": {
"value": "#concat(dataset().TableSchema,'.',dataset().TableName,'.csv')",
"type": "Expression"
},
"folderPath": "Path"
},
"columnDelimiter": ",",
"escapeChar": "\\",
"firstRowAsHeader": true,
"quoteChar": "\""
},
"schema": {
"value": "#dataset().Schema",
"type": "Expression"
}
},
"type": "Microsoft.DataFactory/factories/datasets"
}
However, when I publish, i get the following error:
Error code: BadRequest
Inner error code: InvalidPropertyValue
Message: Invalid value for property 'schema'
Am I doing something wrong? are the docs wrong?

Answer 1

Yes, this is the expected behavior. If you need to set dynamic value for column mapping, please ignore schema in DelimitedText dataset, which is more for a visually display of physical schema information and would not take effect when do copy activity column mapping. The expression setting for it is also not allowed. You could configure mapping as an expression to achieve this goal and pass it a proper value when trigger run.