After some trial and error, and a support call to Microsoft's Data Factory team of engineers, supposedly the product does not support CDATA values returned in XML.
I am using a web sink using XML format to send SOAP headers, and return XML data that contains CDATA formatting. Everything comes back properly in my pipeline, except the XML values containing any sort of CDATA formatting.
Is there a known workaround for this within Azure or Data Factory itself?
Thank you
Related
I have a web service that accepts OData filters and can serve data in json or csv, respecting the Accept content-type header. I now have users trying to use the service from Microsoft Excel OData feed, but Excel does not send any Accept header. I tried returning the data in csv format when I see the User-Agent: PowerPivot but Excel does not seem to like the the csv data.
What is the expected Content-Type and shape of the data returned that makes Excel OData works? Is there any particular schema expected?
we need to convert XML data into csv/excel/table in Azure cloud.
below is the sample xml code.
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
<ns2:getProjectsResponse
xmlns:ns2="http://www.logic8.com/eq/webservices/generated">
<ns2:Project>
<ns2:fileName>P10001</ns2:fileName>
<ns2:alias>project1</ns2:alias>
</ns2:Project>
<ns2:Project>
<ns2:fileName>P10002</ns2:fileName>
<ns2:alias>project2</ns2:alias>
</ns2:Project>
<ns2:Project>
<ns2:fileName>P10003</ns2:fileName>
<ns2:alias>project3</ns2:alias>
</ns2:Project>
</ns2:getProjectsResponse>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Expected output:
Can anyone help me on this.
You could try this way, firstly convert xml to json then use create csv table action to implement it. The below is my test flow.
I use blob to get the xml content. The Compose action input is json(xml(body('Get_blob_content'))), then will get thejson data. Then is the Create CSV table From, ause the from data should be array, so it should be outputs('Compose')['SOAP-ENV:Envelope']['SOAP-ENV:Body']['ns2:getProjectsResponse']['ns2:Project'].
The last thing is customize the header and the value, the ProjectID value should be item()['ns2:fileName'] and the ProjectDescription should be item()['ns2:alias'].
And here is the flow output, suppose this is what you want, hope this could help you.
Do you have any experience with Azure? I ask because from your question it sounds like you’re not sure where to start or which service to use. I’m also curious if you were given a requirement to use Azure or you thought Azure might be the solution yourself. Also where is this XML coming from? It looks like a SOAP request.
If you are a developer I’d consider authoring a Web App in .Net, it can use MVC, Core, Web APIs, and use it to consume this SOAP request and translate it and save the file.
For this I’d consider using an XMLDocument class to load the XML and parse through it.
But if you absolutely need to use Azure, the closest thing that would help automate this is Azure Logic Apps. It offers many “no-code” solutions to plug in connectors that can transform and save data.
https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-enterprise-integration-transform#how-to-use-a-transform
If you elaborate on your situation I’d be happy to offer further solutions
I was trying to understand if there is any feature available that would allow you to export the API query results to formats such as csv, xls, etc. But i did not find it. So i wonder if i am missing something, but the requirement would be to be able to easily export the results of a given query that i execute against my API hosted in API Management. Thanks
Only /reports API support returning results in format other than JSON and only in CSV.
I am calling a web service that is returning a json message.
In my spring integration application i want to have a generic processing of this message (without having to write a domain object) by converting it into pipe delimited key value pair and build the downstream flow based on tokens. How would i approach this in spring-integration?
Thanks!
You could use a json-to-object-transformer to create, e.g. a LinkedHashMap from the JSON, the use a custom transformer to transform the map to your format.
I would like to import weather forecasts in PowerPivot for a specified country by using Yahoo API.
So first of all I got the data stream URL including all requiered parameters to get forecasts (it works in my browser).
https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%3D612977&format=json&diagnostics=true&callback=
Then I tried to use it as "Other data Stream" in PowerPivot to grab the content.
However I got the following strange error about DTD :
Cannot connect to the specified feed. Verify the connection and try
again. Reason: For security reasons DTD is prohibited in this XML
document. To enable DTD processing set the DtdProcessing property on
XmlReaderSettings to Parse and pass the settings into XmlReader.Create
method
Any idea to solve that ?
The easiest was to install PowerQuery for Microsoft Excel then creating a new request to Yahoo Web Services.
After parsing data in PowerQuery (split columns, etc) , formatted data is available in a PowerPivot tab and you can use it as well as direct imported data in PowerPivot.