I am trying to create a data flow in Azure data factory but when I attempt to call the API connection, it throws me an error. This API connection was working fine when I used in copy activity but it does not work in data flow.
Here is the link to screenshot
Please tell me what wrong?
Related
I am using Logic Apps Designer on Azure to move source data to a storage file and I need to connect to a database on my SQL Server. To do this, I am using an on-premise connection througha gateway. I was able to successfully connect to my SQL Server database, however, when I am using Logic Apps action, not all tables are available on Logic Apps. Is there a security/permissions/connections issue that is preventing all tables from the SQL Server database not to return to Logic Apps?
enter image description here
enter image description here
I tried refreshing/restarting Azure and also tried connecting through with other authentications hoping more tables would return, but no luck.
I would suggest you to create a new api connection to sql and try to access tables again. There might be issue with connecting to db.
One of the alternative approaches to get row from sql db is using execute query action in logic app.
As shown in above picture, Execute a SQL query (V2) action is used to query table and retrieve records based on condition.
Reference link
I am new to ADF. I am creating a small demo where I am trying to hit a sample API "https://dummy.restapiexample.com/api/v1/employees" and inserting data into the Employee table in the SQL database.
However, I would also like to add data flow to perform few select and other transformations on other related tables.
Below are the things that I have done so far in azure data factory:
Created a linked service called "EmployeeRestService" to communicate with the above api.
Created a linked service called "AzureSqlDatabase" to communicate with azure sql database
Created a dataset "EmployeeApiDataset" of type "Rest" and linked it with "EmployeeRestService".
Created a dataset of type Azure sql database and Selected "EmployeeTable".
Added one data flow (Now here I am getting the problem)
And here is the error snapshot:
In the data flow --> Source Settings, I selected "EmployeeApiDataset". But when I am trying to preview the data by enabling the debug, I am getting below error (refer above snapshot):
at Source 'EmployeeApi': Rest Sink - Error response received from the server (url:http://dummy.restapiexample.com/api/v1/employees,request body: None, request method: GET, status code: 429), response body: Some(
Is this a correct way of doing adding Rest data source in data flow?
Note: I do not want to create an Azure data lake to store API results.
Can anybody suggest to me why I am getting the above error and what is the correct way of doing this?
As #esqew mentioned in the comments, there is nothing wrong with your dataflow Source settings. As you are using online dummy Rest API as your source dataset, there are high chances that too many requests are made to call the Rest API by users.
I also got the same error when tried.
Also got the same 429 error when tested the URL.
I just hit the refresh and got the data in the source data preview.
I am new to Azure Data Factory and wonder if you can please help me in acheiving with the following scenario:
I want to get data from a REST Endpoint using API. The REST Endpoint is stored in a SQL Database table and therefore I fetch the URL using a Lookup activity
Further on, I am storing the URL value in a variable using a "Set Variables" activity
Post that I am fetching the data from the Endpoint using REST API in a Web activity.
Now , I want to store the output data from the Web activity into a Blob storage. For this, i am using Copy activity , but I am not able to get this working at all. Meaning , I am unable to collect the output from the Web Activity into my Copy activity.
In case any of you have come across this situation then it will be of very good help for me indeed.
Seems you are not using correct output from web activity in copy activity. To check what is the output from web activity follow this GIF and pass those values appropriately to copy activity
We are trying to get the data from On Premise IBM DB2 from Azure Data Factory using Microsoft Integration Runtime. We are able to connect the database also we are able to get the list of tables in the ADF Dataset but when we are trying to execute the query we are getting the below error. Not able to identify the issue. Help me on this.
ROUTINE SQLSTATISTICS (SPECIFIC NAME SQLSTATISTICS) HAS RETURNED AN
ERROR SQLSTATE WITH DIAGNOSTIC TEXT -805
DSN.DSNASPCC.DSNASTAU.0E5F1F1D09F1404 SQLSTATE=38112 SQLCODE=-443
Error Screen Shot
Suggest you follow this technote to bind db2schema.bnd to the target Db2 database, after ensuring your Db2 client has the same version/fixpack as the Db2-server.
I am trying to pull data from web API using HTTP connector (of Azure Data Factory) and load into Azure Data Lake. I am successfully able to create pipeline using Copy Wizard but getting below error while running the pipeline.
Copy activity encountered a user error at Source side: ErrorCode=UserErrorFailedToReadHttpFile,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to read data from http source file.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The underlying connection was closed: An unexpected error occurred on a send.,Source=System,''Type=System.IO.IOException,Message=Authentication failed because the remote party has closed the transport stream.,Source=System,'.
Thanks
Nutan Patel
According to this article, you could find:
The type property must be set to: Http.
So we couldn't use https as the http linked service, I suggest you could firstly changed your linked service's setting to enable http connected and test again.