I am new in web development and trying to learn AWS.
I have made a lambda function for listing.
What I am doing here is I am showing listing, if I get counteId in params(URL) then it shows only data of that counter id else it shows all data.
My lambda function was working fine. But I am having a problem while API integration.
this is how I am accessing pathparameters which are in the event
this is how I am configuring event
and this is my query and response
then I create an API gateway for it.
this is what I did while creating Resource
/{proxy+} - ANY - Setup
I want to get only data of counterId 1, but I am getting whole data.
response
My HTTPmethod is "ANY" and I choose lambda proxy integration in request integration.
I don't know how to send path parameters. Kindly help me.
You have to edit 'Mapping Template' in 'Integration Request' of you method properties in API Gateway.
You could find how to map it in API Gateway Mapping Template Reference article, in the 'Accessing the $input Variable' section.
Your template has to look like next:
{
"name" : "$input.params('name')",
"body" : $input.json('$')
}
Check out more details in my answer to the similar question.
Related
I am trying to call API endpoint as a POST and Content-Type as from-data using azure data factory web activity. Tried different way of passing Body parameters but it failed.
Here is the Postman Request.
Here is the Azure Data Factory Web Activity configurations.(use the body as a json and tried different combinations but all didn't work)
And above is the error message.
Any help would be highly appreciated.
Since your request is seen successfully executing from postman, try copying the entire body from that and use in web activity.
The format for passing body for a POST request from a web activity is shown here.
Also make sure you have entered valid “url–Target endpoint and path”. This is usually seen as Activity requires Public end point but you may have used Private vnet where this is not allowed. Web Activity is supported for invoking URLs that are hosted in a private virtual network as well by leveraging self-hosted integration runtime. The integration runtime should have a line of sight to the URL endpoint.
Note: The activity will timeout at 1 minute with an error if it does not receive a response from the endpoint.
Further going through some similar scenarios it is learnt that;
Mostly the header is passed as string in WebActivity whereas Postman
it is integer/long
In case your API tries redirecting, it seems that the web activity
in Azure Data Factory does not currently support following
redirects, meanwhile Postman and other tools and libraries usually
follow redirects by default or include a option for handling them.
Checkout the supported authentication types in the web activity. If you are trying to authorize your from, try set the following.
URL: https://login.microsoftonline.com/<<tenantid>>/oauth2/token
Headers: Content-Type - "application/x-www-form-urlencoded"
Body: grant_type=client_credentials&client_id=<<clientid>>f&client_secret=<<secret>>&resource=https%3A%2F%2Fmicrosoft.onmicrosoft.com%2F<<resourceId>>
Error code: 2108:
Message: Error calling the endpoint '%url;'. Response status code: '%code;'
Cause: The request failed due to an underlying issue such as network connectivity, a DNS failure, a server certificate validation, or a timeout.
Workaround: Make the API call using Powershell, and call that Powershell script from within Data Factory.
I am trying to set "advanced filter" of BlobEventsTrigger programmatically.
They are reset at each deployment. I need only 3 and having 7 of them are causing the job to start twice. It is super annoying to delete them manually after each deployment.
I have tried to add a field "advancedFilters" or "blobType" to the trigger json file without success.
"typeProperties": {
"blobPathBeginsWith": "/bingofile/blobs/",
"blobPathEndsWith": "/_SUCCESS",
"ignoreEmptyBlobs": false,
"scope": "/subscriptions/bingofilesup/resourceGroups/bingofilesup/providers/Microsoft.Storage/storageAccounts/bingofilesup",
"events": [
"Microsoft.Storage.BlobCreated"
]
I've also tried az eventgrid system-topic event-subscription update but this library does not work when it comes to updating advanced filter. It asks an endpoint (which is normally a facultative argument) , and when provided the existing data factory endpoint, it fails reaching it.
I have checked the documentation about this endpoint and it is said to be the webhook endpoint .
Endpoint where EventGrid should deliver events matching this event
subscription. For webhook endpoint type, this should be the
corresponding webhook URL. For other endpoint types, this should be
the Azure resource identifier of the endpoint. It is expected that the
destination endpoint to be already created and available for use
before executing any Event Grid command.
But it does not work .
Deployment failed. Correlation ID:
95e4fab5-163e-48ab-8cb2-b23432516e53. Webhook validation handshake
failed for [webwook end point provided in the topic]. Http POST
request failed with response code Unknown. For troublehooting, visit
https://aka.ms/esvalidation.
Any observation or suggestion would be great, thanks in advance !
According to my test, the endpoint https://pmeastasia.svc.datafactory.azure.com:4443/triggerevent/BlobEventsTrigger/<> is juts a base URL. When the events are sent to data factory or update subscription, azure will generate an endpoint with the base URL to do auth. So if you want to update the subscription with other tools, I think you need to use fildder to catch the request to get the whole endpoint at first.
I'm trying to fetch data from REST API and save the json string it into DataLake and I'm getting an error. I've followed the steps mentioned here
https://learn.microsoft.com/en-us/azure/data-factory/connector-rest & https://www.alexvolok.com/2019/adfv2-rest-api-part1-oauth2/
The API which I'm trying to connect uses OAuth2 so I need to first get the access token and then do a get request to get actual data.
Below are the steps which I'm following
Creating a Web HTTP request in the pipeline and passing the client_ID, client secret, username, password and grant type in the body of the request. When I debug the pipline I do get the Access_token which I need in step 2.
In Step two I have a copy activity which uses the output(access_token) from web to authenticate the second REST GET request but this is where I'm facing a lot of issues. The code which I'm using is "#concat('Bearer ', activity('GetAccessToken').output.access_token)"
In step 3 I have two datasets and 2 Linked services, Dataset 1 is a REST dataset which has the base url and relative url which is linked to the REST linked service and secondly the sink dataset is connected to AZURE datalake storage.
In Source Dataset I'm passing additional header Authorization = #concat('Bearer ', activity('GetAccessToken').output.access_token) and ideally since the API which I want to call will return empty if no parameters are send so I pass in the parameters inside the "Request Body" is that even correct? the Request body would look something like this "start_date=2020/07/17&end_date=2020/07/18".
The sink is a simple Json dataset stored in DataLake.
When I try to debug I get the error as below
But I'm getting the below error
{
"errorCode": "2200",
"message": "Failure happened on 'Source' side. ErrorCode=UserErrorHttpStatusCodeIndicatingFailure,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The HttpStatusCode 401 indicates failure. { \"Error\": { \"Message\":\"Authentication failed: Invalid headers\", \"Server-Time\":\"2020-07-27T06:59:24\", \"Id\":\"6AAF87BC-5634-4C28-8626-810A19B86BFF\" } },Source=Microsoft.DataTransfer.ClientLibrary,'",
"failureType": "UserError",
"target": "CopyDataFromAPI",
"details": []
}
Please advise if I'm doing anything wrong.
I Knew this was a simple issue,
So for people who are looking for answers.
Please make sure the REST Source URL starts with HTTPS:// instead of HTTP:// I Guess Azure does not pass headers to url which starts with HTTP:// which is strange because POSTMAN and python script has no problem sending the headers.
I have created one API endpoint for lambda function, as - https://XXXXXXXXX.execute-api.us-east-1.amazonaws.com/XXXX/XXXXXXXXXXXX/ which is GET method.
While calling that endpoint from postman it is giving me
{
"message": "'XXXXXXXXX3LPDGPBF33Q:XXXXXXXXXXBLh219REWwTsNMyyyfbucW8MuM7' not a valid key=value pair (missing equal-sign) in Authorization header: 'AWS XXXXXXXXX3LPDGPBF33Q:XXXXXXXXXXBLh219REWwTsNMyyyfbucW8MuM7'."
}
This is a screenshot of the Amazon Lambda Upload Site: http://i.stack.imgur.com/mwJ3w.png
I have Access Key Id & Secret Access Key for IAM user. I used it all but no luck. Can anyone suggest tweak about this.
If you're using the latest version of Postman, you can generate the SigV4 signature automatically. The region should correspond to your API region (i.e. "us-east-1") and the service name should be "execute-api"
This is not a solution but it has helped me more than once:
Double-check that you are actually hitting an existing endpoint! Especially if you're working with AWS. AWS will return this error if you don't have the correct handler set up in your Lambda or if your API Gateway is not configured to serve this resource/verb/etc.
I was able to successfully create a test API and host in Azure. However when I try to create the proxy client, I receive the following error.
[Fatal]Error generating service model: The operation 'Get' has a body
parameter, but did not have a supported MIME type ('application/json')
in its Consumes property.
Exception: There was an error during code
generation when trying to add a client for the Microsoft Azure API App
Generating client code and adding to project failed
I checked the Swagger file and the Contains node was empty. When I change it to
"consumes": [
"application/json",
"application/xml"
]
the proxy creation works. Why did the auto-generated Swagger json not have the Contains property set? I went with the default SwaggerConfig when I created the API app. Am I missing some configuration? Any help will be greatly appreciated.
As I found out from the comments, the solution is to to remove the HttpRequestMessage as parameter of the Action. This will enable the API App Client to generate the code OK.
If you need to mock the object, please follow the documented way from here or another example here.