I am setting up a AWS Lambda function to connect to my DynamoDB. To access it I'm also setting up an API Gateway.
The lambda seems to work when I test it. Because of this I believe the issue to be in the API gateway setup.
For the lambda I have configured the following test event:
I have configured a test event which looks like this:
{
"httpMethod": "GET"
}
This test event gives me the following response:
Response:
{
"statusCode": "200",
"body": "{\"Items\":[{\"id\":1,\"brand\":\"Test brand\",\"title\":\"Test product\"}],\"Count\":1,\"ScannedCount\":1}",
"headers": {
"Content-Type": "application/json"
}
}
For the API Gateway I have tried with the following test:
I have tried automatically creating the API Gateway in the lambda management console. Recreating the lambda and the API Gateway.
Lambda function:
console.log('Loading function');
const doc = require('dynamodb-doc');
const dynamo = new doc.DynamoDB();
/**
* Demonstrates a simple HTTP endpoint using API Gateway. You have full
* access to the request and response payload, including headers and
* status code.
*
* To scan a DynamoDB table, make a GET request with the TableName as a
* query string parameter. To put, update, or delete an item, make a POST,
* PUT, or DELETE request respectively, passing in the payload to the
* DynamoDB API as a JSON body.
*/
exports.handler = (event, context, callback) => {
//console.log('Received event:', JSON.stringify(event, null, 2));
const done = (err, res) => callback(null, {
statusCode: err ? '400' : '200',
body: err ? err.message : JSON.stringify(res),
headers: {
'Content-Type': 'application/json',
},
});
switch (event.httpMethod) {
case 'DELETE':
dynamo.deleteItem(JSON.parse(event.body), done);
break;
case 'GET':
dynamo.scan({ "TableName": "productdb" }, done);
//dynamo.scan({"TableName":"productdb"})
break;
case 'POST':
dynamo.putItem(JSON.parse(event.body), done);
break;
case 'PUT':
dynamo.updateItem(JSON.parse(event.body), done);
break;
default:
done(new Error(`Unsupported method "${event.httpMethod}"`));
}
};
API Gateway logs from the test:
Execution log for request 885e5505-2212-11e9-aee0-7f024016f574
Sun Jan 27 09:04:20 UTC 2019 : Starting execution for request: 885e5505-2212-11e9-aee0-7f024016f574
Sun Jan 27 09:04:20 UTC 2019 : HTTP Method: GET, Resource Path: /
Sun Jan 27 09:04:20 UTC 2019 : Method request path: {}
Sun Jan 27 09:04:20 UTC 2019 : Method request query string: {}
Sun Jan 27 09:04:20 UTC 2019 : Method request headers: {}
Sun Jan 27 09:04:20 UTC 2019 : Method request body before transformations:
Sun Jan 27 09:04:20 UTC 2019 : Endpoint request URI: https://lambda.eu-central-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-central-1:304886708348:function:dynamoDBService/invocations
Sun Jan 27 09:04:20 UTC 2019 : Endpoint request headers: {x-amzn-lambda-integration-tag=885e5505-2212-11e9-aee0-7f024016f574, Authorization=*****************************************************************************************************************************************************************************************************************************************************************************************500617, X-Amz-Date=20190127T090420Z, x-amzn-apigateway-api-id=lqhm3agxxf, X-Amz-Source-Arn=arn:aws:execute-api:eu-central-1:304886708348:lqhm3agxxf/test-invoke-stage/GET/, Accept=application/json, User-Agent=AmazonAPIGateway_lqhm3agxxf, X-Amz-Security-Token=FQoGZXIvYXdzEOH//////////wEaDFvawdYGjH/+gSI14yK9AzQFZtlDghAr2NUHIhLGWmeJkKL8sUP3L6fu0h5PtFPN7wA7hgfWMtUNHCWyGykG0g5Zs81zKx5bUGMLCMK2zuVwD4WMgBRmkx40bZYehHdeS8czOxRTbQIqwP1lfZ0d74l4MqG4g8XpigkcLACLEn6buaq37rO4WYOo+J8ecFeSpti+u+V8OON4idxxXEHiYGJEc23OwjVvf3GTr1EUscB+Lsp/nw58oCWQArUA6LLSwcnGYXYcmnPav2Xs8mJgvqnVowxxYre0N8Gca8D9XBN2Y93/qnVTsOI5nWHSUQOnwaoXSZzgBAXKrUV1S5X+UH3zQI9p [TRUNCATED]
Sun Jan 27 09:04:20 UTC 2019 : Endpoint request body after transformations:
Sun Jan 27 09:04:20 UTC 2019 : Sending request to https://lambda.eu-central-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-central-1:304886708348:function:dynamoDBService/invocations
Sun Jan 27 09:04:20 UTC 2019 : Received response. Integration latency: 17 ms
Sun Jan 27 09:04:20 UTC 2019 : Endpoint response body before transformations: {"statusCode":"400","body":"Unsupported method \"undefined\"","headers":{"Content-Type":"application/json"}}
Sun Jan 27 09:04:20 UTC 2019 : Endpoint response headers: {Date=Sun, 27 Jan 2019 09:04:20 GMT, Content-Type=application/json, Content-Length=108, Connection=keep-alive, x-amzn-RequestId=6c00229e-caa1-4d37-aeaa-7c1cbd0ddd71, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-5c4d7414-e52b0fba267596b50fdbb102;sampled=0}
Sun Jan 27 09:04:20 UTC 2019 : Method response body after transformations: {"statusCode":"400","body":"Unsupported method \"undefined\"","headers":{"Content-Type":"application/json"}}
Sun Jan 27 09:04:20 UTC 2019 : Method response headers: {X-Amzn-Trace-Id=Root=1-5c4d7414-e52b0fba267596b50fdbb102;Sampled=0, Access-Control-Allow-Origin=*, Content-Type=application/json}
Sun Jan 27 09:04:20 UTC 2019 : Successfully completed execution
Sun Jan 27 09:04:20 UTC 2019 : Method completed with status: 200
I expect the result to be the same as the test event in the lambda returns.
When you create an API method, you need to select the option "Use Lambda Proxy integration" in order for the httpMethod field, along with other information from the API Gateway, to be accessible in the event object in your Lambda function.
From the docs:
You can set up a Lambda proxy integration for any API method. But a
Lambda proxy integration is more potent when it is configured for an
API method involving a generic proxy resource. The generic proxy
resource can be denoted by a special templated path variable of
{proxy+}, the catch-all ANY method placeholder, or both. The client
can pass the input to the backend Lambda function in the incoming
request as request parameters or applicable payload. The request
parameters include headers, URL path variables, query string
parameters, and the applicable payload. The integrated Lambda function
verifies all of the input sources before processing the request and
responding to the client with meaningful error messages if any of the
required input is missing.
You can find the "Use Lambda Proxy integration" option here, on the "Create Method" screen in your API Gateway instance:
Edit: For reference, you can tell that the API Gateway method is not using the Lambda proxy integration because under "Integration Request" the type is "LAMBDA", but when using the Lambda proxy integration the type is "LAMBDA_PROXY".
For anyone who couldn't get the above solutions to work, I had to change the switch statement for the http method that the Lambda function was originally built with
from: switch (event.httpMethod) {
to: switch (event.requestContext.http.method) {
I think V1.0 uses event.httpMethod? but I wasn't able to change the version in the API Gateway (message: "API Gateway managed resources are view only"). So I had to change the switch statement to match the event object.
I fixed the issue for HTTP API, by changing the payload format to V1.0 (under the integration details for the route).
Related
I try to POST body to an REST API using fetch and Authorization with Bearer header in React Native App.
I don't understand why, but the the Authorization value on the headers add some backslash (so the header is wrong).
if (!JSON.parse(logoutMode) || customToken) {
params.headers = {
...params.headers,
Authorization: `Bearer ${token || customToken}`
};
}
console.log(params.headers);
console.log(params.headers.Authorization);
console.log(`${token}`);
You can see the console.log() result :
[Sat Oct 17 2020 11:53:17.527] LOG {"Authorization": "Bearer {\"session\":\"v1:whut-xYXgZJRw-tSsKOE=\",\"scopes\":[\":*\"],\"signature\":\"-GLiIN9O0SfAzMofgIQh1GbiI9cMEvQVqJLD98atmBQ=\"}", "Content-Type": "application/json"}
[Sat Oct 17 2020 11:53:17.529] LOG Bearer {"session":"v1:whut-xYXgZJRw-tSsKOE=","scopes":[":*"],"signature":"-GLiIN9O0SfAzMofgIQh1GbiI9cMEvQVqJLD98atmBQ="}
[Sat Oct 17 2020 11:53:17.529] LOG {"session":"v1:whut-xYXgZJRw-tSsKOE=","scopes":[":*"],"signature":"-GLiIN9O0SfAzMofgIQh1GbiI9cMEvQVqJLD98atmBQ="}
The first log is the problem : i don't really understand why the first log add backslash :/
I've try to add replace() methode, but without result.
Anyone can help me ?
Thank you community !
I am not entirely sure if this is a bad thing or not, but i was monitoring (logging) what incoming http request i get with this piece of code (note that running an NodeJS application on the scalable OpenShift platform):
function onRequest(request, response)
{
var date = new Date();
console.log(date.toUTCString() + " A request was made with url: " + request.url + " and header: " + JSON.stringify(request.headers));
// continue handling the request
}
The results i get are the following (every 2 seconds):
Fri, 07 Mar 2014 09:43:59 GMT A request was made with url: / and header: {}
Fri, 07 Mar 2014 09:44:01 GMT A request was made with url: / and header: {}
Fri, 07 Mar 2014 09:44:03 GMT A request was made with url: / and header: {}
So i am wondering if this is normal behaviour for a scalable NodeJS app (with a MongoDB database gear attached) in openshift, or is this something that could cause problems?
Sincerely,
Hylke Bron
If you are running a scaled application, then that is haproxy making sure that your application is up so that it can forward requests to it. You can change haproxy settings in your haproxy/haproxy.cfg file on your main gear.
I am using this method to inform OPENSHIFT that the application is alive
app.head("/", function(req, res, next){
res.status(200).end();
});
Since I didn't want to mess with haproxy.cfg, I did this and it seems to work. In your main app, add a middleware function to abort on the ping
function ignoreHeartbeat(except) {
except = except || 0;
var count = except;
return function(req, res, next) {
if (req.headers["x-forwarded-for"])
return next();
if (except > 0) {
if (--count <= 0) {
count = except;
return next();
}
}
res.end();
}
}
app.use(ignoreHeartbeat(1000));
Place that code before the call to setup the logger (Express 3 example shown below)
app.use(express.logger(...))
This logs out every 1000th ping. Set except to 0 or -1 to ignore all the pings.
I have a simple API upload, it is used to accept upload file from client.
var flg=true;
app.post('/test', function(req, res){
flg=!flg;
var returnJson='{';
if(flg){
req.form.on('part', function (part) {
if(part){
part.resume();
}
returnJson=returnJson+',\"status\":\"0\"}';
res.send(returnJson);
});
}else{
console.log('close');
returnJson=returnJson+',\"status\":\"1\"}';
res.header('Connection', 'close');
res.send(413, returnJson);
}
});
I'd like to test this API with Jmeter. "status":"0" means success. "status":"1" means failure. I write Jmeter script like this:
http://i.imgur.com/vEUJKc8.jpg
Jmeter only displays all the samplers which response contains "status":"0". it seems Jmeter exclude failure sampler response which comes from else section.
http://imgur.com/bkFSpK2
How can I see all samplers which includes all success and failure samplers in Jmeter?
successful Sampler result is :
Thread Name: API 1-1
Sample Start: 2013-12-18 11:46:08 PST
Load time: 7
Latency: 6
Size in bytes: 178
Headers size in bytes: 163
Body size in bytes: 15
Sample Count: 1
Error Count: 0
Response code: 200
Response message: OK
Response headers:
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/html; charset=utf-8
Content-Length: 15
Date: Wed, 18 Dec 2013 19:46:08 GMT
Connection: keep-alive
HTTPSampleResult fields:
ContentType: text/html; charset=utf-8
DataEncoding: utf-8
Any suggestion?
I don't like this stanza:
ContentType: text/html;
Correct ContentType for JSON will be application/json
You can try using HTTP Header Manager to set Content-Type header of your request to be application/json and see what happens.
Also there is a JSON plugin which provides JSON Path Extractor and JSON Path Assertion (Select “Extras with libs set” from the download list).
I am trying to properly handle Accept headers in RESTful API in node.js/restify by using WrongAcceptError as follows.
var restify = require('restify')
; server = restify.createServer()
// Write some content as JSON together with appropriate HTTP headers.
function respond(status,response,contentType,content)
{ var json = JSON.stringify(content)
; response.writeHead(status,
{ 'Content-Type': contentType
, 'Content-Encoding': 'UTF-8'
, 'Content-Length': Buffer.byteLength(json,'utf-8')
})
; response.write(json)
; response.end()
}
server.get('/api',function(request,response,next)
{ var contentType = "application/vnd.me.org.api+json"
; var properContentType = request.accepts(contentType)
; if (properContentType!=contentType)
{ return next(new restify.WrongAcceptError("Only provides "+contentType)) }
respond(200,response,contentType,
{ "uri": "http://me.org/api"
, "users": "/users"
, "teams": "/teams"
})
; return next()
});
server.listen(8080, function(){});
which works fine if the client provides the right Accept header, or no header as seen here:
$ curl -is http://localhost:8080/api
HTTP/1.1 200 OK
Content-Type: application/vnd.me.org.api+json
Content-Encoding: UTF-8
Content-Length: 61
Date: Tue, 02 Apr 2013 10:19:45 GMT
Connection: keep-alive
{"uri":"http://me.org/api","users":"/users","teams":"/teams"}
The problem is that if the client do indeed provide a wrong Accept header, the server will not send the error message:
$ curl -is http://localhost:8080/api -H 'Accept: application/vnd.me.org.users+json'
HTTP/1.1 500 Internal Server Error
Date: Tue, 02 Apr 2013 10:27:23 GMT
Connection: keep-alive
Transfer-Encoding: chunked
because the client is not assumed to understand the error message, which is in JSON, as
seen by this:
$ curl -is http://localhost:8080/api -H 'Accept: application/json'
HTTP/1.1 406 Not Acceptable
Content-Type: application/json
Content-Length: 80
Date: Tue, 02 Apr 2013 10:30:28 GMT
Connection: keep-alive
{"code":"WrongAccept","message":"Only provides application/vnd.me.org.api+json"}
My question is therefore, how do I force restify to send back the right error status code and body, or am I doing things wrong?
The problem is actually that you're returning a JSON object with a content-type (application/vnd.me.org.api+json) that Restify doesn't know (and therefore, creates an error no formatter found).
You need to tell Restify how your responses should be formatted:
server = restify.createServer({
formatters : {
'*/*' : function(req, res, body) { // 'catch-all' formatter
if (body instanceof Error) { // see text
body = JSON.stringify({
code : body.body.code,
message : body.body.message
});
};
return body;
}
}
});
The body instanceof Error is also required, because it has to be converted to JSON before it can be sent back to the client.
The */* construction creates a 'catch-all' formatter, which is used for all mime-types that Restify can't handle itself (that list is application/javascript, application/json, text/plain and application/octet-stream). I can imagine that for certain cases the catch-all formatter could pose issues, but that depends on your exact setup.
I am experiencing occasional retries for a POST request, when there is no response from server due to timeout. All modern browsers have retry logic for idempotent requests (GET, HEAD, etc) but I am unable to reason out why it happens for a POST request.
I am testing this case using a simple node.js server with 3 routes and chrome browser .
/ : gives a html page with jquery and code snippets to fire ajax requests
/hi : gives a text response 'hello'
/sleep : request will timeout without any response
By default, node.js http server times out a request after 2 minutes.
retry.js
var http = require('http');
var server = http.createServer();
server.on('request', function(req, res) {
console.log(new Date() + ' ' + req.method + ' request on ' + req.url);
if (req.url === '/sleep') {
console.log('!!! sleeping');
} else if (req.url === '/') {
html = "$.post('/hi', {'for':'server'}, function() { console.log(arguments) } ).error(function() { console.log(arguments) })";
html += "<br><br>";
html += "$.post('/sleep', {'for':'infinite'}, function() { console.log(arguments) } ).error(function() { console.log(arguments) })";
html += '<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>';
res.writeHead(200, {'Content-Type': 'text/html'});
res.end(html);
} else {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('hello');
}
});
server.listen(2020);
console.log('server listening on port 2020');
run it
$ node retry.js
server listening on port 2020
1
Load this page in browser http://localhost:2020
Fri Mar 01 2013 12:21:59 GMT+0530 (IST) GET request on /
Fri Mar 01 2013 12:21:59 GMT+0530 (IST) GET request on /favicon.ico
2
From Dev console, fire an ajax POST request to /hi using jquery
$.post('/hi', {'for':'server'}, function() { console.log(arguments) } ).error(function() { console.log(arguments) })
Fri Mar 01 2013 12:22:05 GMT+0530 (IST) POST request on /hi
3
Fire a POST request to /sleep, results in a retry after 2 mins and errors out after 4 mins.
$.post('/sleep', {'for':'infinite'}, function() { console.log(arguments) } ).error(function() { console.log(arguments) })
server logs shows 2 requests
Fri Mar 01 2013 12:22:21 GMT+0530 (IST) POST request on /sleep
!!! sleeping
Fri Mar 01 2013 12:24:21 GMT+0530 (IST) POST request on /sleep
!!! sleeping
Firing it again, errors out in 2 mins without any retry.
Fri Mar 01 2013 12:30:01 GMT+0530 (IST) POST request on /sleep
!!! sleeping ?
It's not getting retried until we fire a request to /hi (or any other url) that results in a response. And retry happens for just one subsequent request to /sleep.
In browser, the network tab shows the pattern like
/hi - success
/sleep - cancelled - 4 mins (retry happens)
/sleep - cancelled - 2 mins (no retry)
/sleep - cancelled - 2 mins (no retry)
/hi - success
/sleep - cancelled - 4 mins (retry happens)
/sleep - cancelled - 2 mins (no retry)
/sleep - cancelled - 2 mins (no retry)
Question
Though we need to design our web app to tolerate these extra requests (either by browsers or any other intermediaries), this inconsistent browser retries looks weird. I had observed this behaviour in chrome (v16 & v24) & firefox.
Can someone help me to understand the browser retry logic behind timed out non-idempotent requests ?
Other relevant stackoverflow questions
What happens when no response is received for a request? I'm seeing retries
Browsers retry requests (including POSTs) when the connection is closed before receiving a response from the server. This is defined in the HTTP 1.1 Spec Section 8.2.4.