I am new to NodeJS as well as AWS Lambda. I am creating a lambda function to get pre-signed URL for s3 put object. The problem I am facing here is that, call s3.getSignedUrl is async process which respond signed URL in callback but before this call responds, lambda function controls is reaching to the next line after this call and empty response is being received at client side. here is my code snippet of lambda function.
exports.handler = async (event) => {
// Some stuff like reading inputs from request, initialising s3 object with AWS_ACCESS_KEY, and AWS_SECRET_KEY
var response="default response";
response = s3.getSignedUrl('putObject', {
"Bucket": myBucket,
"Key": fileName,
"ContentType": fileType
}, function (err, url) {
if (err) {
console.log("error")
return {
statusCode: 200,
body: JSON.stringify("Error in creating signed url"),
};
} else {
console.log("generated "+url)
return {
statusCode: 200,
body: JSON.stringify(url),
};
}
});
console.log("at end");
return response;
};
If say in terms of log statements then before statement error or generated url at end log statement is printed and default response is returning. Can some body correct this code to responde the response being prepared inside callbacks of s3.getSignedUrl.
If you are just looking to create presigned url, you can use getSignedUrlPromise method that returns promise
exports.handler = async (event) => {
return s3.getSignedUrlPromise("putObject", {
Bucket: "my bucket",
Key: "my file name",
ContentType: "some file type",
});
};
If you need customize the response
exports.handler = async (event) => {
return new Promise((resolve, reject) => {
s3.getSignedUrl(
"putObject",
{
Bucket: "my bucket",
Key: "my file name",
ContentType: "some file type",
},
(err, url) => {
if (err) reject(err);
if (url)
resolve({
statusCode: 200,
body: url,
});
});
});
};
Easiest by far is to use the Promise API for aws-sdk, and just await getSignedUrl.
Related
I have a lambda and I want to invoke multiple lambdas instance and wait for the longest to complete.
I already checked online and found very interesting threads but none of them helped me achieve what I want.
Here's the code:
Lambda as promise
invokeVideoRenderer = (params) => {
const param = {
FunctionName: 'myFunction',
InvocationType: 'Event',
Payload: JSON.stringify({ body: params })
};
return new Promise(async (resolve, reject) => {
this.lambda.invoke(param, (error, data) => {
if (error) {
reject(error);
}
resolve(data); <--- it always returns { StatusCode: 202, Payload: '' }
});
});
};
the loop where I invoke the different lambdas
const promises = [...Array(totalNumberOfLambdaToInvoke).keys()].map(
async (part: number) => {
const body = {
// body
};
const request = await invokeVideoRenderer(body);
return request;
}
);
const results = await Promise.all(promises); <--- here I want to wait for the longest to complete
The lambda I am invoking:
const myFunction = async (event, context, callback) => {
try {
//...code
callback(null, {
status: 200,
body: JSON.stringify({
status: "SUCCESS",
}),
});
return;
} catch (error) {
callback(null, {
status: 500,
body: JSON.stringify({
status: "FAILED",
errorMessage: error,
}),
});
return;
}
};
What am I doing wrong?
An InvocationType of Event tells Lambda to invoke the child Lambda asynchronously. Hence the Lambda service responds with 202 (Accepted) response immediately. Meanwhile the child Lambda function executes.
If you want it to execute synchronously, then don't provide an InvocationType of Event. Either remove it entirely to get the default, or explicitly set it to RequestResponse (which is the default).
Note: it's typically not ideal to invoke Lambda functions synchronously if you can avoid it, because you are paying for both Lambdas concurrently.
I'm trying to mock a call for putObject in a local Node.JS AWS Lambda function using Jest but for some reason I keep getting 0 number of calls in my expect.
Here is my main func(index.js):
const S3 = require("aws-sdk/clients/s3");
const s3 = new S3();
exports.handler = async (event) => {
putFunction = async (params, callback) => {
await s3.putObject(params, callback).promise();
};
const params = {
Bucket: "some value",
Key: "some key value",
ContentType: "application/json",
Body: "some body value",
};
const callback = {
function(err, data) {
console.log(JSON.stringify(err) + " " + JSON.stringify(data));
},
};
putFunction(params, callback);
}
I've tried adding async to my test function as I thought it was an asynchronous issue but that I still seem to be receiving the same error. Here is my test code(index.test.js):
let myHandler = require("../../src/lambda/index");
const mockedPutObject = jest.fn();
jest.mock("aws-sdk/clients/s3", () => {
return class S3 {
putObject(params, cb) {
mockedPutObject(params, cb);
}
};
});
it("has to mock s3#putObject", () => {
const params = {
Bucket: "test1",
Key: "test2",
ContentType: "application/json",
Body: "test3",
};
const callback = {
function(err, data) {
console.log(JSON.stringify(err) + " " + JSON.stringify(data));
},
};
const putFunc = myHandler.handler.putFunction;
putFunc;
expect(mockedPutObject).toHaveBeenCalledWith(params, callback);
});
Any help would be great.
This is a Jest/Node only answer for those that don't wish to pull in any third-party mocking libraries like aws-sdk-mock.
The problem (without seeing the error itself in your answer) is very likely to be related to the .promise() in your implementation code.
You have added this in the implementation to tell the SDK to return you a promise for whatever operation was called.
await s3.putObject(params, callback).promise();
That returned promise will either reject with an error or resolve with the data.
This means in the promise-based approach you can omit the callback completely.
await s3.putObject(params).promise();
(taken from this AWS blog post)
Fixing the handler...
You can either:
Put that callback logic in subsequent promise chain blocks:
.then((data) => {
// ... do stuff
})
.catch((err) => {
// ... handle error
}
or better still (as it looks like you're already embracing) the
more modern ES6 approach of
Awaiting the putObject promise within in a try-catch block:
try {
const data = await s3.putObject(params).promise()
// ... do things with data on successful response
} catch (err) {
// ... handle error
}
Putting those together
Your handler should look something like this:
const { S3 } = require("aws-sdk");
const s3 = new S3();
exports.handler = async (event) => {
const params = {
Bucket: "some value",
Key: "some key value",
ContentType: "application/json",
Body: "some body value",
};
try {
const data = await s3.putObject(params).promise();
// ... do stuff with data
return {
statusCode: 200,
body: JSON.stringify(data),
// ... etc.
}
} catch (err) {
// ... handle error
return {
statusCode: 400, // or any 4XX, 5XX
body: '...', // whatever you wish to return on error
// ... etc.
}
}
}
Fixing the tests...
Bearing in mind that you can omit the callback, the test code needs to reflect the extra .promise() in the call chain of the putObject in the handler.
In the test file, the SDK mock needs to be configured to:
a) return the top-level S3 constructor
b) have this S3 constructor itself return an object containing the putObject function
c) have this putObject itself return an object containing the promise function
So that it can be invoked as the real SDK would:
const { S3 } = require("aws-sdk"); // require("aws-sdk") returns { S3 }
const s3 = new S3() // returns { putObject }
await s3.putObject(params) // returns { promise }
.promise(); // returns ...your_mock_response
// You need to return the { promise } here even if you don't care about
// mock calls beyond the putObject, because the handler itself calls .promise()
// and will throw "TypeError: Cannot read property 'promise' of undefined"
const putObjectMock = jest.fn(() => ({
promise: jest.fn()
}));
jest.mock('aws-sdk', () => ({
S3: jest.fn(() => ({
putObject: putObjectMock,
})),
}));
// S3 must have jest.fn(...) over an ordinary function otherwise
// the new S3() in the handler will fail.
// Jest does its magic with the function you provide to make it callable as a constructor
const myHandler = require("../../src/lambda/index");
// Don't forget to add the "async" before the "it" callback as your handler is async
it("has to mock s3#putObject", async () => {
const params = {
Bucket: "test1",
Key: "test2",
ContentType: "application/json",
Body: "test3",
};
await handler(); // Call the handler to then assert against the mock params
expect(putObjectMock).toHaveBeenCalledWith(params);
});
Final note - add your handler import after the mock setup to prevent a "Cannot access 'putObject' before initialization" error (caused by the handler's require of the SDK).
Hope this helps!
I'm trying to put data into Dynamodb using serverless deployment. I have added the permission to write to Dynamodb.
The api is always sending {"message": "Internal server error"} but is able to put the data into db once if tried 5,6 times.
Following is the serverless.yaml config
handler: dynamoUpdate.handler
events:
- http:
path: /createdbentry
method: get
cors: true
Following is the code:
const AWS = require('aws-sdk')
AWS.config.update({ region: process.env.REGION || 'us-east-1' })
var ddb = new AWS.DynamoDB({apiVersion: '2012-08-10'});
exports.handler = async (event) => {
return await createDBEntry("i123","Working");
}
const sendRes = (status, body) => {
var response = {
statusCode: status,
headers: {
"Content-Type": "application/json"
},
body: body
};
return response;
};
const createDBEntry = async function(id,result) {
var params = {
TableName: 'emplist',
Item: {
'ID' : {S:id},
'Summary':{S: result},
}
};
ddb.putItem(params, function(err, data) {
console.log("Here comes me.")
if (err) {
console.log("Opps Error");
return sendRes(403,err);
} else {
console.log("Complete")
return sendRes(200,data);
}
});
}
How can it be resolved ?
The problem is you are not returning any promise or awaiting anything async in your function called createDBEntry. Thus your handler returns undefined which makes apigateway return 500 internal server error.
You are mixing callbacks and async/await.
Your createDBEntry function should look like this.
const createDBEntry = async function(id, result) {
var params = {
TableName: 'emplist',
Item: {
'ID' : {S:id},
'Summary':{S: result},
}
};
try {
let data = await ddb.putItem(params).promise()
return sendRes(200, JSON.stringify(data))
} catch (err) {
console.log("Oops Error");
return sendRes(403, err.toString());
}
}
When you return from an async handler in lambda the current execution environment is frozen immediately (unlike when you use a non-async handler with a callback). This is why most of the time the writes to the database does not succeed.
I have a lambda function trying to place an mp3 file into an S3 bucket, however I am not seeing my file uploaded and more strangely do not see any logging/response from the callback.
My lambda/s3 bucket are all on the same AWS account and the bucket name is definitely correct.
Is there something I'm missing here? Or any explanation why my callback is not being fired?
exports.handler = async (event, context, callback) => {
// prior setup
console.log('about to putObject on s3');
const s3BucketData = {
Bucket: 'media-files',
Key: fileName,
Body: fileDataBuffer,
ContentType: 'audio/mp3'
};
await s3.putObject(s3BucketData, (err, data) => {
console.log('putObject callback executing');
if (err) {
console.log('err occurred storing to s3: ', err)
} else{
console.log(`${fileName} succuessfully uploaded`);
}
context.done();
});
};
First of all, it's a bad practice to stick your methods inside of the handler function. Second of all you have some issue with your runtime. I mean that you choosed node 8.10 with await/async support, but you still trying to use callbacks.
I have some comments for you. I hope it's going to help you.
1) You can simple do that:
export async function handler(event)
{
// body of your function
};
2) AWS services promisified. You have to re-write your s3 method. Take a look at the following snippet. And I've got a question. Are sure that you have to use putObject method instead of upload?
try
{
let s3= new AWS.S3({ region: process.env.AWS_REGION, apiVersion: '2006-03-01' });
let params =
{
Bucket: //aws s3 bucket location (a full path),
Key: //file name/key,
Body: //entity to upload,
ACL: 'public-read' // access policy,
ContentType: 'audio/mp3'
};
let s3Response = await s3.upload(params).promise();
// request successed
console.log(`File uploaded to S3 at ${s3Response.Bucket} bucket. File
location: ${s3Response.Location}`);
return s3Response.Location;
}
// request failed
catch (ex)
{
console.error(ex);
}
If you want to stick callbacks then:
s3.upload(params, (err, data) =>
{
console.log('putObject callback executing');
if (err)
{
console.error('err occurred storing to s3: ', err);
return ;
}
console.log(`${fileName} succuessfully uploaded`);
return data;
});
I hope you'll find this helpfull. Cheers!
Below example help you to convert putObject to promise
exports.handler = (event, context, callback) => {
console.log('about to putObject on s3');
const s3BucketData = {
Bucket: 'media-files',
Key: fileName,
Body: fileDataBuffer,
ContentType: 'audio/mp3'
};
S3.putObject(s3BucketData).promise()
.then(data => {
console.log('complete:PUT Object',data);
callback(null, data);
})
.catch(err => {
console.log('failure:PUT Object', err);
callback(err);
});
};
How can my AWS Lambda make an API call to some external service and return the output to the user who invoked the function please?
My piece of code looks like this:
module.exports.oauth_callback = (event, context, callback) => {
oauth2.authorizationCode.getToken({
redirect_uri: 'https://' + event.headers.Host + event.requestContext.path
}).then(results => {
callback(null, {statusCode: 200, body: JSON.stringify(results)});
});
};
With this, I receive a 500 Internal Error because I believe the lambda function is exiting before returning the result.
EDIT Case number 2:
Actually in the case where I add another callback like this:
return oauth2.authorizationCode.getToken({
code: code,
}).then(results => {
createFirebaseAccount(results).then(token => {
const template = signInFirebaseTemplate(token);
return callback(null, { statusCode: 200, body: template.toString() });
});
});
};
My variable template has its value but the next callback never returns and my function times out
I think you might need to return your promise from getToken(), like this:
module.exports.oauth_callback = (event, context, callback) => {
return oauth2.authorizationCode.getToken({
redirect_uri: 'https://' + event.headers.Host + event.requestContext.path
}).then(results => {
callback(null, {statusCode: 200, body: JSON.stringify(results)});
});
};
If that doesn't work, check your CloudWatch log under the monitoring tab of your Lambda.