AWS Lambda not executing Promise outside Handler until Handler is invoked - node.js

Question

I am working on an aws lambda that requires a puppeteer browser to be launched for each new s3 object in a bucket. The browser launch code was taking a very long time on the initial invocation, so I thought I would put the launch code outside the handler and use Provisioned Concurrency to have the browser ready to go when a new file in inserted into the bucket.
It does seem to call the promise because before any actual invocations are made, I'm getting logs saying "Getting executable path from the provisioned concurrency instances. However, it never outputs the message "Launching browser" until an actual invocation of the lambda is made. Why would the promise chromium.executablePath not complete until an invocation is made if it is outside the handler?
let startTime = Date.now();
const chromium = require("chrome-aws-lambda");
const AWS = require("aws-sdk");
const s3 = new AWS.S3();
const { createSSRApp } = require("vue");
const { renderToString } = require("vue/server-renderer");
const path = require("path");
const fs = require("fs");
const manifest = require("../../compiled/ssr-manifest.json");
console.log("Load packages: " + (Date.now() - startTime));
const browserPromise = new Promise((res) => {
const browserStartTime = Date.now();
console.log("Getting executable path");
chromium.executablePath.then((executablePath) => {
console.log("Launching browser");
chromium.puppeteer
.launch({
args: chromium.args,
defaultViewport: chromium.defaultViewport,
executablePath: executablePath,
headless: true,
})
.then((browser) => {
res(browser);
console.log("Start headless browser: " + (Date.now() - browserStartTime));
});
});
});
browserPromise.then(() => console.log("Started Headless Browser"));
/**
* A Lambda function that logs the payload received from S3.
*/
exports.handler = async (event, context) => {
const bucketName = event.Records[0].s3.bucket.name;
const objectKey = event.Records[0].s3.object.key;
const browser = await browserPromise;
... //use browser code
}
If I require this file locally in another node file it runs the promise fine without calling the handler function, so it must be some lambda environment specific thing I'm not understanding. Does anyone have any insight into this? Thanks in advance.

Answer 1

The issue you describe results from poor control of the async code.
Delay instantiation of browser promise
When you instantiate a Promise using the new keyword, execution of the function you provide starts immediately.
const x = new Promise(res => console.log('test'))
This will print 'test' immediately without needing a .then or await. This is why your code prints 'Getting executable path' right away, vs. waiting for a request event from lambda.
To solve this, don't instantiate this promise until a request actually happens. Move your promise construction to a function that you can call from the handler when a request occurs.
async function startBrowser () {
// code to start browser
return browser
}
exports.handler = async (event, context) => {
const bucketName = event.Records[0].s3.bucket.name;
const objectKey = event.Records[0].s3.object.key;
const browser = await startBrowser();
// use browser
}
Fixing async return flow
Secondly, you need to make your startBrowser function actually return a browser. Because you haven't awaited any of the promises created inside your browserPromise, it will trigger the code to start chromium but resolve immediately. It will take some time for the browser to start, which is why you don't see 'Launching browser' until much later.
To fix this, make sure your browser promise doesn't resolve until the browser is ready, and then return the browser object so it can be used.
function startBrowser () {
const browserStartTime = Date.now();
console.log("Getting executable path");
// await this promise
const browser = await chromium.executablePath.then((executablePath) => {
console.log("Launching browser");
// return browser promise
return chromium.puppeteer
.launch({
args: chromium.args,
defaultViewport: chromium.defaultViewport,
executablePath: executablePath,
headless: true,
})
console.log("Start headless browser: " + (Date.now() - browserStartTime));
return browser
}
exports.handler = async (event, context) => {
const bucketName = event.Records[0].s3.bucket.name;
const objectKey = event.Records[0].s3.object.key;
const browser = await startBrowser();
// use browser
}
Improving performance by sharing browser across requests
You can make further performance improvements by saving the browser in a singleton so that a new one doesn't need to be instantiated every request cycle.
let browser; // singleton/global browser object
// start a browser to be used for all requests
// this will take a little time, so hold a reference to the promise so we
// can know when it is ready to use
const browserPromise = startBrowser.then(newBrowser => { browser = newBrowser }
exports.handler = async (event, context) => {
const bucketName = event.Records[0].s3.bucket.name;
const objectKey = event.Records[0].s3.object.key;
// if the first request comes before the browser is ready, we should
// wait for the promise to resolve
if (!browser) await browserPromise
// use browser
}
Notes about memory usage
Browsers can use a lot of memory and can also leak memory. If fixing your async code still does not make the lambda work, consider that loading a webpage can consume gigabytes of memory especially for large and complex pages (Google Maps for example). I'm not experienced with lambda but you may find yourself running into memory limits.