Asynchronously writing files in a loop, how to manage streams - node.js

Question

I am trying to write over 100 png files via node-canvas in a loop. Only 40 files get generated and then the process completes.
I have tried creating a png stream via createPNGStream() and piping the results to a write stream created by fs.createWriteStream().
Write function:
function writeFile(row, col) {
// canvas code ...
const stream = canvas.createPNGStream();
let path = __dirname + '/imgs/heatmapRow' + row + "Col" + col + '.png';
const out = fs.createWriteStream(path);
stream.pipe(out);
out.on('finish', () => console.log('The PNG file was created.'))
}
Calling function:
function generateImages(){
var numRows = 20;
var numCols = 5;
for(let row = 0; row < numRows; ++row) {
for (let col = 0; col < numCols; ++col) {
writeFile(row, col);
}
}
}
The loop runs and completes, and at the end I get a bunch of the following lines all at once:
The PNG file was created.
The PNG file was created.
The PNG file was created.
The PNG file was created.
I'm thinking that on each loop a writeStream is being created and is asynchronous. The process is terminating because I can have only so many streams open.
How can I write all of my files and do so asynchronously to speed up the process time? (I prefer not to write the files synchronously) Do I need to add each writeFile to a queue? How do I determine limit the number of streams I have open and how do I manage them?

Answer 1

You have to use Promises to make your asynchronous calls.
I give you a solution:
function writeFile(row, col) {
// canvas code ...
const stream = canvas.createPNGStream();
let path = __dirname + "/imgs/heatmapRow" + row + "Col" + col + ".png";
return new Promise(resolve => {
const out = fs.createWriteStream(path);
stream.pipe(out);
out.on("finish", () => {
console.log("The PNG file was created.");
resolve();
});
});
}
function generateImages() {
var numRows = 20;
var numCols = 5;
var imagesToGenerate = [];
for (let row = 0; row < numRows; ++row) {
for (let col = 0; col < numCols; ++col) {
imagesToGenerate.push(writeFile(row, col));
}
}
Promise.all(imagesToGenerate).then(() => {
console.log("All images generated");
});
}
Take a look at Promise.all docs if you don't clearly understand how it is working