Bad performance compositing images with Sharp - node.js

I have the task of stacking several pngs one on top of another and exporting it as one png. The images do not have the same size, however they do not need to be resized, just cropped to the available canvas.
The images come from the file system and are sent as a buffer. There may be somewhere between 8 and 30 images.
My tests have been with 10 images, and I have compared it with node-canvas which takes less than 50% of the time that Sharp does. I think one problem with Sharp is that I have to make and extract operation before compositing.
This is my code running with Sharp, which takes ~600ms with 10 images:
export const renderComponentsSHARP = async ( layers )=>{
const images = [];
for(let i=0; i<layers.length; i++){
const layer = sharp(layers[i]));
const meta = await layer.metadata();
if(meta.height > AVATAR_SIZE.height || meta.width > AVATAR_SIZE.width)
layer.extract({ top:0, left:0, ...AVATAR_SIZE });
images.push({input:await layer.toBuffer()});
}
return sharp({
create: {
...AVATAR_SIZE,
channels: 4,
background: { r: 0, g: 0, b: 0, alpha: 0 }
}})
.composite(images)
.png()
.toBuffer();
};
This is my code running with canvas, which takes ~280ms:
export const renderComponentsCANVAS = async ( layers )=>{
const canvas = createCanvas(AVATAR_SIZE.width, AVATAR_SIZE.height);
const ctx = canvas.getContext('2d');
const images = await Promise.all(layers.map( layer=>loadImage(layer)));
images.forEach(image=>{
ctx.drawImage(image, 0, 0);
});
return canvas.toBuffer();
};
Am I doing something wrong with sharp? Or is it not the tool for this job?

Related

is there way to compare two images using jimp for node js

I am a beginner in node js, so I am looking for a way to compare two images using jimp for my project. I want to know is that possible, if it is possible the code and method of doing it, or is there another way to do that.
Here's an example of comparing images, we can compare using pixel distance or hamming distance.
This is very similar to an example from the Jimp docs, though we're loading the images from an online source.
You could play around with the thresholds, we're using the same here as Jimp uses in its demo.
const Jimp = require('jimp');
async function compareImages(image1Url, image2Url) {
const image1 = await Jimp.read(image1Url);
const image2 = await Jimp.read(image2Url);
// Perceived distance
const distance = Jimp.distance(image1, image2);
// Pixel difference
const diff = Jimp.diff(image1, image2);
console.log(`compareImages: distance: ${distance.toFixed(3)}, diff.percent: ${diff.percent.toFixed(3)}`);
if (distance < 0.15 || diff.percent < 0.15) {
console.log("compareImages: Images match!");
return true;
} else {
console.log("compareImages: Images do NOT match!");
return false;
}
}
const usFlag = "https://flaglane.com/download/american-flag/american-flag-small.jpg";
const canadianFlagJpg = "https://flaglane.com/download/canadian-flag/canadian-flag-small.jpg";
const canadianFlagPng = "https://flaglane.com/download/canadian-flag/canadian-flag-small.png";
// These should not match.
compareImages(usFlag, canadianFlagJpg);
// These should match.
compareImages(canadianFlagJpg, canadianFlagPng);

Nodejs sharp library toFile method not going into callback code

I am using the sharp nodejs library found here: https://github.com/lovell/sharp. I am trying to take several screenshots, and then piece the images together using the sharp library.
Here is my code. I am using puppeteer to take screenshots of the page, saving in memory as a binary file and combining those binary files together using sharp's composite() method.
let pagePath = 'path/to/file.png';
let maxScreenshotHeight = 2000;
// Loop over sections of the screen that are of size maxScreenshotHeight.
for (let ypos = 0; ypos < contentSize.height; ypos += maxScreenshotHeight) {
const height = Math.min(contentSize.height - ypos, maxScreenshotHeight);
let image = await page.screenshot({
encoding: 'binary',
clip: {
x: 0,
y: ypos,
width: contentSize.width,
height
}
});
composites.push({input: image, gravity: 'southeast'});
}
sharp()
.composite(composites)
.toFile(pagePath, function(err) {
if (err) {
console.log('fail');
return;
}
console.log('complete');
});
However, in the toFile callback, nothing ever gets logged. Console logging works, as I've added logs before and after the toFile statement, but it seems that this function call never completes. I want to create a png file that I can later download.
How can I merge these multiple screenshots and store them on the server for a later download? Am I using toFile incorrectly?

Max height of 16,384px for headless Chrome screenshots?

No matter what I try, the maximum height of a full-page screenshot with headless Chrome 60 (and 59) is 16,348px.
It is not a memory issue. There are no segfaults and no, for example, FATAL:memory_linux.cc(35) Out of memory. messages. None. Capturing screenshots are "successful". Changing the screenshot format - PNG or JPEG - has no affect.
The screenshot width can vary, but the height of the saved screenshot is always limited to 16,348px. Example,
1600x16384, 1200x16384, 2400x16384, etc.
I'm taking upscaled screenshots with this minimal code (full source):
const upscale = 2;
const viewportWidth = 1200;
const viewportHeight = 800;
....
// Set up viewport resolution, etc.
const deviceMetrics = {
width: viewportWidth,
height: viewportHeight,
deviceScaleFactor: 0,
mobile: false,
fitWindow: false,
scale: 1 // Relative to the upscale
};
await Emulation.setDeviceMetricsOverride(deviceMetrics);
await Emulation.setVisibleSize({width: viewportWidth, height: viewportHeight});
await Emulation.setPageScaleFactor({pageScaleFactor: upscale});
// Navigate to target page
await Page.navigate({url});
const {root: {nodeId: documentNodeId}} = await DOM.getDocument();
const {nodeId: bodyNodeId} = await DOM.querySelector({
selector: 'body',
nodeId: documentNodeId,
});
const {model: {height}} = await DOM.getBoxModel({nodeId: bodyNodeId});
// Upscale the dimensions for full page
await Emulation.setVisibleSize({width: Math.round(viewportWidth * upscale), height: Math.round(height * upscale)});
// This forceViewport call ensures that content outside the viewport is
// rendered, otherwise it shows up as grey. Possibly a bug?
await Emulation.forceViewport({x: 0, y: 0, scale: upscale});
const screenshot = await Page.captureScreenshot({format});
const buffer = new Buffer(screenshot.data, 'base64');
file.writeFile(outFile, buffer, 'base64', function (err) {
if (err) {
console.error('Exception while saving screenshot:', err);
} else {
console.log('Screenshot saved');
}
client.close();
});
I was unable to find any useful Chrome switches either. Is this a hardcoded limit of Chrome? 16384 is a suspicious number (2^14 = 16,384). How can this height be increased?
This is a limitation of Chromium as documented in this bug
https://bugs.chromium.org/p/chromium/issues/detail?id=770769&desc=2

Add image as frame to a gif

So I want to create a gif and add an image as a frame. I am using gifencoder. I've seen the examples and its pretty easy to add colors, txt etc, but I couldnt figure out how to do the same with an image. Do I need something like the png file stream.
ex: from website for color frame
let canvas = new Canvas(width, height);
var ctx = canvas.getContext('2d');
red rectangle
ctx.fillStyle = '#ff0000';
ctx.fillRect(0, 0, 320, 240);
encoder.addFrame(ctx);
Edit:
I tried doing something like below and while the frames appear on my log as wanted the frames dont get added, no errors. Before someone suggests to convert to base64 the png/gif this module cant add such frames, that being said if you have another in mind that does please go ahead and say so.
fs.createReadStream('test.gif')
.pipe(new GIFDecoder) //gif-stream
.pipe(concat(function(frames) { //concat-frames
console.log(frames);
for (var i=0; i<frames.length; i++) {
encoder.addFrame(frames[i]);
}
}));
whatever I have tried at best I get just a black gif, nothing more.
Edit 2:
Okay so the reason I am trying to add from from a gif the frame, was because it simply seemed easier. Below is the progress I have done trying to get an image, convert it to RGBA (i noticed the module accepts rgba format, not rgb, perhaps that's why it was always black) and afterwards add the frame. Adding the frame if I have the data is extremely easy as I simply call a method and push the data in, so I am going to leave that out.
var request = require('request').defaults({ encoding: null });
request.get('https://upload.wikimedia.org/wikipedia/commons/0/01/Quinlingpandabearr.jpg', function (error, response, body) {
if (!error && response.statusCode == 200) {
var img = new Canvas(105, 159);
var context = img.getContext('2d');
img.src = "data:" + response.headers["content-type"] + ";base64," + new Buffer(body).toString('base64');
var img_to_rgba = context.getImageData(0, 0, img.width, img.height);
console.log(img_to_rgba); //always returns 0, like my pic is completely black, its not.
}
});
I've added an image as a frame to a gif, along side a blue square.
Here is the full code, tested and working fine for me:
var GIFEncoder = require('gifencoder');
var Canvas = require('canvas');
var fs = require('fs');
var encoder = new GIFEncoder(320, 240);
encoder.createReadStream().pipe(fs.createWriteStream('myanimated.gif'));
encoder.start();
encoder.setRepeat(0);
encoder.setDelay(500);
encoder.setQuality(10);
var canvas = new Canvas(320, 240);
var ctx = canvas.getContext('2d');
// blue rectangle frame
ctx.fillStyle = '#0000ff';
ctx.fillRect(0, 0, 320, 240);
encoder.addFrame(ctx);
// image frame
var data = fs.readFileSync(__dirname + '/image.jpg');
var img = new Canvas.Image;
img.src = data;
ctx.drawImage(img, 0, 0, 320, 240);
encoder.addFrame(ctx);
encoder.finish();
Note that I'm using readFileSync this time, but you can use readFile too as in my other answer, depending on your code.
I've also change the size of the loaded image, to fit the square.
You can load the image onto the canvas with fs.readFile:
fs.readFile(__dirname + '/image.jpg', function(err, data) {
if (err) throw err;
var img = new canvas.Image;
img.src = data;
ctx.drawImage(img, 0, 0, img.width, img.height);
})

Check for transparency, GraphicsMagick node.js

I am making a code where users can upload a image. The image is converted with GraphicsMagick and uploaded to our cloud. But it will be best if non-transparent images was converted to JPG instead of PNG for transparent images. How can I check if the image contain a alpha channel in GraphicsMagick?
I am not sure you can achieve that using only GraphicsMagick, but it is possible in several other ways. For example with pngjs:
You can check PNG metadata:
const gm = require('gm');
const PNG = require('pngjs').PNG;
gm('/path/to/image')
.stream('png')
.pipe(new PNG({}))
.on('metadata', meta => {
if (meta.alpha) {
// image is transparent
} else {
// image is not transparent
}
});
Or iterate over pixels and decide if it's transparency valuable to you, or you can omit it:
...
.on('parsed', function() {
let isAlphaValuable = false;
for (var y = 0; y < this.height; y++) {
for (var x = 0; x < this.width; x++) {
var idx = (this.width * y + x) << 2;
// this.data[idx] - red channel
// this.data[idx + 1] - green channel
// this.data[idx + 2] - blue channel
// this.data[idx + 3] - alpha channel
// if there is at least one pixel
// which transparent for more than 30%
// then transparency valuable to us
isAlphaValuable |= (1 - this.data[idx + 3] / 255) > 0.3;
}
}
if (isAlphaValuable) {
// keep transparency
} else {
// ignore transparency
}
});
You may also give imagemagick a try
Snippet is in TypeScript and makes use of BPromise.promisify for better readability.
Note that this works for PNG, JPEG the expected way (returning string true/false), but for GIFs it will give you a concatenated 'true'|'false' string (e.g. 'truetruefalse', and apply the alpha check per frame).
I also recommend applying .trim() to the result to get rid of potential useless whitespace returned by imagemagick v0.x. every now and then.
import * as imagemagick from 'imagemagick';
import * as BPromise from 'bluebird';
...
const opaqueAsync: any = BPromise.promisify(imagemagick.identify, {context: imagemagick});
const isOpaqueReturnValue: string = await opaqueAsync(['-format', '%[opaque]', picturePath]);
const isPicTransparent: boolean = 'false' === isOpaqueReturnValue.trim();

Resources