I'm using nodejs and graphicsmagick to process images with text, then streaming the final jpg to S3.
Using postman, I was able to test this flow on my localhost and everything works fine. However, I'm having issues now that I moved it to Elastic Beanstalk. When I post to the endpoint, it uploads a blank file to S3 and there are no errors logged in EB. I think it has something to do with the software but am a bit stuck. Any advice appreciated! Thanks!
Top file is from localhost, bottom file is from Elastic Beanstalk:
http://cl.ly/image/0O231k171N0W
var gm = require('gm');
var appRoot = require('app-root-path').path;
function createImage(caption, res) {
var originalImage = '/images/2015-02-24.jpg';
var textColor = 'white';
gm(appRoot + originalImage)
.fill(textColor)
.font( appRoot + '/fonts/BentonSans-Book.otf')
.drawText(0, 0, caption, 'Center')
.stream(function(err, stdout, stderr) {
sendToS3(err, stdout, stderr, originalImage, res);
});
}
function sendToS3(err, stdout, stderr, originalImage, client_response) {
var imageName = shortId.generate();
var buff = new Buffer('');
stdout.on('data', function(data) {
buff = Buffer.concat([buff, data]);
});
stdout.on('end', function(data) {
var data = {
Bucket: S3_bucket,
Key: imageName + '.jpg',
Body: buff,
ContentType: mime.lookup(originalImage)
};
s3.putObject(data, function(err, res) {
client_response.send('done');
});
});
}
===============================================================
EDIT:
Instead of streaming to S3, I changed it to write directly to the filesystem. The error being thrown in AWS EB logs is:
err { [Error: Command failed: gm convert: Request did not return an
image.] code: 1, signal: null }
I believe I'm missing some dependencies for ImageMagick. Any thoughts?
This is from running convert --version in my local terminal:
Version: ImageMagick 6.8.9-7 Q16 x86_64 2014-08-31
http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC Modules
Delegates: bzlib freetype jng jpeg ltdl lzma png xml zlib
This is from running convert --version in my EC2 instance (The Delegates section is empty):
Version: ImageMagick 6.9.1-1 Q16 x86_64 2015-04-10
http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: DPC OpenMP
Delegates (built-in):
How are you installing GraphicsMagick on your EC2 instance in ElasticBeanstalk? Are you using a custom AMI? The default AMI (at least the ones I've used) didn't have GraphicsMagick, I don't know about ImageMagick though.
You can use container commands to install packages with yum. I used the one below on a project where I needed GraphicsMagick.
Create a folder at the root of your project with the name ".ebextensions." Inside of that folder, create a file called "package.config" with the following contents:
commands:
01-command:
command: yum install -y --enablerepo=epel GraphicsMagick
This will install it when the instance is created. I have a feeling this should resolve your issue, if not you may want to use command line options for yum to use the same version or install the delegates as well:
commands:
01-command:
command: yum install -y --enablerepo=epel GraphicsMagick
02-command:
command: yum install -y --enablerepo=epel GraphicsMagick-devel
I lowered my elasticbeanstalks' nodejs version from node 12 to node 8.15.0, and yum CAN find Graphicsmagick and installs it successfully. (I listed Graphicsmagick in .ebextensions/packages.config)
Hope this will help someone!
Related
I am trying to see if my company can use Azure Functions to automate conversions of TIFF files to a number of JPG and PNG formats and sizes. I am using Functions with Node.js, but other languages could be used.
My problem is, that I can't get GraphicsMagick or ImageMagick to work on Functions. I used the normal procedures for installment using npm install.
It seems to install ok, and the module also seems to load, but nothing happens when I try to process a file. Nothing, as in no errors either.
var fs = require('fs');
var gm = require('gm');
module.exports = function (context, req) {
context.log('Start...');
try {
context.log('Looking for GM...');
context.log(require.resolve("gm"));
} catch(e) {
console.log("GM is not found");
process.exit(e.code);
}
gm('D:/home/site/wwwroot/HttpTriggerJS1/input/870003-02070-main-nfh.jpg')
.resize(240, 240)
.noProfile()
.write('D:/home/site/wwwroot/HttpTriggerJS1/output/resize.jpg',
function (err) {
context.log('TEST');
if (!err) {
context.log('done');
}
}
);
context.done(null, res); };
I'm not sure that it's even possible, but I haven't found any information that states that it can't.
So, can I use ImageMagick, GraphicsMagick or a third image converter in Functions? If yes, is there something special that I need to be aware of when installing?
Is there also a C# solution to this?
Web Apps in Azure is a SaaS (Software as a Service). You deploy your bits to the Azure IIS containers, and Azure do the rest. We don’t get much control.
So we will not have the privilege to install any 3rd party executable file on Azure Functions App (e.g. ImageMagick or GraphicsMagick). If you need to do that, look at Virtual Machines. Another option is using Cloud Services' Web or Worker Role.
Alternatively, there is a good image processing library for Node written entirely in JavaScript, with zero external or native dependencies, Jimp. https://github.com/oliver-moran/jimp
Example usage:
var Jimp = require("jimp");
Jimp.read("lenna.png").then(function (lenna) {
lenna.resize(256, 256) // resize
.quality(60) // set JPEG quality
.greyscale() // set greyscale
.write("lena-small-bw.jpg"); // save
}).catch(function (err) {
console.error(err);
});
There is another node.js library called sharp to achieve your requirement. You may try this way:
First, install the sharp on your local environment, and then deploy your application to Azure with the node_modules folder which contains the compiled module. Finally, upgrade the node executable on Azure App Service to 64-bits.
The similar thread you can refer here.
Example usage:
var sharp = require("sharp");
sharp(inputBuffer)
.resize(320, 240)
.toFile('output.webp', (err, info) => {
//...
});
Azure functions can run custom docker images also
https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-function-linux-custom-image
Not sure which language you are interested in, but you can have a python image with below style Dockerfile
FROM mcr.microsoft.com/azure-functions/python:2.0
RUN apt-get update && \
apt-get install -y --no-install-recommends apt-utils && \
apt-get install -y imagemagick
ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true
COPY . /home/site/wwwroot
RUN cd /home/site/wwwroot && \
pip install -r requirements.txt
And then use the PythonMagick to work with the same
You can use the site extension to make imagemagick work for azure web apps.
You can check the repository for more info: https://github.com/fatihturgut/azure-imagemagick-nodejs
I'm working on a Node.js application and I'm facing a critical problem right now which it is Resizing img after uploading .. the thing is everything goes right starting from uploading to getting the path of the uploaded img but I can't understand why this is happening .. here's my code and the error
var fileName = req.file.filename;
var filpath = "/uploads/" + fileName;
gm(filpath)
.resize(240, 240)
.write('/uploads/'+'resized-'+fileName, function (err) {
if (err) console.log(err);
else console.log("HOOLA");
});
The error is
{ [Error: Command failed: Invalid Parameter - /uploaded_file-1474832730810.png
] code: 4, signal: null }
The log
UPDATE:
first of all I would like to thank #r-a-lucas,and here's the full answer for Windows users:
npm install gm --save
download the latest ImageMagick from here ImageMagic EXE
Make sure it is excusable (advance system settings -> Advance -> Environment Variables -> add Path most likely in C:\Program Files\ImageMagick-*** )
In your JS file make sure to add (__dirname) to get the full path of the img. gm(__dirname+filpath)
It looks like you may need to install imagemagick for Windows. Check out this answer here: imagemagick with nodejs not working. Hopefully that helps. Best.
I'm a Linux & node noob. I'm trying to run FabricJS (which requires node-canvas) in AWS Lambda. I've been able to follow the instructions to get up and running on an AWS Linux EC2, however, Lambda has me at my wits end. Anyone have any tips or pointers on how to get this compiled for AW Lambda?
I found this issue in the Node Canvas GitHub site. The questioner was trying to run FabricJS in Lambda as well. Here is the relevant section with an answer:
Make sure you're compiling this on the same AMI that lambda currently uses:
http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html
Lambda runs at /var/task (that's the path when you unzip), so something.so at the root of the zip file will be at /var/task/something.so.
We then want to build our libraries using an "rpath":
export LDFLAGS=-Wl,-rpath=/var/task/
Compile libraries according to: https://github.com/Automattic/node-canvas/wiki/Installation---Amazon-Linux-AMI-%28EC2%29
You may want to set the prefix= ~/canvas to have all the files in one place.
Install node-canvas with npm
cd node_modules/canvas; node-gyp rebuild
mkdir ~/pkg and cp the .so files (~/canvas/lib/*.so) there, using -L to dereference symlinks.
scp the pkg directory to the local lambda folder, possibly putting the files in the right places. (.so in root, node_modules/canvas with other libs). You'll probably want to rearrange this.
Here is a gulp plugin which could upload your files along with node-canvas and its dependencies binary specifically built for aws lambda.
NPM Package
'use strict';
//This is a sample gulp file that can be used.
//npm install --save gulp gulp-zip gulp-awslambda
const gulp = require('gulp');
const zip = require('gulp-zip');
const path = require('path');
const lambda = require('gulp-awslambda');
const aws_lambda_node_canvas = require('./');
let runtime = 'nodejs4.3' // nodejs or nodejs4.3
const lambda_params = {
FunctionName: 'NodeCanvas', /* Lambda function name */
Description: 'Node canvas function in aws lambda', //Description for your lambda function
Handler: 'main.lambda_handler', //Assuming you will provide main.py file with a function called handler.
MemorySize: 128,
Runtime: runtime,
Role : 'ROLE_STRING',//eg:'arn:aws:iam::[Account]:role/lambda_basic_execution'
Timeout: 50
};
var opts = {
region : 'ap-southeast-2'
}
gulp.task('default', () => {
return gulp.src(['main.js', '!node_modules/**/*','!dist/**/*','!node_modules/aws-lambda-node-canvas/**/*']) //Your src files to bundle into aws lambda
.pipe(aws_lambda_node_canvas({runtime : runtime})) //Adds all the required files needed to run node-canvas in aws lambda
.pipe(zip('archive.zip'))
.pipe(lambda(lambda_params, opts))
.pipe(gulp.dest('dist')); //Also place the uploaded file
});
I wrote a Node.JS application that uses the fluent-ffmpeg module to watermark videos uploaded on the platform. I pushed the code to a my Google Cloud Compute Engine project, and every time I get Error : Cannot Find FFMPEG. I ssh'd into the instance once it was created and ran these commands to install FFMPEG before actually testing out the code. I am not sure what is causing the error because after this I am positive that FFMPEG is installed.
sudo apt-get update
sudo apt-get install -y ffmpeg
export FFMPEG_PATH="/usr/bin/ffmpeg"
export FFPROBE_PATH="/usr/bin/ffprobe"
Below is my FFMPEG code
function generate_thumbnail(name, path){
logging.info("Generating Thumbnail");
ffmpeg(path)
.setFfmpegPath('/usr/bin/ffmpeg')
.setFfprobePath('/usr/bin/ffprobe')
.on('end', function() {
upload_thumbnail(name);
logging.info("Thumbnail Generated and uploaded");
return;
})
.on('error', function(err, stdout, stderr) {
logging.info('ERROR: ' + err.message);
logging.info('STDERR:' + stderr);
})
.on('start', function(commandLine) {
logging.info(commandLine);
})
.screenshots({
count: 1,
filename: name + '_thumbnail.png',
folder: 'public/images/thumbnails/'
});
}
Can you execute /usr/bin/ffmpeg from the shell to verify if it's the correct path?
Also you can set path to the FFMPEG binary manually with setFfmpegPath method.
I realize "Running PhantomJS on Heroku" is a related but slightly different question as I am trying to use a node app.
I'm having trouble deploying a casperJS (based on phantomJS) script for a node app. I've tried deploying to Heroku by placing the PhantomJS 64-bit binary in my app's /bin directory, but I get the following error when I try to run PhantomJS:
phantomjs: error while loading shared libraries: libQtWebKit.so.4: cannot open shared object file: No such file or directory
From what I've read this can be solved by installing the QtWebKit library, but Heroku does not have this installed. Is there another hosting provider I could use that will work or a way to install this package on Heroku?
Relevant code:
app.get('/', function(request, response) {
var sys = require('sys')
var exec = require('child_process').exec;
var child;
//works but gives error while loading shared library libqtwebkit.so.4
child = exec("phantomjs --version | tr -d '\n'", function(error, stdout, stderr) {
sys.print('stdout: ' + stdout);
sys.print('stderr: ' + stderr + '\n');
response.header('Content-Type', 'application/json');
response.send('_testcb(stdout:' + stdout + '\nstderr:' + stderr + ')', 200);
if(error !== null) {
console.log('exec error: ' + error);
}
});
});
I've signed up for beta-testing on Nodester but their documentation is still pretty limited at this point.
EDIT: I was able to get it working by simply copying the lib folder of PhantomJS to the root directory of my node app.
Copy the lib folder of phantomjs to the root directory of your node app
You could also try putting a sym link in bin or sbin
The key is that is has to run from terminal using the same account that node runs on.
Also, node-phantom is a good npm library to utilize phantomjs, once you get it working.