I am using node js and canvas to create an API that writes text on a certain image. I successfully created a route such as -> /:sometext/:fontsize/:color/:position-x/:position-y and sends a static image with the text of the given font-size, color, and position on the canvas.
What I am unable to achieve is that I want to send the font family in the route and have that rendered on the screen. Plus, isn't there any other way that I can load at least google fonts without having to download the .ttf file.
What I have tried:
GetGoogleFonts npm package (which was a bad idea, since it was stuck at the installation)
WebFontLoader (Gives "Window is not defined" error)
Steps to Reproduce
Currently, I am using a ttf file to load the font
router.get('/thumbnail/:name/:fontsize/:color/:posx/:posy', function (req, res, next) {
let name = req.params.name;
let fontsize = req.params.fontsize
let positionx = req.params.posx
let positiony = req.params.posy
let color = req.params.color
let myimage = 'static/image1.jpg'
const canvas = createCanvas(2000, 1000)
const ctx = canvas.getContext('2d')
var str = "hi "+name
registerFont('AvenirNext.ttf', { family: 'Avenir Next' })
loadImage(myimage).then((image) => {
ctx.drawImage(image, 0 , 0, 2000, 1000);
ctx.font = fontsize + "px Avenir Next"
ctx.fillStyle = color
ctx.fillText(str, positionx, positiony);
const out = fs.createWriteStream(__dirname + '/test.jpeg')
const stream = canvas.createJPEGStream()
stream.pipe(res)
out.on('finish', () => console.log('The JPEG file was created.'))
})
});
If you don't want to host the ttf files on your own server you could try to use the Google Font Github repo.
// missing imports
const http = require('http');
const fs = require('fs');
const fontFamily = req.params.fontfamily; // example: ArchivoNarrow
// download font from github
const file = fs.createWriteStream(fontFamily + '.ttf');
const request = http.get('https://github.com/google/fonts/blob/master/ofl/' + fontFamily.toLowerCase() + '/' + fontFamily + '-Regular.ttf?raw=true', function(response) {
response.pipe(file);
});
registerFont(fontFamily + '.ttf', { family: fontFamily });
// delete font after the image is created
try {
fs.unlinkSync(fontFamily + '.ttf');
} catch(err) {
console.error(err);
}
Font I used for this example: ArchivoNarrow
From the docs for registerFont
To use a font file that is not installed as a system font, use registerFont() to register the font with Canvas. This must be done before the Canvas is created.
emphasis not mine
You are calling it after you created your canvas.
// first register the font
registerFont('AvenirNext.ttf', { family: 'Avenir Next' });
// then create the canvas
const canvas = createCanvas(2000, 1000);
const ctx = canvas.getContext('2d');
var str = "hi " + name;
But note that this will try to load a font that would be available on your server. If you want to make it target a font on ://fonts.googleapis.com server, you'd need to pass the full URI to the font file (not for the .css).
Also, you should wrap your family name inside quotes (") since it contains a space:
ctx.font = fontsize + 'px "Avenir Next"';
I have a Node.js server that uses node-canvas to render text on an image on the server-side. Here is the repo: https://github.com/shawninder/meme-generator (just git clone, npm i and npm run dev to run locally).
As you'll notice in the code, I am loading the Anton font, which I got from here with the documented registerFont function provided by node-canvas
registerFont('./fonts/Anton-Regular.ttf', { family: 'Anton' })
Everything works like a charm locally, but when I deploy to Vercel (formerly known as zeit), that line throws an ENOENT error:
no such file or directory, lstat '/var/task/fonts'
Is there a path I can use here that will successfully load the font from within a Vercel function?
Can I find a single path that will work both locally and once deployed?
I had the same problem recently and I finally found a solution. I'm no guru, so someone will probably be able to suggest a better way, but here's what worked for me.
Because of how Vercel runs their serverless functions, a function doesn't really know anything about the rest of the project, or the public folder. This makes sense (because security), but it does make it tricky when you need the actual path to a file. You can import the font file no problem, the build process will give it a new name and put it on the disk (in /var/task ), but you can't access it. path.resolve(_font_name_) can see it, but you can't access it.
I ended up writing a very bad, separate api page that used path.join and fs.readdirSync to see what files are actually visible from the api page. One thing that is visible is a node_modules folder that contains the files for modules used on that api page.
fs.readdirSync(path.join(process.cwd(), 'node_modules/')
So what I did was write a local module, install it in my project, then import it into my api page. In the local module's package.json, I have a line "files": ["*"] so it will bundle all the module files into its node_modules folder (instead of just the .js files). In my module I have my font file and a function that copies it to /tmp (/tmp is readable and writable) then returns the path to the file, /tmp/Roboto-Regular.ttf.
On my api page, I include this module, then run it, and I pass the resultant path to registerfont.
It works. I'd share my code, but it's pretty sloppy right now, and I'd like to clean it up and try a couple things first (like I'm not sure if I need to copy it to /tmp, but I haven't tested it without that step). When I get it straightened out I'll edit this answer.
-- EDIT
Since I haven't been able to improve on my original solution, let me give some more details about what I did.
In my package.json I added a line to include a local module:
"dependencies": {
"canvas": "^2.6.1",
"fonttrick": "file:fonttrick",
In my project root, I have a folder "fonttrick". Inside the folder is another package.json:
{
"name": "fonttrick",
"version": "1.0.6",
"description": "a trick to get canvas registerfont to work in a Vercel serverless function",
"license": "MIT",
"homepage": "https://grumbly.games",
"main": "index.js",
"files": [
"*"
],
"keywords": [
"registerfont",
"canvas",
"vercel",
"zeit",
"nextjs"
]
}
This is the only local module I've ever had to write; the keywords don't do anything, but at first I'd thought about putting it on NPM, so they're there.
The fonttrick folder also contains my font file (in this case "Roboto-Regular.ttf"), and a the main file, index.js:
module.exports = function fonttrick() {
const fs = require('fs')
const path = require('path')
const RobotoR = require.resolve('./Roboto-Regular.ttf')
const { COPYFILE_EXCL } = fs.constants;
const { COPYFILE_FICLONE } = fs.constants;
//const pathToRoboto = path.join(process.cwd(), 'node_modules/fonttrick/Roboto-Regular.ttf')
try {
if (fs.existsSync('/tmp/Roboto-Regular.ttf')) {
console.log("Roboto lives in tmp!!!!")
} else {
fs.copyFileSync(RobotoR, '/tmp/Roboto-Regular.ttf', COPYFILE_FICLONE | COPYFILE_EXCL)
}
} catch (err) {
console.error(err)
}
return '/tmp/Roboto-Regular.ttf'
};
I ran npm install in this folder, and then fonttrick was available as a module in my main project (don't forget to run npm install there, too).
Since I only need to use this for API calls, the module is only used in one file, /pages/api/[img].js
import { drawCanvas } from "../../components/drawCanvas"
import { stringIsValid, strToGameState } from '../../components/gameStatePack'
import fonttrick from 'fonttrick'
export default (req, res) => { // { query: { img } }
// some constants
const fallbackString = "1xThe~2ysent~3zlink~4yis~5wnot~6xa~7xvalid~8zsentence~9f~~"
// const fbs64 = Buffer.from(fallbackString,'utf8').toString('base64')
// some variables
let imageWidth = 1200 // standard for fb ogimage
let imageHeight = 628 // standard for fb ogimage
// we need to remove the initial "/api/" before we can use the req string
const reqString64 = req.url.split('/')[2]
// and also it's base64 encoded, so convert to utf8
const reqString = Buffer.from(reqString64, 'base64').toString('utf8')
//const pathToRoboto = path.join(process.cwd(), 'node_modules/fonttrick/Roboto-Regular.ttf')
let output = null
if (stringIsValid({ sentenceString: reqString })) {
let data = JSON.parse(strToGameState({ canvasURLstring: reqString }))
output = drawCanvas({
sentence: data.sentence,
cards: data.cards,
width: imageWidth,
height: imageHeight,
fontPath: fonttrick()
})
} else {
let data = JSON.parse(strToGameState({ canvasURLstring: fallbackString }))
output = drawCanvas({
sentence: data.sentence,
cards: data.cards,
width: imageWidth,
height: imageHeight,
fontPath: fonttrick()
})
}
const buffy = Buffer.from(output.split(',')[1], 'base64')
res.statusCode = 200
res.setHeader('Content-Type', 'image/png')
res.end(buffy)
}
The important part of what this does is import fonttrick which puts a copy of the font in tmp, then returns the path to that file; the path to the font is then passed to the canvas drawing function (along with some other stuff; what to draw, how big to draw it, etc.)
My drawing function itself is in components/drawCanvas.js; here's the important stuff at the beginning (TLDR version: if it gets called from the API page, it gets a path to the font; if so, it uses that, otherwise the regular system fonts are available):
import { registerFont, createCanvas } from 'canvas';
import path from 'path'
// width and height are optional
export const drawCanvas = ({ sentence, cards, width, height, fontPath }) => {
// default canvas size
let cw = 1200 // canvas width
let ch = 628 // canvas height
// if given different canvas size, update
if (width && !height) {
cw = width
ch = Math.floor(width / 1.91)
}
if (height && width) {
cw = width
ch = height
}
if (height && !width) {
ch = height
cw = Math.floor(height * 1.91)
}
// this path is only used for api calls in development mode
let theFontPath = path.join(process.cwd(), 'public/fonts/Roboto-Regular.ttf')
// when run in browser, registerfont isn't available,
// but we don't need it; when run from an API call,
// there is no css loaded, so we can't get fonts from #fontface
// and the canvas element has no fonts installed by default;
// in dev mode we can load them from local, but when run serverless
// it gets complicated: basically, we have a local module whose only
// job is to get loaded and piggyback the font file into the serverless
// function (thread); the module default function copies the font to
// /tmp then returns its absolute path; the function in the api
// then passes that path here so we can load the font from it
if (registerFont !== undefined) {
if (process.env.NODE_ENV === "production") {
theFontPath = fontPath
}
registerFont(theFontPath, { family: 'Roboto' })
}
const canvas = createCanvas(cw, ch)
const ctx = canvas.getContext('2d')
This API path gets used in the header for my game, in the meta tags to create the image on demand when a page gets shared on facebook or twitter or wherever:
<meta property="og:image" content={`https://grumbly.games/api/${returnString}`} />
Anyway. Ugly and hacky, but it works for me.
I think you were very close with registerFont. Here’s what I got to work using your repo:
In img.js:
import { registerFont, createCanvas, loadImage } from 'canvas'
// …
// Where 'Anton' is the same font-family name you want to use within
// your canvas code, ie. in writeText.js.
registerFont('./pages/fonts/Anton/Anton-Regular.ttf', { family: 'Anton' })
// Make sure this goes after registerFont()
const canvas = createCanvas()
//…
I added a new folder in pages/ called fonts/, and added the Anton folder downloaded from Google Fonts. Click “Download Family” to get the font file from here: https://fonts.google.com/specimen/Anton?query=Anton&selection.family=Anton&sidebar.open
The other file you downloaded (https://fonts.googleapis.com/css?family=Anton&display=swap) is actually the CSS file you’ll want to use the fonts client side in the browser, for your previewer.
At first, I would keep using the hosted version provided by Google Fonts. You can add that to the PreviewMeme.js component:
<link href="https://fonts.googleapis.com/css2?family=Anton" rel="stylesheet" />
<canvas id='meme' ref={canvas}></canvas>
(You might also want to use something like FontFaceObserver client side to make sure the font has loaded before rendering your canvas the first time.)
In writeText.js you’ll also then change the fontFamily to Anton:
const fontFamily = 'Anton'
That will make Anton available client side via the hosted Google Fonts, and it should be available to you as a file on the server for rendering with the server-side canvas package.
Hope that’s helpful!
The solution ended up being
import path from 'path'
registerFont(path.resolve('./fonts/Anton-Regular.ttf'), { family: 'Anton' })`
See path.resolve
I finally got this working, using officially-documented configurations rather than the hacky top answer!
First of all, I'm assuming your serverless function is at api/some_function.js, where the api/ folder is at the project root.
Create a folder in api/ to put static files into, such as api/_files/. For me, I put font and image files.
Put this in vercel.json:
{
"functions": {
"api/some_function.js": {
"includeFiles": "_files/**"
}
}
}
Now in api/some_function.js, you can use __dirname to reference the files:
const { join } = require('path')
registerFont(join(__dirname, '_files/fonts/Anton-Regular.ttf'), { family: 'Anton' })
This is based on this Vercel help page, except I had to figure out where the _files/ folder goes in your project directory structure because they forgot to mention that.
Is there a way to extract text from PDFs in nodejs without any OS dependencies (like pdf2text, or xpdf on windows)? I wasn't able to find any 'native' pdf packages in nodejs. They always are a wrapper/util on top of an existing OS command.
Thanks
Have you checked PDF2Json? It is built on top of PDF.js. Though it is not providing the text output as a single line but I believe you may just reconstruct the final text based on the generated Json output:
'Texts': an array of text blocks with position, actual text and styling informations:
'x' and 'y': relative coordinates for positioning
'clr': a color index in color dictionary, same 'clr' field as in 'Fill' object. If a color can be found in color dictionary, 'oc' field will be added to the field as 'original color" value.
'A': text alignment, including:
left
center
right
'R': an array of text run, each text run object has two main fields:
'T': actual text
'S': style index from style dictionary. More info about 'Style Dictionary' can be found at 'Dictionary Reference' section
After some work, I finally got a reliable function for reading text from PDF using https://github.com/mozilla/pdfjs-dist
To get this to work, first npm install on the command line:
npm i pdfjs-dist
Then create a file with this code (I named the file "pdfExport.js" in this example):
const pdfjsLib = require("pdfjs-dist");
async function GetTextFromPDF(path) {
let doc = await pdfjsLib.getDocument(path).promise;
let page1 = await doc.getPage(1);
let content = await page1.getTextContent();
let strings = content.items.map(function(item) {
return item.str;
});
return strings;
}
module.exports = { GetTextFromPDF }
Then it can simply be used in any other js file you have like so:
const pdfExport = require('./pdfExport');
pdfExport.GetTextFromPDF('./sample.pdf').then(data => console.log(data));
Thought I'd chime in here for anyone who came across this question in the future.
I had this problem and spent hours over literally all the PDF libraries on NPM. My requirements were that I needed to run it on AWS Lambda so could not depend on OS dependencies.
The code below is adapted from another stackoverflow answer (which I cannot currently find). The only difference being that we import the ES5 version which works with Node >= 12. If you just import pdfjs-dist there will be an error of "Readable Stream is not defined". Hope it helps!
import * as pdfjslib from 'pdfjs-dist/es5/build/pdf.js';
export default class Pdf {
public static async getPageText(pdf: any, pageNo: number) {
const page = await pdf.getPage(pageNo);
const tokenizedText = await page.getTextContent();
const pageText = tokenizedText.items.map((token: any) => token.str).join('');
return pageText;
}
public static async getPDFText(source: any): Promise<string> {
const pdf = await pdfjslib.getDocument(source).promise;
const maxPages = pdf.numPages;
const pageTextPromises = [];
for (let pageNo = 1; pageNo <= maxPages; pageNo += 1) {
pageTextPromises.push(Pdf.getPageText(pdf, pageNo));
}
const pageTexts = await Promise.all(pageTextPromises);
return pageTexts.join(' ');
}
}
Usage
const fileBuffer = fs.readFile('sample.pdf');
const pdfText = await Pdf.getPDFText(fileBuffer);
This solution worked for me using node 14.20.1 using "pdf-parse": "^1.1.1"
You can install it with:
yarn add pdf-parse
This is the main function which converts the PDF file to text.
const path = require('path');
const fs = require('fs');
const pdf = require('pdf-parse');
const assert = require('assert');
const extractText = async (pathStr) => {
assert (fs.existsSync(pathStr), `Path does not exist ${pathStr}`)
const pdfFile = path.resolve(pathStr)
const dataBuffer = fs.readFileSync(pdfFile);
const data = await pdf(dataBuffer)
return data.text
}
module.exports = {
extractText
}
Then you can use the function like this:
const { extractText } = require('../api/lighthouse/lib/pdfExtraction')
extractText('./data/CoreDeveloper-v5.1.4.pdf').then(t => console.log(t))
Instead of using the proposed PDF2Json you can also use PDF.js directly (https://github.com/mozilla/pdfjs-dist). This has the advantage that you are not depending on modesty who owns PDF2Json and that he updates the PDF.js base.
I have several arrays that contain data that I would like to export, each array to a txt file, in order to be analyzed using MATLAB.
Let's say my array is:
var xPosition = [];
// some algorithm that adds content to xPosition
// TODO: export array into a txt file let's call it x_n.txt
It would be great to store each element of an array per line.
I have found a guide for the solution to my question in this post. The following code is what I ended up using:
var fs = require('fs');
var xPosition = [];
// some algorithm that adds content to xPosition
var file = fs.createWriteStream('./positions/x_n.txt');
file.on('error', function(err) { /* error handling */ });
xPosition.forEach(function(v) { file.write(v + '\n'); });
file.end();
The solution you found works, but here's how I'd have done it:
var fs = require('fs');
var xPosition = [1,2,3]; // Generate this
var fileName = './positions/x_n.txt';
fs.writeFileSync(fileName, xPosition.join('\n'));
This uses node's synchronous file writing capability, which is ideal for your purposes. You don't have to open or close file handles, etc. I'd use streams only if I had gigabytes of data to write out.
I've written a little script that gathers up all my CSS files and compresses them:
var cssSources = [];
bundle.input.stylesheets.forEach(function(filename) {
if(isLocalPath(filename)) {
var fileContents = FileSystem.readFileSync(filename, {encoding: 'utf8'});
if(/\.css$/.test(filename)) {
cssSources.push(fileContents);
} else if(/\.less$/.test(filename)) {
var parser = new Less.Parser({
paths: [staticDir],
filename: filename
});
parser.parse(fileContents, function(e, tree) {
cssSources.push(tree.toCSS());
});
}
}
});
var cssCode = UglifyCSS.processString(cssSources.join(''));
Which was working well until I tried #importing a font in a CSS file that was not included first. This means that the #import wound up somewhere in the middle of my compressed CSS, which of course doesn't work.
I'm currently using UglifyCSS as you may have noticed; is there a better alternative that will move all the #imports to the top? Preferably available as an npm package.
Update: clean-css has the same problem.