I'm using WebMScore to render audio of music scores (it's a fork of MuseScore that runs in the browser or node).
I can successfully load my own, local .sf2 or .sf3 files, however
Trying to load an .sfz soundfont throws error 15424120. (And error.message is simply 'undefined'.)
Unlike .sf2 and .sf3, which contain the sounds and instructions in a single file, the .sfz format is just a text instruction file that refers to a separate folder of samples.
The reason I need the .sfz is that I need to be able to edit the .sfz file textually and programatically without an intervening Soundfont generator.
Is there a way to use .sfz's? Do I need to specify Zerberus (the Musescore .sfz player)? Do I need a different file structure? Please see below.
My environment is node js, with the following test case and file structure:
File Structure
Project Folder
app.js
testScore.mscz
mySFZ.sfz
samples
one.wav
two.wav
etc.wav
Test Case (Works with .sf3 , errors with .sfz)
const WebMscore = require('webmscore');
const fs = require('fs');
// free example scores available at https://musescore.com/openscore/scores
const name = 'testScore.mscz';
const exportedPrefix = 'exported';
const filedata = fs.readFileSync(`./${name}`);
WebMscore.ready.then(async () => {
const score = await WebMscore.load('mscz', filedata, [], false);
await score.setSoundFont(fs.readFileSync('./mySFZ.sfz'));
try { fs.writeFileSync(`./${exportedPrefix}.mp3`, await score.saveAudio('mp3')); }
catch (err) { console.log(err) }
score.destroy();
});
I have a Node.js server that uses node-canvas to render text on an image on the server-side. Here is the repo: https://github.com/shawninder/meme-generator (just git clone, npm i and npm run dev to run locally).
As you'll notice in the code, I am loading the Anton font, which I got from here with the documented registerFont function provided by node-canvas
registerFont('./fonts/Anton-Regular.ttf', { family: 'Anton' })
Everything works like a charm locally, but when I deploy to Vercel (formerly known as zeit), that line throws an ENOENT error:
no such file or directory, lstat '/var/task/fonts'
Is there a path I can use here that will successfully load the font from within a Vercel function?
Can I find a single path that will work both locally and once deployed?
I had the same problem recently and I finally found a solution. I'm no guru, so someone will probably be able to suggest a better way, but here's what worked for me.
Because of how Vercel runs their serverless functions, a function doesn't really know anything about the rest of the project, or the public folder. This makes sense (because security), but it does make it tricky when you need the actual path to a file. You can import the font file no problem, the build process will give it a new name and put it on the disk (in /var/task ), but you can't access it. path.resolve(_font_name_) can see it, but you can't access it.
I ended up writing a very bad, separate api page that used path.join and fs.readdirSync to see what files are actually visible from the api page. One thing that is visible is a node_modules folder that contains the files for modules used on that api page.
fs.readdirSync(path.join(process.cwd(), 'node_modules/')
So what I did was write a local module, install it in my project, then import it into my api page. In the local module's package.json, I have a line "files": ["*"] so it will bundle all the module files into its node_modules folder (instead of just the .js files). In my module I have my font file and a function that copies it to /tmp (/tmp is readable and writable) then returns the path to the file, /tmp/Roboto-Regular.ttf.
On my api page, I include this module, then run it, and I pass the resultant path to registerfont.
It works. I'd share my code, but it's pretty sloppy right now, and I'd like to clean it up and try a couple things first (like I'm not sure if I need to copy it to /tmp, but I haven't tested it without that step). When I get it straightened out I'll edit this answer.
-- EDIT
Since I haven't been able to improve on my original solution, let me give some more details about what I did.
In my package.json I added a line to include a local module:
"dependencies": {
"canvas": "^2.6.1",
"fonttrick": "file:fonttrick",
In my project root, I have a folder "fonttrick". Inside the folder is another package.json:
{
"name": "fonttrick",
"version": "1.0.6",
"description": "a trick to get canvas registerfont to work in a Vercel serverless function",
"license": "MIT",
"homepage": "https://grumbly.games",
"main": "index.js",
"files": [
"*"
],
"keywords": [
"registerfont",
"canvas",
"vercel",
"zeit",
"nextjs"
]
}
This is the only local module I've ever had to write; the keywords don't do anything, but at first I'd thought about putting it on NPM, so they're there.
The fonttrick folder also contains my font file (in this case "Roboto-Regular.ttf"), and a the main file, index.js:
module.exports = function fonttrick() {
const fs = require('fs')
const path = require('path')
const RobotoR = require.resolve('./Roboto-Regular.ttf')
const { COPYFILE_EXCL } = fs.constants;
const { COPYFILE_FICLONE } = fs.constants;
//const pathToRoboto = path.join(process.cwd(), 'node_modules/fonttrick/Roboto-Regular.ttf')
try {
if (fs.existsSync('/tmp/Roboto-Regular.ttf')) {
console.log("Roboto lives in tmp!!!!")
} else {
fs.copyFileSync(RobotoR, '/tmp/Roboto-Regular.ttf', COPYFILE_FICLONE | COPYFILE_EXCL)
}
} catch (err) {
console.error(err)
}
return '/tmp/Roboto-Regular.ttf'
};
I ran npm install in this folder, and then fonttrick was available as a module in my main project (don't forget to run npm install there, too).
Since I only need to use this for API calls, the module is only used in one file, /pages/api/[img].js
import { drawCanvas } from "../../components/drawCanvas"
import { stringIsValid, strToGameState } from '../../components/gameStatePack'
import fonttrick from 'fonttrick'
export default (req, res) => { // { query: { img } }
// some constants
const fallbackString = "1xThe~2ysent~3zlink~4yis~5wnot~6xa~7xvalid~8zsentence~9f~~"
// const fbs64 = Buffer.from(fallbackString,'utf8').toString('base64')
// some variables
let imageWidth = 1200 // standard for fb ogimage
let imageHeight = 628 // standard for fb ogimage
// we need to remove the initial "/api/" before we can use the req string
const reqString64 = req.url.split('/')[2]
// and also it's base64 encoded, so convert to utf8
const reqString = Buffer.from(reqString64, 'base64').toString('utf8')
//const pathToRoboto = path.join(process.cwd(), 'node_modules/fonttrick/Roboto-Regular.ttf')
let output = null
if (stringIsValid({ sentenceString: reqString })) {
let data = JSON.parse(strToGameState({ canvasURLstring: reqString }))
output = drawCanvas({
sentence: data.sentence,
cards: data.cards,
width: imageWidth,
height: imageHeight,
fontPath: fonttrick()
})
} else {
let data = JSON.parse(strToGameState({ canvasURLstring: fallbackString }))
output = drawCanvas({
sentence: data.sentence,
cards: data.cards,
width: imageWidth,
height: imageHeight,
fontPath: fonttrick()
})
}
const buffy = Buffer.from(output.split(',')[1], 'base64')
res.statusCode = 200
res.setHeader('Content-Type', 'image/png')
res.end(buffy)
}
The important part of what this does is import fonttrick which puts a copy of the font in tmp, then returns the path to that file; the path to the font is then passed to the canvas drawing function (along with some other stuff; what to draw, how big to draw it, etc.)
My drawing function itself is in components/drawCanvas.js; here's the important stuff at the beginning (TLDR version: if it gets called from the API page, it gets a path to the font; if so, it uses that, otherwise the regular system fonts are available):
import { registerFont, createCanvas } from 'canvas';
import path from 'path'
// width and height are optional
export const drawCanvas = ({ sentence, cards, width, height, fontPath }) => {
// default canvas size
let cw = 1200 // canvas width
let ch = 628 // canvas height
// if given different canvas size, update
if (width && !height) {
cw = width
ch = Math.floor(width / 1.91)
}
if (height && width) {
cw = width
ch = height
}
if (height && !width) {
ch = height
cw = Math.floor(height * 1.91)
}
// this path is only used for api calls in development mode
let theFontPath = path.join(process.cwd(), 'public/fonts/Roboto-Regular.ttf')
// when run in browser, registerfont isn't available,
// but we don't need it; when run from an API call,
// there is no css loaded, so we can't get fonts from #fontface
// and the canvas element has no fonts installed by default;
// in dev mode we can load them from local, but when run serverless
// it gets complicated: basically, we have a local module whose only
// job is to get loaded and piggyback the font file into the serverless
// function (thread); the module default function copies the font to
// /tmp then returns its absolute path; the function in the api
// then passes that path here so we can load the font from it
if (registerFont !== undefined) {
if (process.env.NODE_ENV === "production") {
theFontPath = fontPath
}
registerFont(theFontPath, { family: 'Roboto' })
}
const canvas = createCanvas(cw, ch)
const ctx = canvas.getContext('2d')
This API path gets used in the header for my game, in the meta tags to create the image on demand when a page gets shared on facebook or twitter or wherever:
<meta property="og:image" content={`https://grumbly.games/api/${returnString}`} />
Anyway. Ugly and hacky, but it works for me.
I think you were very close with registerFont. Here’s what I got to work using your repo:
In img.js:
import { registerFont, createCanvas, loadImage } from 'canvas'
// …
// Where 'Anton' is the same font-family name you want to use within
// your canvas code, ie. in writeText.js.
registerFont('./pages/fonts/Anton/Anton-Regular.ttf', { family: 'Anton' })
// Make sure this goes after registerFont()
const canvas = createCanvas()
//…
I added a new folder in pages/ called fonts/, and added the Anton folder downloaded from Google Fonts. Click “Download Family” to get the font file from here: https://fonts.google.com/specimen/Anton?query=Anton&selection.family=Anton&sidebar.open
The other file you downloaded (https://fonts.googleapis.com/css?family=Anton&display=swap) is actually the CSS file you’ll want to use the fonts client side in the browser, for your previewer.
At first, I would keep using the hosted version provided by Google Fonts. You can add that to the PreviewMeme.js component:
<link href="https://fonts.googleapis.com/css2?family=Anton" rel="stylesheet" />
<canvas id='meme' ref={canvas}></canvas>
(You might also want to use something like FontFaceObserver client side to make sure the font has loaded before rendering your canvas the first time.)
In writeText.js you’ll also then change the fontFamily to Anton:
const fontFamily = 'Anton'
That will make Anton available client side via the hosted Google Fonts, and it should be available to you as a file on the server for rendering with the server-side canvas package.
Hope that’s helpful!
The solution ended up being
import path from 'path'
registerFont(path.resolve('./fonts/Anton-Regular.ttf'), { family: 'Anton' })`
See path.resolve
I finally got this working, using officially-documented configurations rather than the hacky top answer!
First of all, I'm assuming your serverless function is at api/some_function.js, where the api/ folder is at the project root.
Create a folder in api/ to put static files into, such as api/_files/. For me, I put font and image files.
Put this in vercel.json:
{
"functions": {
"api/some_function.js": {
"includeFiles": "_files/**"
}
}
}
Now in api/some_function.js, you can use __dirname to reference the files:
const { join } = require('path')
registerFont(join(__dirname, '_files/fonts/Anton-Regular.ttf'), { family: 'Anton' })
This is based on this Vercel help page, except I had to figure out where the _files/ folder goes in your project directory structure because they forgot to mention that.
I'm trying to solve this Windows filename issue
Basically our CI job fails, with the 'filename too long' error for Windows.
warning: Could not stat path 'node_modules/data-validator/tests/data/data-examples/ds000247/sub-emptyroom/ses-18910512/meg/sub-emptyroom_ses-18910512_task-noise_run-01_meg.ds/sub-emptyroom_ses-18910512_task-noise_run-01_meg.acq': Filename too long
I've read the docs for Node's path module, which seems like a possible solution. I also read about a Windows prefix (\\?\) to bypass the MAX_PATH...but have no idea how to implement these in a clean way.
This part of the codebase with the tests that are failing. The hardcoded path (testDatasetPath) is likely part of the problem.
function getDirectories(srcpath) {
return fs.readdirSync(srcpath).filter(function(file) {
return (
file !== '.git' && fs.statSync(path.join(srcpath, file)).isDirectory()
)
})
}
var missing_session_files = //array of strings here
const dataDirectory = 'data-validator/tests/data/'
function createDatasetFileList(path) {
const testDatasetPath = `${dataDirectory}${path}`
if (!isNode) {
return createFileList(testDatasetPath)
} else {
return testDatasetPath
}
}
createFileList function
function createFileList(dir) {
const str = dir.substr(dir.lastIndexOf('/') + 1) + '$'
const rootpath = dir.replace(new RegExp(str), '')
const paths = getFilepaths(dir, [], rootpath)
return paths.map(path => {
return createFile(path, path.replace(rootpath, ''))
})
}
tl;dr A GitLab CI Job fails on Windows because the node module filenames become too long. How can I make this nodejs code OS agnostic?
This is a known error in the Windows Environment, however, there is a fix..
If you're using an NTFS based filesystem, you should be able to enable long paths in
Local Computer Policy > Computer Configuration > Administrative Templates > System > Filesystem > NTFS
This is also specified in the document you just linked, and theoretically, should work. However, the path shouldn't really be longer than 32 bits,
This will come with some performance hits, but should get the job done.
Also, you should preferably switch to a better package or search for alternatives. If this is your own package, maybe restructure it?
Finally, if none of these work, move your project folder directly to your data drive, (C:\) this will cut down on the other nested and parent folders.
This is inherently bad, and you may run into issues during deployment, if you choose to do it.
I use browserify in order to be able to use require. To use fs functions with browserify i need to transform it with brfs but as far as I understood this results in only being able to input static strings as parameters inside my fs function. I want to be able to use variables for this.
I want to search for xml files in a specific directory and read them. Either by searching via text field or showing all of their data at once. In order to do this I need fs and browserify in order to require it.
const FS = require('fs')
function lookForRoom() {
let files = getFileNames()
findSearchedRoom(files)
}
function getFileNames() {
return FS.readdirSync('../data/')
}
function findSearchedRoom(files) {
const SEARCH_FIELD_ID = 'room'
let searchText = document.getElementById(SEARCH_FIELD_ID).value
files.forEach((file) => {
const SEARCHTEXT_FOUND = file.includes(searchText.toLowerCase())
if (SEARCHTEXT_FOUND) loadXML(file)
})
}
function loadXML(file) {
const XML2JS = require('xml2js')
let parser = new XML2JS.Parser()
let data = FS.readFile('../data/' + file)
console.dir(data);
}
module.exports = { lookForRoom: lookForRoom }
I want to be able to read contents out of a directory containing xml files.
Current status is that I can only do so when I provide a constant string to the fs function
The brfs README contains this gotcha:
Since brfs evaluates your source code statically, you can't use dynamic expressions that need to be evaluated at run time.
So, basically, you can't use brfs in the way you were hoping.
I want to be able to read contents out of a directory containing xml files
If by "a directory" you mean "any random directory, the name of which is determined by some form input", then that's not going to work. Browsers don't have direct access to directory contents, either locally or on a server.
You're not saying where that directory exists. If it's local (on the machine the browser is running on): I don't think there are standardized API's to do that, at all.
If it's on the server, then you need to implement an HTTP server that will accept a directory-/filename from some clientside code, and retrieve the file contents that way.
I want to use factor-bundle to find common dependencies for my browserify entry points and save them out into a single common bundle:
https://www.npmjs.org/package/factor-bundle
The factor-bundle documentation makes it seem very easy to do on the command line, but I want to do it programmatically and I'm struggling to get my head around it.
My current script is this (I'm using reactify to transform react's jsx files too):
var browserify = require('browserify');
var factor = require('factor-bundle')
var glob = require('glob');
glob('static/js/'/**/*.{js,jsx}', function (err, files) {
var bundle = browserify({
debug: true
});
files.forEach(function(f) {
bundle.add('./' + f);
});
bundle.transform(require('reactify'));
// factor-bundle code goes here?
var dest = fs.createWriteStream('./static/js/build/common.js');
var stream = bundle.bundle().pipe(dest);
});
I'm trying to figure out how to use factor-bundle as a plugin, and specify the desired output file for each of the input files (ie each entry in files)
This answer is pretty late, so it's likely you've either already found a solution or a work around for this question. I'm answering this as it's quite similar to my question.
I was able to get this working by using factor-bundle as a browserify plugin. I haven't tested your specific code, but the pattern should be the same:
var fs = require('fs'),
browserify = require('browserify'),
factor = require('factor-bundle');
var bundle = browserify({
entries: ['x.js', 'y.js', 'z.js'],
debug: true
});
// Group common dependencies
// -o outputs the entry files without the common dependencies
bundle.plugin('factor-bundle', {
o: ['./static/js/build/x.js',
'./static/js/build/y.js',
'./static/js/build/z.js']
});
// Create Write Stream
var dest = fs.createWriteStream('./static/js/build/common.js');
// Bundle
var stream = bundle.bundle().pipe(dest);
The factor-bundle plugin takes output options o which need to have the same indexes as the entry files.
Unfortunately, I haven't figured out how to do anything else with these files after this point because I can't seem to access factor-bundle's stream event. So for minification etc, it might need to be done also via a browserify plugin.
I have created grunt-reactify to allow you to have a bundle file for a JSX file, in order to make it easier to work with modular React components.
All what you have to do is to specify a parent destination folder and the source files:
grunt.initConfig({
reactify: {
'tmp': 'test/**/*.jsx'
},
})