Output an executable file with webpack - node.js

I'm currently writing a node CLI tool and using webpack to bundle all of my assets. The entry point for this application is the js file where I actually parse process.argv and run a command (For reference, I'm using tj/commander). This way, once the bundling is complete, I can enter ./<outputFile> and it will run my application. The entry file looks like this:
import cli from './cli';
cli.parse(process.argv);
// If nothing was supplied
if (!process.argv.slice(2).length) {
cli.outputHelp();
}
The bundling works fine but I can't get webpack to output the file as an executable. Once I run chmod +x <outputFile>, everything works perfectly. Is there a way that I can tell webpack what permissions to grant an output file?

I'm surprised no one said a thing about webpack's BannerPlugin. I do something similar than #oklas, but using BannerPlugin to add the specific node shebang:
{
plugins: [
new webpack.BannerPlugin({
banner: '#!/usr/bin/env node',
raw: true,
}),
],
}
Then I simply add the execution permissions just adding chmod to my package.json file:
"scripts": {
"build": "webpack && chmod +x dist/mycommand"
}
Anyway, if you'd like to just use webpack you can use the WebpackShellPlugin, as said by oklas (note that using this forces you to add a new dependency, that's why I avoid using this approach):
const WebpackShellPlugin = require('webpack-shell-plugin')
{
// [...]
plugins: [
new WebpackShellPlugin({
onBuildEnd:['chmod +x dist/mycommand'],
}),
],
}
If you want to avoid including WebpackShellPlugin as a dependency, you can try to define a custom plugin based on fs, as said by #taylorc93

One simple way is to use npm. Do you have an package.json in your project?
Add "build": "webpack && chmod +x outputFile" to the scripts section of your package.json and build your project by running npm run build.
Another way is to add one of these solutions to your webpack.config.js:
simple plugin from this answer which has pre and post build handlers
use on-build-webpack plugin, which executes js code at the end of the webpack build process
Whatever you choose, you'll need to add this piece of code:
var chmod = require('chmod');
chmod("outputFile", 500);

You'll need to append #!/usr/bin/env node on top of the file.
I ended up with this webpack plugin using shelljs
plugins: [
// ...plugins,
function () {
this.plugin('done', () => {
shell
.echo('#!/usr/bin/env node\n')
.cat(`${__dirname}/build/outputfile.js`)
.to(`${__dirname}/commandname`)
shell.chmod(755, `${__dirname}/commandname`)
})
},
]

Although #oklas's solution worked perfectly for me, I really wanted to try and keep all of this within webpack. I realized after a little more thought that this could all be done by a very simple plugin:
plugins: [
// ...plugins,
function() {
this.plugin('done', () => {
fs.chmodSync('bin/program-name.js', '755');
// When the webpack output doesn't have a .js extension, minification fails :(
fs.renameSync('bin/program-name.js', 'bin/program-name');
})
},
]
Use whichever way suits your needs!

This is how I did it with Webpack 5:
import { promises as fs } from 'fs';
plugins: [
new webpack.BannerPlugin({
banner: '#!/usr/bin/env node',
raw: true,
entryOnly: true
}),
function() {
this.hooks.done.tapPromise('Make executable', async () => {
await fs.chmod(`${__dirname}/dist/app.js`, '755');
});
}
]

Related

How to disable warnings when node is launched via a (global) shell script

I am building a CLI tool with node, and want to use the fs.promise API. However, when the app is launched, there's always an ExperimentalWarning, which is super annoying and messes up with the interaction prompts. How can I disable this warning/all warnings?
I'm testing this with the latest node v10 lts release on Windows 10.
To use the CLI tool globally, I have added this to my package.json file:
{
//...
"preferGlobal": true,
"bin": { "myapp" : "./index.js" }
//...
}
And have run npm link to link the ./index.js script. Then I am able to run the app globally simply with myapp.
After some research I noticed that there are generally 2 ways to disable the warnings:
set environmental variable NODE_NO_WARNINGS=1
call the script with node --no-warnings ./index.js
Although I was able to disable the warnings with the 2 methods above, there seems to be no way to do that while directly running myapp command.
The shebang I placed in the entrance script ./index.js is:
#!/usr/bin/env node
// my code...
I have also read other discussions on modifying the shebang, but haven't found a universal/cross-platform way to do this - to either pass argument to node itself, or set the env variable.
If I publish this npm package, it would be great if there's a way to make sure the warnings of this single package are disabled in advance, instead of having each individual user tweak their environment themselves. Is there any hidden npm package.json configs that allow this?
Any help would be greatly appreciated!
I am now using a launcher script to spawn a child_process to work around this limitation. Ugly, but it works with npm link, global installs and whatnot.
#!/usr/bin/env node
const { spawnSync } = require("child_process");
const { resolve } = require("path");
// Say our original entrance script is `app.js`
const cmd = "node --no-warnings " + resolve(__dirname, "app.js");
spawnSync(cmd, { stdio: "inherit", shell: true });
As it's kind of like a hack, I won't be using this method next time, and will instead be wrapping the original APIs in a promise manually, sticking to util.promisify, or using the blocking/sync version of the APIs.
I configured my test script like this:
"scripts": {
"test": "tsc && cross-env NODE_OPTIONS=--experimental-vm-modules NODE_NO_WARNINGS=1 jest"
},
Notice the NODE_NO_WARNINGS=1 part. It disables the warnings I was getting from setting NODE_OPTIONS=--experimental-vm-modules
Here's what I'm using to run node with a command line flag:
#!/bin/sh
_=0// "exec" "/usr/bin/env" "node" "--experimental-repl-await" "$0" "$#"
// Your normal Javascript here
The first line tells the shell to use /bin/sh to run the script. The second line is a bit magical. To the shell it's a variable assignment _=0// followed by "exec" ....
Node sees it as a variable assignment followed by a comment - so it's almost a nop apart from the side effect of assigning 0 to _.
The result is that when the shell reaches line 2 it will exec node (via env) with any command line options you need.
New answer: You can also catch emitted warnings in your script and choose which ones to prevent from being logged
const originalEmit = process.emit;
process.emit = function (name, data, ...args) {
if (
name === `warning` &&
typeof data === `object` &&
data.name === `ExperimentalWarning`
//if you want to only stop certain messages, test for the message here:
//&& data.message.includes(`Fetch API`)
) {
return false;
}
return originalEmit.apply(process, arguments);
};
Inspired by this patch to yarn

Sharing code between React Native + Node

I am using React Native and Node.js. I want to share code between the two. My folder structure is as so.
myreactnativeapp/
mynodeserver/
myshared/
In the react native and node apps I have included the
package.json
"dpendencies" : {
"myshared": "git+https://myrepository/ugoshared.git"
}
This can then be included in each project via require/import etc. This all works fine and for production I'm happy with it. (Though I'd love to know a better way?)
The issue I'm facing is in development it's really slow.
The steps for a change to populate are:
Make changes in Shared
Commit Changes to git
Update the npm module
In development, I really want the same codebase to be used rather than this long update process. I tried the following:
Adding a symlink in node_models/shared - doesn't work in react-native package mangaer
Using relative paths ../../../shared - doesn't work in react-native package mangaer
Any other ideas?
Update 1
I created a script.sh which I run to copy the files into a local directory before the package manager starts. It's not ideal but at least I only have to restart the packager instead of messing with git etc.
#myreactnativeapp/start.sh
SOURCE=../myshared
MODULE=myshared
rm -rf ./$MODULE
mkdir ./$MODULE
find $SOURCE -maxdepth 1 -name \*.js -exec cp -v {} "./$MODULE/" \;
# create the package.json
echo '{ "name": "'$MODULE'" }' > ./$MODULE/package.json
# start the packager
node node_modules/react-native/local-cli/cli.js start
Then in my package.json I update the script to
"scripts": {
"start": "./start.sh",
},
So, the process is now.
Make a change
Start/Resetart the packager
Automatic:
Script copies all .js files under myshared/ -> myreactnativeapp/myshared/
Script creates a package.json with the name of the module
Because I've added the package.json to the copied files with the name of the module, in my project I can just include the items the same as I would if the module was included via the package manager above. In theory when I switch to using the package in production I wont have to change anything.
Import MyModule from 'myshared/MyModule'
Update 2
My first idea got tiresome restarting the package manager all the time. Instead i created a small node script in the shared directory to watch for changes. Whenever there is a change it copies it to the react native working directory.
var watch = require('node-watch')
var fs = require('fs')
var path = require('path')
let targetPath = '../reactnativeapp/myshared/'
watch('.', { recursive: false, filter: /\.js$/ }, function(evt, name) {
console.log('File changed: '+name+path.basename(__filename))
// don't copy this file
if(path.basename(__filename) === name) {
return
}
console.log(`Copying file: ${name} --> ${targetPath+name}`);
fs.copyFile(name, targetPath+name, err => {
if(err) {
console.log('Error:', err)
return;
}
console.log('Success');
})
});
console.log(`Starting to watch: ${__dirname}. All files to be copied to: ${targetPath}`)

Add git information to create-react-app

In development, I want to be able to see the build information (git commit hash, author, last commit message, etc) from the web. I have tried:
use child_process to execute a git command line, and read the result (Does not work because browser environment)
generate a buildInfo.txt file during npm build and read from the file (Does not work because fs is also unavailable in browser environment)
use external libraries such as "git-rev"
The only thing left to do seems to be doing npm run eject and applying https://www.npmjs.com/package/git-revision-webpack-plugin , but I really don't want to eject out of create-react-app. Anyone got any ideas?
On a slight tangent (no need to eject and works in develop),
this may be of help to other folk looking to add their current git commit SHA into their index.html as a meta-tag by adding:
REACT_APP_GIT_SHA=`git rev-parse --short HEAD`
to the build script in the package.json and then adding (note it MUST start with REACT_APP... or it will be ignored):
<meta name="ui-version" content="%REACT_APP_GIT_SHA%">
into the index.html in the public folder.
Within react components, do it like this:
<Component>{process.env.REACT_APP_GIT_SHA}</Component>
I created another option inspired by Yifei Xu's response that utilizes es6 modules with create-react-app. This option creates a javascript file and imports it as a constant inside of the build files. While having it as a text file makes it easy to update, this option ensures it is a js file packaged into the javascript bundle. The name of this file is _git_commit.js
package.json scripts:
"git-info": "echo export default \"{\\\"logMessage\\\": \\\"$(git log -1 --oneline)\\\"}\" > src/_git_commit.js",
"precommit": "lint-staged",
"start": "yarn git-info; react-scripts start",
"build": "yarn git-info; react-scripts build",
A sample component that consumes this commit message:
import React from 'react';
/**
* This is the commit message of the last commit before building or running this project
* #see ./package.json git-info for how to generate this commit
*/
import GitCommit from './_git_commit';
const VersionComponent = () => (
<div>
<h1>Git Log: {GitCommit.logMessage}</h1>
</div>
);
export default VersionComponent;
Note that this will automatically put your commit message in the javascript bundle, so do ensure no secure information is ever entered into the commit message. I also add the created _git_commit.js to .gitignore so it's not checked in (and creates a crazy git commit loop).
It was impossible to be able to do this without ejecting until Create React App 2.0 (Release Notes) which brought with it automatic configuration of Babel Plugin Macros which run at compile time. To make the job simpler for everyone, I wrote one of those macros and published an NPM package that you can import to get git information into your React pages: https://www.npmjs.com/package/react-git-info
With it, you can do it like this:
import GitInfo from 'react-git-info/macro';
const gitInfo = GitInfo();
...
render() {
return (
<p>{gitInfo.commit.hash}</p>
);
}
The project README has some more information. You can also see a live demo of the package working here.
So, turns out there is no way to achieve this without ejecting, so the workaround I used is:
1) in package.json, define a script "git-info": "git log -1 --oneline > src/static/gitInfo.txt"
2) add npm run git-info for both start and build
3) In the config js file (or whenever you need the git info), i have
const data = require('static/gitInfo.txt')
fetch(data).then(result => {
return result.text()
})
My approach is slightly different from #uidevthing's answer. I don't want to pollute package.json file with environment variable settings.
You simply have to run another script that save those environment variables into .env file at the project root. That's it.
In the example below, I'll use typescript but it should be trivial to convert to javascript anyway.
package.json
If you use javascript it's node scripts/start.js
...
"start": "ts-node scripts/start.ts && react-scripts start",
scripts/start.ts
Create a new script file scripts/start.ts
const childProcess = require("child_process");
const fs = require("fs");
function writeToEnv(key: string = "", value: string = "") {
const empty = key === "" && value === "";
if (empty) {
fs.writeFile(".env", "", () => {});
} else {
fs.appendFile(".env", `${key}='${value.trim()}'\n`, (err) => {
if (err) console.log(err);
});
}
}
// reset .env file
writeToEnv();
childProcess.exec("git rev-parse --abbrev-ref HEAD", (err, stdout) => {
writeToEnv("REACT_APP_GIT_BRANCH", stdout);
});
childProcess.exec("git rev-parse --short HEAD", (err, stdout) => {
writeToEnv("REACT_APP_GIT_SHA", stdout);
});
// trick typescript to think it's a module
// https://stackoverflow.com/a/56577324/9449426
export {};
The code above will setup environment variables and save them to .env file at the root folder. They must start with REACT_APP_. React script then automatically reads .env at build time and then defines them in process.env.
App.tsx
...
console.log('REACT_APP_GIT_BRANCH', process.env.REACT_APP_GIT_BRANCH)
console.log('REACT_APP_GIT_SHA', process.env.REACT_APP_GIT_SHA)
Result
REACT_APP_GIT_BRANCH master
REACT_APP_GIT_SHA 042bbc6
More references:
https://create-react-app.dev/docs/adding-custom-environment-variables/#adding-development-environment-variables-in-env
If your package.json scripts are always executed in a unix environment you can achieve the same as in #NearHuscarl answer, but with fewer lines of code by initializing your .env dotenv file from a shell script. The generated .env is then picked up by the react-scripts in the subsequent step.
"scripts": {
"start": "sh ./env.sh && react-scripts start"
"build": "sh ./env.sh && react-scripts build",
}
where .env.sh is placed in your project root and contains code similar to the one below to override you .env file content on each build or start.
{
echo BROWSER=none
echo REACT_APP_FOO=bar
echo REACT_APP_VERSION=$(git rev-parse --short HEAD)
echo REACT_APP_APP_BUILD_DATE=$(date)
# ...
} > .env
Since the .env is overridden on each build, you may consider putting it on the .gitignore list to avoid too much noise in your commit diffs.
Again the disclaimer: This solution only works for environments where a bourne shell interpreter or similar exists, i.e. unix.
You can easily inject your git information like commit hash into your index.html using CRACO and craco-interpolate-html-plugin. Such way you won't have to use yarn eject and it also works for development server environment along with production builds.
After installing CRACO the following config in craco.config.js worked for me:
const interpolateHtml = require('craco-interpolate-html-plugin');
module.exports = {
plugins: [
{
plugin: interpolateHtml,
// Enter the variable to be interpolated in the html file
options: {
BUILD_VERSION: require('child_process')
.execSync('git rev-parse HEAD', { cwd: __dirname })
.toString().trim(),
},
},
],
};
and in your index.html:
<meta name="build-version" content="%BUILD_VERSION%" />
Here are the lines of code to add in package.json to make it all work:
"scripts": {
"start": "craco start",
"build": "craco build"
}

How can I bundle a precompiled binary with electron

I am trying to include a precompiled binary with an electron app. I began with electron quick start app and modified my renderer.js file to include this code that is triggered when a file is dropped on the body:
spawn = require('child_process').spawn,
ffmpeg = spawn('node_modules/.bin/ffmpeg', ['-i', clips[0], '-an', '-q:v', '1', '-vcodec', 'libx264', '-y', '-pix_fmt', 'yuv420p', '-vf', 'setsar=1,scale=trunc(iw/2)*2:trunc(ih/2)*2,crop=in_w:in_h-50:0:50', '/tmp/out21321.mp4']);
ffmpeg.stdout.on('data', data => {
console.log(`stdout: ${data}`);
});
ffmpeg.stderr.on('data', data => {
console.log(`stderr: ${data}`);
});
I have placed my precompiled ffmpeg binary in node_modules/.bin/. Everything works great in the dev panel, but when I use electron-packager to set up the app, it throws a spawn error ENOENT to the console when triggered. I did find a very similar question on SO, but the question doesn't seem to be definitively answered. The npm page on electron-packager does show that they can be bundled, but I cannot find any documentation on how to do so.
The problem is that electron-builder or electron-packager will bundle your dependency into the asar file. It seems that if the dependency has a binary into node_modules/.bin it is smart enough to not package it.
This is the documentation for asar packaging for electron-builder on that topic. It says
Node modules, that must be unpacked, will be detected automatically
I understand that it is related to existing binaries in node_modules/.bin.
If the module you are using is not automatically unpacked you can disable asar archiving completely or explicitly tell electron-builder to not pack certain files. You do so in your package.json file like this:
"build": {
"asarUnpack": [
"**/app/node_modules/some-module/*"
],
For your particular case
I ran into the same issue with ffmpeg and this is what I've done:
Use ffmpeg-static. This package bundles statically compiled ffmpeg binaries for Windows, Mac and Linux. It also provides a way to get the full path of the binary for the OS you are running: require('ffmpeg-static').path
This will work fine in development, but we still need to troubleshoot the distribution problem.
Tell electron-builder to not pack the ffmpeg-static module:
"build": {
"asarUnpack": [
"**/app/node_modules/ffmpeg-static/*"
],
Now we need to slightly change the code to get the right path to ffmpeg with this code: require('ffmpeg-static').path.replace('app.asar', 'app.asar.unpacked') (if we are in development the replace() won't replace anything which is fine).
If you are using webpack (or other javascript bundler)
I ran into the issue that require('ffmpeg-static').path was returning a relative path in the renderer process. But the issue seemed to be that webpack changes the way the module is required and that prevents ffmpeg-static to provide a full path. In the Dev Tools the require('ffmpeg-static').path was working fine when run manually, but when doing the same in the bundled code I was always getting a relative path. So this is what I did.
In the main process add this before opening the BrowserWindow: global.ffmpegpath = require('ffmpeg-static').path.replace('app.asar', 'app.asar.unpacked'). The code that runs in the main process is not bundled by webpack so I always get a full path with this code.
In the renderer process pick the value this way: require('electron').remote.getGlobal('ffmpegpath')
I know I'm a bit late but just wanted to mention ffbinaries npm package I created a while ago exactly for this purpose.
It'll allow you to download ffmpeg/ffplay/ffserver/ffprobe binaries to specified location either during application boot (so you don't need to bundle it with your application) or in a CI setup. It can autodetect platform, you can also specify it manually.
If anyone happens to need an answer to this question: I do have a solution to this, but I have no idea if this is considered best practice. I couldn't find any good documentation for including 3rd party precompiled binaries, so I just fiddled with it until it finally worked. Here's what I did (starting with the electron quick start, node.js v6):
From the app directory I ran the following commands to include the ffmpeg binary as a module:
mkdir node_modules/ffmpeg
cp /usr/local/bin/ffmpeg node_modules/ffmpeg/
ln -s ../ffmpeg/ffmpeg node_modules/.bin/ffmpeg
(replace /usr/local/bin/ffmpeg with your current binary path, download it from here) Placing the link allowed electron-packager to include the binary I saved to node_modules/ffmpeg/.
Then to get the bundled app path I installed the npm package app-root-dir by running the following command:
npm i -S app-root-dir
Since I could then get the app path, I just appended the subfolder for my binary and spawned from there. This is the code that I placed in renderer.js:.
var appRootDir = require('app-root-dir').get();
var ffmpegpath=appRootDir+'/node_modules/ffmpeg/ffmpeg';
console.log(ffmpegpath);
const
spawn = require( 'child_process' ).spawn,
ffmpeg = spawn( ffmpegpath, ['-i',clips_input[0]]); //add whatever switches you need here
ffmpeg.stdout.on( 'data', data => {
console.log( `stdout: ${data}` );
});
ffmpeg.stderr.on( 'data', data => {
console.log( `stderr: ${data}` );
});
This is how I would do it:
Taking cues from tsuriga's answer, here is my code:
Note: replace or add OS path accordingly.
Create a directory ./resources/mac/bin
Place you binaries inside this folder
Create file ./app/binaries.js and paste the following code:
'use strict';
import path from 'path';
import { remote } from 'electron';
import getPlatform from './get-platform';
const IS_PROD = process.env.NODE_ENV === 'production';
const root = process.cwd();
const { isPackaged, getAppPath } = remote.app;
const binariesPath =
IS_PROD && isPackaged
? path.join(path.dirname(getAppPath()), '..', './Resources', './bin')
: path.join(root, './resources', getPlatform(), './bin');
export const execPath = path.resolve(path.join(binariesPath, './exec-file-name'));
Create file ./app/get-platform.js and paste the following code:
'use strict';
import { platform } from 'os';
export default () => {
switch (platform()) {
case 'aix':
case 'freebsd':
case 'linux':
case 'openbsd':
case 'android':
return 'linux';
case 'darwin':
case 'sunos':
return 'mac';
case 'win32':
return 'win';
}
};
Add the following code inside the ./package.json file:
"build": {
....
"extraFiles": [
{
"from": "resources/mac/bin",
"to": "Resources/bin",
"filter": [
"**/*"
]
}
],
....
},
import binary file path as:
import { execPath } from './binaries';
#your program code:
var command = spawn(execPath, arg, {});
Why this is better?
Most of the answers require an additional package called app-root-dir
The original answer doesn't handle the (env=production) build or the pre-packed versions properly. He/she has only taken care of development and post-packaged versions.

Create and use Babel plugin without making it a npm module

In my project, I'm using Babel 6 with the require hook. I need to load a custom babel plugin that I wrote. But do I really need to publish my plugin using npm first, and then include the plugin name in my main project's .babelrc?
Is there any way to just directly load the plugin code? In other words, can I just load the following directly?
export default function({types: t }) {
return {
visitor: {
...
}
};
}
Where you list your plugins in your .babelrc, provide the path to your plugin instead of your standard published plugin name.
"plugins": ["transform-react-jsx", "./your/plugin/location"]
When exporting your plugin function, you'll probably need to use module.exports = instead of export default, since ES2015 modules haven't been fully implemented in Node yet.
This is my entire babel.config.js file.
module.exports = function (api) {
api.cache(true);
const presets = ["#babel/preset-env", "#babel/preset-react"];
const plugins = [
["#babel/plugin-proposal-pipeline-operator", { "proposal": "minimal" }],
"c:\\projects\\my-babel-plugin"
];
return {
presets,
plugins
};
}
First item in the plugins array is a plugin with options in form of an array. Second item is my own local plugin.
Inside of my-babel-plugin folder there needs to be a package.json with the "main" entry, usually "main": "lib/index.js" or "main": "src/index.js".

Resources