I'm attempting to build a package.json so that when running a NodeJS app on Heroku it will run the scripts.postinstall step using an environment variable. For example:
...
"scripts": {
"postinstall": "command $ENV_VAR"}
},
...
I've looked at the docs and wasn't able to find something saying I can.
Is this even possible? Is this even desirable and "I'm Doing It Wrong"™?
Ignore the nay-sayers. You can do this in a cross-platform manner using cross-var:
"scripts": {
"postinstall": "cross-var command $ENV_VAR"
}
Updated answer due to new packages having been written
You can use the cross-var package to do this in a clean way:
...
"scripts": {
...
"postinstall": "cross-var command $ENV_VAR",
...
},
"dependencies": {
...
"cross-var": "^1.1.0",
...
}
...
Original answer
To answer the last questions, because they're the most important one: yes, no, and absolutely, because you've just broken cross-platform compatibility. There is no guarantee your environment syntax works for all shells on all operating systems, so don't do this.
We have a guaranteed cross-platform technology available to us already: Node. So, create a file called something like bootstrap.js, and then make npm run node bootstrap as your postinstall script. Since the code inside bootstrap.js will run like any other node script, it'll have access to process.env in a fully cross-platform compatible way, and everyone will be happy.
And many, many, many things that use common utils have node equivalents, so you can npm install them, locally rather than globally, and then call them in an npm script. For instance mkdir -p is not cross-platform, but installing the mkdirp module is, and then an npm script like "ensuredirs": "mkdirp dist/assets" works fine everywhere when run as npm run ensuredirs
And for convenience, the most common unix utilities have their own runner package, shx, which is fully cross-platform and makes the lives of devs even easier, with the "if you're writing code" equivalent being fs-extra.
Related
I would like to know more about how NPM scripts work.
For example:
package.json
"scripts": {
"build": "set NODE_ENV=production&& webpack --config webpack.config.js",
}
When I execute npm run build:
What happens? I know a Node process will be initiated somewhere and webpack binary file will be called, right? But what are the steps before that? Does that depend whether I'm on Windows, Linux, PowerShell or Git bash? How does that process relate to the OS and the CMD/CLI that is being used?
The npm source code helpfully has the whole run-script functionality separated into its own module and repository so you can review the source code if the documentation does not answer your questions.
Speaking of your questions:
I know a Node process will be initiated somewhere and webpack binary file will be called, right?
The webpack executable will be run. And since webpack is a Node.js script, it will be run with node.
If, however, your "build" value consisted of shell/CLI commands, those commands would be run. Node.js is not necessarily invoked.
But what are the steps before that?
The "steps before that" include certain lifecycle scripts that might also be defined. In particular, if there is a "prebuild" script, it will run before the "build" script.
Does that depend whether I'm on Windows, Linux, PowerShell or Git bash? How does that process relate to the OS and the CMD/CLI that is being used?
npm (and node) make efforts to make Windows and Linux experiences comparable. Differences occur for sure, but without more details, I'm not sure I'd want to speculate about what specifics might be of interest to you beyond that.
Powershell and GitBash: Again, npm will make efforts to smooth out differences, but I'm sure they come up. One thing to be aware of is that your PATH (and other environment variables) might be set differently and that may affect behavior. (It may especially impact which version of node gets executed, if you have more than one version installed.)
I am not a Windows expert, but I have seen a lot of npm scripts that assume a UNIX-like environment. So, if given the choice and all else being equal (which it never is), a bash-like environment is probably going to be a bit smoother.
The answer I was looking for was something like this:
When you call npm run, npm will initiate a shell to run those instructions.
And which shell npm will use is defined in your npm settings.
You can see your npm config by running:
>>> npm config ls
In the example above, npm will run the script on git-bash.
You can change the npm shell by calling:
npm config set script-shell "C:\\Program Files\\git\\bin\\bash.exe"
When expressing the dependencies of a Debian package, you can use syntax like exim | mail-transport-agent to indicate that your package needs either exim or mail-transport-agent, but it doesn't care which.
I want to express something similar in NPM. Is there a way to do it? Specifically suppose I want my application to express a dependency on either mikesthing-impl1 v1.7 better or mikesthing-impl2 v2.1 or better. I'd like to be able to say something like:
dependencies: {
"mikesthing": {
"mikesthing-impl1": "^1.7",
"mikesthing-impl2": "^2.1"
}
}
Is there a way?
No, there is no functionality within a package.json to specify that sort of logic. However, you can implement a postinstall script in the scripts that will be executed after all other dependencies have been installed and in which you can script out this kind (or any kind) of behavior.
e.g. (in package.json)
"scripts": {
"postinstall": "./bin/postinstall"
A good place to start is to run npm view {package} to get back a JSON object that details what versions are available in the registry.
This is really weird, I have the following scripts in my package.json:
"scripts": {
"lint": "./node_modules/tslint/bin/tslint src/js/**/*",
"lint:fix": "./node_modules/tslint/bin/tslint src/js/**/* --fix"
},
When I run npm run lint I don't get any errors and running echo $? immediately after shows 0.
However, if I run tslint src/js/**/* I do get linting errors.
How come?
There are a host of well-known issues in npm arising from the use of globbing. Many of them exclusively impact Windows, while others are "merely" shell-specific.
Try the following.
"scripts": {
"lint": "./node_modules/tslint/bin/tslint \"src/**/*.ts\"",
},
If this didn't immediately persuade you that computers have been a disaster for the human race, you can learn more about why these issues occur in the fantastic The Linux Programming Interface, which covers a surprising number non-Linux portability issues such as this one.
using npm run lint defaults to the tslint in the node_modules of your local project directory, while using tslint src/js/**/* defaults to the the one in your global, you should check if there is a version mismatch which could cause differences in rules
With npm-run-scripts you can omit the ./node_modules/.bin as npm will first look in there.
I want to force using yarn install instead of npm install. I want to raise an error in npm install. What should I do in package.json?
UPDATE: Alexander's answer is the better solution and uses the same technique I describe here. I am leaving my answer in tact for posterity. The original point of my answer was to show that you can execute a small node script which should work on all platforms.
In your preinstall script you can run a mini node script which should work on all platforms, whereas things like pgrep (and other common *nix commands and operators) won't work on Windows until Windows 10 has received widespread adoption.
I tested the below script on Node v4.7.0 (npm v2.15.11) and Node v7.2.1 (npm v3.10.10). I assume it works on everything in between. It works by checking the environment variables on the currently running process - the npm_execpath is the path to the currently running "npm" script. In the case of yarn, it should point to /path/to/yarn/on/your/machine/yarn.js.
"scripts": {
"preinstall": "node -e \"if(process.env.npm_execpath.indexOf('yarn') === -1) throw new Error('You must use Yarn to install, not NPM')\""
}
You can read more about npm scripts here: https://docs.npmjs.com/misc/scripts
As far as the npm_execpath environment variable, while not documented I doubt that it will ever change. It's been around for multiple major releases of npm and it doesn't really pass the "there's a better name for this" test.
Most of the answers here involve hacky scripts but there's a built in way to achieve this which I posted over on the Yarn github issue. Unlike soe of the other ways, this works for any and all NPM commands -- actually a bug in npm means it blocks npm install but not npm install <package>. Hopefully though the developers suspicions would already be raised from doing an npm install.
You add a fake engine version like so in package.json (you may want to tweak the yarn and node entries):
"engines": {
"npm": "please-use-yarn",
"yarn": ">= 1.17.3",
"node": ">= 12.5.0"
}
Then you add an .npmrc file to the project root with this:
engine-strict = true
Running NPM then raises an error:
npm ERR! code ENOTSUP
npm ERR! notsup Unsupported engine for root#: wanted: {"npm":"please-use-yarn","yarn":">= 1.17.3","node":">= 12.5.0"} (current: {"node":"12.9.1","npm":"6.10.2"})
npm ERR! notsup Not compatible with your version of node/npm: root#
Like the other answers, I'd recommend using a preinstall script and checking your environment. For a portable solution that won't have false-positives if another npm process happens to be running, using node -e 'JS_CODE' is probably the best option.
In that JS code, you can check the package manager's path using the following:
process.env.npm_execpath
Yarn's binary is yarn.js, compared to npm-cli.js used by NPM. We can use a regex like the following to check that this string ends with yarn.js.
/yarn\.js$/
By using this regex, we can be sure it won't accidentally match somewhere earlier in the file system. Most-likely yarn won't appear in the file path, but you can never be too sure.
Here's a minimal example:
{
"name": "test",
"version": "1.0.0",
"scripts": {
"preinstall": "node -e 'if(!/yarn\\.js$/.test(process.env.npm_execpath))throw new Error(\"Use yarn\")'"
}
}
Of course, the user will still be able to get around this check be editing the JSON or using the --ignore-scripts options:
npm install --ignore-scripts
After trying these options and not being very satisfied, I recommend only-allow.
Just add:
{
"scripts": {
"preinstall": "npx only-allow yarn"
}
}
I like that it provides a clear warning message, and instructions how to install yarn:
Credit to Adam Thomas' answer for providing the thread recommending this.
You can use the preinstall hook along with some shell script to achieve this.
sample package.json:
"scripts": {
"preinstall": "pgrep npm && exit 1"
}
I've just released a module that includes a CLI for this (useful for npm preinstall scripts): https://github.com/adjohnson916/use-yarn
Also, I've just released a helper for Danger to check for missing yarn.lock changes on CI:
https://github.com/adjohnson916/danger-yarn-lock
See also discussion here:
https://github.com/yarnpkg/yarn/issues/1732
https://github.com/alexanderwallin/use-yarn-instead/issues/1
If you want to simply test whether packages are being installed under yarn or npm, I tweaked Alexander O'Mara's answer slightly since it worked for me on OS X:
"scripts": {
"preinstall": "if node -e \"process.exitCode=!/yarn\\.js$/.test(process.env.npm_execpath)\" ; then echo yarn ; else echo npm ; fi",
"postinstall": ""
}
There are quite a few concepts happening in this short snippet:
The \\. portion is escaped so that \\ becomes \ and results in a properly escaped \. to detect a period in the regex.
process.exitCode= can be used to set the process's exit code and is safer than calling process.exit(N) due to the asynchronous nature of Node.js.
In Alexander's example, throw new Error(\"Use yarn\") caused node to exit with code 1 and print the stack trace to stderr. You can try running these on the console to see how that works: node -e 'throw new Error("Oops")' and node -e 'throw new Error("Oops")' 2> /dev/null (which directs the stderr stream to /dev/null). Then you can verify that the exit code was 1 with echo $? (which prints the last exit code).
The shell's if XXXX ; then YYYY ; else ZZZZ ; fi conditional logic checks the exit code of XXXX and goes to the then case for 0 (any other value goes to the else case). So if the regex detects yarn.js at the end of process.env.npm_execpath then it returns true. This must be negated so that the node process exits with code 0 and satisfies the if.
You could also console.log() the regex result and compare the output in the shell (this is just a little more verbose). Here are some examples of how to do that: https://unix.stackexchange.com/a/52801 and https://superuser.com/a/688902
You can append true ; or false ; to any shell statement to set the exit code manually. For example you can try true ; echo $? or false ; echo $?.
You can also leave off the else echo npm ; portion entirely if you don't need it.
With all of that out of the way, you can substitute the echo yarn and echo npm portions with other commands. For example, you could put multiple commands in a subshell like (echo yarn) or echo $(echo yarn).
In my case, I needed to work around an issue where one of the packages installed but had bugs under yarn so I had to run an npm install --ignore-scripts in the success case. Note that this should probably never be done in production, but can be a lifesaver if you just need to get something done or don't have control over which package manager will be used down the road.
I haven't tried this on Windows, so if someone can test the syntax there I will update my answer with what works. It would be best if the preinstall script is identical under both Windows and the Mac/Linux shell.
Found an alternate solution on Reddit. I added this to the end of my .zshenv file:
NPM_PATH=$(which npm)
npm () {
if [ -e yarn.lock ]
then
echo "Please use yarn with this project"
else
$NPM_PATH "$#"
fi
}
It now stops me from absentmindedly running commands like npm i on any yarn project on my Mac.
As some answers have already showed, you can use the only-allow package like so:
{
"scripts": {
"preinstall": "npx only-allow [npm|cnpm|pnpm|yarn]"
}
}
However, NodeJS v16.9.0 and v14.19.0 support a new experimental packageManager field in the package.json file.
Type: <string>
{
"packageManager": "<package manager name>#<version>"
}
The "packageManager" field defines which package manager is expected to be used when working on the current project. It can be set to any of the supported package managers, and will ensure that your teams use the exact same package manager versions without having to install anything else other than Node.js.
This field is currently experimental and needs to be opted-in; check the Corepack page for details about the procedure.
When deploying to Modulus.io (this probably applies to other PAAS as well), they will install the required packages from the packages.json file. As part of the install process, some npm scripts might be called as well. For example postinstall. However, these scripts might not be able to run (or should not run) on production. Be it because of scripts that are only available locally or do not make any sense on production.
How can I detect the environment and execute or not execute certain npm scripts? Can I access the process.env object and handle the scripts appropriatly or is there a better way?
Unfortunately, you can't in your package.json define script only for specific environment.
Let's say you have a postinstall script declared like this in package.json:
{
"scripts": {
"postinstall": "node postInstall.js"
},
}
The "easy" way would be to add your logic regarding the environment in this postInstall.js script:
if (process.env.NODE_ENV === 'production') {
// Do not run in production
process.exit(1);
}
If you're running in the production environment, you just instructs Node.js to terminate the process as quickly as possible with the specified exit code for example.
You could also if you're running multiple scripts in the postinstall hook, move all your scripts execution in a wrapper having the same mechanism to exit on certain environment, if not, executes all the other scripts.
Another approach if you're always running on Unix systems is to check directly the Node.js environment using a Bash condition:
{
"scripts": {
"postinstall": "[ \"$NODE_ENV\" != production ] && node postInstall.js"
},
}
In this case, if the node environment is not production, then you're running your postInstall.js script. You can adjust it to other conditions like only in development, etc.