My workflow is npm scripts, running commands in my node_modules along with simple shell commands.
Unfortunately this makes it difficult for windows users due to my using backticks in some commands (see Example below). I have a pull request volunteering to convert to shellJS/shx for my build so my repo will build cross-platform but we can't figure out a solution for backticks in npm scripts.
Question:
What shell does npm use? On windows it appears to not support backticks.
Is there a workaround? Piping doesn't help, alas, rm, mkdir etc don't use stdin.
Example backtick use in package.json:
"mkdirs": [
"dist/AS",
"libs",
"models/scripts"
],
"scripts": {
"mkdirs": "mkdir -p `bin/pkgkey.js mkdirs`",
....
.. where the bin/pkgkey.js mkdirs script simply returns the mkdirs array. This may seem odd but it's great for organizing npm-style workflow.
The pkgkey script (simplified):
#!/usr/bin/env node
const fs = require('fs')
const json = JSON.parse(fs.readFileSync('package.json'))
const key = process.argv[2]
let val = json[key]
if (Array.isArray(val)) val = val.join(' ')
process.stdout.write(val)
Check cross-env, don't know if it will work for your use case but I use it to make npm scripts platform independent
npm i --save-dev cross-env
"mkdirs": "cross-env mkdir -p ..."
https://www.npmjs.com/package/cross-env
Related
Our team has built a small CLI used for maintenance. The package.json specifies a path for with the bin property, and everything works great; "bin": { "eddy": "./dist/src/cli/entry.js"}
Autocompletion is achived by using yargs#17.0.1. However we recently converted the project to use es6 modules, because of a migration to Sveltekit, i.e. the package.json now contains type: module. Because of this, the CLI now only works if we run with:
what works
node --experimental-specifier-resolution=node ./dist/src/cli/entry.js help
However, if we run this without the flag, we get an error "module not found":
Error [ERR_MODULE_NOT_FOUND]: Cannot find module...
So the question is
Can we somehow "always" add the experimental-specifier-resolution=node to the CLI - so we can continue to use the shorthand eddy, and utilize auto completion?
There are two probable solutions here.
Solution 1
Your entry.js file should start with a shebang like #!/usr/bin/env node. You cannot specify the flag directly here, however, if you could provide the absolute path to node directly in the shebang, you can specify the flag.
Assuming you have node installed in /usr/bin/node, you can write the shebang in entry.js like:
#!/usr/bin/node --experimental-specifier-resolution=node
(Use which node to find the absolute path)
However, this is not a very portable solution. You cannot always assume everyone has node installed in the same path. Also some may use nvm to manage versions and can have multiple version in different path. This is the reason why we use /usr/bin/env to find the required node installation in the first place. This leads to the second solution.
Solution 2
You can create a shell script that would intern call the cli entry point with the required flags. This shell script can be specified in the package.json bin section.
The shell script (entry.sh) should look like:
#!/usr/bin/env bash
/usr/bin/env node --experimental-specifier-resolution=node ./entry.js "$#"
Then, in your package.json, replace bin with:
"bin": { "eddy": "./dist/src/cli/entry.sh"}
So when you run eddy, it will run the entry.js using node with the required flag. The "$#" in the command will be replaced by any arguments that you pass to eddy.
So eddy help will translate to /usr/bin/env node --experimental-specifier-resolution=node ./entry.js help
Just add a script to your package.json:Assuming index.js is your entry point and package.json is in the same directory
{
"scripts": {
"start": "node --experimental-specifier-resolution=node index.js"
}
}
Then you can just run on your console:
npm start
I want to force using yarn install instead of npm install. I want to raise an error in npm install. What should I do in package.json?
UPDATE: Alexander's answer is the better solution and uses the same technique I describe here. I am leaving my answer in tact for posterity. The original point of my answer was to show that you can execute a small node script which should work on all platforms.
In your preinstall script you can run a mini node script which should work on all platforms, whereas things like pgrep (and other common *nix commands and operators) won't work on Windows until Windows 10 has received widespread adoption.
I tested the below script on Node v4.7.0 (npm v2.15.11) and Node v7.2.1 (npm v3.10.10). I assume it works on everything in between. It works by checking the environment variables on the currently running process - the npm_execpath is the path to the currently running "npm" script. In the case of yarn, it should point to /path/to/yarn/on/your/machine/yarn.js.
"scripts": {
"preinstall": "node -e \"if(process.env.npm_execpath.indexOf('yarn') === -1) throw new Error('You must use Yarn to install, not NPM')\""
}
You can read more about npm scripts here: https://docs.npmjs.com/misc/scripts
As far as the npm_execpath environment variable, while not documented I doubt that it will ever change. It's been around for multiple major releases of npm and it doesn't really pass the "there's a better name for this" test.
Most of the answers here involve hacky scripts but there's a built in way to achieve this which I posted over on the Yarn github issue. Unlike soe of the other ways, this works for any and all NPM commands -- actually a bug in npm means it blocks npm install but not npm install <package>. Hopefully though the developers suspicions would already be raised from doing an npm install.
You add a fake engine version like so in package.json (you may want to tweak the yarn and node entries):
"engines": {
"npm": "please-use-yarn",
"yarn": ">= 1.17.3",
"node": ">= 12.5.0"
}
Then you add an .npmrc file to the project root with this:
engine-strict = true
Running NPM then raises an error:
npm ERR! code ENOTSUP
npm ERR! notsup Unsupported engine for root#: wanted: {"npm":"please-use-yarn","yarn":">= 1.17.3","node":">= 12.5.0"} (current: {"node":"12.9.1","npm":"6.10.2"})
npm ERR! notsup Not compatible with your version of node/npm: root#
Like the other answers, I'd recommend using a preinstall script and checking your environment. For a portable solution that won't have false-positives if another npm process happens to be running, using node -e 'JS_CODE' is probably the best option.
In that JS code, you can check the package manager's path using the following:
process.env.npm_execpath
Yarn's binary is yarn.js, compared to npm-cli.js used by NPM. We can use a regex like the following to check that this string ends with yarn.js.
/yarn\.js$/
By using this regex, we can be sure it won't accidentally match somewhere earlier in the file system. Most-likely yarn won't appear in the file path, but you can never be too sure.
Here's a minimal example:
{
"name": "test",
"version": "1.0.0",
"scripts": {
"preinstall": "node -e 'if(!/yarn\\.js$/.test(process.env.npm_execpath))throw new Error(\"Use yarn\")'"
}
}
Of course, the user will still be able to get around this check be editing the JSON or using the --ignore-scripts options:
npm install --ignore-scripts
After trying these options and not being very satisfied, I recommend only-allow.
Just add:
{
"scripts": {
"preinstall": "npx only-allow yarn"
}
}
I like that it provides a clear warning message, and instructions how to install yarn:
Credit to Adam Thomas' answer for providing the thread recommending this.
You can use the preinstall hook along with some shell script to achieve this.
sample package.json:
"scripts": {
"preinstall": "pgrep npm && exit 1"
}
I've just released a module that includes a CLI for this (useful for npm preinstall scripts): https://github.com/adjohnson916/use-yarn
Also, I've just released a helper for Danger to check for missing yarn.lock changes on CI:
https://github.com/adjohnson916/danger-yarn-lock
See also discussion here:
https://github.com/yarnpkg/yarn/issues/1732
https://github.com/alexanderwallin/use-yarn-instead/issues/1
If you want to simply test whether packages are being installed under yarn or npm, I tweaked Alexander O'Mara's answer slightly since it worked for me on OS X:
"scripts": {
"preinstall": "if node -e \"process.exitCode=!/yarn\\.js$/.test(process.env.npm_execpath)\" ; then echo yarn ; else echo npm ; fi",
"postinstall": ""
}
There are quite a few concepts happening in this short snippet:
The \\. portion is escaped so that \\ becomes \ and results in a properly escaped \. to detect a period in the regex.
process.exitCode= can be used to set the process's exit code and is safer than calling process.exit(N) due to the asynchronous nature of Node.js.
In Alexander's example, throw new Error(\"Use yarn\") caused node to exit with code 1 and print the stack trace to stderr. You can try running these on the console to see how that works: node -e 'throw new Error("Oops")' and node -e 'throw new Error("Oops")' 2> /dev/null (which directs the stderr stream to /dev/null). Then you can verify that the exit code was 1 with echo $? (which prints the last exit code).
The shell's if XXXX ; then YYYY ; else ZZZZ ; fi conditional logic checks the exit code of XXXX and goes to the then case for 0 (any other value goes to the else case). So if the regex detects yarn.js at the end of process.env.npm_execpath then it returns true. This must be negated so that the node process exits with code 0 and satisfies the if.
You could also console.log() the regex result and compare the output in the shell (this is just a little more verbose). Here are some examples of how to do that: https://unix.stackexchange.com/a/52801 and https://superuser.com/a/688902
You can append true ; or false ; to any shell statement to set the exit code manually. For example you can try true ; echo $? or false ; echo $?.
You can also leave off the else echo npm ; portion entirely if you don't need it.
With all of that out of the way, you can substitute the echo yarn and echo npm portions with other commands. For example, you could put multiple commands in a subshell like (echo yarn) or echo $(echo yarn).
In my case, I needed to work around an issue where one of the packages installed but had bugs under yarn so I had to run an npm install --ignore-scripts in the success case. Note that this should probably never be done in production, but can be a lifesaver if you just need to get something done or don't have control over which package manager will be used down the road.
I haven't tried this on Windows, so if someone can test the syntax there I will update my answer with what works. It would be best if the preinstall script is identical under both Windows and the Mac/Linux shell.
Found an alternate solution on Reddit. I added this to the end of my .zshenv file:
NPM_PATH=$(which npm)
npm () {
if [ -e yarn.lock ]
then
echo "Please use yarn with this project"
else
$NPM_PATH "$#"
fi
}
It now stops me from absentmindedly running commands like npm i on any yarn project on my Mac.
As some answers have already showed, you can use the only-allow package like so:
{
"scripts": {
"preinstall": "npx only-allow [npm|cnpm|pnpm|yarn]"
}
}
However, NodeJS v16.9.0 and v14.19.0 support a new experimental packageManager field in the package.json file.
Type: <string>
{
"packageManager": "<package manager name>#<version>"
}
The "packageManager" field defines which package manager is expected to be used when working on the current project. It can be set to any of the supported package managers, and will ensure that your teams use the exact same package manager versions without having to install anything else other than Node.js.
This field is currently experimental and needs to be opted-in; check the Corepack page for details about the procedure.
Is there a way to measure the performance of an npm scripts similar to the way time-grunt works?
I am moving some of my critical build tasks to use npm instead of Grunt as writing my own build script is more flexible than using some of the Grunt plugins like grunt-browserify for example.
I have tried using console.time() but it finishes before the script is done, I assume because the methods are asynchronous. I also tried running the npm script as a Grunt task like this:
grunt.registerTask('collectify', function () {
grunt.util.spawn({
cmd: 'npm',
args: ['run', 'collectify:app']
});
});
But the output is different than if I run npm run collectify:app from the command line, perhaps because of pwd issues.
Coloured bars would be nice but at the very least I'd like to see the time in numbers.
Have you tried prepending the time command before your npm run command ?
So if your command is:
npm run collectify:app
It becomes:
time npm run collectify:app
And it'll output 3 lines e.g.
real 0m11.580s
user 0m7.400s
sys 0m1.304s
Let me know if it helps!
Your best option is likely pre[foo] and post[foo] scripts.
So if I have a NPM script called "foobar" then I can create a script called "preboofar" and "postfoobar" and they will be executed automatically before and after "foobar" is executed.
So in "pre" you can touch a file with a timestamp and in "post" you can read that file and calculate the difference.
I'm attempting to build a package.json so that when running a NodeJS app on Heroku it will run the scripts.postinstall step using an environment variable. For example:
...
"scripts": {
"postinstall": "command $ENV_VAR"}
},
...
I've looked at the docs and wasn't able to find something saying I can.
Is this even possible? Is this even desirable and "I'm Doing It Wrong"™?
Ignore the nay-sayers. You can do this in a cross-platform manner using cross-var:
"scripts": {
"postinstall": "cross-var command $ENV_VAR"
}
Updated answer due to new packages having been written
You can use the cross-var package to do this in a clean way:
...
"scripts": {
...
"postinstall": "cross-var command $ENV_VAR",
...
},
"dependencies": {
...
"cross-var": "^1.1.0",
...
}
...
Original answer
To answer the last questions, because they're the most important one: yes, no, and absolutely, because you've just broken cross-platform compatibility. There is no guarantee your environment syntax works for all shells on all operating systems, so don't do this.
We have a guaranteed cross-platform technology available to us already: Node. So, create a file called something like bootstrap.js, and then make npm run node bootstrap as your postinstall script. Since the code inside bootstrap.js will run like any other node script, it'll have access to process.env in a fully cross-platform compatible way, and everyone will be happy.
And many, many, many things that use common utils have node equivalents, so you can npm install them, locally rather than globally, and then call them in an npm script. For instance mkdir -p is not cross-platform, but installing the mkdirp module is, and then an npm script like "ensuredirs": "mkdirp dist/assets" works fine everywhere when run as npm run ensuredirs
And for convenience, the most common unix utilities have their own runner package, shx, which is fully cross-platform and makes the lives of devs even easier, with the "if you're writing code" equivalent being fs-extra.
Inside my composer.json, there's a postinstall hook setup like the following:
"scripts" : {
"dist" : "node dist; node_modules/.bin/doccoh src/package.js",
"postinstall" : "node_modules/.bin/grunt setup || true; node_modules/.bin/bower install",
"start" : "node server.js"
}
Whenever I run it (on Win from Git/Gnu Bash CLI), I end with
command not found. either the command was written wrong or couldn't be found
Rough translation from German CLI error.
I tried splitting it into multiple ;/semicolon separated parts and first cd into that directory, but it simply ends up with the same error message. Replacing the whole postinstall command set with a simple ls does work. So I guess the problem might be the semicolon separation or a wrong usage of commands. But overall I got no idea what's wrong.
Note: I got grunt-cli version 0.1.9 and grunt version 0.4.1 installed globally.
I'm a bit late to answer, but if you're on Windows, multiple commands on a single line are executed with the use of &&
postinstall: "some command && some other -c"
I ran into this looking for something and thought this may help other people. I have found it easier to move to postinstall.js files as things get a little complicated. This makes it easier to deal with moving forward.