In node 14 I had the following script
scripts: {
"test": "node test.js",
}
Then I ran it with the following command npm run test -- --value '10' this gave the output
node test.js "--value" "10"
But when I switch to node 16 every argument that starts with -- gets removed
So whenI run npm run test -- --value '10' this gives the output
node test.js "10"
Where did "--value" go?
I'm not sure 100% about the root cause, but I can now explain the semi-root causes and ways to work around it.
It appears that while on my machine there is only an npm.cmd file in C:\Program Files\nodejs, on yours there is also an npm.ps1 file. It appears this file is created only in some install/upgrade paths, and your node upgrade seems to cause this file to be created.
That means that when you run npm in PowerShell, it is invoking an ExternalScript and not an Application. For this reason, semantics for calling cmdlets apply to the invocation and not semantics for calling native commands/applications.
These semantics include the use of -- to stop parsing further arguments. That means that in this case PowerShell will consume the -- token so it never arrives to npm! And in turn, npm now being called with npm run test --value 10 without -- will swallow --value because it's considered an argument to npm and not to the script you want to invoke.
Quote from the docs:
The end-of-parameters token (--)
The end-of-parameters token (--) indicates that all arguments following it are to be passed in their actual form as though double quotes were placed around them. For example, using -- you can output the string -InputObject without using quotes or having it interpreted as a parameter
It doesn't state here that it applies only to cmdlets and external scripts but my testing showed that that's the case.
There are four possible solutions I can think of:
Create an alias to manually point npm to npm.cmd instead of npm.ps1: Set-Alias -Name npm -Value npm.cmd
Rename/remove npm.ps1 to make sure PowerShell uses npm.cmd instead
Add another -- in front which is consumed by PowerShell to allow the following -- to be consumed by npm: npm run test -- -- --value 10 or npm -- run test -- --value 10
Quote the --: npm run test '--' --value 10
Related
I would like to know more about how NPM scripts work.
For example:
package.json
"scripts": {
"build": "set NODE_ENV=production&& webpack --config webpack.config.js",
}
When I execute npm run build:
What happens? I know a Node process will be initiated somewhere and webpack binary file will be called, right? But what are the steps before that? Does that depend whether I'm on Windows, Linux, PowerShell or Git bash? How does that process relate to the OS and the CMD/CLI that is being used?
The npm source code helpfully has the whole run-script functionality separated into its own module and repository so you can review the source code if the documentation does not answer your questions.
Speaking of your questions:
I know a Node process will be initiated somewhere and webpack binary file will be called, right?
The webpack executable will be run. And since webpack is a Node.js script, it will be run with node.
If, however, your "build" value consisted of shell/CLI commands, those commands would be run. Node.js is not necessarily invoked.
But what are the steps before that?
The "steps before that" include certain lifecycle scripts that might also be defined. In particular, if there is a "prebuild" script, it will run before the "build" script.
Does that depend whether I'm on Windows, Linux, PowerShell or Git bash? How does that process relate to the OS and the CMD/CLI that is being used?
npm (and node) make efforts to make Windows and Linux experiences comparable. Differences occur for sure, but without more details, I'm not sure I'd want to speculate about what specifics might be of interest to you beyond that.
Powershell and GitBash: Again, npm will make efforts to smooth out differences, but I'm sure they come up. One thing to be aware of is that your PATH (and other environment variables) might be set differently and that may affect behavior. (It may especially impact which version of node gets executed, if you have more than one version installed.)
I am not a Windows expert, but I have seen a lot of npm scripts that assume a UNIX-like environment. So, if given the choice and all else being equal (which it never is), a bash-like environment is probably going to be a bit smoother.
The answer I was looking for was something like this:
When you call npm run, npm will initiate a shell to run those instructions.
And which shell npm will use is defined in your npm settings.
You can see your npm config by running:
>>> npm config ls
In the example above, npm will run the script on git-bash.
You can change the npm shell by calling:
npm config set script-shell "C:\\Program Files\\git\\bin\\bash.exe"
I'm trying to pass environment variables to npm scripts as arguments
with no success.
export ENVIRONMENT=test.proxy.json
npm run test
I'm trying to do something like this in package.json
npm run test --proxy-config-file $ENVIRONMENT
When you do this:
export ENVIRONMENT=test.proxy.json
npm run test
or this:
ENVIRONMENT=test.proxy.json npm run test
then you will pass the "test.proxy.json" string as a value of the environment variable named ENVIRONMENT.
If you want to pass arguments to npm scripts then you may need to use:
npm run test -- --proxy-config-file $ENVIRONMENT
Keep in mind that if you pass the argument to the npm script, it doesn't necessarily mean that it will be passed to other scripts that this script is executing. With environment variables it's the other way around - by default they should be passed from one script to the other but there is still no guarantee as the caller may decide what environment variables to pass, if any.
But it's hard to tell from your question what is your real problem here - the phrase "with no success" is too general to know what is the problem here.
I want to force using yarn install instead of npm install. I want to raise an error in npm install. What should I do in package.json?
UPDATE: Alexander's answer is the better solution and uses the same technique I describe here. I am leaving my answer in tact for posterity. The original point of my answer was to show that you can execute a small node script which should work on all platforms.
In your preinstall script you can run a mini node script which should work on all platforms, whereas things like pgrep (and other common *nix commands and operators) won't work on Windows until Windows 10 has received widespread adoption.
I tested the below script on Node v4.7.0 (npm v2.15.11) and Node v7.2.1 (npm v3.10.10). I assume it works on everything in between. It works by checking the environment variables on the currently running process - the npm_execpath is the path to the currently running "npm" script. In the case of yarn, it should point to /path/to/yarn/on/your/machine/yarn.js.
"scripts": {
"preinstall": "node -e \"if(process.env.npm_execpath.indexOf('yarn') === -1) throw new Error('You must use Yarn to install, not NPM')\""
}
You can read more about npm scripts here: https://docs.npmjs.com/misc/scripts
As far as the npm_execpath environment variable, while not documented I doubt that it will ever change. It's been around for multiple major releases of npm and it doesn't really pass the "there's a better name for this" test.
Most of the answers here involve hacky scripts but there's a built in way to achieve this which I posted over on the Yarn github issue. Unlike soe of the other ways, this works for any and all NPM commands -- actually a bug in npm means it blocks npm install but not npm install <package>. Hopefully though the developers suspicions would already be raised from doing an npm install.
You add a fake engine version like so in package.json (you may want to tweak the yarn and node entries):
"engines": {
"npm": "please-use-yarn",
"yarn": ">= 1.17.3",
"node": ">= 12.5.0"
}
Then you add an .npmrc file to the project root with this:
engine-strict = true
Running NPM then raises an error:
npm ERR! code ENOTSUP
npm ERR! notsup Unsupported engine for root#: wanted: {"npm":"please-use-yarn","yarn":">= 1.17.3","node":">= 12.5.0"} (current: {"node":"12.9.1","npm":"6.10.2"})
npm ERR! notsup Not compatible with your version of node/npm: root#
Like the other answers, I'd recommend using a preinstall script and checking your environment. For a portable solution that won't have false-positives if another npm process happens to be running, using node -e 'JS_CODE' is probably the best option.
In that JS code, you can check the package manager's path using the following:
process.env.npm_execpath
Yarn's binary is yarn.js, compared to npm-cli.js used by NPM. We can use a regex like the following to check that this string ends with yarn.js.
/yarn\.js$/
By using this regex, we can be sure it won't accidentally match somewhere earlier in the file system. Most-likely yarn won't appear in the file path, but you can never be too sure.
Here's a minimal example:
{
"name": "test",
"version": "1.0.0",
"scripts": {
"preinstall": "node -e 'if(!/yarn\\.js$/.test(process.env.npm_execpath))throw new Error(\"Use yarn\")'"
}
}
Of course, the user will still be able to get around this check be editing the JSON or using the --ignore-scripts options:
npm install --ignore-scripts
After trying these options and not being very satisfied, I recommend only-allow.
Just add:
{
"scripts": {
"preinstall": "npx only-allow yarn"
}
}
I like that it provides a clear warning message, and instructions how to install yarn:
Credit to Adam Thomas' answer for providing the thread recommending this.
You can use the preinstall hook along with some shell script to achieve this.
sample package.json:
"scripts": {
"preinstall": "pgrep npm && exit 1"
}
I've just released a module that includes a CLI for this (useful for npm preinstall scripts): https://github.com/adjohnson916/use-yarn
Also, I've just released a helper for Danger to check for missing yarn.lock changes on CI:
https://github.com/adjohnson916/danger-yarn-lock
See also discussion here:
https://github.com/yarnpkg/yarn/issues/1732
https://github.com/alexanderwallin/use-yarn-instead/issues/1
If you want to simply test whether packages are being installed under yarn or npm, I tweaked Alexander O'Mara's answer slightly since it worked for me on OS X:
"scripts": {
"preinstall": "if node -e \"process.exitCode=!/yarn\\.js$/.test(process.env.npm_execpath)\" ; then echo yarn ; else echo npm ; fi",
"postinstall": ""
}
There are quite a few concepts happening in this short snippet:
The \\. portion is escaped so that \\ becomes \ and results in a properly escaped \. to detect a period in the regex.
process.exitCode= can be used to set the process's exit code and is safer than calling process.exit(N) due to the asynchronous nature of Node.js.
In Alexander's example, throw new Error(\"Use yarn\") caused node to exit with code 1 and print the stack trace to stderr. You can try running these on the console to see how that works: node -e 'throw new Error("Oops")' and node -e 'throw new Error("Oops")' 2> /dev/null (which directs the stderr stream to /dev/null). Then you can verify that the exit code was 1 with echo $? (which prints the last exit code).
The shell's if XXXX ; then YYYY ; else ZZZZ ; fi conditional logic checks the exit code of XXXX and goes to the then case for 0 (any other value goes to the else case). So if the regex detects yarn.js at the end of process.env.npm_execpath then it returns true. This must be negated so that the node process exits with code 0 and satisfies the if.
You could also console.log() the regex result and compare the output in the shell (this is just a little more verbose). Here are some examples of how to do that: https://unix.stackexchange.com/a/52801 and https://superuser.com/a/688902
You can append true ; or false ; to any shell statement to set the exit code manually. For example you can try true ; echo $? or false ; echo $?.
You can also leave off the else echo npm ; portion entirely if you don't need it.
With all of that out of the way, you can substitute the echo yarn and echo npm portions with other commands. For example, you could put multiple commands in a subshell like (echo yarn) or echo $(echo yarn).
In my case, I needed to work around an issue where one of the packages installed but had bugs under yarn so I had to run an npm install --ignore-scripts in the success case. Note that this should probably never be done in production, but can be a lifesaver if you just need to get something done or don't have control over which package manager will be used down the road.
I haven't tried this on Windows, so if someone can test the syntax there I will update my answer with what works. It would be best if the preinstall script is identical under both Windows and the Mac/Linux shell.
Found an alternate solution on Reddit. I added this to the end of my .zshenv file:
NPM_PATH=$(which npm)
npm () {
if [ -e yarn.lock ]
then
echo "Please use yarn with this project"
else
$NPM_PATH "$#"
fi
}
It now stops me from absentmindedly running commands like npm i on any yarn project on my Mac.
As some answers have already showed, you can use the only-allow package like so:
{
"scripts": {
"preinstall": "npx only-allow [npm|cnpm|pnpm|yarn]"
}
}
However, NodeJS v16.9.0 and v14.19.0 support a new experimental packageManager field in the package.json file.
Type: <string>
{
"packageManager": "<package manager name>#<version>"
}
The "packageManager" field defines which package manager is expected to be used when working on the current project. It can be set to any of the supported package managers, and will ensure that your teams use the exact same package manager versions without having to install anything else other than Node.js.
This field is currently experimental and needs to be opted-in; check the Corepack page for details about the procedure.
Is there a way to measure the performance of an npm scripts similar to the way time-grunt works?
I am moving some of my critical build tasks to use npm instead of Grunt as writing my own build script is more flexible than using some of the Grunt plugins like grunt-browserify for example.
I have tried using console.time() but it finishes before the script is done, I assume because the methods are asynchronous. I also tried running the npm script as a Grunt task like this:
grunt.registerTask('collectify', function () {
grunt.util.spawn({
cmd: 'npm',
args: ['run', 'collectify:app']
});
});
But the output is different than if I run npm run collectify:app from the command line, perhaps because of pwd issues.
Coloured bars would be nice but at the very least I'd like to see the time in numbers.
Have you tried prepending the time command before your npm run command ?
So if your command is:
npm run collectify:app
It becomes:
time npm run collectify:app
And it'll output 3 lines e.g.
real 0m11.580s
user 0m7.400s
sys 0m1.304s
Let me know if it helps!
Your best option is likely pre[foo] and post[foo] scripts.
So if I have a NPM script called "foobar" then I can create a script called "preboofar" and "postfoobar" and they will be executed automatically before and after "foobar" is executed.
So in "pre" you can touch a file with a timestamp and in "post" you can read that file and calculate the difference.
I have these two node.js command line commands:
$ NODE_ENV=dev_local npm start --fp data_for_testing/csvfile.csv --mptp map_ivr_itg
$ NODE_ENV=dev_local node start_script --fp data_for_testing/csvfile.csv --mptp map_ivr_itg
I am using nconf the command line and environment variable parser for node.js.
The problem is the command line arguments --fp and --mptp seem to disappear when using npm start.
Furthermore, as an aside, does any program interpret --fp as a force flag, as NPM is warning?
Your command line flags (e.g., --fp) are being sent to npm and not the script that results from running npm start. To send them to the resulting script as arguments, first send -- by itself as an argument. That indicates that the remaining arguments are for the resulting script and not for npm itself.
npm start -- --fp ...