Replace env variables in the .npmrc with dotenv before `npm install` - node.js

I have to work with some packages in the private registry. So, in my package.json in the dependencies section I have a lines like this one:
...
"dependencies": {
"#myco/my-awesome-package": "^0.4.5",
...
}
...
There is authentication required for the private registry, so I have to create the .npmrc file in my project:
registry=https://registry.npmjs.org/
#myco:registry=https://myco-registry-path/
//myco-registry-path/:username=${MYCO_REGISTRY_USER}
//myco-registry-path/:_password=${MYCO_REGISTRY_PASSWORD_BASE64}
Yes, I know about _authToken, but in my case it is easier to use user and password.
Anyway, here you can see two env variables: ${MYCO_REGISTRY_USER} and ${MYCO_REGISTRY_PASSWORD_BASE64} which I have to replace before npm install.
I know the very simple solution for this problem: put them to the "global" env variables for example to my .bash_profile (or any terminal profile of your choice).
But I do not want to keep variables like this in the "global" scope because the are important only for the current project. What I want to do is to use dotenv. I want to create a .env file in the root of my project:
MYCO_REGISTRY_USER=myco-registry-username-value
MYCO_REGISTRY_PASSWORD_BASE64=myco-registry-password-value-base64
I want that this values replace env variables in my .npmrc on the install action. But when I try npm install I get an error: Error: Failed to replace env in config: ${MYCO_REGISTRY_USER}. I can understand why it happens. Possibly because npm reads .npmrc values first and try to replace env variables and fails, because in this moment it know nothing about dotenv.
My question is how to deal with it?
Short summary:
I do not want to keep env variables in the terminal profile, instead I want to put it in the .env file inside my project.
I have to replace env variables in the .npmrc file with dotenv before npm install

I know this answer might come too late, but in case anyone else is looking for answers, here's a solution:
You need to prepend your scripts with dotenv-cli as so:
dotenv npm install
or in my case where the file was not .env:
dotenv -e .env.local npm install
The problem is that you cannot save this anywhere so that someone can use it with "npm install" somehow. Definitely npm preinstall is run after reading .npmrc so it fails too.
You will need to either document it well or just include a small shell script, but if you're supporting different OSs then it can get funny really fast...
Happily so, CD platforms like Netlify allow you to set environment variables manually.
But I guess this must not be the nicest of starts if someone clones your repo and the first they've got is a failing npm install 🤷‍♂️
Also, check this one out: locking-the-vault-on-font-awesome-npm-tokens

Related

Can I run "npm ci" with a custom npmrc file?

I want to have different paths for different situations for npmrc. For example I want to have a "npmrc.dev" for devs and one for CI "npmrc.ci".
Is this possible? I would be happy with any hack, param, environment variable or some npm module.

How To Setup Private NPM Module With Firebase Cloud Functions .npmrc?

I have created a private typings npm module that I am using for my firebase functions and app projects. When I went to deploy firebase functions, I get a big error for every function that basically says ERR! remote: Invalid username or password.
For what I have read, it looks like I need to create a .npmrc file and put it in the /functions directory. (https://cloud.google.com/functions/docs/writing/specifying-dependencies-nodejs#using_private_modules)
I cannot however find proper instructions on how to do this anywhere. From what I found, I have done the following:
ran npm login
ran npm token create --read-only
This then gave me a token that looks like this: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.
I then create a file called .npmrc in my functions directory, and placed //registry.npmjs.org/:_authToken=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX in it.
Additionally, I saw that the error message was trying to use ssh to install my private repo package, I have not setup ssh and am using https instead, because of this I changed my package file to git+https://github.com/accounts-name/repo#commit-num so that it uses HTTPS.
However, I still get the same error message. What am I missing? The above is what I have strung together from lots of google searching.
It seems that you have made too many different changes while trying to make it work, so let's just go through the whole process step by step.
Check the package.json of your npm module and publish it:
Remove "private" property or set it to false because private modules in the npm are meant to be never published. That is not obvious but that is true.
Next step is to apply restricted access to the package. In order to do that, add such property in the package.json file:
"publishConfig": {
"access": "restricted"
},
Make sure that npm account you use for publishing supports private packages.
Now open the terminal in the root directory of your package, type npm login then sign in to npm. Check if you put the proper version in the package.json.
Run npm publish. The package should be published in few seconds. No worries, thanks to publishConfig property nobody can access it.
Now it is time to allow package installation in your project
Go to the directory of the project and open package.json file
Check that you have the name and version of your package in the dependencies list
Open browser, navigate to https://npmjs.com, login to your account, navigate to settings page of your account and open the tokens tab
Create new token and copy it
Now again go to the directory of your project, on same level where package.json file is situated (that is important!) and create .npmrc file there.
Put such string in the .npmrc file:
//registry.npmjs.org/:_authToken=TOKEN_HERE
You are done!
Deployment with remote CI/CD services
The easiest approach is not add .npmrc into .gitignore. In such case the file will be always in repository, so npm install will run smoothly on any machine where project was cloned
If you don't want to have token string in the repository, you can move it to the environment variable of your CI/CD service and then link .npmrc file to that variable. For example, you can put generated token into the NPM_TOKEN env variable (Just token from npmjs, not the whole string from .npmrc!)
And then change the .npmrc file in the next way:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}.
So, with those steps you should be able to install your restricted packages without any issues. Good luck!
If you are trying to deploy you functions with firebase deploy from a CI and your .npmrc file looks like this.
#acmecorp:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=${NPM_REGISTRY_TOKEN}
You will run into the problem even if you have the env var set.
Build failed: Error: Failed to replace env in config: ${NPM_REGISTRY_TOKEN}
Firebase for some reason needs access to that private repo. But the env var is not sent over to firebase.
Solution I've implemented was to replace ${NPM_REGISTRY_TOKEN} in the .npmrc file on every run of the CI pipeline.
sed -i.bak "s/\${NPM_REGISTRY_TOKEN}/${NPM_REGISTRY_TOKEN}/g" .npmrc
This breaks if you use Yarn. Took me a while to find a thread pointing to npm install in the firebase cli predeploy step. If there's no package-lock.json and you only use yarn, this will break. Remove yarn.lock and install using npm to resolve the issue.

npm config file not reading environment variables

I'm not able to get my project .npmrc file to recognize any of the environment variables set at three different scopes (project, user, global).
The only way I'm able to install a private module is by hardcoding the api key into the .npmrc file, which is obviously unacceptable since .npmrc is watched by git.
I've tried creating environment variables as the npm-config docs suggest, i.e.:
in a project .env file, where both a .npmrc file and a .env file are siblings of package.json, i.e.: fontawesome_pro_token=ABC123
in a user config file, i.e.: $ npm set fontawesome_pro_token ABC123
in a global config file, i.e.: $ npm set fontawesome_pro_token ABC123 --global
When I reference the env variable in the project .npmrc file, i.e.:
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=${fontawesome_pro_token}
I get this error:
Error: Failed to replace env in config: ${fontawesome_pro_token}
When I remove the curly braces around the variable name (as this stack overflow answer suggests), I get the following error:
npm ERR! 401 Unauthorized
Any advice on how to config npm to read env variables?
Incidentally -- if deploying private modules to Netlify, Netlify expects the .npmrc file to use the curly braces for env var syntax, see this gist. I can confirm that using the curly brace syntax in a git watched npmrc file, along with setting a build env var in the netlify project admin dashboard, indeed works.
Use your env vars without curly braces like:
$fontawesome_pro_token
Change your casing to UPPERCASE when refer to your .env file or handle environment variables
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=${FONTAWESOME_PRO_TOKEN}

.npmrc config file not reading environment variable to download private Node module

When I hardcode my auth token into my .npmrc file, the private module installs as expected.
When I replace the hardcoded token with an environment variable, the private module fails to install.
I've tried multiple ways of writing the variable name, as well as the syntax of the variable in the .npmrc file, due to the following resources:
npm-config docs on env vars
a counter answer to the npm config docs above about the .npmrc variable syntax
npm blog post re: private modules that many folks link to as the source of truth on the matter
Example .npmrc files:
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=$NPM_TOKEN
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=${NPM_TOKEN}
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=${npm_token}
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=$npm_token
And, example .env files, both with and without strings around the value:
npm_config_npm_token=ABC123
npm_token=ABC123
NPM_CONFIG_NPM_TOKEN=ABC123
NPM_TOKEN=ABC123
Nothing has worked.
Any clues?
EDIT
I'm using npm v6.9.0.
I figured out the error in my assumption, and have come to a solution.
tl;dr - create shell-based persistant environment variables, and use the curly braces variable syntax in a .npmrc file.
The error was assuming npm reads project .env files. Apparently npm does not read .env files located in the a project root.
I wanted npm to read from a .env file so that I could keep all relevant data for a project contained within the project.
Instead, I created a shell-based environment variable that is always available. The following code blocks show how to add environment variables to your (oh-my-zsh) shell, even if you git watch and publish your dot files.
# ~/.oh-my-zsh/custom/env.zsh
# be sure this file is gitignored!
export TOKEN="ABC123"
# ~/.zshrc
source $ZSH/custom/env.zsh
# example .npmrc
#fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=${TOKEN}
Thanks to this npm forum answer that nudged me towards this shell-based solution.
ps - because the native OS X man pages file export and source under the builtin category, and don't actually provide any details on how each command works, here are links the man page for each:
export
source (or dot operator)
Assuming you're using macOS, try using the source command in the terminal to refresh the env file.
source .env
I had the same problem and looked and the same posts you did but none worked until I tried that. I got the hint from here. I also used curly brackets around my variable in the .npmrc file unlike suggested in the posts you linked. Good luck!

NodeJS: add options to npmrc

I am currently running my script as
node --no-deprecation main.js
I was wondering if there is a way to add this option to ~/.npmrc or otherwise (environment variable), such that I do not need to add the option to the command line. I would like to just run my script as node main.js and to be shown no deprecation warnings.
To make this question more general, is there a general way in NodeJS of setting flag values in a config file rather than adding it to the command (say I am interested in running with --trace-warnings as well.
I am looking for something cleaner than say alias node="node --no-deprecation".
I already tried adding this line inside ~/.npmrc
no-deprecationss=true
Reference:
https://nodejs.org/dist/latest-v7.x/docs/api/cli.html#cli_no_deprecation
If you're looking for a global configuration, setting the --no-deprecation flag in the NODE_OPTIONS environment variable in your .bashrc or other shell config file is what you're looking for, as long as you're using node v8 or above.
https://nodejs.org/docs/latest-v10.x/api/cli.html#cli_node_options_options
You can do it via package.json. e.g
"scripts": {
"main": "node --no-deprecation main.js"
},
and run it via
npm run main
or
yarn main
I know this is an old question, but for anyone new who comes across it, one reproducible way to do this without having to add it to every npm script and without affecting other projects (which is what would happen if you added it to .bashrc or .profile), is to add a .npmrc file in your project with node-options set in it.
It accepts the same flags as NODE_OPTIONS except applied to every npm script in your project without affecting others and keeping your scripts DRY.
e.g.
# Sets the flags in NODE_OPTIONS when running `npm run myScript`
node-options='--no-deprecation'

Resources