.env variables not available in jest testing environment - node.js

When I run my code normally I can access my .env file with const newVar = process.env.MY_DOTENV_VARIABLE, but when I run jest everything becomes undefined. Is this normal for jest? If so, what is the best practice for storing variables?
Is it simply to create a set up file, eg:
// jest.config.ts
setupFiles: [
"<rootDir>/.jest/setEnvVars.ts",
],
# .env
MY_DOTENV_VARIABLE=exampleString

I just needed to install dotenv. I think I got confused where process.env was working previously without the dotenv packaging. This was either due to me setting the env variables with scripts in my package.json file, eg "scripts":"NODE_ENV=test ..." and/or some packages were making changes. (I'm using various aws packages, and I've read that they can change environment variables)

Related

Environment variables in NodeJS depenency

Environment variables in NodeJS depenency
I have a NodeJS application which has a dependency of mine
my-base-module: git+https://myuser:mytoken#gitlab.com/organization/my-base-module.git#v1.0.0
I am also using the dependency dotenv, which in development I use the file .env and in other environments I pass the variables through docker environment variables
The problem I have, and I do not know why and how to solve it is that in my-base-module I do not share the same environment variables (and I need it to). It is like the variables loose the reference
require('dotenv').config();
const env = process.env.NODE_ENV;
for example in that bit of code the NODE_ENV variable inside my-base-module is undefined. However in the container is defined and with the right value
Update 1
I am requiring dotenv in both my-application and my-base-module. If I enter to the container and I do $ echo $NODE_ENV. I get production.
If my application process.env.NODE_ENV also holds production. But in the dependency process.env.NODE_ENV is undefined
I will try to do some github repositories to reproduce it
I was wrong, this works I made this small POC with these public repositories
https://github.com/agusgambina/try-base-module..https://github.com/agusgambina/try-application-dotenv
https://github.com/agusgambina/try-application-dotenv

Node js express project how to add environment variables

I am a java script full stack developer.
In the vue js I create two files in the root.
env.development
env.production
In my main.js file, I can access environment variables like this
process.env.VUE_APP_MY_VARIABLE_X
How production and development differs in the vue js is,
if I use npm run serve project loaded the development environment variables. If I use npm run build it takes production environment variables. That's all in vue.
But in my expressjs project,
"start": "nodemon server.js --exec babel-node -e js",
This command will always responsible for the run project. I couldn't find two commands like in vue. I go though the tutorials every body says use the package called dotenv.
I couldn't figure out that how could this package identify the environments this is production and this is development.
End of the day I want to set my db password to 123456 in my local machine and root#123456 in the server. How could I achieve this?
Vue.js handles modes automatically, depending on the command (e.g. serve sets NODE_ENV=development, while build sets NODE_ENV=production).
Depending on the mode, it also automatically loads files with variables from disk.
Environment variables in Node.JS
However, Node.JS does not do that by default, and you have to set up your application to handle environment variables.
In Node.JS you can pass variables via the shell command, for example:
MY_VARIABLE=x node myapp.js (by prepending the command with variables), and that will give you process.env.MY_VARIABLE inside your myapp.js application.
Following this behavior, there's a consensus on using NODE_ENV variable for setting up the environment mode.
For production use, you'd start your application using NODE_ENV=production node myapp.js, while for development NODE_ENV=development node myapp.js.
Of course, if your machine already has these environment variables set up (for instance in .bash_profile) you do not need to prepend your command with them.
As a practice, in development machines, you'd have these variables already set up on your machine, while production machines (e.g. docker containers, etc) start clean and you pass the environment variables when starting your application.
For example, Heroku (and other deployment services) allow you to set up environment variables which are set at machine start.
There's also the method of storing variables in files (such as the .env, or other files), but those you'd have to read from disk when your application starts.
And this is where dotenv (and config package) come in play.
What dotenv does, is it reads .env file stored in your application execution path (root of the app), and sets any variables defined there into process.env for Node.JS to use.
This is extremely useful for development machines, but on production it's recommended to not have files that store sensitive information in variables, but rather use system environment variables.
There is also the option, for production machines, to construct or load into the machine an .env file at machine setup time, into the app directory.
Note:
never commit .env or config files that store passwords and sensitive information, to your repository
never store development and production variables in the same file
The dotenv approach
The best way to go about it, would be to use dotenv, and have different .env files, one on your development machine, and a different one on your production machine. That way, when your application starts, it reads the variables that are stored in the adjacent .env. This is the most secure and reliable way, in absence of means to pass environment variables to your production machine from a machine management application/interface.
The easiest way to set up your app with dotenv would be to set your start command to something like:
"start": "nodemon --exec \"node -r dotenv/config\" server.js"
and then in the .env file stored on your local/development machine you'd have something similar to:
DATABASE_PASSWORD=1235
DATABASE_USERNAME=joe-dev
...
and in the .env file on your production machine (server):
DATABASE_PASSWORD=root#1235
DATABASE_USERNAME=root
...
according to my understanding
you don't have created and setup a config file and required it in your app.js or any other file. let see for example create config file
const config = {
"development":{
"host":"localhost",
"dbport":"27017",
"port":"",
"username":"",
"password":"",
"userdb":"",
"passworddb":"123456",
"authSource":"",
"database":"devDB"
},
"staging":{
"host":"",
"dbport":"",
"port":"",
"username":"",
"password":"",
"passworddb":"#123456",
"database":"stageDB",
},
"production":{
"host":"",
"dbport":"",
"port":"",
"userdb":"",
"passworddb":"#123456",
"username":"",
"password":"",
"database":"prodDB",
}
};
module.exports = config;
then you have you setup a variable in your .env file like NodeENV = 'prod'
then import it into your files like
var config = require('./config')["process.env.NodeENV"];
and one more last thing dotenv is only used to loads environment variables from a .env file into process.env.

Using environment variables in Node

I have been trying to get a streamline way of having different environment variables for local and production web apps, but I haven't come across the "ideal" solution yet.
There's the option of having a config.js file like so:
//config.js
{
"secretKey": "SDFDASFFSFD",
"facebook": {
"clientID": "EFGFDGBGDGFS",
"clientSecret": "EGDFNHFG"
}
}
And accessing via ES6 imports
Or using .env files like so:
SOME_KEY=someValue
HELLO=world
FACEBOOK_SECRET=435SDFSF5DZVD7S
And accessing the variables via process.env in the code using dotenv.
Obviously no matter what route you go down, the file will need to be omitted from version control which is fine. Each of these ways are great, but they only seem to work well for local development.
So how do you then have a separate file for a production environment? The dotenv docs say they strongly recommend against a .local.env and .prod.env situation.
Also, how is best to push to a remote server? I have my own server with Gulp tasks which run on a Git post-receive hook. How is best to pass up the production environment variables to here?
Thanks
You could have own config file for each environment:
- environments
- index.js
- deveplopment.json
- staging.json
- production.json
To use appropriate config file, run the app with required NODE_ENV:
NODE_ENV=production node index
In environments/index.js determinate the current NODE_ENV and use config:
process.env.NODE_ENV = process.env.NODE_ENV || 'development';
module.exports = require('./' + process.env.NODE_ENV);
If config file doesn't include secret info (apiKeys, etc), it can be pushed to repo. Otherwise add it to .gitignore and use environment variables on the server.
Note:
For advanced configuration use such packages as nconf.
It allows to create hierarchical node.js configuration with files, environment variables, command-line arguments, and atomic object merging.
Have you thought about having a keys.js file with a .gitignore on it?
I've used this in the past to use module.exports{} to get my variables usable but not going to version control for the security side of things!

create react app cannot read environment variable after build

I have a react app , created with create-react-app then I build the app with command: npm run build
It's using serve to run the app after build, if we start the app with development code by running ENV=production npm run start it can read the process.env.ENV variable beacause I'm adding this plugins to webpack dev config
new webpack.DefinePlugin({
'process.env':{
'ENV': JSON.stringify(process.env.ENV),
}
}),
I also add the script above to webpack prod config, but if I try this command after build ENV=prod serve -s build, it cannot read the environment variable
How to fix this?
If you set all the environment variables inside the app.config.js, you can replace them after the build in the main.????????.chunk.js file.
A sample app.config.js could look like:
export default {
SOME_URL: "https://${ENV_VAR_1}"
SOME_CONFIGURATION: "${ENV_VAR_2}",
}
Leave the app.config.js file as is, without replacing the environment variables with their actual values. Then, create the optimized production build:
npm ci # if not already installed
npm run build
If the default webpack configurations are used, the contents of app.config.js will be bundled in build/static/js/main.????????.chunk.js. The values of the environment variables can be be envsubst, with a bash script like this:
main_chunk=$(ls build/static/js/main.*.js)
envsubst <$main_chunk >./main_chunk_temp
cp ./main_chunk_temp $main_chunk
rm ./main_chunk_temp
Note: In the above example, envsubst reads the actual variables set in the environment at runtime and literally replaces ${ENV_VAR_1} and ${ENV_VAR_2} with them. So, you can only run this once as the chunk is being over-written.
The reason why you can not read the ENV var is because:
(1) In development mode webpack watches your files and bundles you app on the fly. It also will read (because of the DefinePlugin) your process.env.ENV and will add it as a global variable. So it is basically piping variables from process.env to your JS app.
(2) After you've build your app (with webpack) everything is already bundled up into one or more files. When you run serve you just start a HTTP server that serves the static build files. So there is no way to pipe the ENV to you app.
Basically what the DefinePlugin does is add a var to the bundle. E.g.
new webpack.DefinePlugin({
'token': '12356234ga5q3aesd'
})
will add a line similar to this:
var token = '12356234ga5q3aesd';
since the JS files is static there is no way to change this variable after you've build/bundled it with webpack. Basically, when you do npm run build you're creating the compiled binary/.dll/.jar/... file and can no longer influence its contents via the plugin.
You can add a .env file to the root of your project and define your environment variables there. That will be your default (production) environment variables definition. But then you can have a local file called .env.local to override values from the default.
When defining your environment variables, make sure they start with REACT_APP_ so your environment variable definitions would look like this:
REACT_APP_SERVER_URL=https://my-awesome-app.herokuapp.com
Also, add this to .gitignore so you don't commit your local overrides:
.env*.local
Reference:
Adding Development Environment Variables In .env (create-react-app)
From create-react-app documentation:
Your project can consume variables declared in your environment as if
they were declared locally in your JS files. By default you will have
NODE_ENV defined for you, and any other environment variables starting
with REACT_APP_.
You can read them from process.env inside your code:
render() {
return (
<div>
<small>You are running this application in <b>{process.env.NODE_ENV}</b> mode.</small>
<form>
<input type="hidden" defaultValue={process.env.REACT_APP_NOT_SECRET_CODE} />
</form>
</div>
);
}

Setting Environment Variables for Node to retrieve

I'm trying to follow a tutorial and it says:
There are a few ways to load credentials.
Loaded from environment variables,
Loaded from a JSON file on disk,
The keys need to be as follows:
USER_ID, USER_KEY
...This means that if you properly set your environment variables, you
do not need to manage credentials in your application at all.
Based on some Googling, it appears that I need to set the variables in process.env? How and where do I set these credentials? Example Please.
Environment variables (in this case) are being used to pass credentials to your application. USER_ID and USER_KEY can both be accessed from process.env.USER_ID and process.env.USER_KEY respectively. You don't need to edit them, just access their contents.
It looks like they are simply giving you the choice between loading your USER_ID and USER_KEY from either process.env or some specificed file on disk.
Now, the magic happens when you run the application.
USER_ID=239482 USER_KEY=foobar node app.js
That will pass the user id 239482 and the user key as foobar. This is suitable for testing, however for production, you will probably be configuring some bash scripts to export variables.
I highly recommend looking into the dotenv package.
https://github.com/motdotla/dotenv
It's kind of similar to the library suggested within the answer from #Benxamin, but it's a lot cleaner and doesn't require any bash scripts. Also worth noting that the code base is popular and well maintained.
Basically you need a .env file (which I highly recommend be ignored from your git/mercurial/etc):
FOO=bar
BAZ=bob
Then in your application entry file put the following line in as early as possible:
require('dotenv').config();
Boom. Done. 'process.env' will now contain the variables above:
console.log(process.env.FOO);
// bar
The '.env' file isn't required so you don't need to worry about your app falling over in it's absence.
You can set the environment variable through process global variable as follows:
process.env['NODE_ENV'] = 'production';
Works in all platforms.
Just provide the env values on command line
USER_ID='abc' USER_KEY='def' node app.js
If you want a management option, try the envs npm package. It returns environment values if they are set. Otherwise, you can specify a default value that is stored in a global defaults object variable if it is not in your environment.
Using .env ("dot ee-en-vee") or environment files is good for many reasons. Individuals may manage their own configs. You can deploy different environments (dev, stage, prod) to cloud services with their own environment settings. And you can set sensible defaults.
Inside your .env file each line is an entry, like this example:
NODE_ENV=development
API_URL=http://api.domain.com
TRANSLATION_API_URL=/translations/
GA_UA=987654321-0
NEW_RELIC_KEY=hi-mom
SOME_TOKEN=asdfasdfasdf
SOME_OTHER_TOKEN=zxcvzxcvzxcv
You should not include the .env in your version control repository (add it to your .gitignore file).
To get variables from the .env file into your environment, you can use a bash script to do the equivalent of export NODE_ENV=development right before you start your application.
#!/bin/bash
while read line; do export "$line";
done <source .env
Then this goes in your application javascript:
var envs = require('envs');
// If NODE_ENV is not set,
// then this application will assume it's prod by default.
app.set('environment', envs('NODE_ENV', 'production'));
// Usage examples:
app.set('ga_account', envs('GA_UA'));
app.set('nr_browser_key', envs('NEW_RELIC_BROWSER_KEY'));
app.set('other', envs('SOME_OTHER_TOKEN));
It depends on your operating system and your shell
On linux with the shell bash, you create environment variables like this(in the console):
export FOO=bar
For more information on environment variables on ubuntu (for example):
Environment variables on ubuntu
Windows-users: beware! These commands are recommended for Unix. But on Windows they don't persist, they only set a variable in your current shell, and it'll be gone when you restart.
SET TEST="hello world"
$env:TEST = "hello world"
3 ways to set a persistent environment variable on Windows:
A) .env file in your project - The best method. As you can just copy that file to any computer and get the same config when running the project.
Create an .env file in your project folder root with the content: TEST="hello world"
Write some node code that will read that file. I suggest installing dotenv ( npm install dotenv --save) and then add require('dotenv').config(); during your node setup code.
process.env.TEST is now usable in node
Env-files are a good way of keeping api-keys out of your codebase
B) Use Powershell - this will create a variable that will be accessible in other terminals. But it sucks as it'll be lost after you restart your computer.
[Environment]::SetEnvironmentVariable("TEST", "hello world", "User")
This method is widely recommended on Windows forums, people seem unaware it doesn't persist after a system restart....
C) Use the Windows GUI
Search for "Environment Variables" in the Start Menu Search or in the Control Panel, Select "Edit the system environment variables". A dialogue opens and you click the button "Environment Variables" at the bottom of the dialogue to open an edit-view where you can click the "New" button to add a new environment variable. Easy. And persists even after a restart. But not something you should use to config a specific codebase.
Like ctrlplusb said, I recommend you to use the package dotenv, but another way to do this is creating a js file and requiring it on the first line of your app server.
env.js:
process.env.VAR1="foo"
process.env.VAR2="bar"
app.js:
require('./env') // env.js relative path.
console.log(process.env.VAR1) // foo
Step 1: Add your environment variables to their appropriate file. For example, your staging environment could be called .env.staging, which contains the environment variables USER_ID and USER_KEY, specific to your staging environment.
Step 2: In your package.json file, add the following:
"scripts": {
"build": "sh -ac '. ./.env.${REACT_APP_ENV}; react-scripts build'",
"build:staging": "REACT_APP_ENV=staging npm run build",
"build:production": "REACT_APP_ENV=production npm run build",
...
}
then call it in your deploy script like this:
npm run build:staging
Super simple set up and works like a charm!
Source: https://medium.com/#tacomanator/environments-with-create-react-app-7b645312c09d
Make your life easier with dotenv-webpack. Simply install it npm install dotenv-webpack --save-dev, then create an .env file in your application's root (remember to add this to .gitignore before you git push). Open this file, and set some environmental variables there, like for example:
ENV_VAR_1=1234
ENV_VAR_2=abcd
ENV_VAR_3=1234abcd
Now, in your webpack config add:
const Dotenv = require('dotenv-webpack');
const webpackConfig = {
node: { global: true, fs: 'empty' }, // Fix: "Uncaught ReferenceError: global is not defined", and "Can't resolve 'fs'".
output: {
libraryTarget: 'umd' // Fix: "Uncaught ReferenceError: exports is not defined".
},
plugins: [new Dotenv()]
};
module.exports = webpackConfig; // Export all custom Webpack configs.
Only const Dotenv = require('dotenv-webpack');, plugins: [new Dotenv()], and of course module.exports = webpackConfig; // Export all custom Webpack configs. are required. However, in some scenarios you might get some errors. For these you have the solution as well implying how you can fix certain error.
Now, wherever you want you can simply use process.env.ENV_VAR_1, process.env.ENV_VAR_2, process.env.ENV_VAR_3 in your application.
For windows users this Stack Overflow question and top answer is quite useful on how to set environement variables via the command line
How can i set NODE_ENV=production in Windows?
Came across a nice tool for doing this.
node-env-file
Parses and loads environment files (containing ENV variable exports) into Node.js environment, i.e. process.env - Uses this style:
.env
# some env variables
FOO=foo1
BAR=bar1
BAZ=1
QUX=
# QUUX=
If you are using a mac/linux and you want to retrieve local parameters to the machine you're using, this is what you'll do:
In terminal run nano ~/.bash_profile
add a line like: export MY_VAR=var
save & run source ~/.bash_profile
in node use like: console.log(process.env.MY_VAR);
As expansion of #ctrlplusb answer,
I would suggest you to also take a look to the env-dot-prop package.
It allows you to set/get properties from process.env using a dot-path.
Let's assume that your process.env contains the following:
process.env = {
FOO_BAR: 'baz'
'FOO_🦄': '42'
}
Then you can manipulate the environment variables like that:
const envDotProp = require('env-dot-prop');
console.log(process.env);
//=> {FOO_BAR: 'baz', 'FOO_🦄': '42'}
envDotProp.get('foo');
//=> {bar: 'baz', '🦄': '42'}
envDotProp.get('foo.🦄');
//=> '42'
envDotProp.get('foo.🦄', {parse: true});
//=> 42
envDotProp.set('baz.foo', 'bar');
envDotProp.get('', {parse: true});
//=> {foo: {bar: 'baz', '🦄': 42}, baz: {foo: 'bar'}}
console.log(process.env);
//=> {FOO_BAR: 'baz', 'FOO_🦄': '42', BAZ_FOO: 'bar'}
envDotProp.delete('foo');
envDotProp.get('');
//=> {baz: {foo: 'bar'}}
console.log(process.env);
//=> {BAZ_FOO: 'bar'}
This helps you to parse the environment variables and use them as a config object in your app.
It also helps you implement a 12-factor configuration.
A very good way of doing environment variables I have successfully used is below:
A. Have different config files:
dev.js // this has all environment variables for development only
The file contains:
module.exports = {
ENV: 'dev',
someEnvKey1 : 'some DEV Value1',
someEnvKey2 : 'some DEV Value2'
};
stage.js // this has all environment variables for development only
..
qa.js // this has all environment variables for qa testing only
The file contains:
module.exports = {
ENV: 'dev',
someEnvKey1 : 'some QA Value1',
someEnvKey2 : 'some QA Value2'
};
NOTE: the values are changing with the environment, mostly, but keys remain same.
you can have more
z__prod.js // this has all environment variables for production/live only
NOTE: This file is never bundled for deployment
Put all these config files in /config/ folder
<projectRoot>/config/dev.js
<projectRoot>/config/qa.js
<projectRoot>/config/z__prod.js
<projectRoot>/setenv.js
<projectRoot>/setenv.bat
<projectRoot>/setenv.sh
NOTE: The name of prod is different than others, as it would not be used by all.
B. Set the OS/ Lambda/ AzureFunction/ GoogleCloudFunction environment variables from config file
Now ideally, these config variables in file, should go as OS environment variables (or, LAMBDA function variables, or, Azure function variables, Google Cloud Functions, etc.)
so, we write automation in Windows OS (or other)
Assume we write 'setenv' bat file, which takes one argument that is environment that we want to set
Now run "setenv dev"
a) This takes the input from the passed argument variable ('dev' for now)
b) read the corresponding file ('config\dev.js')
c) sets the environment variables in Windows OS (or other)
For example,
The setenv.bat contents might be:
node setenv.js
The setenv.js contents might be:
// import "process.env.ENV".js file (dev.js example)
// loop the imported file contents
// set the environment variables in Windows OS (or, Lambda, etc.)
That's all, your environment is ready for use.
When you do 'setenv qa', all qa environment variables will be ready for use from qa.js, and ready for use by same program (which always asks for process.env.someEnvKey1, but the value it gets is qa one).
Hope that helps.
Pretty much like some others answers but without any lib nor (bash) export.
I have some encrypted variables then I need to generate them on the fly.
The magic happens with set -a && ... && set +a which can be some content or a file.
#!/bin/sh
set -a
SOMEVAR_A="abcd"
SOMEVAR_B="efgh"
SOMEVAR_C=123456
set +a
# or
set -a && . ./file && set +a
I have a docker-entrypoint.sh with:
#!/bin/sh
node app/config/set-environment.js
ENVFILE=/tmp/.env
if [[ ! -f "$ENVFILE" ]] ; then
echo "File $ENVFILE is not there, aborting."
exit
fi
# here is where things happen
set -a && . $ENVFILE && set +a
if [ "${NODE_ENV}" = "development" ]; then
npx nodemon app/server.js
else
node app/server.js
fi
exec "$#"
While set-environment.js generates a (tmp) .env file
I was getting undefined after setting a system env var. When I put APP_VERSION in the User env var, then I can display the value from node via process.env.APP_VERSION
in case you're using visual studio code debugging feature, you can add "envFile": "${workspaceRoot}/.env" to launch configuration. This way you don't have to use dotenv.
{
"cwd": "${workspaceRoot}",
"command": "npm start",
"name": "Run be",
"request": "launch",
"type": "node-terminal",
"envFile": "${workspaceRoot}/.env"
},
Make a file called local-env and populate it with variables
PORT=80
DB_NAME=foo
SOME_URL=example.com
Now run node thusly:
source ./local_env ; node index.js
Use cross-env. It will save you a lot of headache
npm i -S cross-env
cross-env PARAM=value node ./index.js
That's usually good for non-credentials. For things like credentials and keys
it's better not to store hardcoded user id and password but use .env file which is not in repo and dotenv

Resources