I'm new to nodejs and I can't figure out why environment variables are not always available.
Env vars in my .env are parsed by docker-compose automatically. So I'm not using the dotenv package.
console.log(process.env.API_URL) // outputs undefined
But in an asynchronous function (maybe it doesn't work in all async functions)
async function foo() {
console.log(process.env.API_URL) // outputs http://example.com
}
After moving a few ways the console.log in my index.ts, I've found the line where env vars become available :
// index.ts
import {createConnection} from 'typeorm'
console.log(process.env.API_URL) // outputs undefined
createConnection()
console.log(process.env.API_URL) // outputs http://example.com
Any idea on why this is happening ?
Edit :
I'm using this docker-compose.yml :
version: '3.7'
services:
server:
build:
context: ./docker-build
target: server-develop
ports:
- "3200:3000"
volumes:
- .:/var/app
command: "yarn run dev"
Docker docs seems to tell that the .env file is parsed by default when env_file is not specified. Screenshot from docker documentation :
However I don't see env vars when using docker-compose exec server env.
But if I specify the env_file in the docker-compose.yml (see below), environment vars are correctly loaded.
version: '3.7'
services:
server:
build:
context: ./docker-build
target: server-develop
env_file: .env # the only change
ports:
- "3200:3000"
volumes:
- .:/var/app
Maybe I misunderstood what actually docker does when It says
The .env file is loaded by default
You can use dotenv from here https://www.npmjs.com/package/dotenv
Probably you are using nodemon and as far as I know it only loads the NODE_ENV variable. By using dotenv you can just:
create a new file .env in root folder where you put your env variables
as early as possible you load require('dotenv').config()
You'll find more information on the npmjs package page itself like changing your startup command so it can automatically load .env file and more
Related
First I want to use my application .env variables on my webpack.config.prod.js, so I did this in my webpack file.
I am successfully able to access process.env.BUILD variables.
My application’s env has this configuration -
My nodejs web app is running fine locally, no problem at all. I want to build docker image of this application and need to use docker-compose to create the container.
I built my docker image and everything good so far.
now to create container, instead of docker run. I am using separate folder which consists of docker-compose.yml and .env files. Attached the screenshot below
My docker-compose.yml has this code -
version: '3.9'
services:
api:
image: 'api:latest'
ports:
- '17000:17000'
env_file:
- .env
volumes:
- ./logs:/app/logs
networks:
- default
My docker-compose .env has this redis details
My application has this logs -
I started my docker container by doing docker-compose up. Containers created and up and running, but the problem is
In the console, after connecting to redis.. process.env.REDIS_HOST contains the value called ‘localhost’ (which came from the first env where I used to build docker image). Docker compose .env is not getting accessed.
After spending 5+ hours. I found the culprit, It was webpack. On my initial code, I added some env related things in my webpack right? Once I commented those, taken a new build. Everything is working fine.
But my problem is how I can actually use process.ENV in webpack, also docker-compose need to use the .env from its directory.
Updated -
My DockerFile looks like this:
Just, It will copy the dist which contains bundle.js, npm start will do - pm2 run bundle.js.
From what I know webpack picks up the .env at build time, not at runtime. This means that it needs the environment variables when the image is built.
The one you pass in docker-compose.yml is not used because by then your application is already built. Is that correct? In order to user your .env you should build the image with docker-compose and pass the env variables as build arguments to your Dockerfile.
In order to build the image using your docker-compose.yml, you should do something like this:
version: '3.9'
services:
api:
image: 'api:latest'
build:
context: .
args:
- REDIS_HOST
- REDIS_PORT
ports:
- '17000:17000'
volumes:
- ./logs:/app/logs
Note: the context above points to the current folder. You can change it to point to a folder where you Dockerfile (and the rest of the project is) or you can put your docker-compose.yml together with the rest of the project directly and then context stays ..
In your Dockerfile you need to specify these arguments:
FROM node:14 as build
ARG REDIS_HOST
ARG REDIS_PORT
...
With these changes you can build and run with docker-compose:
docker-compose up -d --build
I have a React App running using Docker. I want to deploy the same docker image on dev, staging and prod environment.
The value of my React environment variable (REACT_APP_PARAM) is different for each environment.
My dockerfile look like this :
…
COPY . .
ARG REACT_APP_PARAM
ENV REACT_APP_PARAM $ REACT_APP_PARAM
…
I use this Dockecompose file to build my image
version: "3"
services:
frontend:
image: my-image
build:
context: .
args:
- REACT_APP_PARAM=BuildParam
environment:
- REACT_APP_PARAM=${REACT_APP_PARAM}
ports:
- "80"
- "443"
When deploy and run my image on dev environment I use this dockercompose file
version: "3"
services:
frontend:
image: my-image
restart: always
environment:
- REACT_APP_PARAM=DevelopmentParam
ports:
- "80"
- "443"
When I debug my application the value of REACT_APP_PARAM is still BuildParam
But went I list my container environment variable with command docker exec my-image env, the value of REACT_APP_PARAM is DevelopmentParam.
Any ideas how I can solve this or how is the best approach to archive that?
Thanks
After several searches, I found that Environment variables are embedded into the build.
The official React documentation :
The environment variables are embedded during the build time. Since
Create React App produces a static HTML/CSS/JS bundle, it can’t
possibly read them at runtime. To read them at runtime, you would need
to load HTML into memory on the server and replace placeholders in
runtime, as described here. Alternatively you can rebuild the app on
the server anytime you change them.
https://create-react-app.dev/docs/adding-custom-environment-variables/
I don't like the idea to build different image for each environment.
Having the following docker-compose file:
db:
build: .
environment:
- MYSQL_ROOT_PASSWORD=password
- ENV=test
env_file: .env
Is there any way to use the env variables declared in docker-compose.yml (either as environment or declared in the env_file) as part of Dockerfile without declaring them in the Dockerfile? Something like this:
FROM java:7
ADD ${ENV}/data.xml /data/
CMD ["run.sh"]
Although this question was asked long ago, there is an answer to a similar question here: Pass environment variables from docker-compose to container at build stage
Basically, to use variables at the container's build time one has to define the variable in docker-compose.yml:
build:
context: .
args:
MYSQL_ROOT_PASSWORD: password
ENV: test
and then reference it in the Dockerfile using ARG:
ARG MYSQL_ROOT_PASSWORD
ARG ENV
ADD ${ENV}/data.xml /data/
Concerning environment variables defined in an *.env file, I believe that they can't be passed to the container at build time.
It works ok this way:
docker-compose.yml
version: '3.5'
services:
container:
build:
context: .
args:
ENV: ${ENV} # from .env file
env_file:
- .env
Dockerfile
# from compose args
ARG ENV
ADD ${ENV}/data.xml /data/
.env
ENV=myenv
Thus all the values are taken from .env file
This approach goes against the 'build once, run anywhere' theory behind Docker and most DevOps approaches. With this approach you'll need to build a container for every environment you expect to use. By doing so you can't safely say if a container works in the dev environment it will work in staging and production since you aren't using the same container.
You'd be better off adding all config files you need on to the container and writing an entrypoint script that selects/copies the data for that environment to the correct location when the container starts. You can also apply this approach to other config on the container, like templated Apache config using jinja2 templates etc.
I'm building a node/express service using docker and knex for database interaction. I have an env_file (defined in docker-compose file) that has some environment variables defined. The app is reading them correctly, as a console.log(process.env.DATABASE_USER); will log the correct value.
I followed the knex documentation to setup a knexfile that looks like so:
module.exports = {
development: {
client: 'pg',
connection: {
host: process.env.DATABASE_HOST,
port: process.env.DATABASE_PORT,
user: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME_DEV,
},
migrations: {
directory: __dirname + '/db/migrations',
},
seeds: {
directory: __dirname + '/db/seeds',
},
},
};
If I hardcode the values into the knexfile, all is well. I can connect to the database, run migrations, etc.
When I use my environment variables (like above), they return undefined. Why is that?
UPDATE:
My docker compose file--api.env is just a basic .env file:
version: '3.3'
services:
db:
container_name: db
build:
context: ./services/api/src/db
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
nginx:
container_name: nginx
build: ./services/nginx
restart: always
ports:
- 80:80
depends_on:
- api
links:
- api
api:
container_name: api
build:
context: ./services/api
dockerfile: Dockerfile
volumes:
- './services/api:/usr/src/app'
- './services/api/package.json:/usr/src/app/package.json'
ports:
- 8887:8888
env_file: ./api.env
depends_on:
- db
links:
- db
client:
container_name: client
build:
context: ./services/client
dockerfile: Dockerfile
volumes:
- './services/client:/usr/src/app'
ports:
- 3007:3000
environment:
- NODE_ENV=development
depends_on:
- api
links:
- api
Dockerfile for api service:
FROM node:latest
RUN mkdir /usr/src/app
WORKDIR /usr/src/app
ENV PATH /usr/src/app/node_modules/.bin:$PATH
ADD package.json /usr/src/app/package.json
RUN npm install
CMD ["npm", "start"]
dotenv reads the vars from the files on same level as it is, so if your knexfile is not on the same directory level with you .env, you may want to set the path. Something like this:
import dotenv from 'dotenv'
dotenv.config({ path: '../../.env' })
You can read the docs for more details https://www.npmjs.com/package/dotenv
You are not setting environment variables correctly in your setup. Maybe you are referring env_file wrong or you might have typo in your environment variable setups.
You need to give more information how is your docker compose file etc. Your knexfile.js looks correct.
EDIT:
From your docker-compose file it looks like you are passing database connection details only to your API container. Nothing shows how are you running knex migrations and how are you passing those variables for that call.
EDIT2:
bash -c "sleep 10 && npm run knex migrate:latest"
Command opens new shell, which does not have environment variables having connection details set. You need to pass environment variables with connection details to your shell.
Probably easiest for you would be to write run-migrations.sh script, which sets and passes variables correctly before running the migrate. Like:
for i in $(cat api.env); do
export $i;
done;
sleep 10;
npm run knex migrate:latest;
When I am using env_file in docker-compose.yml it builds correctly, but when I am trying to use docker-compose my node app can´t find env_file variables inside the process.env object.
Here is my docker-compose file:
node1:
container_name: node01
env_file: ./env/node1.production.env
#environment:
#- SOME_VALUE=9599
build:
context: ./node1
dockerfile: dockerfile
ports:
- "3000:3000"
networks:
- dev_net
Here is my node1.production.env file:
SOME_VALUE=9599
When I use environment instead, my node app works fine:
DOCKER Version : 17.03
DOCKER COMPOSE Version : 1.14
OS : CentOS
It should work. I guess that you might have defined variables more than once in node1.production.env file. Verify if the env file is correct.
From the code you gave, it seems there are no errors in the syntax you are using, and if there were, they would have been reported before build could even be started. In my case, I use env file as follows:
env_file:
- .env
where .env named file is present in the base directory.