Knexfile not reading environment variables? - node.js

Question

I'm building a node/express service using docker and knex for database interaction. I have an env_file (defined in docker-compose file) that has some environment variables defined. The app is reading them correctly, as a console.log(process.env.DATABASE_USER); will log the correct value.
I followed the knex documentation to setup a knexfile that looks like so:
module.exports = {
development: {
client: 'pg',
connection: {
host: process.env.DATABASE_HOST,
port: process.env.DATABASE_PORT,
user: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME_DEV,
},
migrations: {
directory: __dirname + '/db/migrations',
},
seeds: {
directory: __dirname + '/db/seeds',
},
},
};
If I hardcode the values into the knexfile, all is well. I can connect to the database, run migrations, etc.
When I use my environment variables (like above), they return undefined. Why is that?
UPDATE:
My docker compose file--api.env is just a basic .env file:
version: '3.3'
services:
db:
container_name: db
build:
context: ./services/api/src/db
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
nginx:
container_name: nginx
build: ./services/nginx
restart: always
ports:
- 80:80
depends_on:
- api
links:
- api
api:
container_name: api
build:
context: ./services/api
dockerfile: Dockerfile
volumes:
- './services/api:/usr/src/app'
- './services/api/package.json:/usr/src/app/package.json'
ports:
- 8887:8888
env_file: ./api.env
depends_on:
- db
links:
- db
client:
container_name: client
build:
context: ./services/client
dockerfile: Dockerfile
volumes:
- './services/client:/usr/src/app'
ports:
- 3007:3000
environment:
- NODE_ENV=development
depends_on:
- api
links:
- api
Dockerfile for api service:
FROM node:latest
RUN mkdir /usr/src/app
WORKDIR /usr/src/app
ENV PATH /usr/src/app/node_modules/.bin:$PATH
ADD package.json /usr/src/app/package.json
RUN npm install
CMD ["npm", "start"]

Answer 1

dotenv reads the vars from the files on same level as it is, so if your knexfile is not on the same directory level with you .env, you may want to set the path. Something like this:
import dotenv from 'dotenv'
dotenv.config({ path: '../../.env' })
You can read the docs for more details https://www.npmjs.com/package/dotenv

Answer 2

You are not setting environment variables correctly in your setup. Maybe you are referring env_file wrong or you might have typo in your environment variable setups.
You need to give more information how is your docker compose file etc. Your knexfile.js looks correct.
EDIT:
From your docker-compose file it looks like you are passing database connection details only to your API container. Nothing shows how are you running knex migrations and how are you passing those variables for that call.
EDIT2:
bash -c "sleep 10 && npm run knex migrate:latest"
Command opens new shell, which does not have environment variables having connection details set. You need to pass environment variables with connection details to your shell.
Probably easiest for you would be to write run-migrations.sh script, which sets and passes variables correctly before running the migrate. Like:
for i in $(cat api.env); do
export $i;
done;
sleep 10;
npm run knex migrate:latest;