What's the best way to set environment variables when developing nodejs on a local machine for App Engine Flex environment? If they're set in app.yaml then they don't get set during local development. Is there a way to force this, or should I use something like dotenv and keep track of the same environment variables in 2 places?
Sensitive data (e.g. API Keys) should not be committed to source code.
The way I went around that is storing a .env file in Google Storage. Then you can use #google-cloud/storage to download it in production (using a prestart hook) and dotenv to load variables into memory.
You may find a full guide here: http://gunargessner.com/gcloud-env-vars/
PS: I would go for Aidan's answer for storing any data that is not sensitive. I have myself used dotenv satisfactorily in the past.
Similar to that, there's nconf, the package that gcloud itself uses for examples. Pretty neat!
Option 1:
require('dotenv').config({path: '/custom/project/root/app.yaml'})
Option 2:
Maintain both a .env file and a .yaml file with the same keys but different values (local and GAE, respectively). In app.yaml, I make sure not to deploy my .env file by adding the following line:
skip_files : .env
You will then need to add a check on ('dotenv').config() to make sure that it doesn't error or overwrite your process variables if no .env file is detected.
Aidan's suggestion is good.
As configurations should be different on GAE and local, I would suggest option 2 is best - a separate .ENV for local and .YAML for GAE environments.
One minor point though. I'd suggest to add a .gcloudignore files, something like the below:
.gcloudignore
.git
.gitignore
.env
staging.yaml
node_modules/
Related
I am working on a NodeJS application which is a containerized application. I use Jenkins to build and deploy it. I have an environment .env file and along with this, also have .env files based on environments.
For example: .env.DEV, .env.SQA, .env.STG and .env.PROD.
Each file has different values for the variables based on environments.
When I am deploying my application, it always fetches variables from the .env file instead of specific environment file i.e. .env.DEV (if deploying on DEV server).
How do we use specific environment file while doing the deployment on Jenkins?
Note - I followed this great content on dotenv library but I didn't find anything helpful for my use-case. I even Googled a lot but didn't find much on this. Any help would be greatly appreciated.
You can use dotenv-flow which does exactly this, given the value of NODE_ENV environment value it will load the expected environment.
You also will need to make sure that the container receives the proper environment values from jenkins, this might help.
I've been using K8S for a year or so and continue to revisit a problem.
My app is running in K8S and I now need to debug it. I have a NodeJS App that I'm asking about. But similar questions could be asked about Java SpringBoot apps (but this question is just for NodeJS).
I want to use my favorite IDE (IntelliJ or VSCode) to run the app but the app is currently getting it's configuration (inside K8S) using ConfigMaps and Secrets.
(Q) Is there a "best practice" or "pattern" that can be followed that supports the DRY principle and has configuration in one place that can be used for both K8S and when running locally.
Background
I have a NodeJS app that I decided to use ENVIRONMENT variables to hold configuration information because that worked well in IntelliJ IDE, in Docker and in K8S.
I used npm dotenv and created .env.local, .env.stage, .env.prod files to support running in different environments. This worked well enough until it was running in K8S and someone wanted to tweak the configuration and didn't believe that rebuilding the image was the best way to support this. Instead the K8S experts told me I should use ConfigMaps and Secrets, so I converted from the dotenv approach to use the K8S ConfigMaps and Secrets.
I kept the old .env files around just in case and I can use them but the source code no longer call uses dotenv package.
require('dotenv').config()
process.env.myConfigVariable
So I need to either add that code back to support debugging, or manually set the environment variables. I'm wondering if there is a better approach.
I have yaml files templates to make it easy to recreate the deployment from scratch if/when needed.
.env.local
deploy/
helm/
create-configmap.yaml
create-secret.yaml
src/
common/*
appMain.js
Some of the approaches I've considered:
(a) Accept it and have two configs (one for local and one for K8S). Leave the code for dotenv but don't deploy a .env file when deploying to K8S.
(b) Run local k8s (like minikube or k3s) and use my ConfigMap and Secrets as I would with K8S. I then need to figure out how to connect from my IDE to the local K3S environment and open ports in the k3s environment to support this. Some solutions include: Bridge to Kubernetes, YouTube Video Remote Debugging in Kubernetes with Cloud Code,
Debug Java Microservices in Kubernetes with IntelliJ, and I'm sure several others.
(c) Use a JSON config file instead of dotenv. For example, use a JSON config file for everything and map that to /app/config.json and that same config file can be used in both environments. I could have config-local.json, config-stage.json, and config-prod.json to support the different environments.
(d) You tell me. What's another way?
Thanks!
I was just wondering if there was any standardized convention for .env environment variable files. If I had multiple setups (e.g. development, staging, production), what should they be titled?
I've seen:
.env.development, ...
development.env, ...
settings.development.env, ...
.env, .env.staging, .env.production, ...
If there isn't a standard, are there arguments for which to use (kinda like "" vs '' for strings in JS)? Which do you use and why?
I prefer .env at the end ex: .production.env because last word after . indicates extension. VSCode does not recognize the file as .env file if you do .env.production
VSCode Documentation:
https://code.visualstudio.com/docs/python/environments#_environment-variables
There is only one standard in node ecosystem - use production in NODE_ENV variable for production. It's used by different tools for optimizing.
The rest is on you. But personally, I don't see reasons to make it complex. If you need to differentiate instances further than just production, development, you can use other environment variables.
Also you need to be careful when using different NODE_ENV for staging and production. Because the main purpose of staging is to test everything as close to production as possible. So changing NODE_ENV can be a reason to production fail.
I'm using this convention since I can't find any other suggestions: https://rubyonjets.com/docs/env-files/
.env
.env.development
.env.test
.env.production
I like maintaining the .env at the front and this seems reasonable to me.
I've been told to implement Continuous Integration for an existing application (FrontEnd: Node.js - BackEnd: .Net API).
The API endpoints are currently hardcoded in the .js files, that get "uglyfied" after the build (webpack). I want to move them to a config file, that gets copied to the dist folder, so they can be changed at deployment time (just like a Web.config file in the API).
I have zero experience with Node. Is this possible? How?
Look into a dotenv file and use process.env.MY_ENV_VARIABLE in your code to access environment variables. Here's one library https://github.com/motdotla/dotenv but I'm sure there others (can't remember which one I used in the past).
Edit: If you're using Webpack to bundle your front end app you might need to configure it to pull in environment variables, you can use the define plugin: https://webpack.js.org/plugins/define-plugin/
I've come across people using both methods to do config management.
What are the pros and cons of each approach?
If I have a lot of variables that I store in my config object, will I have to set them all one by one in an upstart script before executing the node app?
You generally use envvar to keep an application stateless. The same codebase should work in dev, staging, test and production environment.
You will put var like MySQL config, API keys, if log is enabled or not, if debug is on or not ...
Config file are used for variables which are not dependent of the environment. For instance, name of the application, number of items per page ...
I guess you can use config.json file for storing big configs. ENV I've usually use for passing application port or something very important for normal application start. For example if you use some external lib, it's better to be able to pass custom path to lib executor in ENV.
P.S. You should never save config.json in SVN.