I've come across people using both methods to do config management.
What are the pros and cons of each approach?
If I have a lot of variables that I store in my config object, will I have to set them all one by one in an upstart script before executing the node app?
You generally use envvar to keep an application stateless. The same codebase should work in dev, staging, test and production environment.
You will put var like MySQL config, API keys, if log is enabled or not, if debug is on or not ...
Config file are used for variables which are not dependent of the environment. For instance, name of the application, number of items per page ...
I guess you can use config.json file for storing big configs. ENV I've usually use for passing application port or something very important for normal application start. For example if you use some external lib, it's better to be able to pass custom path to lib executor in ENV.
P.S. You should never save config.json in SVN.
Related
I've been using K8S for a year or so and continue to revisit a problem.
My app is running in K8S and I now need to debug it. I have a NodeJS App that I'm asking about. But similar questions could be asked about Java SpringBoot apps (but this question is just for NodeJS).
I want to use my favorite IDE (IntelliJ or VSCode) to run the app but the app is currently getting it's configuration (inside K8S) using ConfigMaps and Secrets.
(Q) Is there a "best practice" or "pattern" that can be followed that supports the DRY principle and has configuration in one place that can be used for both K8S and when running locally.
Background
I have a NodeJS app that I decided to use ENVIRONMENT variables to hold configuration information because that worked well in IntelliJ IDE, in Docker and in K8S.
I used npm dotenv and created .env.local, .env.stage, .env.prod files to support running in different environments. This worked well enough until it was running in K8S and someone wanted to tweak the configuration and didn't believe that rebuilding the image was the best way to support this. Instead the K8S experts told me I should use ConfigMaps and Secrets, so I converted from the dotenv approach to use the K8S ConfigMaps and Secrets.
I kept the old .env files around just in case and I can use them but the source code no longer call uses dotenv package.
require('dotenv').config()
process.env.myConfigVariable
So I need to either add that code back to support debugging, or manually set the environment variables. I'm wondering if there is a better approach.
I have yaml files templates to make it easy to recreate the deployment from scratch if/when needed.
.env.local
deploy/
helm/
create-configmap.yaml
create-secret.yaml
src/
common/*
appMain.js
Some of the approaches I've considered:
(a) Accept it and have two configs (one for local and one for K8S). Leave the code for dotenv but don't deploy a .env file when deploying to K8S.
(b) Run local k8s (like minikube or k3s) and use my ConfigMap and Secrets as I would with K8S. I then need to figure out how to connect from my IDE to the local K3S environment and open ports in the k3s environment to support this. Some solutions include: Bridge to Kubernetes, YouTube Video Remote Debugging in Kubernetes with Cloud Code,
Debug Java Microservices in Kubernetes with IntelliJ, and I'm sure several others.
(c) Use a JSON config file instead of dotenv. For example, use a JSON config file for everything and map that to /app/config.json and that same config file can be used in both environments. I could have config-local.json, config-stage.json, and config-prod.json to support the different environments.
(d) You tell me. What's another way?
Thanks!
I develop an application with nodejs and react. I use dotenv for configuration in my different environment.
I use TFS 2017 for build and release my application.
What is the best practise for add my .env file of production environment?
Production configs can be difficult to maintain. You can use any module or even write your own custom module to load environment variable in to your application.
However, maintaining production .env files locally for each product (i.e. committing them or putting them inside your docker image is a bad idea. If you ever happen to change some of those configs, you will have to manually change in the code for each application, build docker image (if required) and redeploy. If there are just a couple of applications, it might be easy. But if the number of related applications grow in size (as usually happens in case of microservice based architecture) - all of them sharing credentials, ips for database connections etc. it becomes a hectic task to manually change code and redeploy all the applications.
Usually, developers tend to keep a central repository of all credentials and environment variables. For examples, AWS offers parameter store and MS Azure offers Azure Key vault. In such a case, all the parameters are fetched during start time, and all we have to do is restart the application.
So usually people set only one global varibale NODE_ENV (as prod or dev), dynamically fetch all environment variables based on NODE_ENV running something like env $(node read-env-variables.js) node app.js.
I am using pm2 to run several node applications. Problem is that I am using config files for every node application and so what I want to do is easily have say a json file outside of all the node application folders in which they can ALL point all for common database connections etc...
Prefer to not use linux environment variables unless there is an easy and great way of setting it up
pm2 does have the ecosystem, but it doesn't seem to be very well documented to me
what other solutions?
pm2 ecosystem // this generates .config.js not a .json
Create json or yml file. Put it in your root projects folder. And write "configProvider" which will read the file and populate configuration. It works really well for us. Especially this file can also been shared between different languages, not only javascript.
I am building a web app which will be sending out emails for sign up verification. I will be using https://github.com/RGBboy/express-mailer. I wanted to know whether it is safe for me to display the email password in the code and push it to the server (Heroku, AWS etc.) where the app is hosted. If not, what alternative methods should I use to 'hide' the password?
It is usually considered bad practice to have plaintext secrets/credentials stored under version control. As that could lead to security issues
Usually these sorts of info are set as environment variables. Heroku has a pretty straightforward way of doing this configuration. You can either use their web admin, or set them via command line.
As for other cases, like your development setup, this could be done with the use of .env files, which are loaded and have its values exposed to your running code. Since [express-mailer][2] is a node application, I suggest using some npm package like dotenv or node-env-file automatically do this loading.I personally prefer dotenv which I feel is simpler.
You should also check out this article regarding the use of .env files. The basic idea is to have your .gitignore(or equivalent) to ignore your .env file, thus ensuring your secret credentials are never introduced in your version control. And then setup an .env.sample file to show the developer which data needs to be declared on said .env file.
Example:
.env
EMAIL=foo#bar.com
PASSWORD=AahUbf796
S3_TOKEN=ASVNS7843NCA87SDVNBRT9
.env.sample
EMAIL=<email to access the account>
PASSWORD=<secret password>
S3_TOKEN=<s3 token for application foobar>
You shouldn't ever store secrets in version control.
One alternative (which I personally like the best) is setting secrets as environment variables for your application in your production environment. Heroku I think supports this. This is also the approach that for example Rails takes. Dev/test "secrets" (which are not actually real credentials to anything valuable) can still of course be stored in your VCS.
Another option is to encrypt the user credentials in your source code and decrypt them from your source code.
I am building a Node.js application and need to store database credentials (and other runtime properties) in such a way that they can be read when deployed on Heroku. My source is available in a public GitHub repository.
I am currently using environment variables, configured using heroku config:add, but am looking to understand if there are any alternatives. I would potentially like to use Cloud9 IDE, but it does not currently support environment variables.
Another option is to store the parameters in a config. file, but I believe the file would need to be checked in to Git (and as such, be publicly available) in order to be pushed to Heroku.
Thanks for your help.
ENV vars are generally considered the way to go, and the way Heroku do it themselves for database_urls and the like.
As you and your app are the only people with access to the env vars, you're generally OK security wise.
Putting credentials in Git or similar is a bad idea as it's another place that needs to be secured.
The one way I know of to solve the problem for development using command-line arguments. These can be specified in your run/debug configuration. You can then access the parameters in process.argv. Of course this means that they will be stored in your Cloud9IDE dev environment. You could then use the ENV variables in a retail production. This will at least prevent the credentials from being visible in source or config files.