I'm a newbie with Google Cloud App Engine and I want to deploy my simple Node.js application. I need to configure it for "production" environment. For example, I want to provide some credentials for email box, which my application is going to query for new emails.
I like to use nconf npm package, it gives me powerful and flexible configuration via command-line arguments or environment variables. I don't need to hard-code possibly secure values in source code.
Is there a way to configure environment variables or command-line arguments for an application in Google Developer Console?
P. S.
I found from documentation, that one can inline environment variables in app.yml config file:
env_variables:
FOO: 'myapp.bar'
But this is still not suitable for me, since app.yml is staged in git.
Related
I'm developing an application with my team with Firebase, and I want to deploy my app both in development and production environments.
To achieve this, I have done several things. (Details are below)
But NODE_ENV keeps set as 'development' when I deploy it to firebase functions app.
How can I deploy it as 'development' to firebase?
Those are what I have done to achieve my goal.
Make another firebase project and function for 'production' (Use current firebase project as 'development')
firebase console
Edit .firebaserc file and set aliases for my projects (to use firebase --use command)
.firebaserc
Separate RDS Instance and Slack Monitoring app
Separate .env files with .env.dev and .env.prod and use secrets based on NODE_ENV
set .env files
Add 'dev' script to package.json to deploy as 'NODE_ENV = development'
package.json scripts
This is the code I wrote to find out is which environment my server is running
node_env console.log
And this is from my github actions log and firebase console log
github actions and firebase console log
When I run my app in local with 'serve' command, console.log prints 'development' as I expected.
I guess 'firebase deploy' command automatically changes my NODE_ENV to production when it is deployed.
Or did I make some mistakes to do this?
The recommended way to have a development and production environment is to have two separate Firebase projects, which you are already making use of. For the sake of an example, let's assume you have hyositive-app as your production project and hyositive-dev as your development project.
Defining Deployed Environment Variables
Use of environment variables with Cloud Functions for Firebase are based on .env files compatible with the dotenv package. By default, the following files are supported (ignoring others!):
Name
Purpose
.env
Environment variables common to all environments
.env.<projectID>
Environment variables used when deployed to the named project (e.g. .env.hyositive-app)
.env.<alias>
Environment variables used when deployed to the aliased project (e.g. .env.dev).Note: default is the only alias configured out of the box. dev/prod/etc. are not defined.
.env.local
Environment variables used when using emulating functions on your system using the Cloud Functions emulator.
To use .env.dev and .env.prod, you will need to define them as project aliases in your .firebaserc file (or you can continue using development and production aliases and just update the filenames to match):
{
"projects": {
"default": "hyositive-dev",
"dev": "hyositive-dev",
"prod": "hyositive-app"
}
}
This then allows you to use .env.dev and .env.prod instead of .env.hyositive-dev and .env.hyositive-app respectively.
Using Environment Variables
The Cloud Functions runtime (the process that runs your deployed code) defines a number of built-in environment variables that have various purposes (such as allowing you to use initializeApp() with no arguments).
In addition to these variables, a handful of language-specific variables are also defined by the runtime to aid in painless deployment of code. However, the Cloud Functions documentation states to not rely on their values unless you set the values yourself.
The Node.js Cloud Functions runtime is built using the Cloud Functions Framework (however, it is not guaranteed to perfectly match this open source version). Because this runtime executes using Node.js and makes use of other packages such as express, it sets NODE_ENV to production, to minimise unexpected behaviour that depends on its value. But as mentioned above, this behaviour should not be relied on even though it is unlikely to change.
To override NODE_ENV to development, you would add it into .env.dev, .env.hyositive-dev and/or .env.local (as appropriate). Similarly, you should also define NODE_ENV as production in .env.prod or .env.hyositive-app (as appropriate).
Rather than rely on NODE_ENV, I would recommend defining behaviour around another variable that you have complete control over (such as HYOSITIVE_ENV) or compare against the executing project ID to determine whether it is running in the production project or not.
const PROD_PROJECT_ID = "hyositive-app",
// DEV_PROJECT_ID = "hyositive-dev",
PROJECT_ID = JSON.parse(process.env.FIREBASE_CONFIG).projectId,
isProduction = PROJECT_ID === PROD_PROJECT_ID;
Note: This other thread may have some useful background information.
I've been using K8S for a year or so and continue to revisit a problem.
My app is running in K8S and I now need to debug it. I have a NodeJS App that I'm asking about. But similar questions could be asked about Java SpringBoot apps (but this question is just for NodeJS).
I want to use my favorite IDE (IntelliJ or VSCode) to run the app but the app is currently getting it's configuration (inside K8S) using ConfigMaps and Secrets.
(Q) Is there a "best practice" or "pattern" that can be followed that supports the DRY principle and has configuration in one place that can be used for both K8S and when running locally.
Background
I have a NodeJS app that I decided to use ENVIRONMENT variables to hold configuration information because that worked well in IntelliJ IDE, in Docker and in K8S.
I used npm dotenv and created .env.local, .env.stage, .env.prod files to support running in different environments. This worked well enough until it was running in K8S and someone wanted to tweak the configuration and didn't believe that rebuilding the image was the best way to support this. Instead the K8S experts told me I should use ConfigMaps and Secrets, so I converted from the dotenv approach to use the K8S ConfigMaps and Secrets.
I kept the old .env files around just in case and I can use them but the source code no longer call uses dotenv package.
require('dotenv').config()
process.env.myConfigVariable
So I need to either add that code back to support debugging, or manually set the environment variables. I'm wondering if there is a better approach.
I have yaml files templates to make it easy to recreate the deployment from scratch if/when needed.
.env.local
deploy/
helm/
create-configmap.yaml
create-secret.yaml
src/
common/*
appMain.js
Some of the approaches I've considered:
(a) Accept it and have two configs (one for local and one for K8S). Leave the code for dotenv but don't deploy a .env file when deploying to K8S.
(b) Run local k8s (like minikube or k3s) and use my ConfigMap and Secrets as I would with K8S. I then need to figure out how to connect from my IDE to the local K3S environment and open ports in the k3s environment to support this. Some solutions include: Bridge to Kubernetes, YouTube Video Remote Debugging in Kubernetes with Cloud Code,
Debug Java Microservices in Kubernetes with IntelliJ, and I'm sure several others.
(c) Use a JSON config file instead of dotenv. For example, use a JSON config file for everything and map that to /app/config.json and that same config file can be used in both environments. I could have config-local.json, config-stage.json, and config-prod.json to support the different environments.
(d) You tell me. What's another way?
Thanks!
I am building a web application that utilises environment variables locally, and I want to put this on production (online). I am trying to find out how to set environment variables on AWS.
This is a node.js application and I am using elastic beanstalk on AWS for deployment. I have looked through https://docs.aws.amazon.com/cloud9/latest/user-guide/env-vars.html#env-vars-env-list , but I'm unsure of which option applies to me.
The .env file I have locally contains lines like
PASSWORD=MYPASSWORD
and I am using the dotenv package, with require('dotenv').config(); in the appropriate files. I am accessing environment variables in my code through things like process.env.PASSWORD (using the aforementioned example of a line in the .env file).
I've tried searching several places and am presented with various options, but I'm not sure which one applies to my environment variables.
The link you are following may help you in ec2 machine that is mangages by you, but if you are working Elasticbeanstalk I will recommend using Environment variable configuration provided by elasticbeanstalk.
I am not in favour of .env in case of Elasticbeanstalk, ECS and many other services where AWS provide easy and out of the box feature to set environment variable in the configuration and .env write environment to file which is less secure then system environment variable.
The interesting part of Elasticbeanstalk ENV is, the system environment variable has higher periphery then .env environment variable but better to not place dotenv on elasticbeanstalk.
Environment Properties and Other Software Settings
You can use environment properties to pass secrets, endpoints, debug
settings, and other information to your application. Environment
properties help you run your application in multiple environments for
different purposes, such as development, testing, staging, and
production.
elasticbeanstalk-deploy_nodejs
Example .ebextensions/options.config
option_settings:
aws:elasticbeanstalk:application:environment:
API_ENDPOINT: www.example.com/api
Now all you need
var endpoint = process.env.API_ENDPOINT
Environment Properties
The Environment Properties section lets you specify environment
configuration settings on the Amazon EC2 instances that are running
your application. These settings are passed in as key-value pairs to
the application.
Inside the Node.js environment running in AWS Elastic Beanstalk, you
can access the environment variables using process.env.ENV_VARIABLE
similar to the following example.
var endpoint = process.env.API_ENDPOINT
I develop an application with nodejs and react. I use dotenv for configuration in my different environment.
I use TFS 2017 for build and release my application.
What is the best practise for add my .env file of production environment?
Production configs can be difficult to maintain. You can use any module or even write your own custom module to load environment variable in to your application.
However, maintaining production .env files locally for each product (i.e. committing them or putting them inside your docker image is a bad idea. If you ever happen to change some of those configs, you will have to manually change in the code for each application, build docker image (if required) and redeploy. If there are just a couple of applications, it might be easy. But if the number of related applications grow in size (as usually happens in case of microservice based architecture) - all of them sharing credentials, ips for database connections etc. it becomes a hectic task to manually change code and redeploy all the applications.
Usually, developers tend to keep a central repository of all credentials and environment variables. For examples, AWS offers parameter store and MS Azure offers Azure Key vault. In such a case, all the parameters are fetched during start time, and all we have to do is restart the application.
So usually people set only one global varibale NODE_ENV (as prod or dev), dynamically fetch all environment variables based on NODE_ENV running something like env $(node read-env-variables.js) node app.js.
I am building a Node.js application and need to store database credentials (and other runtime properties) in such a way that they can be read when deployed on Heroku. My source is available in a public GitHub repository.
I am currently using environment variables, configured using heroku config:add, but am looking to understand if there are any alternatives. I would potentially like to use Cloud9 IDE, but it does not currently support environment variables.
Another option is to store the parameters in a config. file, but I believe the file would need to be checked in to Git (and as such, be publicly available) in order to be pushed to Heroku.
Thanks for your help.
ENV vars are generally considered the way to go, and the way Heroku do it themselves for database_urls and the like.
As you and your app are the only people with access to the env vars, you're generally OK security wise.
Putting credentials in Git or similar is a bad idea as it's another place that needs to be secured.
The one way I know of to solve the problem for development using command-line arguments. These can be specified in your run/debug configuration. You can then access the parameters in process.argv. Of course this means that they will be stored in your Cloud9IDE dev environment. You could then use the ENV variables in a retail production. This will at least prevent the credentials from being visible in source or config files.