Tech Stack
Bitbucket Pipelines
Docker
Node
Webpack
VueJS 2
Firebase
Question
Which piece of technology above do I use to reconfigure the Firebase APIKeys before building? I think it should be done in Webpack. Can you point me to an example? I'm a Webpack Noob.
The Issue
When I use Webpack to build the Vue project, I need to change the $FIREBASE_API_KEY and other config options based on the branch that Bitbucket Pipelines was triggered from. If I check-in a change to "inflight" on Bitbucket, Pipelines should build and deploy with the $FIREBASE_API_KEY (and other props) that match the firebase project, "dev".
// Firebase Config
let config = {
apiKey: "$FIREBASE_API_KEY",
authDomain: "my-project.firebaseapp.com",
databaseURL: "https://my-project.firebaseio.com",
databasecloudfunctionsUrl: "https://my-project.firebaseio.com",
projectId: "my-project",
storageBucket: "my-project.appspot.com",
messagingSenderId: "xyz123"
}
Details
The config info you see above currently lives within my-project\src\validation-service.js
Seems, I need to define $FIREBASE_API_KEY (and the other props) in a separate file that Webpack can manipulate.
Does Docker have a roll in updating the APIKeys and other configs for deploying Dev/Prod from Inflight/Master?
Examples - similar tech
https://github.com/bartw/multi_env_webpack_travis_app
https://hackernoon.com/continuous-deployment-of-a-webpack-app-to-multiple-environments-using-travic-ci-d2c6f22eac50
https://www.atlassian.com/continuous-delivery/tips-for-scripting-tasks-with-Bitbucket-Pipelines
Answer
Webpack
Webpack makes it possible to configure Client API Keys for different environments, like Prod or Dev.
I created this project with vue-cli and chose the webpack option. This creates folders and files to configure Continuous Integration for multiple environments.
In my specific case, I saved my two Client API Keys in dev.env.js and prod.env.js. These .js files are in the config folder created by vue-cli.
I used This Tutorial to gain an understanding of how Webpack uses files in the build folder. For vue-cli, keep and eye on NODE_ENV and '"production"'
More
There are other Secret API Keys that this application needs access to when deploying. For this, I use Environmental Variables in Bitbucket Pipelines. Although Bitbucket Pipelines uses Docker Images to build and deploy, it is the bitbucket-pipelines.yml that can reference encrypted environmental variables from Bitbucket, i.e., $PROD_FIREBASE_API_KEY, $DEV_FIRBASE_API_KEY.
Related
I'm developing an application with my team with Firebase, and I want to deploy my app both in development and production environments.
To achieve this, I have done several things. (Details are below)
But NODE_ENV keeps set as 'development' when I deploy it to firebase functions app.
How can I deploy it as 'development' to firebase?
Those are what I have done to achieve my goal.
Make another firebase project and function for 'production' (Use current firebase project as 'development')
firebase console
Edit .firebaserc file and set aliases for my projects (to use firebase --use command)
.firebaserc
Separate RDS Instance and Slack Monitoring app
Separate .env files with .env.dev and .env.prod and use secrets based on NODE_ENV
set .env files
Add 'dev' script to package.json to deploy as 'NODE_ENV = development'
package.json scripts
This is the code I wrote to find out is which environment my server is running
node_env console.log
And this is from my github actions log and firebase console log
github actions and firebase console log
When I run my app in local with 'serve' command, console.log prints 'development' as I expected.
I guess 'firebase deploy' command automatically changes my NODE_ENV to production when it is deployed.
Or did I make some mistakes to do this?
The recommended way to have a development and production environment is to have two separate Firebase projects, which you are already making use of. For the sake of an example, let's assume you have hyositive-app as your production project and hyositive-dev as your development project.
Defining Deployed Environment Variables
Use of environment variables with Cloud Functions for Firebase are based on .env files compatible with the dotenv package. By default, the following files are supported (ignoring others!):
Name
Purpose
.env
Environment variables common to all environments
.env.<projectID>
Environment variables used when deployed to the named project (e.g. .env.hyositive-app)
.env.<alias>
Environment variables used when deployed to the aliased project (e.g. .env.dev).Note: default is the only alias configured out of the box. dev/prod/etc. are not defined.
.env.local
Environment variables used when using emulating functions on your system using the Cloud Functions emulator.
To use .env.dev and .env.prod, you will need to define them as project aliases in your .firebaserc file (or you can continue using development and production aliases and just update the filenames to match):
{
"projects": {
"default": "hyositive-dev",
"dev": "hyositive-dev",
"prod": "hyositive-app"
}
}
This then allows you to use .env.dev and .env.prod instead of .env.hyositive-dev and .env.hyositive-app respectively.
Using Environment Variables
The Cloud Functions runtime (the process that runs your deployed code) defines a number of built-in environment variables that have various purposes (such as allowing you to use initializeApp() with no arguments).
In addition to these variables, a handful of language-specific variables are also defined by the runtime to aid in painless deployment of code. However, the Cloud Functions documentation states to not rely on their values unless you set the values yourself.
The Node.js Cloud Functions runtime is built using the Cloud Functions Framework (however, it is not guaranteed to perfectly match this open source version). Because this runtime executes using Node.js and makes use of other packages such as express, it sets NODE_ENV to production, to minimise unexpected behaviour that depends on its value. But as mentioned above, this behaviour should not be relied on even though it is unlikely to change.
To override NODE_ENV to development, you would add it into .env.dev, .env.hyositive-dev and/or .env.local (as appropriate). Similarly, you should also define NODE_ENV as production in .env.prod or .env.hyositive-app (as appropriate).
Rather than rely on NODE_ENV, I would recommend defining behaviour around another variable that you have complete control over (such as HYOSITIVE_ENV) or compare against the executing project ID to determine whether it is running in the production project or not.
const PROD_PROJECT_ID = "hyositive-app",
// DEV_PROJECT_ID = "hyositive-dev",
PROJECT_ID = JSON.parse(process.env.FIREBASE_CONFIG).projectId,
isProduction = PROJECT_ID === PROD_PROJECT_ID;
Note: This other thread may have some useful background information.
In our Node.JS application, we have been specifying application environments (dev, qa, stage, prod, etc.) in package.json as below:
"start-prod": "NODE_ENV=prod node ./bin/www",
"start-qa": "NODE_ENV=qa node ./bin/www"
etc.
However, we have realized recently that using NODE_ENV in this manner to specify app environments is not a good practice. So, I am trying to use the npm package node-config
for this purpose as well as for loading environment-specific configuration which is currently coming from AWS Parameter Store since there seem multiple advantages of using overriding config files over repeated configuration in AWS Parameter Store.
Therefore, I have created different config.json files, e.g., dev.config.json, qa.config.json, etc. But I am not sure how to specify/read the environment (qa, dev, stage, prod, etc.) and load configuration from the corresponding config.json file.
It may be a very stupid question but I am not able to find a good implementation example for node-config. Any help will be highly appreciated.
As documented on the node-config wiki, you can create:
default.js
prod.js
dev.js
qa.js
I've been using K8S for a year or so and continue to revisit a problem.
My app is running in K8S and I now need to debug it. I have a NodeJS App that I'm asking about. But similar questions could be asked about Java SpringBoot apps (but this question is just for NodeJS).
I want to use my favorite IDE (IntelliJ or VSCode) to run the app but the app is currently getting it's configuration (inside K8S) using ConfigMaps and Secrets.
(Q) Is there a "best practice" or "pattern" that can be followed that supports the DRY principle and has configuration in one place that can be used for both K8S and when running locally.
Background
I have a NodeJS app that I decided to use ENVIRONMENT variables to hold configuration information because that worked well in IntelliJ IDE, in Docker and in K8S.
I used npm dotenv and created .env.local, .env.stage, .env.prod files to support running in different environments. This worked well enough until it was running in K8S and someone wanted to tweak the configuration and didn't believe that rebuilding the image was the best way to support this. Instead the K8S experts told me I should use ConfigMaps and Secrets, so I converted from the dotenv approach to use the K8S ConfigMaps and Secrets.
I kept the old .env files around just in case and I can use them but the source code no longer call uses dotenv package.
require('dotenv').config()
process.env.myConfigVariable
So I need to either add that code back to support debugging, or manually set the environment variables. I'm wondering if there is a better approach.
I have yaml files templates to make it easy to recreate the deployment from scratch if/when needed.
.env.local
deploy/
helm/
create-configmap.yaml
create-secret.yaml
src/
common/*
appMain.js
Some of the approaches I've considered:
(a) Accept it and have two configs (one for local and one for K8S). Leave the code for dotenv but don't deploy a .env file when deploying to K8S.
(b) Run local k8s (like minikube or k3s) and use my ConfigMap and Secrets as I would with K8S. I then need to figure out how to connect from my IDE to the local K3S environment and open ports in the k3s environment to support this. Some solutions include: Bridge to Kubernetes, YouTube Video Remote Debugging in Kubernetes with Cloud Code,
Debug Java Microservices in Kubernetes with IntelliJ, and I'm sure several others.
(c) Use a JSON config file instead of dotenv. For example, use a JSON config file for everything and map that to /app/config.json and that same config file can be used in both environments. I could have config-local.json, config-stage.json, and config-prod.json to support the different environments.
(d) You tell me. What's another way?
Thanks!
I'm currently having a look at JHipster.
What I could not figure out is how JHipster handles environments like dev, testing and production.
This includes:
server (configs like db or other app specific configurations)
client (Angular4, configs like api endpoint)
In angular-cli projects I'm used to environment specific configuration files (environment.dev.ts, environment.prod.ts) that can be defined at build (ng build --env prod) but since angular.cli does not seem to be the prefered way on JHipster there must be another solution to this.
So my questions is:
how to configure environments (server and client)?
how to specify the environment (server and client) during build?
If you are creating a standalone application or using the JHipster gateway bundle architecture, you can parameterize your Angular 2 app through webpack by using DefinePlugin. I detailed the steps to this process here: Environment-based properties for Angular 2 App Served by Webpack?
JHipster packages the client within the server as an executable jar so there is no need for different API endpoints between different environments on client side as long as you use /api.
On server side, JHipster uses Spring profiles which can be set dynamically and point to external configuration files (application*.yml files). By default JHipster comes with 2 main profiles dev and prod, but there are also others like swagger and no-liquibase that you can combine with dev or prod. You can define your own profiles too.
If your client app needs to access some external API endpoints (e.g. Google) that are different between environments, you can do it in 2 ways:
- retrieve the active profiles from server API using /api/profile-info and then initializes your endpoints in client code.
- create a new endpoint in server (e.g. /api/configuration) to fetch your client configuration
Then there are 2 kind of builds triggered by maven/gradle profiles: dev and prod.
For a newcomer, the most important thing is probably to understand Spring Profiles and application properties, these are not specific to JHipster.
I'm still new to web development and I'm using Firebase to handle all my data right now.
I have everything up and running, but how do I make it so my Firebase website updates whenever I make a change to my files? Do I have to manually call firebase deploy after each change in order to see the updated site?
To deploy your changes to the Firebase Hosting server, you will indeed have to run firebase deploy.
But normally when I develop an application, I run a local web server for the most part. I then only push the changes to Firebase Hosting when I have finished the feature/bugfix that I'm working on.
For local execution, I either use http-server or a gulp script that also packs the files. The latter have the advantage that they can watch your local files for changes and execute the correct steps based on that.
I'm working on a Angular 4 app with Firebase as a backend, so the steps are
$ ng build --prod
$ firebase deploy
It really depends on what you are doing and what you're trying to deploy.
There's three different areas to deploy to:
Hosting - this is just a simple web server in which to house your HTML, JavaScript and any other static files
Database - your Firebase access rules are placed in here
Storage - access rules to the file store, typically user submitted files
Typically you'll be developing your HTML and JavaScript files locally and testing them there. When you're ready to deploy to the hosting environment you'll typically deploy via firebase deploy, this will deploy all of the local files and rules to the Firebase servers.
If your question relates to just the database rules then there is no local version or instance of this, you need to deploy changes as you make them in order to make them active.
You can perform a rules update by issuing the command firebase deploy --only database. Just make sure you have a firebase.json file with "database": { "rules": "firebase.rules.json" }, or similar defined in it.
Bonus: Use BOLT to build the rules, it transpiles into a Firebase JSON rules file but makes development so much easier especially when your rules inevitably become more complicated.