Historically on most large projects I have worked on we have a build script of some sort which is run by a developer to setup their environment, such as setting up iis, database migrations, templating configuration files etc.
As each environment is different, I would generally have a set of configuration files for each environment, such as dev.config, qa.config or release-candidate.config, then when the build process runs it would use the <environment>.config file to find out what its settings should be. This way my dev environment can build fine and the build server can do its stuff fine too in an automated fashion.
Anyway in .net we have configuration files such as web.config which can be templated, is there any notion of this within the nodejs world? as I want to have a file which contains all the configuration values required for the application to run without putting them in the application code.
You can do that very simple, actually. Just place all your configuration in .js file, like
{
mongodb: {
connection: 'http://mongodb.com/mydb:12323/user/password',
options: {
keepConnection: true
}
},
app: {
deploymentKey: '1234'
},
// etc
}
So, you will have 2-3 files, depending on the number of environments you have.
/config/development.js
/config/production.js
/config/test.js
/config/staging.js
The environment type is typically exposed by NODE_ENV variable. Then you have really simple module, to load the configuration:
var util = require('util');
var env = process.env.NODE_ENV || 'development';
var config = util.format('/%s.config.js', env);
module.exports = require(__dirname + config);
Checkout some real code here;
This is why I created the properties module.
Check the environment example.
Related
I am trying to deploy my test Express.js app on Heroku using GitHub for resources and mlab for my database. In development th app doesn't have problems when I pass mLab connection string but in production... How must my production environment look?
Here is my config.js:
const env=require('dotenv').config();
module.exports = {
development: {
port: process.env.PORT|| 3000,
dbPath: process.env.DB_CONNECTION,
},
production: {
port: process.env.PORT|| 3000,
dbPath: process.env.DB_CONNECTION_MLAB,
}
};
Your .env file probably isn't (and shouldn't be) used in production. It should be ignored in your Git repository.
This means your production database configuration needs to come from somewhere else. If you're using the official mLab addon you can access your connection string via the MONGODB_URI environment variable, which the addon sets automatically.
If you're not using the official addon you should set the appropriate environment variable yourself, e.g. via
heroku config:set MONGODB_URI=...
In either case, make sure the name of the environment variable in your code matches what's set in the environment. Generally there is no need for separate development and production variables since they are set in different environments. I recommend using MONGODB_URI everywhere.
I am trying to move the tiny node-express app I made into firebase functions.
The file have dotenv variables. Earlier I thought If I just deploy and put dotenv in dependency, It will work but that didn't happen so..
So, I went to environment configuration article of firebase to understand how I can set .env
Which states to set things by doing something like this
firebase functions:config:set someservice.key="THE API KEY" someservice.id="THE CLIENT ID"
But I have so many environment configuration and doing that some what seems to be cumbersome task.
So let's say this is environment file
# App port Address
PORT = 8080
# Google Secret
GOOGLE_CALLBACK_URL = http://localhost:8080/auth/google/callback
GOOGLE_CLIENT_ID = 4048108-bssbfjohpu69vl6jhpgs1ne0.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET = lTQHpjzY57oQpO
# Client Address
CLIENT_ADDRESS = http://localhost:3000/
# Meetup Secret
MEETUP_CALLBACK_URL = http://localhost:8080/auth/meetup/callback
MEETUP_CLIENT_ID = ef6i9f7m6k0jp33m9olgt
MEETUP_CLIENT_SECRET = sk3t5lnss2sdl1kgnt
#EventBrite Secret
EVENTBRITE_CALLBACK_URL = http://localhost:8080/auth/eventbrite/callback
EVENTBRITE_CLIENT_ID = UU2FXKGYHJRNHLN
EVENTBRITE_CLIENT_SECRET = NA55QG52FAOF6GDMLKSJBKYOPIGQU4R46HHEU4
How Can I best set up so that when I do firebase firebase serve --only functions,hosting it doesn't throw any errors such as
OAuth2Strategy requires a clientID option
As of Feb 16, 2022 Firebase now supports .env, .env.prod, .env.dev, .env.local files natively!
https://firebase.google.com/docs/functions/config-env
Set your variables in the corresponding environment, and then run firebase use dev or firebase use prod before you deploy.
Your variables can be accessed via process.env.VARIABLE_NAME
UPDATED 2019-06-04
I'm very sorry. This solution is wrong.
I found the correct way.
https://stackoverflow.com/a/45064266/1872674
You should put a .runtimeconfig.json into the functions directory. Your dotenv variables move to .runtimeconfig.json with json format.
This is my solution.
const functionConfig = () => {
if (process.env.RUN_LOCALLY) {
const fs = require('fs');
return JSON.parse(fs.readFileSync('.env.json'));
} else {
return functions.config();
}
};
The functionConfig() was called by your Firebase Function.
exports.helloWorld = functions.https.onRequest((request, response) => {
response.send("someservice id is: " + functionConfig().someservice.id);
});
.env.json is like:
{
"someservice": {
"key":"THE API KEY",
"id":"THE CLIENT ID"
}
}
Finally, run the command with the RUN_LOCALLY variable.
RUN_LOCALLY=1 firebase serve
When we will deploy functions,
don't forget to update the environment configuration in Firebase using the .env.json.
The Firebase CLI currently doesn't allow you to set process environment variables on deployment. This may change in the future. The configuration vars it supports today (that you linked to) are not actually process environment variables - they stored somewhere else that's not actually the process environment.
If you absolutely need to be able to set process environment variables, you will have to deploy your function with gcloud, which means that you also won't be able to use the firebase-functions module to define your function. Start with the Google Cloud Functions documentation to learn about deployment from a Cloud perspective.
If you want to use the Firebase tools, I'd recommend that you find a different way to configure your function that doesn't involve process environment variables.
If you want to have your functions use the process.env variables, you you can set them by going to google cloud console and cloud functions. You will be able to find the deployed firebase functions there. You can select each function one by one and then set the environment variables there.
what I did was create a env.json file into functions FOLDER:
//env.json
{
"send_email_config": {
"FOW_ADMIN_EMAIL": "admin.example#gmail.com",
"FOW_ADMIN_EMAIL_PASSWORD": "adminPassExample",
"FOW_ADMIN_RECEIVER_EMAIL": "adminReceiver.example#gmail.com"
}
}
then I created a file called env.js where I created a function to return the value of functions.config() which is kind of env module
//env.js
// TO UPDATE functions.config().env RUN INSIDE functions FOLDER:
// firebase functions:config:set env="$(cat env.json)"
const functions = require('firebase-functions');
const getEnvConfig = () => {
/*
I return functions.config().env cause I set the env.json values into env
property running firebase functions:config:set env="$(cat env.json)"
*/
return functions.config().env
}
exports.getEnvConfig = getEnvConfig;
exports.PROCESS_ENV = getEnvConfig();
then I just call PROCESS_ENV to access the values that I set on env.json file for example:
const { PROCESS_ENV } = require('./utils/env');
exports.mailCredentials = {
main: {
email: PROCESS_ENV.send_email_config.FOW_ADMIN_EMAIL,
password: PROCESS_ENV.send_email_config.FOW_ADMIN_EMAIL_PASSWORD
},
receiver: {
email: PROCESS_ENV.send_email_config.FOW_ADMIN_RECEIVER_EMAIL
}
}
IMPORTANT!!!!
for this to work you have to deploy functions.config().env with the values of the env.json file
to deploy functions.config().env you just have to run INSIDE the functions FOLDER the next command: firebase functions:config:set env="$(cat env.json)"
and also don't forget to add env.json in your .gitignore
if you have firebase functions FOLDER inside your react project just add */env.json into your react .gitignore file
your .env file need to be in the "functions" folder
Here you have the documentation about this
https://firebase.google.com/docs/functions/config-env
you can access variables like this :
This is your .env :
PLANET=Earth
AUDIENCE=Humans
and a firebase functions
// Responds with "Hello Earth and Humans"
exports.hello = functions.https.onRequest((request, response) => {
response.send(`Hello ${process.env.PLANET} and ${process.env.AUDIENCE}`);
});
For your local environment you can have a .env.local file and the contents of .env.local take precedence over .env when you are using the emulators suite
I've been developing a nodejs app on C9 for some time and now I'm trying to make a copy of it on my remote host. So far, in the new environment node app.js works in the console but I am unable to view the website in my browser.
It seems that it is a port issue.
My app.js file goes like this :
var express = require("express"),
app = express();
(...)
app.listen(process.env.PORT, process.env.IP, function(){
console.log(process.env.PORT);
console.log("The YelpCamp Server Has Started!");
});
In the C9 environment, the log tells me that process.env.PORT is 8080. But in the new environment, the log tells me that process.env.PORT is undefined.
How can I fix this ?
This is similar to this older question except that my remote OS is Linux not Windows. The answer to this question says that one should "modify the web.config file", but I couldn't find it on my remote host and I'm not sure it works the same way under Linux and under Windows.
process.env variables are set by the "Environment". Basically, they act as access to your systems environment variables. When trying to access process.env.PORT it'll return undefined if you've not setup that environment variable in the shell that you're trying to run your system.
You can set the environment variable up before you run node app.js with the following.
$ PORT=8080
$ node app.js
In this case we set the environment variable within the existing shell, then we call node with the app.js file. Environment variables are passed from parent processes into processes they start, if you run those two in sequence, you'll be setting up an environment variable within the current shell, and node will then receive that when it starts up (along side all other environment variables).
To see all environment variables avalible to the existing node process you can run console.log(process.env);.
This will be the same for process.env.IP as well.
Side note: C9 and other similar environments often have a lot of pre-set environment variables. This is why it was available on C9. The same is true for Heroku as well, this is because their system must dictate the port your service should use so that their load balancers / reverse proxies can be pre-configured for that port.
I ran into something similar and ended up providing a default port number if one couldn't be read from the environment.
const PORT = process.env.PORT || 8080;
app.listen(PORT, ...)
After deploying your node project files, you'll need to make sure any env variables are also transferred.
1. Manually copy-paste env variables
From the original host config to the new host config. Use the dashboards on either end. Simple and secure.
--- or ---
2. Manually transfer .env file from project root
(Not recommended for production environments) Node projects often have a .env file in the root, which is commonly excluded from file transfers (for obvious security reasons). You'd need to make sure the new host also has the identical .env file in root. For production environments, this is usually not desired for security reasons, and it's better to use option #1.
Projects using this method are likely using the dotenv package which detects .env and makes the parameters available to your app. It's mainly for development convenience.
All the Node.js tutorials that I have followed have put everything in one file. It includes importing of libraries, routing, database connecting and starting of the server, by say, express.js:
var app = require('express');
app.get('/somePath', blah blah);
app.listen(...);
Now, I have 4 node servers behind an Nginx load balancer. It then becomes very difficult to have the source code updated on all the four servers.
Is there a way to keep the source code out of the server creation code in such a way that I can deploy the source code on the servers as one package? The server creation code should not know anything about routing or database connections. It should only be listening to changes in a folder and the moment a new module meta file appears, it starts hosting that web application.
Much like how we deploy a Java code packaged as war by Maven and deployed to the webapp of Tomcat, because Tomcat instantiation is not part of the source code. In node.js it seems server is also part of the source code.
For now, the packaging is not my concern. My concern is how to separate the logic and how do I point all my servers to one source code base?
Node.js or JavaScript for that matter doesn't have a concept like WAR. But what it does have is something similar. To achieve something WAR like, you would essentially bundle the code into one source file using something like webpack. However, this will probably not work with Node.js modules like http (Express uses `http since it likely calls or relies on native V8/C++ functions/libraries.
You could also use Docker and think of the Docker containers as WARs.
Here is what I figured out as a work around:
Keep the servers under a folder say, "server_clusters" and put different node servers there, namely: node1.js, node2.js, node3.js, node4.js, etc (I know, in the real world, the clusters would be different VMs or CPUs altogether but for now, I simply want to separate server creation logic from source code). These files would have this code snippet:
var constants = require('./prop');
var appBasePath = constants.APP_BASE_DIR;
var appFilePath = appBasePath + "/main";
var app = require(appFilePath);
//each server would have just different port number while everything else would remain constant
app.listen(8080, function (req, res) {
console.log("server started up");
});
Create a properties file that would have the path to the source code and export the object. That simple. This is what is used on line#1 in the above code
Create the source directory project wherever you want on the machine and just update its home directory in the constant file above. The source code directory can export one landing file that will provide the express app to the servers to start:
var express = require('express');
var app = express();
module.exports = app;
With this, there are multiple servers that are pointing to the same source code.
Hope this helps to those who are facing the same problem.
Other approaches are welcome.
Node-red documentation here gives info on how to embed a single node-red app inside a nodejs app - http://nodered.org/docs/embedding
We wanted our site's users to have their own node-red's on different ports for some custom programming. Is it possible to embed multiple node-red apps in a nodejs applicaiton?
I tried repeating same steps for embedding by changing settings of each call with different port but only one time it is created. First time, a node-red instance is created based on settings. Next time we call, we get port in use. I assume this has something to do with node require doing caching and all... Any workaround for this issue?
If you're interested I created a fork of the node-red project allowing this feature.
this is how you would initiate it:
var http = require('http');
var express = require("express");
var RED = require("node-red")();
var RED2 = require("node-red")();
// Create an Express app
var app = express();
// Add a simple route for static content served from 'public'
app.use("/",express.static("public"));
// Create a server
var server = http.createServer(app);
// Create the settings object - see default settings.js file for other options
var settings = {
httpAdminRoot:"/red1",
httpNodeRoot: "/api",
userDir:"./hhh",
functionGlobalContext: { } // enables global context
};
// Initialise the runtime with a server and settings
RED.init(server,settings);
console.log(RED2.settings === RED.settings, 888, RED2.settings.userSettings);
// Serve the editor UI from /red
app.use(settings.httpAdminRoot,RED.httpAdmin);
// Serve the http nodes UI from /api
app.use(settings.httpNodeRoot,RED.httpNode);
server.listen(8005);
// Start the runtime
RED.start();
var app2 = express();
app2.use("/",express.static("public"));
var server2 = http.createServer(app2);
var settings2 = {
httpAdminRoot:"/red2",
httpNodeRoot: "/api",
userDir:"./hhhh",
functionGlobalContext: { }
};
RED2.init(server2,settings2);
app2.use(settings2.httpAdminRoot,RED2.httpAdmin);
app2.use(settings2.httpNodeRoot,RED2.httpNode);
RED2.start();
server2.listen(8006);
console.log(RED.settings.httpAdminRoot);
console.log(RED2.settings.httpAdminRoot);
console.log(RED2.settings === RED.settings);
also, works on the same port. but make sure to use different paths is so.
https://github.com/aryeharmon/node-red
No, currently Node-RED has no multi-user capabilities and no way to instantiate multiple instances in one process.
You'll have to run separate instances of the application for each user. Have a look at something like FRED for an example of this. This runs individual instances and proxies them to make the integration look like it's all on the same port/domain