Change connection used on runtime - node.js

I hope you can help me (us). I'm working on a API project which have two databases :
Production DB : api.myapp.fr
Testing DB : test.api.myapp.fr
Theses two databases are writable by the user.
When a user call our API, he can set the authorization header whichever he needs. For example :
Authorization: s_0
Will perform operations on api.myapp.fr and
Authorization: s_t_0
Will perform operations on test.api.myapp.fr .
My question is : How can I do that with sails ?
Actually, I have a policie which check if the user is using a production key or a testing key, and I override the default models with the one for testings purposes, like this :
if (!is_production) {
req.session.isProd = false;
req.session.logs.environment = "test";
User = UserTest;
Payment = PaymentTest;
PayzenStatus = PayzenStatusTest;
Transaction = TransactionTest;
Card = CardTest;
Doc = DocTest;
}
But you can see the problem if a user makes a test request and then a production request, the models are still the tests ones...
I use my models in services and policies, therefor I can't do
req.models = {};
// If not in production, use the test models
if (!is_production) {
req.session.isProd = false;
req.session.logs.environment = "test";
req.models.User = UserTest;
req.models.Payment = PaymentTest;
req.models.PayzenStatus = PayzenStatusTest;
req.models.Transaction = TransactionTest;
req.models.Card = CardTest;
req.models.Doc = DocTest;
}
// Otherwise use the production models
else {
req.models.User = User;
req.models.Payment = Payment;
req.models.PayzenStatus = PayzenStatus;
req.models.Transaction = Transaction;
req.models.Card = Card;
req.models.Doc = Doc;
}
If you have any idea on how ton achieve this (whatever the way, we can still perform deep changes in our code), I would be really happy to ear it.
Thanks

Two different ways of doing this.
First you could set an environment variable on your production host and check that environment variable to see if you are running in prod. If you are running in prod then use the URI to the production database.
Secodonly, which is probably the better way of doing this creating a config.js file that allows you to read environment variables. What I do for all my apps is set environment variables for connection info to databases and API keys. When running locally/testing I have some defaults in my app but when they environment variables are set they are read and used. So set the environment variable in production to point to your production databases.
The config.js file Im posting below contains references to VCAP, which assumes you are running on Cloud Foundry.
config.js
var VCAP_SERVICES = process.env["VCAP_SERVICES"],
vcapServices;
if (VCAP_SERVICES) {
vcapServices = JSON.parse(VCAP_SERVICES);
}
function getEnv(propName, defaultValue) {
if (process.env[propName]) {
return process.env[propName];
} else {
return defaultValue;
}
}
module.exports = function() {
return {
getEnv : getEnv,
couchDbURL: function() {
// Default to a local couch installation for development
if (VCAP_SERVICES) {
return vcapServices["cloudantNoSQLDB"][0].credentials.url;
}
else {
return "http://localhost:5984";
}
},
couchDbName: function() {
return getEnv("COUCHDB_NAME", "mydb");
}
};
};
app.js
var config = require("./config")();
console.log(config.couchDbURL());

Related

Ensuring Azure keyvault secrets are loaded to config (node-config) at application startup

I have a NodeJS application that uses Node-Config (https://www.npmjs.com/package/config) to load application configurations. What I'm trying to do is to load secrets from Azure Keyvault to the config during startup, and ensure these are available before required (e.g. connecting to databases etc).
I have no problem connecting to and retrieving values from the Keyvault, but I am struggling with the non-blocking nature of JS. The application startup process is continuing before the config values have completed loaded (asynchronously) to the config.
One strategy could be to delay application launch to await the keyvault secrets loading How to await in the main during start up in node?
Another would be to not load them in Config but instead modify code where-ever secrets are used to load these asynchronously via promises
It seems like this will be a common problem, so I am hoping someone here can provide examples or a design pattern of the best way of ensuring remote keyvault secrets are loaded during startup.
Thanks in advance for suggestions.
Rod
I have now successfully resolved this question.
A key point to note is setting process.env['ALLOW_CONFIG_MUTATIONS']=true;
Configs are immutable by default (they can't be changed after initial setting). Since async is going to resolve these later, it's critical that you adjust this setting. Otherwise you will see asynchronous configs obtaining correct values from the keystore, but when you check with config.get they will not have been set. This really should be added to the documentation at https://github.com/node-config/node-config/wiki/Asynchronous-Configurations
My solution: first, let's create a module for the Azure keystore client - azure-keyvault.mjs :
import { DefaultAzureCredential } from '#azure/identity';
import { SecretClient } from '#azure/keyvault-secrets';
// https://learn.microsoft.com/en-us/azure/developer/javascript/how-to/with-web-app/use-secret-environment-variables
if (
!process.env.AZURE_TENANT_ID ||
!process.env.AZURE_CLIENT_ID ||
!process.env.AZURE_CLIENT_SECRET ||
!process.env.KEY_VAULT_NAME
) {
throw Error('azure-keyvault - required environment vars not configured');
}
const credential = new DefaultAzureCredential();
// Build the URL to reach your key vault
const url = `https://${process.env.KEY_VAULT_NAME}.vault.azure.net`;
// Create client to connect to service
const client = new SecretClient(url, credential);
export default client;
In the config (using #node-config) files:
process.env['ALLOW_CONFIG_MUTATIONS']=true;
const asyncConfig = require('config/async').asyncConfig;
const defer = require('config/defer').deferConfig;
const debug = require('debug')('app:config:default');
// example usage debug(`\`CASSANDRA_HOSTS\` environment variable is ${databaseHosts}`);
async function getSecret(secretName) {
const client = await (await (import('../azure/azure-keyvault.mjs'))).default;
const secret = await client.getSecret(secretName);
// dev: debug(`Get Async config: ${secretName} : ${secret.value}`);
return secret.value
}
module.exports = {
//note: defer just calculates this config at the end of config generation
isProduction: defer(cfg => cfg.env === 'production'),
database: {
// use asyncConfig to obtain promise for secret
username: asyncConfig(getSecret('DATABASE-USERNAME')),
password: asyncConfig(getSecret('DATABASE-PASSWORD'))
},
...
}
Finally modify application startup to resolve the async conferences BEFORE config.get is called
server.js
const { resolveAsyncConfigs } = require('config/async');
const config = require('config');
const P = require('bluebird');
...
function initServer() {
return resolveAsyncConfigs(config).then(() => {
// if you want to confirm the async configs have loaded
// try outputting one of them to the console at this point
console.log('db username: ' + config.get("database.username"));
// now proceed with any operations that will require configs
const client = require('./init/database.js');
// continue with bootstrapping (whatever you code is)
// in our case let's proceed once the db is ready
return client.promiseToBeReady().then(function () {
return new P.Promise(_pBootstrap);
});
});
}
I hope this helps others wishing to use config/async with remote keystores such as Azure. Comments or improvements on above welcome.
~ Rod

nodejs server cannot export service

Im currently running a nodejs server. The situation is this, I have a services folder, inside the folder, there are many service.js files. In each service, I will export the service and its functions.
eg.
var service = {};
service.fun1 = fun1;
service.fun2 = fun2;
...
module.exports = service;
function fun1(params) { ... };
Now, I have to require one service from another. For example, whenever I need to use the user service from another service, I will do
var userService = require('services/user.service');
And then I can use it as
userService.fun1(params).then(function(data){ ... });
I have a coupon service that need to require the user service. In the coupon service I can require all other service, except for the user service. And I got this error
TypeError: userService.getById is not a function
So I try to console log the userService, it is {}, which means it is not exported somehow. But from other services , I can require user service in the same way and it works fine.
So I try to require other service in the coupon service and all are working. So only the user service in coupon service is giving the error.
This is making me crazy.
------update -------
The code the throws the error is from coupon_service.js:
var userService = require('services/user.service');
function getAllCoupons(params) {
var deferred = Q.defer();
if (params.field1 && params.field2 && params.field3 && params.start_index && params.page_length) {
userService.fun1(params.field2).then(function(data) {
if (data.response1 == params.expected1 && helper.userIsAdmin(params.response1, data.validate)) {
deferred.resolve(getAll(params.field2, params.start_index, params.page_length, params.filter));
} else {
deferred.resolve(getCoupons(params));
}
});
} else {
deferred.resolve(getCoupons(params));
}
return deferred.promise;
}
Assuming both services are in the same /services/ folder, you should just be able to do:
var userService = require('./user.service');
since the path for the imported service should be relative to the importing service.

Web app running on Azure is not picking up and using an environment variable connection string (ASP,NET Core RC1)

Github link for reproduction.
I have an ASP.NET Core (RC1) application that works fine locally. The issue I'm having is that my connection string is not being picked up by my Azure app. I've asked similar questions to this, but I've narrowed down the issue on my end in this app. Note, it requires an app on Azure to reproduce it.
Here's the issue I'm seeing.
First, my configuration is setup as such:
public Startup()
{
var builder = new ConfigurationBuilder()
.AddJsonFile("config.json")
.AddEnvironmentVariables();
mConfiguration = builder.Build();
}
And EF7 is setup here:
public void ConfigureServices(IServiceCollection services)
{
services.AddEntityFramework()
.AddSqlServer()
.AddDbContext<FooDbContext>(options =>
{
// I'm assuming it's failing here.
// I'm not sure how to debug it running on Azure.
// All the developer exception page shows is:
// 500 Internal Server Error "An error occurred while starting the application."
options.UseSqlServer(mConfiguration["Data:ConnectionStringTest:ConnectionString"]);
});
services.AddScoped<IFooDataService, FooSqlDataService>();
}
My config.json has:
{
"Data": {
"ConnectionStringTest": {
"ConnectionString": "Data Source=(localdb)\\mssqllocaldb;Initial Catalog=ConnectionStringTest"
}
}
}
And this should be overridden by the connection string I've setup in Azure:
When going to the Kudu SCM and looking at the environment variables on the Azure web app instance, I see the following:
SQLAZURECONNSTR_Data:ConnectionStringTest:ConnectionString = my_connection_string_here
I am assuming this is the class that is being used under the hood when my environment variable is used at runtime: EnvironmentVariablesConfigurationProvider
Ok here's what I found, and this feels awkward.
It seems that you need to use Data:{my_connection_string_key}:ConnectionString everywhere EXCEPT in Azure. This environment variable converter will construct the proper connection string using this format automatically if the connection string is prefixed with SQLAZURECONNSTR_.
This means when you setup your connection string in Azure, you need to omit EVERYTHING except the key to your connection string. Do not insert Data: or :ConnectionString... simply use {connection_string_key} (refer to the above format) instead. If you include the entire format in your Azure key/value pair, the EnvironmentVariablesConfigurationProvider will add another Data: and :ConnectionString around it, resulting in something like Data:Data:{my_connection_string_key}:ConnectionString:ConnectionString.
In ConfigureServices(...), use the format that ASP expects:
... options.UseSqlServer(mConfiguration["Data:ConnectionStringTest:ConnectionString"]);
You can therefore locally use this for your local JSON, for testing in development/fallback:
{
"Data": {
"ConnectionStringTest": {
"ConnectionString": "Data Source=(localdb)\\mssqllocaldb;Initial Catalog=ConnectionStringTest"
}
}
}
Just make sure your Azure connection string has the middle part of that format (ConnectionStringTest using this example).
This will make your environment variable in Azure look like this in raw format:
SQLAZURECONNSTR_ConnectionStringTest = {insert connection string here}
And the EnvironmentVariablesConfigurationProvider will strip off the Azure prefix string, and wrap your key in the hardcoded format: Data:{0}:ConnectionString
Experiment Results
To augment your excellent answer, I did a local experiment to confirm that the SQLAZURECONNSTR_connection_string_key environmental variable becomes this configuration:
mConfiguration["Data:connection_string_key:ConnectionString"]
Local Experiment
A local environmental variable emulates an Azure SQL Database connection string named connection_string_key.
PS> $env:SQLAZURECONNSTR_connection_string_key = "an azure conn string"
The following code dumps all the environmental variables and configuration sections to the page.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
app.Run(async (context) =>
{
await context.Response.WriteAsync("# Environmental Variables \r\n");
await DumpAllEnvVariables(context, Environment.GetEnvironmentVariables());
await context.Response.WriteAsync("# Configuration Sections \r\n");
await DumpAllConfigItems(context, mConfiguration.GetChildren());
});
}
private async Task DumpAllEnvVariables(HttpContext context, IDictionary envVariables)
{
foreach (var envVar in envVariables.Cast<DictionaryEntry>())
{
await context.Response.WriteAsync($"{envVar.Key}"); // : {envVar.Value}
await context.Response.WriteAsync($"\r\n");
}
}
private async Task DumpAllConfigItems(HttpContext context,
IEnumerable<IConfigurationSection> sections, string prefix = "")
{
foreach (var section in sections)
{
await context.Response.WriteAsync($"{prefix}{section.Key}"); // : {envVar.Value}
await context.Response.WriteAsync($"\r\n");
if(section.GetChildren().Any())
{
await DumpAllConfigItems(context, section.GetChildren(), prefix + " ");
}
}
}

Read environment neutral Configuration Value from NodeJS

I have an env. neutral configuration which is EnableEmailCheck. If the EnableEmailCheck= true then the email validation will takes place and if it is set to false application will not do the email validation.
I need to understand how to do something like that in nodejs
What format the configuration file should be ? I think I can keep the key value pair it in .json file
How can I read this configuration value from nodejs
If the value of the configuration file is changed is it required to restart the nodejs app server to reflect the changes
This configuration value will get changed if there is an issue in the emailvalidation functionality. Then the value will be switched off; in this case it will be set to false by and application engineer.
I would save it in json format and add a watch on it:
config.json content:
{
"env":
{
"EnableEmailCheck": true;
}
}
Loading config.json file at start and when it changes:
var config = null;
var fs = require('fs');
function loadConfig(fileName )
{
var data = fs.readFileSync( fileName );
return JSON.parse( data );
}
function useConfig( )
{
if( config.env.EnableEmailCheck )
{
// do your stuff;
}
}
//load it at start
config = loadConfig("config.json");
useConfig();
//add watch for changes
fs.watchFile("config.json", function ()
{
config = loadConfig("config.json");
useConfig();
});

Overriding delete operation on Azure Mobile Services table

I'd like to override delete operation on my Azure Mobile Services table to make it more like update then real delete. I have additional column named IsDeleted and I'd like to set it's value to true when delete operation is executed.
I figured out, that what I need is:
fire my own 'update' inside del function,
delete current request.execute()
prepare and sent response by myself
That meens my del function should look like that:
function del(id, user, request) {
// execute update query to set 'isDeleted' - true
// return standard response
request.respond();
}
As you can see I'm missing the first part of the function - the update one. Could you help me writing it? I read Mobile Services server script reference but there is no info about making additional queries inside a server script function.
There are basically two ways to do that - using the tables object, and using the mssql object. The links point to the appropriate reference.
Using mssql (I didn't try it, you may need to update your SQL statement):
function del(id, user, request) {
var sql = 'UPDATE <yourTableName> SET isDeleted = true WHERE id = ?';
mssql.query(sql, [id], {
success: function() {
request.respond(statusCodes.OK);
}
});
}
Using tables (again, only tested in notepad):
function del(id, user, request) {
var table = tables.getTable('YourTableName');
table.where({ id: id }).read({
success: function(items) {
if (items.length === 0) {
request.respond(statusCodes.NOT_FOUND);
} else {
var item = items[0];
item.isDeleted = true;
table.update(item, {
success: function() {
request.respond(statusCodes.OK, item);
}
});
}
}
});
}
There is a Node.js driver for SQL Server that you might want to check out.
The script component of Mobile Services uses node.js. You might want to check out the session from AzureConf called Javascript, meet cloud

Resources