My aws config doesn't work unless I set it externally through env variable
db connection works only if I set the credentials externally like,
export AWS_ACCESS_KEY_ID=abcde
export AWS_SECRET_ACCESS_KEY=abcde
export AWS_REGION=ap-south-1
export AWS_DYNAMODB_ENDPOINT="http://localhost:8000"
It doesn't work if I don't set these externally. For example if I set it in code like the following, it does not work.
dynamoose.AWS.config.update({
accessKeyId:'abcde',
secretAccessKey:'abcde',
region:'ap-south-1',
endpoint:'http://localhost:8000'
});
I don't want to set config in any variable externally. Is there a way to just manage this in nodejs code?
These are the alternates I have tried/considered
Setting env variable in the code, this doesn't work either
process.env.AWS_REGION='ap-south-1';
I read about dotenv package. But it is recommended that it should be used for dev only and not production (I am not sure if that will work)
Please help me resolve this. How do I manage the config in code only?
The problem is probably that you are creating or requiring your Dynamoose models before you run the dynamoose.AWS.config.update method.
Make sure that dynamoose.AWS.config.update is the very first method you call, and you haven't created or initialized any Dynamoose related things before.
For example.
const dynamoose = require('dynamoose');
dynamoose.AWS.config.update({
accessKeyId:'abcde',
secretAccessKey:'abcde',
region:'ap-south-1',
endpoint:'http://localhost:8000'
});
const Model = require('./models/MyModel'); // should happen after `dynamoose.AWS.config.update`
Another trick I would try to do is enable debug logging and go through the logs to see what is happening. You can enable Dynamoose logging by running export DEBUG=dynamoose*, then rerunning the script.
If you are working with newer version syntax is changed and could be find here .
https://dynamoosejs.com/guide/Dynamoose/#dynamooseawssdk
const sdk = dynamoose.aws.sdk; // require("aws-sdk");
sdk.config.update({
"accessKeyId": "AKID",
"secretAccessKey": "SECRET",
"region": "us-east-1"
});
Related
I am working on BE side of a project. I have many service bus URLs in my env and wanted to get them dynamically.
Env looks like this
SB1 = 'Endpoint=link1'
SB2 = 'Endpoint=link2'
sample code of mine
const serviceBus = "SB1";
const connectionString = process.env[serviceBus];
This logic is working fine when I run locally, but when I deploy my code. It is not working.
Please correct my approach or suggest me a better approach to do this.
Hi first make sure you have the .env file also uploaded on the server because many times by default project setting .env files are added to gitignore.
Also if you are using express.js then make sure that you make your .env file available by require('dotenv').config()
and install dotenv by command npm i dotenv
you can use it like process.env.serviceBus
As you may see, I have my db connection file and another "protected" file, where my credentials are, and this file is included in .gitignore. I import it and reach the data. Quite basic. Therefore my questions are:
Is this the right way to do it?
If not, how should I do it? Plus: how could I add extra security to my account,connection?
Let's suppose I have a private collection, that no one should see, how could I protect specially this collection? I mean, with a password or a two step verification let's say.
Current code:
const mongoose = require("mongoose");
const mongoCredentials = require("../protected/mongoCredential");
const URI = `mongodb+srv://${mongoCredentials.username}:${mongoCredential.password}
#firstcluster-eldi8.mongodb.net/culturapp?retryWrites=true&w=majority`;
mongoose.connect(URI, { useUnifiedTopology: true, useNewUrlParser: true })
.then(db => console.log("MongoDB is connected"))
.catch(err => console.log(">> ERROR: ",err));
module.exports = mongoose;
...I have my db connection file and another "protected" file, where my credentials are, and this file is included in .gitignore. I import it and reach the data..
The correct way to do it is to use envrironmental variables.
Use environmental variables
Environmental variables are set on the environment, i.e your local development machine or the remote production server.
Then, within your app, you read the environment variables and use them appropriately.
There's (at least) a couple reasons it's usually done like this:
The credentials don't exist in a file that can be read by someone viewing the repository contents. Someone cloning the repository doesn't need to know your database credentials.
The credentials are likely different between environments. You are likely using a different database on your local development machine and a different database in your remote production server.
Here's how you set environment variables (this is for Linux, other OS's might be different):
$ export MONGO_DB_USERNAME=foo
$ export MONGO_DB_PASSWORD=bar
and here's how you read them within Node.js:
console.log(process.env.MONGO_DB_USERNAME) // logs 'foo'
console.log(process.env.MONGO_DB_PASSWORD) // logs 'bar'
or pass variables to the process when starting up
Alternatively, you can pass variables when starting up the process like so:
$ MONGO_DB_USERNAME=foo MONGO_DB_PASSWORD=bar node app.js
However that's generally discouraged since you're most probably starting your process through the npm start script. Since package.json, where the npm start command is defined, is always committed to the repository it defeats the whole purpose of hiding the credentials.
Like you mentioned along lines, using environment variables is more like security through obfuscation.
I would try to have the credentials in a separate configuration file. With a bit of design, encrypt this file and store those keys in secure enclaves or TPMs.
Check this thread.
So I'm using cPanel with Setup Node.js App plugin for a Next.js app. (don't asky why cPanel)
Everything is working as expected in development, except for environment variables in production, I set up them manually from the cPanel interface, I restarted/stopped the app, logging the process.env on the server and I don't see the env variables there (not to say when trying to call them they are undefined).
When doing
res.json(JSON.stringify(process.env)); i get a bunch of variables except for the one I manually wrote in cPanel variables interface.
It is important for me to store these variables as secret key because they are API credentials.
Anyone know what I might have misconfigured or had this problem?
Never mind, found the answer, apparenlty was a Next.js misconfiguration. I had to add the following lines of code inside next.config.js in order to read env variables on build version.
require('dotenv').config();
module.exports = {
env: {
EMAIL_NAME: process.env.EMAIL_NAME,
EMAIL_PASSWORD: process.env.EMAIL_PASSWORD,
GETRESPONSE_API_KEY: process.env.GETRESPONSE_API_KEY
}
};
Where EMAIL_NAME, EMAIL_PASSWORD, GETRESPONSE_API_KEY were the variables defined by me on cPanel interface
As you may see, I have my db connection file and another "protected" file, where my credentials are, and this file is included in .gitignore. I import it and reach the data. Quite basic. Therefore my questions are:
Is this the right way to do it?
If not, how should I do it? Plus: how could I add extra security to my account,connection?
Let's suppose I have a private collection, that no one should see, how could I protect specially this collection? I mean, with a password or a two step verification let's say.
Current code:
const mongoose = require("mongoose");
const mongoCredentials = require("../protected/mongoCredential");
const URI = `mongodb+srv://${mongoCredentials.username}:${mongoCredential.password}
#firstcluster-eldi8.mongodb.net/culturapp?retryWrites=true&w=majority`;
mongoose.connect(URI, { useUnifiedTopology: true, useNewUrlParser: true })
.then(db => console.log("MongoDB is connected"))
.catch(err => console.log(">> ERROR: ",err));
module.exports = mongoose;
...I have my db connection file and another "protected" file, where my credentials are, and this file is included in .gitignore. I import it and reach the data..
The correct way to do it is to use envrironmental variables.
Use environmental variables
Environmental variables are set on the environment, i.e your local development machine or the remote production server.
Then, within your app, you read the environment variables and use them appropriately.
There's (at least) a couple reasons it's usually done like this:
The credentials don't exist in a file that can be read by someone viewing the repository contents. Someone cloning the repository doesn't need to know your database credentials.
The credentials are likely different between environments. You are likely using a different database on your local development machine and a different database in your remote production server.
Here's how you set environment variables (this is for Linux, other OS's might be different):
$ export MONGO_DB_USERNAME=foo
$ export MONGO_DB_PASSWORD=bar
and here's how you read them within Node.js:
console.log(process.env.MONGO_DB_USERNAME) // logs 'foo'
console.log(process.env.MONGO_DB_PASSWORD) // logs 'bar'
or pass variables to the process when starting up
Alternatively, you can pass variables when starting up the process like so:
$ MONGO_DB_USERNAME=foo MONGO_DB_PASSWORD=bar node app.js
However that's generally discouraged since you're most probably starting your process through the npm start script. Since package.json, where the npm start command is defined, is always committed to the repository it defeats the whole purpose of hiding the credentials.
Like you mentioned along lines, using environment variables is more like security through obfuscation.
I would try to have the credentials in a separate configuration file. With a bit of design, encrypt this file and store those keys in secure enclaves or TPMs.
Check this thread.
So i was migrating my app from node express to firebase-functions!
In my node-express app I have .env file which contains all the data, FOr starters let's consider this as my .env file
GOOGLE_CLIENT_ID = 4046108-bssbfjohpj94l0dhpu69vpgs1ne0.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET = lTQHpj3yY57oQpO
And then in my passport strategy, I have something like this
passport.use(new GoogleStrategy({
clientID: process.env.GOOGLE_CLIENT_ID,
clientSecret: process.env.GOOGLE_CLIENT_SECRET,
callbackURL: "/auth/google/callback",
userProfileURL: 'https://www.googleapis.com/oauth2/v3/userinfo',
accessType: 'offline',
passReqToCallback: true
},
Now,
Question: 1 - Firebase-functions probably don't support .env file, so can we set env variable without manually adding it using set flag? let's say I have lot variable in my environment
Question - 2: Can I access the variable I set by doing something like this
firebase functions:config:set slack.url=https://hooks.slack.com/services/XXX
using
process.env.slack.url
or we have to do (necessary)
functions.config().slack.url
Question:3 From Firebase Docs, it is written
There are environment variables that are automatically populated in
the functions runtime and in locally emulated functions, including:
process.env.GCLOUD_PROJECT: Provides the Firebase project ID
process.env.FIREBASE_CONFIG: Provides the following Firebase project
config info:
What do they mean when they mean? and if the answer to the question two is false then how are they using process.env.FIREBASE_CONFIG:
Answer for question 1:
Please note that, at this time, there doesn't appear to be a supported way to deploy true environment variables along with your function(s) using the firebase CLI tool. You can instead provide it as function.config() information, which is their recommended alternative to environment variables.
If you really really want to avoid function config, "pure" google cloud functions support setting environment variables. This page is a walkthrough of how to use a .env.yaml file and ultimately access those values from process.env in your code base. I know your file wasn't *.yaml, but this is a good solution (with some minor refactoring to YAML format).
The short version:
gcloud functions deploy FUNCTION_NAME --env-vars-file .env.yaml FLAGS
Answer for question 2
There is a difference between Firebase function config key/value pairs (i.e. function.config()) and process.env access. They are similar in purpose, but persist data in different places (i.e. in Firebase's services vs actual environment variables). Thus, the syntax for accessing each one is different.
Answer for question 3
Firebase has some process.env variables available to you by convention. They're simply documenting that for your convenience, so you can assume those are there and available as true environment variables (and not function.config() values).
Again, as of right now, the CLI tool doesn't seem to let you set true environment variables. So you'll have to go with function config, or do some other hack similar to this, which takes your function config key/values and sets them as environment variables at runtime.
const functions = require('firebase-functions');
const config = functions.config();
// Porting envs from firebase config
for (const key in config.envs){
process.env[key.toUpperCase()] = config.envs[key];
}
There's a better way as of Feb 16, 2022: Firebase now supports .env, .env.prod, .env.dev files natively!
Documentation: https://firebase.google.com/docs/functions/config-env
You can create your env files and then use firebase use dev or firebase use prod before you deploy.
These variables can be accessed via process.env.VARIABLE_NAME
Reminder
Unlike dotenv Firebase environment files should start with .env.
So, file.env will have to be named be .env.file for it to work in firebase functions.
I just had a need for this and came to a different solution than what was provided here.
First, there are a lot of discussions about why you should separate secrets from configuration. Firebase allows us to do that with the .env, .env.local, .env.dev, etc files. It is a best practice to store configuration here, not secrets.
Straight from their docs
Environment variables stored in .env files can be used for function configuration, but you should not consider them a secure way to store sensitive information such as database credentials or API keys. This is especially important if you check your .env files into source control.
What hasn't been discussed is how to store secrets. The values from the OP appear to be secrets and not configuration so in my opinion this is a relevant addition to this question.
Firebase functions have access to Cloud Secret Manager.
To help you store sensitive configuration information, Cloud Functions for Firebase integrates with Google Cloud Secret Manager. This encrypted service stores configuration values securely, while still allowing easy access from your functions when needed.
You can import runWith and defineSecret to define and access your secret.
import { runWith } from 'firebase-functions/v1';
import { defineSecret } from 'firebase-functions/params';
const secretApiKey = defineSecret('MY_API_KEY');
const functionName = runWith({ secrets: [secretApiKey] })
// ...
client.setApiKey(process.env.MY_API_KEY);
// ...
export default functionName;
This works with https functions, as well as auth, storage, and firebase triggers.
const onFirestore = runWith({ secrets: [secretApiKey] })
.firestore.document('/the/{docId}')
.onCreate(async (snap) => {
const onStorage = runWith({ secrets: [secretApiKey] })
.storage().bucket().object()
.onCreate(async (object) => {
Some note worthy caveats
The service account deploying the function needs the secret manager admin role.
Whenever you set a new value for a secret, you must redeploy all functions that reference that secret for them to pick up the latest value.
If you delete a secret, make sure that none of your deployed functions references that secret. Functions that use a secret value that has been deleted will fail silently.