node and github secrets. how to test locally - node.js

i'm using node and github actions for a simple twitter api script.
twitter credentials are stored in github secrets
other than hard coding the twitter credentials and then changing the code to use secrets before pushing to github how do i test locally?
I need some way to retrieve the credentials locally if running locally and from secrets if running on github.
what is best practice way to do this please?

Essentially, what David is trying to say is that you'll need to store your credentials in two separate places:
Inside GitHub Actions Secrets
Your local filesystem inside a gitignored .env file
GitHub Actions Secrets
GitHub Actions Secrets uses encryption to store the secret value, and this information is only decrypted while the workflow is running. Note that the secrets are not stored in the codebase. Instead, this is managed entirely via the GitHub web interface, and perhaps the GitHub API. I do want to emphasise again that none of this is stored inside your git repository as that would essentially be considered a security leak.
Now, this works fine when running in GitHub Actions, but the question you have is how do you run locally. This is where the .env file comes into play.
Running Locally with .env and dotenv
Create the following file, and add the same value you used as the secret in GitHub Actions secrets inside the .env file as a key/value pair. Let's assume the secret name is MY_SECRET_CRED and the value is abcdefg:
.env:
MY_SECRET_CRED=abcdefg
Next, add this file to .gitignore. You do not want to commit this, because it would violate your security, and you also don't need it to be committed, since GitHub has stored the credential as a secret. You only need this file locally.
.gitignore:
.env
The purpose of adding the file to gitignore is because you do not want to commit this file. This file is local to your workstation only.
Next, install the dotenv npm module:
$ npm i dotenv
In your main file, be sure to require the dotenv module. This module reads the contents of .env and loads the key value pairs up as environment variables:
require('dotenv').config()
Tying it all together
In your application code, you're probably already using the secret in GitHub Actions as some kind of environment variable. If running in GitHub Actions, you can load the secret as an environment variable with the same name you used in the .env file:
- name: Test application
env:
MY_SECRET_CRED: ${{ secrets.MY_SECRET_CRED }}
run: npm run test
In the above example, the Node.js process can access the secret as the environment variable MY_SECRET_CRED
doSomethingWithSecret(process.env.MY_SECRET_CRED);
But do note that, if you try to print it, GitHub will mask it, since it knows it's a secret that shouldn't be leaked.
When running locally, the environment variable is loaded into the Node.js process via the dotenv module.
Downside to .env files
The downside to .env files is that the secret is stored in plain text in your workstation, and it's difficult to share with other developers. Ideally, we don't want to share secrets through chat systems or email for the same reasons we don't want to commit and push them.
One solution is to use a secret manager or key management store to encrypt secrets locally. Your local application then decrypts them when running the application. This also means you can commit the encrypted data directly to the GitHub repository. One example of this is Google KMS.

Related

React NPM Build and Environment Variables

When deploying a ReactJS + NodeJS application, what do you do about environment variables used on the React-side? Running NPM BUILD folds any secret API keys into the final code, making it visible to anyone who inspects the source code. Moving the .evn information to a .env file located server-side before using the npm build command doesn't solve the issue. I've tried to search around but I can't find any actual solutions.
Thanks!
You cannot make secret keys invisible on the client side. Even receiving it as an environment variable from the server, the keys will be incorporated into the build and will be publicly accessible.
So that your React app is not vulnerable, (correcting: not the only way) a possibility is to save keys and secrets in the backend. That is, the backend is the only one with access to keys and secrets. Client makes requests without storing sensitive data.
Thanks folks, all extremely helpful! I'll be moving the secret keys to the server side and changing the way my code works to compensate the adjustment. The other issue was Google Maps API, which can't be moved in the code but can be restricted to specific IP or URL use in the Google Cloud Platform.
https://cloud.google.com/blog/products/maps-platform/google-maps-platform-best-practices-restricting-api-keys

Why and how to put secrets in environment variables in Node.js?

I'm relatively new to Node.js and Express and I was following a guide on adding authentication using JWT to my site found here: Medium. I'm confused by one point of this guide, about using secrets, that says:
This secret will be read by JWT library while creating and validating tokens. In production, we need to store this secret in environment variable instead of a file.
In the guide, they placed the secret inside a config file. I wish the guide described why exactly the JWT secret should be an environment variable instead of in a config file, but it's pretty vague. So why should secrets be environment variables, and what are other best practices for using secrets? I've decided to use Passport & Express Sessions for authentication instead of JWT, but I'd think it still applies to session secrets.
My next question is how would I exactly set a secret in an environment variable? I'm not using the guide anymore, but how would somebody that is using it convert that config file into an environment variable? And lastly, what do you usually use environment variables for in a typical Node.js app?
Why should a secret be an environment variable, instead of being stored in a config file?
While working on projects, you will end up uploading your code on github, which is accessible to everyone. If you store your secrets in a config file, anybody with a github account will be able to read it, and hence it is a security risk. Storing the secrets as environment variables ensure their safe-keeping. Your config file should read these values from the process.env object. Something like this:
const jwtSecretKey = process.env.JWT_SECRET_KEY;
const googleApiKey = process.env.GOOGLE_API_KEY;
const serverPort = process.env.PORT || 8000; // 8000 is the default value in case if the env variable has not been set
module.exports = {
jwtSecretKey: jwtSectetKey,
googleApiKey: googleApiKey,
serverPort: serverPort
}
And any other piece of code using these secrets should require the config file.
const config = require('./config');
...
jwt.verify(token, config.jwtSecretKey);
Along with storing secrets as environment variables, you should also store any environment specific values as environment variables. For example, say your NodeJS server (thats is connected to a hosted Database) is run on three environments - Development, QA and PROD. Each of these environments will have different information regarding the DB HOST and PORT, the environment should connect to. Environment Variables are a good way to store the DB Host and Port on each individual environment, and to use the same code across all environments to connect to different databases by reading the environment variable.
How to store the values as environment variables - One way of doing it is manually, though I wouldn't recommend it. One way is use of shell scripts. It actually depends upon your infrastructure.
You should also add your *.env to the .gitignore file So pushing it to the public will ignore the file and you have only one file with all the environment variables.
Maybe the original question was more about why you should not hard-code secrets into your codebase? Hopefully that is fairly obvious to most people. So ideally your secret and non secret "app config" should live separate from your codebase making your app "configurable". The standard way to do this:
have a development config file (e.g. config.js or .env) that is added to .gitignore so it is never checked in.
have a development sample config file (e.g. config.sample.js or sample.env) for new developers to use as a template for their own development configs (this is checked in and does not contain real secrets).
in production deployment this config file is generated or provisioned with production config and secrets by the devops team.
the path to the config file (location of file is not a secret) might be specified by an environment variable for the app process.
If you are concerned about security you won't put secrets into ENV.
(Yes, I know this is a controversial topic for some people. Sorry.)
You can't really prevent unprivileged users from getting access to ENV information. So if you put secrets in env, the bar is lowered for an attacker because they don't necessarily need a privileged account to see your app's secrets running under a different user.
if your app spawns any subprocesses, your secrets will be exposed to them because they inherit your environment.
if you have debug process logging at a system level, process ENV can be exposed to unprivileged users because the system doesn't consider ENV to be secret.
If you are provisioning a production server, you can just as easily put your secrets in a .env file as a systemd EnvironmentFile using the familiar environment variable format.
However the trick is you want to treat this as secret app-loaded config rather than loading it into process environment (via systemd, bash, npm scripts, etc) or process.env within your app.
dotenv by default (when you call .config()) populates process.env. so if you want to use dotenv securely, you can parse your .env config using dotenv.parse so it doesn't put the secrets from your .env into process.env.
This would allow you to provision a .env file with production secrets and use a development .env file that you add to .gitignore so you don't commit your development file. However application developers need to remember to use .parse instead of .config or else it is a security risk.
Alternatively if you don't want to use dotenv since its primary design is focused on loading up ENV... just use a json or js config file that exports a config object (e.g. config.js). Same as .env you can have a dev version that is not committed and a production one that is auto-provisioned. This can be required within the app where needed and avoids the risk (you have with dotenv) of accidentally populating secrets into the environment. Of course bad app code can always still leak secrets, but at least they won't be leaked by default due to how the secrets are exposed to the app.
The downside of all this is that ENV gave us a very standard way to pass config to any type of app/process/script (ie not just nodejs) in production environments. ENV being insecure means we have to come up with a (ideally standard) mechanism for passing secure config to a variety of apps and scripts in production.
In bash (as an example) you can source a .env file and as long as those vars aren't exported they are just "local" variables to the current bash process (ie not inherited by sub-processes). Not using export is akin to not populating process.env. Parsing json config in simple bash scripts...not so fun without helper apps.
You would like application developers to not have to think about production secrets/config or security and give them a standard config retrieval mechanism that works for local development, testing and production.
You would like this standard mechanism to be straight forward and relatively low-risk of them inadvertently opening up security issues on the app side (exposing production secrets to sub-processes). Of course nothing protects you 100% as if the app has access to the secret there are many ways it could leak it (writing to file, making http calls, etc)...but at least it wouldn't be due to the config loading/passing mechanism.
We are currently looking at options for addressing this for a variety of apps in production...so very happy if someone has well thought out suggestions or experience solving this in a clean, general purpose way.
I would be inclined to stick with the .env because we already support EnvironmentFile provisioning and every app can deal with that file format (e.g. can source a .env with bash)...the problem is how to prevent apps/developers from accidentally using dotenv.config instead of dotenv.parse to keep this out of the app environment.
So we are probably just sticking with config.sample.{env,json,js} so the devlops team has a template to populate with production secrets. Config is loaded by the node app via require so it doesn't populate process.env. We also are adding command-line flag (or ENV) to our apps to allow the location of this config file to be specified so devops has the option to place the production config file outside of the app deploy directory (which can make continuous deployment a bit easier). The downside is that a config.js won't work easily for other languages, but devops can fairly easily script converting an ENV style variable format into other targets.
As I said, keeping secrets out of your ENV is for people that are concerned about security. Many blogs/guides/tutorials online show secret passing through ENV. The popularity of dotenv which (by default) loads secrets into the process environment also might lead you to believe this is the right way. Security is always a tradeoff between risk, effort/cost and inconvenience. Personally I think if you auto-provision production it isn't a big deal to fix this...the issue is more about coming up with a semi-standard mechanism that can work across a variety of apps/code-languages.
References:
https://systemd-devel.freedesktop.narkive.com/ibRtoqPS/environment-variable-security
https://github.com/motdotla/dotenv#parse
https://github.com/motdotla/dotenv/pull/378

In Travis Public Repository how to add a Secure variable that works on Pull requests too

I have Travis-ci on a public repository. After finishing the execution it generates a image that I want to upload to cloudinary.com, but it could be any other service.
The problem is that to do it, I need to add in .travis.yml the auth token. But I don't want to expose it publicly, and for that travis offers a way to secure Env variables: http://docs.travis-ci.com/user/environment-variables/#Secure-Variables. However they do not work on PULL requests:
Secure Env variables are not available on pull requests from forks due
to security risk of exposing such information to unknown code.
Encryption and decryption keys are tied to the repository. If you fork
a project and add it to Travis CI, it will have different keys to the
original.
Anyone has any idea about how could I add an hidden value that is available for PUSH and PULL REQUESTS?
As you already wrote in your question: according to the official Travis CI documentation https://docs.travis-ci.com/user/environment-variables you won't have access to these variables from untrusted builds such as pull requests. This makes sense, since someone could submit a pull request to your repository containing malicious code which then exposes your secret value.
Bottom line: if you want to make secret values available to pull requests, you have to assume they're not secret anymore - therefore you could also just hard code the unencrypted value to your .travis.yml and use it from there. Which doesn't seem like a good idea. ;-)
Possible solution in your case: you could just use an image hoster which provides anonymous uploading? You wouldn't need an auth key, so your pull requests would be able to upload, too.

Best practice for managing web service credentials for Node.JS?

We're planning a secure Node.JS server, which uses several third-party web services. Each requires credentials that will need to be configured by the operations team.
Clearly they could simply put them in plain text in a configuration file.
Microsoft .NET seems to offer a better option with DPAPI (Data Protection API) - see Credential storage best practices. Is there a way to make this available through IISNode? Or is there any other option to secure such credentials within Node-JS configuration?
There's an extensive discussion of several options here, including the two suggested by xShirase:
http://pmuellr.blogspot.co.uk/2014/09/keeping-secrets-secret.html
User-defined services solves the problem, but only for Cloud Foundry.
This blog http://encosia.com/using-nconf-and-azure-to-avoid-leaking-secrets-on-github/ points out that you can often set environment variables separately on servers, and suggests using nconf to read them and config files separately.
I still wonder if there are specials for IIS?
There is 2 ways to do it securely :
First one is to use command line parameters when you launch your app.
These parameters are then found in process.argv
So, node myapp.js username password would give you :
process.argv[0]=node
process.argv[1]=/.../myapp.js (absolute path)
process.argv[2]=username
process.argv[3]=password
Second is to set the credentials as ENV variables. It is generally considered as the best practice as only you have access to these variables.
You would have to set the variables using the export command, than you'd access it in process.env
I currently had to do the exact same thing for my External API credentials. this is what i did
install node-config module
create a folder and file called config/config.js
here require(config) module
In local box it reads the configuation from local.json file
i have dummy values in local.json for api key and shared secret
on my QA environment i export two variables NODE_ENV="QA" and NODE_CONFIG_DIR="path to my configuation folder on qa server"
node-config module reads configuation from "path to your config folder / QA.json"
now i have real api key and credential in QA.json
here you can use an encryption to encrypt these values and put it back in QA.json
in your app get these config values and decrypt use it in your rest call
hope this helps.
so your config can live in the same container as node code.
refer to this for encryption and decryption
http://lollyrock.com/articles/nodejs-encryption/

where to store admin password in sinatra + heroku app?

I have a small Sinatra app I'm running on Heroku that uses a single admin password, plus a couple API authentication keys.
Where's the best place to store these things? Do I put them in environment variables, and use
heroku config:add ADMIN_PASSWORD=foobar
? Or do I use a config file that contains them, and I simply don't commit the config file?
I stick API keys and that sort of thing in a config yaml, like so
development:
twitter_api_key: stringstringstring
chunky: bacon
production:
twitter_api_key: gnirtsgnirtsgnirts
foo: bar
then use Sinatra's builtin set to handle the data.
configure do
yaml = YAML.load_file(settings.config + "/config.yaml")[settings.environment.to_s]
yaml.each_pair do |key, value|
set(key.to_sym, value)
end
end
And I can then access them from the settings object. I'm not sure why you wouldn't commit the config file, though . . . there's no major security risk here, since only those paths that you've explicitly defined can be accessed via the web. I guess the admin password could be stored in the same manner if you don't want to put it in a database, but I would at least encrypt it with a salt.
Just be careful not to step on Sinatra's Configuration settings when defining your own.
EDIT:
I think I just realized why you would prefer not to commit the config file. If you're working on an open source project, you certainly wouldn't want to commit the config file to your open source repo, but you would need to commit the file to Heroku in order for it to work. If this is the case, I'd either:
Use two separate local repos: one for the open source project, and one for the heroku project. Just set the open source project as an upstream repository in the Heroku project, then you can fetch changes.
Put both the API keys and encrypted/salted password in a database; MongoHQ offers a free tier to Heroku users as an addon for simple nosql storage using MongoDB.

Resources