My Google App Engine application includes privileged credentials (like OAuth and cookie signing keys) that I don't want to check in to source control (i.e. Git).
I've considered storing these as system properties or environment variables in the appengine-web.xml file, however this file includes a number of other things that I do want to version. Where would be a good place to store "secret" application data so that I can easily exclude it from source control?
What I usually do it store the credentials in a secure repository (a file that is only accessibly by a superuser that I can trust) and then have a post-build script. Within source control, I have a dummy password in a properties file and then then post-build script runs after a build and replaces the dummy passwords with the real ones. That way the passwords are not in source control, but show up upon every build. This does depend on an automated build/deployment process, but it is very effective.
Put those things into another file/directory and exclude them with .gitignore.
For Java, I recommend the strategy of spring-boot's externalized configuration. The idea is a stack of configuration sources with the ones toward top override the ones below.
Command-line arguments
System properties
Environment variables
Local configuration files (application.yml or application.properties)
Configuration files at the classpath
No. 4 is usually in a hidden folder in the user's home directory, e.g ~/.myapp/application.yml. Only the user can read that file.
For your question, the non-secret, version-controlled properties are put directly in the source code (No. 5 above). Secrets can be also put in 5 but with dummy values. Real secrets overwrite the dummy values at runtime from 1, 2, 3, or 4.
You can either use spring-boot or write your own code following that same strategy.
Related
I would like to share my NodeRed Code in Github as part of an IOT project, however I would like to keep some of my data private (wifi password, database password, etc).
I could manually erase this information, but this would not be very practical, since I would have to do it every time I update the file.
In my python scripts I use os.environ.get to save my passwords and upload to GitHub securely.
¿Is there some way to set up Node-Red to easily push to GitHub without revealing sensitive information?
Node properties can be pulled from environment variables using the ${} syntax. This is implemented by the runtime so should work in all nodes. If the node uses the typed input widget then they should also support environment variables directly.
Details are in the documentation here: https://nodered.org/docs/user-guide/environment-variables
Secondly any node options that are explicitly tagged as credentials are stored in a separate encrypted file _creds.json. By default this is encrypted in a randomly generated key on first run and stored in a hidden file in the userDir, but you can set your own key in the settings.js file.
If you use the Projects feature to store your flows in a git repository then you will be prompted for the encryption key needed as part of setting up the project or when you check it out. Documentation for projects is here:
https://nodered.org/docs/user-guide/projects/
I know that a good way to store data like db passwords, etc. is via environment variables, but setting environment variables manually for every server instance created is time consuming.
I'm planning to deploy my project to the cloud (using aws ebs or heroku).
where should I store my db password?
I think the .ebextensions file isn't a good option because it's tracked in vcs
Don't ever store secrets in source control. A common practice is to either put them in a secure file or in something like https://www.vaultproject.io/ then inject them (programmatically via a script or some other deployment/configuration tool) into the environment when you bring up your VM (or container or whatever).
My recommendation is to create a properties file which can be stored in the resources folder of your application and the code can access the resources. do not need environment variable. One property file can contain all db's userid and passwords. Deploy job based on url mapping in the properties file. For example, look at a spring hibernate example project which uses a property file. Or look at ant deploy scripts. Hope it helps.
I've started working on an Azure project. In terms of config, I currently have three files: ServiceConfiguration.Cloud.cscfg, ServiceConfiguration.Local.cscfg and ServiceDefinition.csdef.
ServiceDefinition.csdef is the template file for the csfg files. ServiceConfiguration.Cloud.cscfg contains all the actual Azure configuration, including DB passwords, SAS keys etc.
Should ServiceConfiguration.Cloud.cscfg be checked into source control? I wouldn't have thought so but a quick search on github for the file shows that it is.
If it should be checked in, how should the sensitive password data be managed?
I typically check in the configurations. The reason is that the behavior of your application will change dramatically depending on these configurations. For example -> number of roles for a distributed application directly affects how you process incoming messages and the vmsize directly affects how much memory you have. You may encounter issues debugging problems if each developer is using a different configuration. This standardizes your deployment.
Anything with plain-text password information shouldn't be checked into a public repo unless you want people to have access to that information.
You can add this file to the .gitignore file and prevent it from being checked in.
Provide a different ServiceConfiguration.Cloud.cscfg named something like ServiceConfiguration.Cloud.cscfg.template with all the config info of your cloud service minus the password values. If someone forks your project they need to use that and fill in the appropriate values and rename the file.
Do this and change all your passwords to something else. Even if you delete this file from the repo, it still exists in the history and anyone can view it.
I'm trying out a yesod applications which I will eventually put up on github or similar.
I will use oauth2 with google which means I have to provide an email and secret token. Which I obviously do not want up on github.
What is a good place to store these in a yesod scaffolded application? I'm hoping to store it in a seperate, config/secret.yml for example, so I can put that into the ignore file of git/mercurial and never commit it.
But i can't find out how to include such a file. Or if such a file already is provided by yesod. config/settings.yml seemed possible, but there's entries there which I would like in github.
So my question is, in a yesod scaffolded application. Where can I store secret keys in a way I can easily exclude it from version control systems?
There are many approaches to this, mostly depending on what flavor of devops/hosting your prefer. One option is to put a dummy value in the config file and override it with an environment variable at runtime (see: https://github.com/yesodweb/yesod/wiki/Configuration#overriding-configuration-values-with-environment-variables). You can also having an extra settings file for production that overrides the values in the default config file, which is how the test suite works. A completely different approach would be to use a system like vault in production and query it for your secure credentials.
EDIT To spell out one of the approaches:
Create a new YAML file with the settings you won't to override, e.g. in config/production.yml:
copyright: This is a copyright notice for production
When you run the application, pass in a command line argument giving the location of the config file
The majority of our C# projects configuration is kept in *.ini files. Mainly these files hold
many sections affecting all aspects of programs behaviour. But besides of regular configuration data some of sections are vulnerable like db connection string or server password. We try to keep this sections in following forms:
[Database]
user=testuser
database=testdb
password=
But when developer is testing application he must fill the config in order to start application. It is quite common that some of the passwords are commited into version control.
Because these files are indispensable for application they cannot be included in .svnignore.
Probably what I'm looking for is some kind of script (maybe in powershell). That would scan all *.ini files and erase all passwords. The most interesting solution would be adding some external password storage that can be used both to encode and decode passwords in *.ini files.
I always push to store configuration template files in subversion, but not actual configuration files. So if the configuration file is "config.ini" then I'll check in a "config.ini.template" populated with non-working sample data.
Then to prevent multiple developers from checking in their individual "config.ini" files, I'll add the actual configation file name to the svn:ignore properties list.
This forces the developer to copy the file and modify it appropriately for their environment, but eases the work of that task by not forcing them to find out which fields need to be present. If you have the time, you can even embed comments into the template file to simplify the meanings of some of the configuration options.
At the top of the file, include the directions of how to configure the system using the template, which should read something like:
# *** CONFIGURATION TEMPLATE --- DO NOT MODIFY THIS FILE ***
# 1. Make a copy of this file in the same directory with the command "copy config.ini.template config.ini"
# 2. Edit the new copy and follow the rest of the instructions
#
# Change "this.system.hostname" to the hostname of this system
Hostname = this.system.hostname
# Set the answer "23" to "42"
Answer = 23
You get the idea....
If you have problems (or think you might have problems) with people checking in their configuration options over the config.ini.template file, then I'd recommend using "svn lock" on the template file. However, with the appropriate warning, I've never found it necessary.
I'll not answer your question and instead recommend a different approach, assuming it's not too late to change the relevant design.
You should not store passwords in the same files as the rest. Have the application read a dedicated password file (or retrieve the password from a password storage service) in addition to the regular configuration file. This is not just about not storing passwords in svn, but also about not having passwords exposed to shoulder surfing, accidentally mailed or posted when someone asks for help with a non-working configuration, etc.