While my understanding is that using environment variables for configuring applications in different deployment environments is best practice, I don't know of a good method for managing these environments and populating the variables in them.
Here are the approaches I'm considering:
Populating them in the Upstart script we use to run our app. We use Ansible to provision our servers which currently copies over a static upstart scrip, however this could be templated with environment variables.
Same approach but with /etc/environment
Using something like envdir and once again using ansible to populate the files.
The other issue is where to store the values, I'm thinking redis, but am open to suggestion. Ansible has a "Vault" that I'm yet to look at which may be an option.
The values are things like API keys and database urls.
I'm really just wondering what approaches other people use. I'm open to all suggestions.
I think this question is going to solicit a lot of opinions, and probably a lot of conflicting opinions, but with that said here's some of my opinions:
/etc/environment is part of the OS and intended for configuration of interactive user shells. Don't use it for applications.
A templatized upstart config via ansible seems pretty reasonable to me. Just ensure the filesystem permissions are suitably locked-down to root read only if you intend to store sensitive data there.
You could also use a templatized application-specific config file such as /etc/myapp/config which has worked pretty well for many programs for a few decades. The whole environment-variables-are-better-than-config files position is really coming more from a PaaS perspective (heroku I believed popularized this approach by way of their 12-factor app site). So if you're deployment is PaaS or PaaS-style, envirnoment is convenient. But if you are installing your app on your own servers via Ansible, IMHO a straight-up config file is simpler to troubleshoot for the reasons I outline in my blog post environment variables considered harmful
Related
Using a lot of (official and non official) terraform providers, I'm looking for a tool to perform security analysis on terraform providers before executing terraform plan/apply commands (and so executing providers code). I want to prevent malicious code from providers to be executed blindly.
I'm basically executing terraform providers mirror command to save local copies of required providers and I'm wondering if I can security scan that result.
I tested kics, checkov and tfsec but they are all looking for security issues in my terraform static code but not in providers.
Do you have any good advices regarding this topic ?
This is actually quite a good question. There are many other problems that can be reduced to same generic question - how to make sure that the thing you downloaded from the internet does not do anything malicious to you like e.g.:
How to make sure that a minecraft plugin does not hack you?
How to make sure that a spring boot dependency does not hack you?
How to make sure that a library xxx you attach to your project does not do harm to you?
Should you use docker image yyy in your project?
Truth is: everything you use has the potential to explode right in your face (or more correctly: right into the face of the system owner). That's why the system owner (usually a company) defines a set of rules to follow what is allowed and what is not allowed. No set of rules you are aware of? Below a set of rules we came up with ourselves when thinking about on-boarding a new library for some projects to use:
Do not take random stuff from github. Take only products with longer history, small bug backlog, little to none past issues in the CVE list, actively maintained.
Do static code analysis yourself. Sometimes it is possible to have tools that work on binaries level do that for you. Sometimes you can do it on source level only. In case of Java libraries, check what tools like Dependency Track think about the library and version you are about to use.
Run the code and see how it works: what does it write, what does it read, what URLs does it communicate with (do a TCP dump if necessary).
Document everything you have done somewhere.
This gives you no 100% confidence that things will not go terribly wrong. But this is a systematic approach that will reduce the risk of doing something stupid.
The rule-of-thumb/best practice which I only occasionally see challenged is that you should never commit environment-specific config files (e.g. .env, .ini, etc) to version control. The closest accepted practice I can find is that you may commit a default config file, but with the expectation that you manually edit the config on initial deployment.
However, in a DevOps role I'm not just writing app logic, I'm also automating the deployment of my code and its multiple environments. As a result, there's a specific need to keep track of what my configs look like so I may (re-)deploy my app if /when an environment needs to be recreated. For the same reason as with traditional code then, the most appealing solution is to store my config in the repo, but the question is what's the best and most scalable way to do so?
Obviously I'm not talking about storing any secrets in my config file. I'm also not against a solution that doesn't involve the repo. But I think discounting the repo outright is a bit silly and is a case of adhering to practice out of tradition more than its practical value.
How have others tackled this issue?
EDIT: Ideally, I think, there would be some extension for git that would allow env-specific configs to be associated with their app's repo, but would be segregated (stored in a separate repo?) in such a way as to avoid downloading an env's config when forking/branching a project. That seems well outside the scope of what's available though.
There are two sets of approaches for this. One uses a configuration from a secret store, such as Vault, to store the configuration of your data independent of your repository and inject it through the environment. This lives outside of the repository entirely, but can be configured for different environments and ensures your data is securely encrypted.
The other, where you want to store some configuration in the repository, usually consists of storing the file in a separate directory as a sort of template and then copying it into place and editing it. The place it is used in production is typically ignored. You may choose to use a script for editing it or edit it by hand.
You can also store configuration in a separate, highly restricted repository, but that has all of the problems of checking secrets into a repository.
Does Digital Ocean have something similar to Credstash or AWS Secrets Manager (both AWS services)?
Trying to decide on the most secure way to store environmental variables with sensitive information (like database access codes, for example).
Locally, I have .env file that is named in my .gitignore to prevent it being version controlled.
If just in a .env file or environment variables, what’s the best way to keep that secure for the app to run appropriately?
Much appreciated :)
Cheers
ADDITIONAL INFO:
I have a fullstack SPA (MongoDB, Node, React, Express) on the digital ocean droplet.
This is quite a loaded question to get this done right, and implementation cost depends on what your risk tolerance is and a variety of other things (i.e. threat modeling). Below are just some things to consider.
At the very minimum, you're going to want to ensure your sensitive configuration is encrypted at rest on the disk and assess whether this is going to end up in your backups as well, as part of infrastructure management, etc. Even if you used a third party system for credential management, you're still going to be maintaining API credentials on your host to connect to that system, else it will still be accessible locally via mount, or in-memory, etc.
You have to also consider how strings might or might not be garbage collected/copied in memory and what your risk tolerance is there.
It also goes without saying that part of your post-commit/CI should explicitly ensure file permissions are being set correctly as intended over your sensitive configuration files (i.e. chmod 0400).
You also want to run through an application compromise scenario where an attacker is able to read the file system (not memory, and not application code injection) with your application. So do a su to impersonate your application process' user on the host to see what it can actually do with your config file. If it's encrypted at rest and is unable to discover the decryption keys or decrypt it altogether, well you're probably in a better place than 99% of WordPress websites out there.
There's definitely more to discuss, but that should get you started. You might also consider popping over to https://security.stackexchange.com/ with this question.
Best wishes!
It's best practice to put sensitive environment variables into env.yml and reference them in serverless.yml. Of course, this also means not checking env.yml into a code repository.
So where's a safe place to store a backup of env.yml? We have a number of microservices, so we're accumulating several env.yml files for our projects. Even sharing them among devs and keeping them updated can become a bit of an issue - they really could benefit from version control but security trumps convenience so we keep them out of git.
I'd be interested to hear how others manage secrets config in general.
While the question was specifically about management of env.yml files, the bigger underlying question is how to manage sensitive environment variables. The link in the comment from Alex is all I needed. Our solution is so AWS-oriented that the AWS Parameter Store is worth exploring.
Alex DeBrie's article
Yan Cui's article on referencing parameter store values at runtime
I've been reading up on a few node tutorials but there are a couple of best/common practices that I would like to ask about for those out there that have built real node apps before.
Who do you run the node application as on your linux box? None of the tutorials I've read mention anything about adding a node user and group so I'm curious if it's because they just neglect to mention it or because they do something else.
Where do you keep your projects? '/home/'? '/var/'?
Do you typically put something in front of your node app? Such as nginx or haproxy?
Do you run other resources, such as storage(redis, mongo, mysql, ...), mq, etc..., on the same machine or separate machines?
I am guessing this question is mostly about setting up your online server and not your local development machine.
In the irc channel somebody answered the same question and said that he uses a separate user for each application. So I am guessing that this is a good common practice.
I mostly do /home/user/apps
I see a lot of nginx examples so I am guessing that is what most people use. I have a server with varnish in front of the a node.js application and that works well and was easy to setup. There are some pure node.js solutions but for something as important as your reversed proxy I would go for something that is a little more battle-tested.
To answer this correctly you probably have to ask your self. What are my resources? Can I afford many small servers? How important is your application? Will you lose money if your app goes down?
If you run a full stack on lets say one VPS then if there is a problem with that VPS then only one of your apps is affected.
In terms of maintenance having for example one database server for multiple apps might seem attractive. You could reason that if you need to update your database to patch a security hole you only need to do it in one place. On the other hand you now have a single point of failure for all the apps depending on that database server.
I personally went for many full stack server and I am learning how to automate deployment and maintenance. Tools like Puppet and Chef seem to be really helpful for this.
I only owned my own Linux servers for the last 3 months and have been a Linux user for 1.5 years. So before setting up a server park based on these answers make sure you do some additional research.
Here's what I think:
Using separate user for each app is the way I'm doing this.
I keep it in /home/user/ to make sure that only user (and root of course) has access to the app.
Some time ago I've created my own reverse proxy in Node JS based on node-http-proxy module. If you don't want to use reverse proxy then there's no point in putting anything in front of Node. There's even more: it may harm the app, since for example nginx can't use HTTP/1.1 (at least at the moment).
All resources I run on the same machine. Only when I actually need to distribute my app between separate machines I start thinking about seperate machines. There's no need to preoptimize. App's code is a different thing, though.
Visit the following links::
nettuts
nodetuts
lynda nodejs tutorials
Best practice seems to be to use the same user/group as you would for Apache or a similar web server.
On Debian, that is www-data:www-data
However, that can be problematic with some applications that might require higher permissions. For example, I've been trying to write something similar to Webmin using Node and this requires root permissions (or at least adm group) for a number of tasks.
On Debian, I use /var/nodejs (I use /var/www for "normal" web applications such as PHP)
One of the reasons I'm still reluctant to use Node (apart from the appalling lack of good quality documentation) is the need to assign multiple IP Ports when running multiple applications. I think that for any reasonably sized production environment you would use virtual servers to partition out the Node server processes.
One thing that Node developers seem to often forget is that, in many enterprise environments, IP ports are very tightly controlled. Getting a new port opened through the firewall is a very painful and time-consuming task.
The other thing to remember if you are using a reverse proxy is that web apps often fail when run from behind a proxy - especially if mapping a virtual folder (e.g. https://extdomain/folder -> http://localhost:1234), you need to keep testing.
I'm just running a single VPS for my own systems. However, for a production app, you would need to understand the requirements. Production apps would be very likely to need multiple servers if only for resilience and scalability.