process.env.PATH undefined in Passenger node app (production mode) - node.js

Question

I recently deployed a node application with Phusion Passenger for nginx, and encountered a pretty quirky error in the process:
My code threw an error from trying to spawn a child_process. I did a bit of debugging and eventually concluded that the problem arose from the $PATH environment variable being undefined in node, and I could solve the problem with a passenger_env_var directive like this (showing an extract of my nginx config):
server {
listen 80;
server_name blargh.com;
root /home/user/blargh.com/build;
passenger_enabled on;
# For some reason $PATH isn't loaded into node, and we can't spawn child processes without it
passenger_env_var PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games;
}
I still haven't figured out what caused this problem though - setting passenger_load_shell_envvars on; didn't help, and the www-data user did have a $PATH envvar defined in the shell. Moreover, other environment variables (like $SHELL) seems to have been loaded by node, adding to the mystery of why $PATH was excluded.
Does anybody know what could cause this problem?

Answer 1

tl;dr
Specify global envvars that you expect to be defined at system boot (like PATH) in /etc/default/nginx. Use something like dotenv properly and write environment specific config for your app in a text file that's not checked in. Environment variables are pretty evil in general.
I felt this one deserved a fairly lengthy answer, since environment variables has caused recurring problems for me during the last couple of months.
Storing your config as environment variables is one of the rules that 12 factor app lays out for writing scalable web applications. They're good because they let you separate your config from your code in a flexible manner. However, a problem with them is that the way we encounter them normally, when we export MYVAR=myvalue or set them in our ~/.pam_environment or ~/.bashrc, the scope of them is our current terminal session.
This causes issues as we start to use solutions like Phusion Passenger to start our apps at system boot - their startup scripts don't care about user shell environments. They also don't care about the global /etc/environment apparently, which is what caused my problems with PATH being undefined.
Phusion Passenger actually has some documentation on making global environment variables persist:
If you installed Nginx through the Debian or Ubuntu packages, then you can define environment variables in /etc/default/nginx. This is a shell script so you must use the export FOO=bar syntax.
So by setting the PATH envvar in /etc/default/nginx, I could solve that issue. But I was still having trouble with the other environment variables - I had to set them in my nginx config to have them passed on to my node app. It was clear to me that this wasn't the right way to do it.
At this point I was already using dotenv, but I had misunderstood its purpose slightly. I had checked in the .env file and thought of it as a way to provide default values for envvars that would be overridden by the environment as needed. This isn't how the authors themselves envisioned this module to be used:
We strongly recommend against committing your .env file to version control. It should only include environment-specific values such as database passwords or API keys.
It started becoming clear to me that people often don't define the envvars for their apps in the actual environment. I found an article by Peter Lyons that suggests storing config in a text file instead of in envvars, and that's when it clicked for me.
My final solution was to uncommit my .env file, and write a specific one for each environment. I left a .env.template in my repo as a reference to what configuration my app expected to be defined at run-time.