Linux System.d Unit Service Doesn't See My Specific Environment Variable - linux

I have a basic "service unit" file like the following.
[Unit]
Description=Certprovider service
After=network.target
[Service]
Type=simple
Restart=always
RestartSec=5s
ExecStart=/home/mert/certprovider/certprovider
WorkingDirectory=/home/mert
User=root
Group=root
[Install]
WantedBy=multi-user.target
I have the .env file in the root of the project.
CA_DIR_URL=https://acme-v02.api.letsencrypt.org/directory
EMAIL=mertsmsk0#gmail.com
HOST=127.0.0.1
PORT=8557
I load this file with the following lines.
err := godotenv.Load()
if err != nil {
log.Fatalln("Error loading .env file")
}
Service has been working very well but I cannot reach the PORT environment variable. Thus I cannot start the webserver because that port cannot listen. I print all the environment variables that in the .env excluding PORT. I changed its name to APP_PORT but it same thing.
The mystery part is I can reach other variables in the .env file. In addition to that when I add the following line in the unit file, I can reach that variable but I don't understand that why should I add only the PORT variable in the unit file?
[Service]
Environment=PORT=8557
It's happening when I try to run it as a binary file. Because I can reach the variables with the following command.
go run .

If you call Load without any args it will default to loading env in the current path.
Your current path is configured here:
WorkingDirectory=/home/mert
And yet, you say (emphasis added)
I have the .env file in the root of the project.
But that's not the current working directory.
root of the project
That concept is not meaningful to the application runtime. Unlike interpreted languages like, say, PHP, Go compiles to a static binary that is functionally entirely distinct from the set of libraries and sources that define it. In PHP (or python, ruby, etc) those libraries have no other place to be, than the root of some project directory.
In go, that stuff is only relevant for development and testing. The fact that your executable appears to be in your "root of the project" is entirely incidental and completely meaningless.
If you really want to put the runtime configuration in tht particular file, in that particular place, just set that as the working directory:
ExecStart=/home/mert/certprovider/certprovider
WorkingDirectory=/home/mert/certprovider/certprovider
I would put that stuff in /usr/local so I didn't accidentally break my let's encrypt while fiddling with stuff in my home directory - doubly so for let's encrypt because it might take up to 90 days to realize your certs weren't being refreshed. I'd put the config outside of my home directory for the same reason.
Actually, for this case I'd probably put all the config in the unit file. Why not put it there? But of course, that's a matter of opinion. If you really want to use the automatic .env discovery then you should dedicate a directory to containing that hidden file. It doesn't make much sense to put a config specific to one application in ~/.env.
Wherever you put .env, make sure that's your working directory so it will be discovered.
I print all the environment variables that in the .env excluding PORT. I changed its name to APP_PORT but it same thing. [...]
The mystery part is I can reach other variables in the .env file.
Respectfully, that sounds like an assumption on your part. Without evidence to the contrary, it's easy to conclude that you have set defaults for these values or that they're coming from some other source or behavior. That's more parsimonious than concluding the godotenv library read some, but not all, the values from a file.
It's happening when I try to run it as a binary file. Because I can reach the variables with the following command. [go run .]
Go always runs as a binary. go run . simply automatically builds the binary in a temp location and then runs it. Why is it recommended to use `go build` instead of `go run` when running a Go app in production? talks about why go run is often contraindicated on SO.

Related

How can I make node application see system variables on Google Cloud?

I have variable set in my .bash_rc file:
whoami#cloudshell:~/source/NodePrototype (x-alcove-9999999)$ echo $APP_ENVIRONMENT
LIVE
Yet node.js application out of:
const app_environment_config=require('./APP_ENVIRONMENT/' + process.env.APP_ENVIRONMENT)
produce
2019-02-21 14:18:16 default[20190221t141628] Error: Cannot find module './APP_ENVIRONMENT/undefined'
Eventhough when I enter node shell:
whoami#cloudshell:~/source/NodePrototype (x-alcove-9999999)$ node
> process.env.APP_ENVIRONMENT
'LIVE'
The same part works locally.
It depends on how your Node app is being launched, because looks like is not running in an environment where that variable exists, to make sure print all your current env vars to make this sure: console.log(process.env).
Also, a good practice, when you need something like that, is to use .env files with this module: https://www.npmjs.com/package/dotenv is a good practice to pass configuration to your Node apps.

Deploying Flask APP with UWSGI,Nginx,direnv and systemd

I have created an API using FLASK which I am trying to deploy on a linux server by creating a systemd service.
I have used direnv to setup input parameters to the app like database connections. Below is what the file looks like :
The uwsgi config is as below :
The systemd file has the following entries:
I get the follwing error in my uwsgi logs whenever I try to reach the service on my browser :
--- no python application found, check your startup logs for errors ---
[pid: 23791|app: -1|req: -1/3] 192.168.9.180 () {44 vars in 719 bytes} [Thu Oct 11 14:35:09 2018] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (1 switches on core 0)
My understanding is the ExecStart command in the systemd file is not able to invoke the direnv set variables , hence i added the ExecStartPre entry but even does not seem to work.
Any hints/ideas are appreciated.
Note: The application is accessible without errors when I run the uwsgi via command line from my python virtual environment :
uwsgi --socket 0.0.0.0:5000 --protocol=http -w app:app
i have a few advises that may help you, probably only the first one is the one you need...
1) Either move all your env variable defined in direnv to the systemd unit as Environment or move them into a special file (similar to the you already have) without the "source activate" line and the export, and then pass that file as EnvironmentFile , this is the doc for that https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Environment
2) Your ExecStartPre does nothing really, even tho you do "cd" into the path, that is lost and is not persistent. you should remove it.
3) By setting your PATH to only that path, you are restricting your self, i would recommend see the value of your current PATH and then set it to that value. but otherwise at least add "/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin". now spoiler alert, you probably dont need to set it.
4) put the socket in /run//socket.socket directory and let systemd manage your /run/<yourapp> with RuntimeDirectory directive.
good luck!

process.env.PATH undefined in Passenger node app (production mode)

I recently deployed a node application with Phusion Passenger for nginx, and encountered a pretty quirky error in the process:
My code threw an error from trying to spawn a child_process. I did a bit of debugging and eventually concluded that the problem arose from the $PATH environment variable being undefined in node, and I could solve the problem with a passenger_env_var directive like this (showing an extract of my nginx config):
server {
listen 80;
server_name blargh.com;
root /home/user/blargh.com/build;
passenger_enabled on;
# For some reason $PATH isn't loaded into node, and we can't spawn child processes without it
passenger_env_var PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games;
}
I still haven't figured out what caused this problem though - setting passenger_load_shell_envvars on; didn't help, and the www-data user did have a $PATH envvar defined in the shell. Moreover, other environment variables (like $SHELL) seems to have been loaded by node, adding to the mystery of why $PATH was excluded.
Does anybody know what could cause this problem?
tl;dr
Specify global envvars that you expect to be defined at system boot (like PATH) in /etc/default/nginx. Use something like dotenv properly and write environment specific config for your app in a text file that's not checked in. Environment variables are pretty evil in general.
I felt this one deserved a fairly lengthy answer, since environment variables has caused recurring problems for me during the last couple of months.
Storing your config as environment variables is one of the rules that 12 factor app lays out for writing scalable web applications. They're good because they let you separate your config from your code in a flexible manner. However, a problem with them is that the way we encounter them normally, when we export MYVAR=myvalue or set them in our ~/.pam_environment or ~/.bashrc, the scope of them is our current terminal session.
This causes issues as we start to use solutions like Phusion Passenger to start our apps at system boot - their startup scripts don't care about user shell environments. They also don't care about the global /etc/environment apparently, which is what caused my problems with PATH being undefined.
Phusion Passenger actually has some documentation on making global environment variables persist:
If you installed Nginx through the Debian or Ubuntu packages, then you can define environment variables in /etc/default/nginx. This is a shell script so you must use the export FOO=bar syntax.
So by setting the PATH envvar in /etc/default/nginx, I could solve that issue. But I was still having trouble with the other environment variables - I had to set them in my nginx config to have them passed on to my node app. It was clear to me that this wasn't the right way to do it.
At this point I was already using dotenv, but I had misunderstood its purpose slightly. I had checked in the .env file and thought of it as a way to provide default values for envvars that would be overridden by the environment as needed. This isn't how the authors themselves envisioned this module to be used:
We strongly recommend against committing your .env file to version control. It should only include environment-specific values such as database passwords or API keys.
It started becoming clear to me that people often don't define the envvars for their apps in the actual environment. I found an article by Peter Lyons that suggests storing config in a text file instead of in envvars, and that's when it clicked for me.
My final solution was to uncommit my .env file, and write a specific one for each environment. I left a .env.template in my repo as a reference to what configuration my app expected to be defined at run-time.

Problems with EXEC pplcd from PeopleSoft Application Engine

On a Unix server, I am running an application engine via the process scheduler.
In it, I am attempting to use a "zip" Unix command from within an "Exec" pplcode function.
However, I only get the error
PS_Exec(P): Error executing batch command with reason: No such file or directory (2)
I have tried it several ways. The most logical approach I thought was to change directory back to the root, then change to the specified directory so that I could easily use the zip command, such as the following...
Exec("cd / && cd /opt/psfin/pt850/dat/PSFIN1/PYMNT && zip INVREND INVREND.XML");
1643 12.20.34 0.000048 72: Exec("cd /opt/psfin/pt850/dat/PSFIN1/PYMNT");
1644 12.20.34 0.001343 PS_Exec(P): Error executing batch command with reason: No such file or directory (2)
I've even tried the following....just to see if anything works from within an Exec...
Exec("ls");
Sure enough, it gave the same error.
Now, some of you may be wondering, does the account that is associated with the process scheduler actually have authority on this particular directory path on the server ? Well, I was able to create the xml file given in the previous command with no problems.
I just cannot seem to be able to modify it with the Exec issuance of Unix commands.
I'm wondering if this is an error of rights and permissions from the unix server with regards to the operator id that the process scheduler is running from. However, given that it can create and write to a file there, I cannot understand why the Exec command would be met with any resistance....Just my gut shot in the dark...
Any help would be GREATLY appreciated!!!
Thanks,
Flynn
Not sure if you're still having an issue, but in your Exec code, adding the optional %FilePath_Absolute constant should help. When that constant is left off, PS automatically prefixes all commands with <PS_HOME>. You'll have to specify absolute paths with this flag on though. I've changed the command to something that should work.
Exec("zip /opt/psfin/pt850/dat/PSFIN1/PYMNT/INVREND /opt/psfin/pt850/dat/PSFIN1/PYMNT/INVREND.XML", %FilePath_Absolute);
The documentation at PeopleBooks is a little confusing sometimes, but it explains it fairly well in this case.
You can always store the absolute location in a variable and prefix that to your commands so you don't have to keep typing out /opt/psfin/pt850/dat/PSFIN1/PYMNT/.

Deployment specific files in NodeJS

I am running my NodeJS project on DotCloud. Sadly, DotClouds deployment is "project-intrusive" that is it requires a supervisord.conf file to reside in the app-root. My deployment setup looks like this (using git repos).
project-deploy.git/prod/dotcloud.yml
project-deploy.git/prod/project -> project.git
(/prod/project use project.git as a submodule to access the code)
Now, my though of this is that I eventually would end up having different environments like this, e.g. dev, test and stage. The dev environment wouldn't even have a dotcloud.yml file since it is expected to run everything locally.
Well this works pretty well. But the problem is the supervisord.conf file which is just for deployment to dotcloud, now it resides in the project.git repo, but it doesn't belong there since it is just for deployment.
Are there any modules or NodeJS scripts that let you put deployment configuration files elsewhere, and maybe even specify what the target environment is, e.g. node deploy.js --production, or something like that?
There is a way to get rid of supervisord.conf. Assuming that you want to run e.g. node app.js, you can put the following in dotcloud.yml:
www:
type: nodejs
process: node app.js
Now, of course, it doesn't solve the problem of the dotcloud.yml file itself; but at least it reduces clutter a little bit -- removing it from the approot.

Resources