I'm setting up a Flask RESTapi for NET-SNMP because i'm doing most development in Windows.
The problem is that Supervisor does not find my installed mibs.
Everything works if i run the flask app manually with Gunicorn.
But with supervisor i get exceptions saying it cannot locate mibs-files.
Installed mibs are in /home/web/.snmp
supervisor conf
[program:app]
command = /home/web/www/venv/bin/python3.6 /home/web/www/venv/bin/gunicorn app:app -b localhost:8000
directory = /home/web/www/app
user = web
Solved.
added environment to the config
[program:app]
command = /home/web/www/venv/bin/python3.6 /home/web/www/venv/bin/gunicorn
app:app -b localhost:8000
directory = /home/web/www/app
user = web
environment=HOME="/home/web",USER="web"
Related
I am using this Dockerfile on Azure Linux App Service:
FROM ruby:2.3.3
ENV "GEM_HOME" "/home/gems"
ENV "BUNDLE_PATH" "/home/gems"
EXPOSE 3000
WORKDIR /home/webapp
CMD bundle install && bundle exec rails server puma -b 0.0.0.0 -e production
As you can see the gems folder is located in the home folder. The home folder is shared with the host system of the App Service. Now my problem the App Service LogFiles/docker/docker_***_out.log indicates that bundle install is called multiple times (probably from different containers). This leads to that some gems are never successfully installed.
Is there some setting which runs just one container so that my gems can be installed successfully and not interferring with each other? Or am I making wrong assumptions here? Maybe the problem isn't that there a multiple containers started?
Is there an easier way to install the gems the first time in the shared folder of the host system?
Currently when I start a build in GitlabCI it is running under gitlab-runner user. I want to change it the company's internal user. I didn't find any parameter to the /etc/gitlab-runner/config.toml which is solve that.
My current configuration:
concurrent = 1
[[runners]]
name = "deploy"
url = ""
token = ""
executor = "shell"
Running ps aux | grep gitlab you can see:
/usr/bin/gitlab-ci-multi-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --service gitlab-runner --syslog --user gitlab-runner
Service is running with option --user.
So let's change this, it depends on what distro. you are running it. If systemd, there is a file:
/etc/systemd/system/gitlab-runner.service:
[Service]
StartLimitInterval=5
StartLimitBurst=10
ExecStart=/usr/bin/gitlab-ci-multi-runner "run" "--working-directory" "/home/gitlab-runner" "--config" "/etc/gitlab-runner/config.toml" "--se
Bingo, let's change this file now:
gitlab-runner uninstall
gitlab-runner install --working-directory /home/ubuntu --user ubuntu
reboot the machine or reload the service (i.e. systemctl daemon-reload), et voilà!
Note that when installing with a specific user (--user), whenever you update, it will revert back to the original systemd script and so, back to using gitlab-runner user.
in order to keep the user change across updates, using systemd overrides (centos7) you can use these steps (assuming service is at /etc/systemd/system/gitlab-runner.service):
Create a /etc/systemd/system/gitlab-runner.service.d directory.
Create a /etc/systemd/system/gitlab-runner.service.d/exec_start.conf file, with content:
[Service]
ExecStart=
ExecStart=/usr/lib/gitlab-runner/gitlab-runner "run" "--working-directory" "/home/ubuntu" "--config" "/etc/gitlab-runner/config.toml" "--service" "gitlab-runner" "--syslog" "--user" "ubuntu"
Execute systemctl daemon-reload
Now to check this is working, you can do this:
Reinstall GitLab Runner package gitlab-runner uninstall and then gitlab-runner install
Check ps aux | grep gitlab and confirm the right user is being used
source: https://gitlab.com/gitlab-org/gitlab-runner/issues/3675
Once the gitlab-runner is registered (yes, it will be installed under the user gitlab-runner and working directory /home/gitlab-runner ) you can execute the following to change the runner's user
gitlab-runner uninstall
gitlab-runner install --working-directory <existing-path> --user <any-existing-user>
# eg: gitlab-runner install --working-directory /home/ec2-user --user ec2-user
then restart the service
service gitlab-runner restart
NOTE: you don't need to edit /etc/systemd/system/gitlab-runner.service for this, as it is being updated once the service is restarted as above
to check if the configurations are reflecting, run
ps aux | grep gitlab
[DEPRECATED ANSWER]
I found a solution, which is not best pactrice but solved it. I need to use the ssh executer and ssh to localhost. It is require to add gitlab-runner id_rsa.pub to the user's authorized_keys what you want to use. There is my extended code:
concurrent = 1
[[runners]]
name = "deploy"
url = ""
token = ""
executor = "ssh"
[runners.ssh]
user = "user"
host = "localhost"
port = "22"
identity_file = "/home/gitlab-runner/.ssh/id_rsa"
Just for future reference, I was doing a test with a cloned version of my setup, if the domainname is not pointing to the server you are working with, gitlab might consider your runners offline. If you have another (copied) instance running at the ip the domain is pointing at and there is no firewall blocking, the gitlab-runner verify command will say your runners are alive.
a solution could be adding your domain pointing to 127.0.0.1 to your hosts file. you'll have to restart your gitlab instance and runners.
For recent version of gitlab-runner you should modify the system arguments in the /etc/default/gitlab-runner file.
Here example for docker gitlab-runner:
Build your own runner image based on Dockerfile with following content
FROM gitlab/gitlab-runner
# add new user (if needed)
RUN useradd -u 998 gitlab-www && mkdir /home/gitlab-www && \
chown gitlab-www /home/gitlab-www && chmod u+rwx /home/gitlab-www
# need to replace entrypoint to force new created user over gitlab-runner
ENTRYPOINT /usr/bin/dumb-init /entrypoint run --user=gitlab-www --working-directory=/home/gitlab-www
(update -u 998 and gitlab-www as you need)
.gitlab-ci.yml scripts are running as user gitlab-www now. If this one has same uid as host mounts, you are also able to deploy directly to host folders.
I have a node application which runs fine if I manually putty into the gcloud computeVM and run it.
Here are the complications (all realted to unix) :
1.) I have a domain name. So I added the dns zone record to point to the above VM.
2.) For the compute VM to respond, there should be process listening on 80
3.) If we follow the https://cloud.google.com/nodejs/getting-started/run-on-compute-engine#download_app , it specifies to run the app on 8080.
4.) For ports < 1024, it requires root privileges to open up ports.
5.) So from npm start, I changed the start up script to use "sudo npm start"
6.) Then it gave the following error : my-app-instance supervisord: nodeapp sudo: no tty present and no askpass program specified
7.) If I have to "sudo visudo" everytime and add the "username ALL = NOPASSWD:" everytime I restart the instance after deployment , its something which I would least prefer.
I have included the relevant portion of the stratup-script for more info :
# Install app dependencies
cd /myrepo/opt/app/servers
sudo npm install
# Create a nodeapp user. The application will run as this user.
useradd -m -d /home/nodeapp nodeapp
chown -R nodeapp:nodeapp /myrepo/opt/app/servers
# Configure supervisor to run the node app.
cat >/etc/supervisor/conf.d/node-app.conf << EOF
[program:nodeapp]
directory=/myrepo/opt/app/servers
command=sudo npm start
autostart=true
autorestart=true
user=nodeapp
environment=HOME="/home/nodeapp",USER="nodeapp",NODE_ENV="production"
stdout_logfile=syslog
stderr_logfile=syslog
EOF
A.) My requirement is simple : My google domain points to the above compute VM now. whenever the user types www.domainname.com, it should take him to the website without any port numbers in the url. How to open port 80 with a simple modification of start-up script(preferred) ?
B.) And also if I have to go with deploy.sh specified in the tutorial, will it get executed automatically ? Or if I have to execute it automatically , whats the procedure.
Note : I am not unix expert. Any help would be appreciated.
Look into using a reverse proxy. This allows you to run your app without root privileges on a port like 8080, and have a privileged HTTP server (like Apache or Nginx) running on port 80 and proxying traffic to your app. This is common practice, and much more secure than running your app with root privileges.
I found an old PC and i want to use it as a dedicated Node.js test machine.
Basically i wanna write my apps on a win machine then copy them over samba to the node folder and launch them via ssh. Later, I would add an upstart script and copy it with samba to the server so that when i reboot the app starts automatically every time.
What do I need to install in order to properly run Node.js apps on my network on a dedicated Ubuntu server? Here is the list I came up with, please correct me if I'm wrong. Is there anything else?
ssh
samba (ftp or sftp should be the way to go but as it's a closed internal network and i have to access it from various os's samba is the simplest way to share files not considering security issues..most of the time i use a simple text editor)
"basic ubuntu server" files?
"LAMP" (?)
node.js
node package manager.
how do i install the latest Node.js, npm, and the init files on Ubuntu server. I saw that there was no simple sudo apt-get install nodejs npm.
What kind of script do I need to launch my apps and where do i put them (prefer native scripts)?
EDIT
After some testing i'm at a good point now, and here is what i did:
I installed ubuntu from a minimal CD
when it comes to choose the packages i selected ONLY ssh & samba
update the system
install the dependencies that u need to run node.js
install latest node from git
setup samba in my case i created the folder /var/nodejs for the scripts
put your testApp.js in the nodejs folder
start your testApp.js from ssh. *it won't work
3-update the system
sudo apt-get update && sudo apt-get upgrade
4-dependancies
sudo apt-get install g++ curl libssl-dev apache2-utils git-core make
5-install node
git clone git://github.com/ry/node.git
cd node
./configure
make
sudo make install
6-setup samba sudo nano /etc/samba/smb.conf
[nodejs]
comment = nodejs
workgroup = WG
security = USER
path = /var/nodejs
server string =Node JS
browsable = yes
read only = no
writeable = yes
create mask = 0777
7-testApp.js
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello Node.js\n');
}).listen(80, "192.168.0.1");
console.log('Server running at http://192.168.0.1:80/');
8-Now everything should run...but:
You can run nodejs only as administrator appending "sudo" in front of the launch command
else as a normal user u don't have access to most of the ports under 1000.
A. How can i lauch my app on port 80 without using sudo?
And obviously if u launch you app with the command sudo node /var/nodejs/testApp.js
if u close the terminal the app will stop.
For that we use a init script.
After some reading i found that upstart is natively installed in ubuntu server and it's probably the best way to launch your apps.
B. I know u need to put the script into /etc/init/ with your appname and .conf extension.but how does that work?
what do i need to install to properly run node.js apps on my network on a dedicated ubuntu server?
You just need to install nodejs. nodejs can run on any port, so you don't need Apache or anything else.
how do i install the latest nodejs,npm,and the init files on ubuntu
server
Try to follow the steps outlined in this guide: http://howtonode.org/how-to-install-nodejs . Use the instructions for Ubuntu.
when i reboot the app starts automatically every time
One way to do this is to write a small script that will run on boot. The script would contain the instruction:
nodejs /path/to/app/app.js
Check out this SO answer on how to run a script on boot: https://stackoverflow.com/questions/3036/files-and-scripts-that-execute-on-boot
By your question, you sound about as lazy and impatient as I am, therefore use PPAs instead of building from the source. Just follow the node.js ubuntu directions.
In-fact I'm so lazy, I refuse to type in port numbers, hence I proxy all my node.js applications with nginx. (This is also the best way, and only way I can tell to have multiple servers "listening" on port 80). [Nginx's install guide.] Once you get nginx up, follow Chris Lea's guide.(http://wiki.nginx.org/Install) for the proxy.
BTW if you installed apache, make sure you purge it sudo apt-get purge apache*. This will most likely break your php apps, but that's why you're running node right? Just google how to run php with nignx.
Now for upstart & monit. Just follow this guide. NOTE: The guide has a typo so read the comments carefully.
As for samaba, you're on your own there.
TL;DR
Answer A: guide
Answer B: sudo cp my-node-app.conf /etc/init; sudo service my-node-app start
Edit 1
Upstart is Ubuntu's native utiltiy for starting background processes. Read all about it here.
#!upstart
description "node-app"
author "me"
env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
respawn
start on runlevel [23]
script
#set enviroment vars here
export NODE_ENV=production
#Uncommit if you need a pid file for monit
#echo $$ > /var/run/node-app.pid
exec /usr/bin/node /path/to/app.js 2>&1 >> /path/to/log/file/app.log
end script
#Logs start and stop time timestamps to the file
pre-start script
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Starting" >> /path/to/log/file/app.log
end script
pre-stop script
#Uncomment if you need a pid file for monit
#rm /var/run/yourprogram.pid
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Stopping" >> /path/to/log/file/app.log
end script
Now start and stop your process by using service:
sudo service node-app start
Switch start with stop or status if needed.
If you are not using monit, just remove the pid lines. I really recommend using monit because you can configure it to give you email alert if your process dies or an error occurs in the log file.
What i would do:
install ubuntu
install apache and SVN, with svn repos accessible via http://
create a svn for each project
create a svn commit hook to autodeploy scripts to a folder
/svn//hooks/post-commit
REPOS="$1"
REV="$2"
cd /var/target/path
svn cleanup /var/target/path/
svn checkout -q --force file:///svn/ /var/target/path
svn cleanup /var/target/path/
exit 0
/svn//hooks/post_commit.sh
#!/bin/bash
REPOS="$1"
REV="$2"
# Files and directories can be distinguished, as directory paths are displayed with a trailing "/" character.
LOOK=/usr/bin/svnlook
SVN=/svn/
DEV=/var/target/path/
#mkdir /var/tmp/svn
cd /var/tmp/svn
for changes in `$LOOK changed $REPOS | awk '{print $1 "=" $2;}'`;
do
len=${#changes}
idx=`expr index "$changes" =`;
directory=${changes:$idx};
action=${changes:0:$idx-1};
if [ ${changes:len-1} = '/' ]
then
case "$action" in
"A" ) \
mkdir --mode=775 -p $DEV/$directory;
chown nobody:nobody $DEV/$directory;
chmod 775 $DEV/$directory;
;;
"D" ) \
rmdir $DEV/$directory;
;;
esac
else
case "$action" in
"A"|"U"|"UU" ) \
$SVN export --force --non-interactive -r HEAD -q file://$REPOS/$directory;
BASE=`basename $directory`;
DIR=`dirname $directory`;
chown nobody:nobody $BASE;
chmod 775 $BASE;
mkdir --mode=775 -p $DEV/$DIR;
cp -f --preserve=ownership $BASE $DEV/$DIR;
unlink $BASE;
;;
"D" ) \
rm -f $DEV/$directory;
;;
esac
fi
done
echo Updated dev subdomain
exit 1
install nodejs
install nodemon with npm install -g nodemon (out of my head - please check the manual)
create an upstart file for the node js script in /etc/init
description "nodejs with nodemonn"
author "etc"
start on startup
respawn
script
cd /var/target/project/dir/
exec nodemon /var/target/project/dir/main.js
end script
now all you have to do is work on the code and commit
when you commit the code is updated and the nodejs script is restarted
have fun coding !
I don't think samba is required, unless you want to develop on the server via samba. If you want to do that just skip the svn part and install samba, but setup nodemon upstart scripts - it will save you alot of hassle os ssh-ing.
I work this way because it allows me to write and test locally and then comit code to dev/test/prod servers quick and easy.
To solve the port 80 issue, just setup a firewall rule to redirect incoming tcp/80 to tcp/8080 (for instance) and listen in your nodejs script on that port.
More information here: https://serverfault.com/questions/112795/how-can-i-run-a-server-on-linux-on-port-80-as-a-normal-user
p.s. i did not add a complete step-by-step instruction for each item because there are plenty of guides out there that would a much better job than i can
on Ubuntu: startup booting Nodejs + socket.io by ubuntu user:
more /etc/init/noded.conf
# Ubuntu upstart file at /etc/init/noded.conf
description "noded.conf"
author "Nguyen Thanh Binh"
start on runlevel [2345]
stop on runlevel [06]
respawn
script
su - ubuntu -c "NODE_ENV=test exec sudo /usr/bin/node /home/ubuntu/server.js" >> /home/ubuntu/log.log &
end script
You can use forvere module to start a nodejs application in the background.
for full info see this http://blog.nodejitsu.com/keep-a-nodejs-server-up-with-forever.
and I am just running an Ubuntu dedicated server with nodejs and it's working without a problem
I would like to automatically run node server when instances are created (using forever). I am on Ubuntu 11.10 (Canonical), I followed the instructions here exactly on creating launch config using user script: http://alestic.com/2011/11/ec2-schedule-instance
I can't seem to get this to work. Below is my startup script:
#!/bin/bash
set -e -x
/home/MyUserName/node_modules/.bin/forever stopall
/home/MyUserName/node_modules/.bin/forever start node.js/app.js
The launch config is created using this cmd:
as-create-launch-config MyLC --image-id ami-b6a3f8f2 --user-data-file user-data-script.sh --instance-type m1.small
Found the issue, I have to run forever as the user, not root, wonder why...like so:
sudo -u MyUserName /home/MyUserName/node_modules/.bin/forever start node.js/app.js
Are you fully qualifying the app.js file? Could it just be this line?
/home/MyUserName/node_modules/.bin/forever start /home/MyUserName/node.js/app.js