I have an app that I want to run on a single self contained Docker image.
I had it running fine on an Ubuntu based image, but the same script now causes me trouble on Alpine.
Here is my docker file :
FROM julienlengrand/alpine-node-rethinkdb
# Preparing
# RUN ln -snf /bin/bash /bin/sh
# # Define mountable directories.
VOLUME ["/data"]
# # Define working directory.
WORKDIR /data
# # Install app dependencies
COPY package.json /data
RUN npm install
# # Bundle app source
COPY . /data
# # Expose rethinkdb ports.
# - 8080: web UI
# - 28015: process
# - 29015: cluster
EXPOSE 8080
#EXPOSE 28015
#EXPOSE 29015
# Expose node app ports
EXPOSE 4567
CMD [ "/bin/sh", "/data/startApp.sh" ]
My startApp script is relatively simple :
#!/bin/sh
rethinkdb --bind all & sleep 1; node dbCreate.js; sleep 2; nohup node workers/worker.js & node app.js
But when I try to run it, I get the following error:
module.js:442
throw err;
^
'rror: Cannot find module '/data/app.js
at Function.Module._resolveFilename (module.js:440:15)
at Function.Module._load (module.js:388:25)
at Module.runMain (module.js:575:10)
at run (bootstrap_node.js:352:7)
at startup (bootstrap_node.js:144:9)
at bootstrap_node.js:467:3
This happens whether I run it automatically, or directly within the image using the shell.
I have checked and everything is correctly placed in the data folder.
Additionally, if I run all the commands one after the other directly in the sh shell everything runs as expected.
I have also tried to simplify my script as such :
#!/bin/sh
rethinkdb --bind all & sleep 1; node dbCreate.js; sleep 2; node app.js
bu the same issue happens.
Any idea what can go wrong? What could make my /data folder unavailable when running via the startApp script? Could it be that it is a specificity from Alpine?
Thanks,
The error message you are receiving looks like a classic carriage return issue as the quote after app.js has moved to the start of the line
'rror: Cannot find module '/data/app.js
Node should normally be able to deal with both line endings but shell scripts aren't so kind.
I generally default all projects/files/editors/git to a Unix \n unless there are specific requirements not too.
You can convert existing files with dos2unix or one of the answers in the question jlengrand found. I like perl -pi -e 's/\r\n/\n/g', because pie
Related
I have created a simple Flask app which I am trying to deploy to Docker.
The basic user interface will load on localhost, but when I execute a command which calls a specific function, it keeps showing:
"Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."
Looking at Docker logs I can see the problem is that the file cannot be found by the subprocess.popen command:
"FileNotFoundError: [Errno 2] No such file or directory: 'test_2.bat': 'test_2.bat'
172.17.0.1 - - [31/Oct/2019 17:01:55] "POST /login HTTP/1.1" 500"
The file certainly exists in the Docker environment, within the container I can see it listed in the root directory.
I have also tried changing:
item = subprocess.Popen(["test_2.bat", i], shell=False,stdout=subprocess.PIPE)
to:
item = subprocess.Popen(["./test_2.bat", i], shell=False,stdout=subprocess.PIPE)
which generated the alternative error:
"OSError: [Errno 8] Exec format error: './test_2.bat'
172.17.0.1 - - [31/Oct/2019 16:58:54] "POST /login HTTP/1.1" 500"
I have added a shebang to the top of both .py files involved in the Flask app (although I may have done this wrong):
#!/usr/bin/env python3
and this is the Dockerfile:
FROM python:3.6
RUN adduser lighthouse
WORKDIR /home/lighthouse
COPY requirements.txt requirements.txt
# RUN python -m venv venv
RUN pip install -r requirements.txt
RUN pip install gunicorn
COPY templates templates
COPY json_logs_nl json_logs_nl
COPY app.py full_script_manual_with_list.py schema_all.json ./
COPY bq_load_indv_jsons_v3.bat test_2.bat ./
RUN chmod 644 app.py
RUN pip install flask
ENV FLASK_APP app.py
RUN chown -R lighthouse:lighthouse ./
USER lighthouse
# EXPOSE 5000
CMD ["flask", "run", "--host=0.0.0.0"]
I am using Ubuntu and WSL2 to run Docker on a Windows machine without a virtual box. I have no trouble navigating my Windows file system or building Docker images so I think this configuration is not the problem - but just in case.
If anyone has any ideas to help subprocess locate test_2.bat I would be very grateful!
Edit: the app works exactly as expected when executed locally via the command line with "flask run"
If anyone is facing a similar problem, the solution was to put the command directly into the Python script rather than calling it in a separate file. It is split into separate strings to allow the "url" variable to be iteratively updated, as this all occurs within a for loop:
url = str(i)
var_command = "lighthouse " + url + " --quiet --chrome-flags=\" --headless\" --output=json output-path=/home/lighthouse/result.json"
item = subprocess.Popen([var_command], stdout=subprocess.PIPE, shell=True)
item.communicate()
As a side note, if you would like to run Lighthouse within a container you need to install it just as you would to run it on the command line, in a Node container. This container can then communicate with my Python container if both deployed in the same pod via Kubernetes and share a namespace. Here is a Lighthouse container Dockerfile I've used: https://github.com/GoogleChromeLabs/lighthousebot/blob/master/builder/Dockerfile
I have been stuck trying to figure out how to edit a python flask code after pulling from a Docker Hub repository on a different computer. I want to create a Folder in my Linux Desktop that contains all of the packages the image has when running as a container (Dockerfile, requirements.txt, app.py) that way I can edit the app.py regardless of what computer I have or even if my classmates want to edit it they can simply just pull my image, run the container, and be able to have a copy of the code saved on their local machine for them to open it using Visual Studio Code (or any IDE) and edit it. This is what I tried.
I first pulled from the Docker hub:
sudo docker pull woonx/dockertester1
Then used this command to run the image as a container and create a directory:
sudo docker run --name=test1 -v ~/testfile:/var/lib/docker -p 4000:80 woonx/dockertester1
I was able to create a local directory called testfile but it was an empty folder when I opened it. No app.py, dockerfile, nothing.
The example code I am using to test is from following the example guide on the Docker website: https://docs.docker.com/get-started/part2/
Dockerfile:
# Use an official Python runtime as a parent image
FROM python:2.7-slim
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY . /app
# Install any needed packages specified in requirements.txt
RUN pip install --trusted-host pypi.python.org -r requirements.txt
# Make port 80 available to the world outside this container
EXPOSE 80
# Define environment variable
ENV NAME World
# Run app.py when the container launches
CMD ["python", "app.py"]
requirements.txt:
Flask
Redis
app.py:
from flask import Flask
from redis import Redis, RedisError
import os
import socket
# Connect to Redis
redis = Redis(host="redis", db=0, socket_connect_timeout=2, socket_timeout=2)
app = Flask(__name__)
#app.route("/")
def hello():
try:
visits = redis.incr("counter")
except RedisError:
visits = "<i>cannot connect to Redis, counter disabled</i>"
html = "<h3>Hello {name}!</h3>" \
"<b>Hostname:</b> {hostname}<br/>" \
"<b>Visits:</b> {visits}"
return html.format(name=os.getenv("NAME", "world"), hostname=socket.gethostname(), visits=visits)
if __name__ == "__main__":
app.run(host='0.0.0.0', port=80)
What I do is;
First, I issue docker run command.
sudo docker run --name=test1 -v ~/testfile:/var/lib/docker -p 4000:80 woonx/dockertester1
At this stage, files are created in container. Then I stop the container (lets say container id is 0101010101) .
docker container stop 0101010101
What I do is simply copying those files from container to the appropriate directory on my machine by using :
docker cp <container_name>:/path/in/container /path/of/host
or
cd ~/testfile
docker cp <container_name>:/path/in/container .
So, You have the files craeted by docker run on you local host. Now you can use them with -v option.
sudo docker run --name=test1 -v ~/testfile:/var/lib/docker -p 4000:80 woonx/dockertester1
Normally, when you change a setting in your configuration, it should be enough to stop/start container to take in action.
I hope this approach solves your problem.
Regards
I've installed an npm package / script in a JAIL on FreeNAS 9.10. (FreeBSD based)
It works perfectly if I run "npm start" in the directory where the scripts are installed.
However, I need this to be auto-starting when the jail starts. I don't know now to do that. Do I need to create an rc script?
Basically all I need to do is give the "npm start" in the correct directory on start up. How do I do that?
thanks
Yes, you can place an rc script within the jail and enable it using the jail's /etc/rc.conf file.
But, for a quick and dirty solution, you could create a /etc/rc.local script (also within the jail's environment) and put your startup commands in there.
See the manual page here.
Don't know about npm start, but for node.js I made such RC srcipt:
#!/bin/sh
# $FreeBSD: 340872 2014-01-24 00:14:07Z mat $
#
# PROVIDE: SERVICENAME
# REQUIRE: NETWORKING
# KEYWORD: shutdown
#
# Add the following line to /etc/rc.conf to enable SERVICENAME:
#
# SERVICENAME_enable="YES"
#
. /etc/rc.subr
name="SERVICENAME"
rcvar=SERVICENAME_enable
pidfile=${SERVICENAME_pidfile:-"/var/run/SERVICENAME.pid"}
command="/usr/sbin/daemon"
#command_args="-r -u USERNAME -P /var/run/SERVICENAME.pid /usr/local/bin/node /home/USERNAME/PROGDIR" # cjayho: restart if crashed
command_args="-u USERNAME -P /var/run/SERVICENAME.pid /usr/local/bin/node /home/USERNAME/PROGDIR"
load_rc_config $name
: ${SERVICENAME_enable:="NO"}
run_rc_command "$1"
name this file something like SERVICENAME and put to /usr/local/etc/rc.d
to enable automatic startup run command as root:
sysrc SERVICENAME_enable="YES"
do not forget to replace SERVICENAME, USERNAME and PROGDIR to your values, and add
process.chdir('/home/USERNAME/PROGDIR')
to your entry js file.
I want to be able to run node inside a docker container, and then be able to run docker stop <container>. This should stop the container on SIGTERM rather than timing out and doing a SIGKILL. Unfortunately, I seem to be missing something, and the information I have found seems to contradict other bits.
Here is a test Dockerfile:
FROM ubuntu:14.04
RUN apt-get update && apt-get install -y curl
RUN curl -sSL http://nodejs.org/dist/v0.11.14/node-v0.11.14-linux-x64.tar.gz | tar -xzf -
ADD test.js /
ENTRYPOINT ["/node-v0.11.14-linux-x64/bin/node", "/test.js"]
Here is the test.js referred to in the Dockerfile:
var http = require('http');
var server = http.createServer(function (req, res) {
console.log('exiting');
process.exit(0);
}).listen(3333, function (err) {
console.log('pid is ' + process.pid)
});
I build it like so:
$ docker build -t test .
I run it like so:
$ docker run --name test -p 3333:3333 -d test
Then I run:
$ docker stop test
Whereupon the SIGTERM apparently doesn't work, causing it to timeout 10 seconds later and then die.
I've found that if I start the node task through sh -c then I can kill it with ^C from an interactive (-it) container, but I still can't get docker stop to work. This is contradictory to comments I've read saying sh doesn't pass on the signal, but might agree with other comments I've read saying that PID 1 doesn't get SIGTERM (since it's started via sh, it'll be PID 2).
The end goal is to be able to run docker start -a ... in an upstart job and be able to stop the service and it actually exits the container.
My way to do this is to catch SIGINT (interrupt signal) in my JavaScript.
process.on('SIGINT', () => {
console.info("Interrupted");
process.exit(0);
})
This should do the trick when you press Ctrl+C.
Ok, I figured out a workaround myself, which I'll venture as an answer in the hope it helps others. It doesn't completely answer why the signals weren't working before, but it does give me the behaviour I want.
Using baseimage-docker seems to solve the issue. Here's what I did to get this working with the minimal test example above:
Keep test.js as is.
Modify Dockerfile to look like the following:
FROM phusion/baseimage:0.9.15
# disable SSH
RUN rm -rf /etc/service/sshd /etc/my_init.d/00_regen_ssh_host_keys.sh
# install curl and node as before
RUN apt-get update && apt-get install -y curl
RUN curl -sSL http://nodejs.org/dist/v0.11.14/node-v0.11.14-linux-x64.tar.gz | tar -xzf -
# the baseimage init process
CMD ["/sbin/my_init"]
# create a directory for the runit script and add it
RUN mkdir /etc/service/app
ADD run.sh /etc/service/app/run
# install the application
ADD test.js /
baseimage-docker includes an init process (/sbin/my_init) which handles starting other processes and dealing with zombie processes. It uses runit for service supervision. The Dockerfile therefore sets the my_init process as the command to run on boot, and adds a script /etc/service for runit to pick it up.
The run.sh script is simple:
#!/bin/sh
exec /node-v0.11.14-linux-x64/bin/node /test.js
Don't forget to chmod +x run.sh!
By default, runit will automatically restart the service if it goes down.
Following these steps (and build, run, and stop as before), the container properly responds to requests for it to shutdown, in a timely fashion.
I'm setting up a container with the following Dockerfile
# Start with project/baseline
FROM project/baseline # => image with mongo / nodejs / sailsjs
# Create folder that will contain all the sources
RUN mkdir -p /var/project
# Load the configuration file and the deployment script
ADD init.sh /var/project/init.sh
ADD src/ /var/project/ # src contains a list of folder, each one being a sails app
# Compile the sources / run the services / run mongodb
CMD /var/project/init.sh
The init.sh script is called when the container runs.
It should start a couple of webapp and mongodb.
#!/bin/bash
PROJECT_PATH=/var/project
# Start mongodb
function start_mongo {
mongod --fork --logpath /var/log/mongodb.log # attempt to have mongo running in daemon
}
# Start services
function start {
for service in $(ls);do
cd $PROJECT_PATH/$service
npm start # Runs sails lift on each service
done
}
# start mongodb
start_mongo
# start web applications defined in /var/project
start
Basically, there is a couple of nodejs (sailsjs) application in /var/project.
When I run the container, I got the following message:
$ sudo docker run -t -i projects/test
about to fork child process, waiting until server is ready for connections.
forked process: 10
and then it remains stuck.
How can mongo and the sails processes can be started and the container to remain in a running state ?
UPDATE
I now use this supervisord.conf file
[supervisord]
nodaemon=false
[program:mongodb]
command=/usr/bin/mongod
[program:process1]
command=/bin/bash "cd /var/project/service1 && node app.js"
[program:process2]
command=/bin/bash "cd /var/project/service2 && node app.js"
it is called in the Dockerfile like:
# run the applications (mongodb + project related services)
CMD ["/usr/bin/supervisord"]
As my services are dependent upon mongo starting correctly, supervisord does not wait that long and the services are not started then. Any idea to solve that ?
By the way, it that a so best practice to use mongo in the same container ?
UPDATE 2
I went back to a service.sh script that is called when the container is running. I know this is not clean (but I'll say it's temporary so I can fix the pb I have in supervisor), but I'm doing the following:
run nohup mongod &
wait 60 sec
run my node (forever) processes
The thing is, the container exit right after the forever processes are ran... how can it be kept active ?
If you want to cleanly start multiple services inside a container, one option is to use a process supervisor of some sort. One option is documented here, in the official Docker documentation.
I've done something similar using runit. You can see my base runit image here, and a multi-service application image using that here.