debug Fast Api with docker compose in pycharm - python-3.x

I am trying to debug mode from pycharm configurations but still no success. From terminal with docker-compose up it runs successfully in run mode and not in debug. Any ideas what is happening and what might be the issue?
Here's the docker file:
FROM python:3.8-slim-buster
EXPOSE 80
WORKDIR api
CMD apt-get --assume-yes update \
&& apt-get --assume-yes upgrade \
&& apt-get --assume-yes install libpq-dev build-essential python3-dev
COPY api/ .
CMD ls
RUN pip install -r requirements.txt
ENTRYPOINT chmod +x ./scripts/start.sh \
&& ./scripts/start.sh
start.sh script is as below:
#! /usr/bin/env sh
set -e
# Unicorn is used for local development, gunicorn for production
# Pre-start script is used to execute commands that need to be run before
# opening the server such as migrations etc
U_EXEC_PATH=${U_EXEC_PATH:-./api/scripts/start-uvicorn.sh}
G_EXEC_PATH=${G_EXEC_PATH:-./api/scripts/start-gunicorn.sh}
PRE_START_PATH=${PRE_START_PATH:-./api/scripts/pre-start.sh}
# Common variables
MODULE_NAME=${MODULE_NAME:-api.app.main}
VARIABLE_NAME=${VARIABLE_NAME:-app}
export APP_MODULE=${APP_MODULE:-"$MODULE_NAME:$VARIABLE_NAME"}
export S_HOST=${S_HOST:-0.0.0.0}
export S_PORT=${S_PORT:-80}
export S_LOG_LEVEL=${S_LOG_LEVEL:-info}
export S_APP_ENV=${S_APP_ENV:-local}
echo "Checking for script in $PRE_START_PATH"
if [ -f $PRE_START_PATH ] ; then
echo "Running script $PRE_START_PATH"
. "$PRE_START_PATH"
else
echo "There is no script $PRE_START_PATH"
fi
cd ..
if [ $S_APP_ENV = "local" ]
then
echo "Local environment."
echo "Starting start-uvicorn.sh"
. "$U_EXEC_PATH"
else
echo "$S_APP_ENV environment."
echo "Starting start-gunicorn.sh"
. "$G_EXEC_PATH"
fi
and start-uvicorn.sh
#! /usr/bin/env sh
set -e
# Start Uvicorn with live reload
echo "host": $S_HOST
echo "port": $S_PORT
echo "module": $APP_MODULE
echo "log level": $S_LOG_LEVEL
exec uvicorn --reload --host $S_HOST --port $S_PORT --log-level $S_LOG_LEVEL "$APP_MODULE"

Related

Unable to connect Filestore from Cloudrun

I want to connect Filestore from CloudRun , I have defined it on my run.sh script that run the node app and mount
command to connect to the filestore , my node app is running on cloud run but not able to mount to the filestore , I have
attached a link to my nodejs codes , also in my script after the node command no other command runs.
I am following the official Google doc.
Problem on my run script:
node /app/index.js //working on cloudrun
mkdir -p $MNT_DIR //not working on cloudrun
chmod 775 $MNT_DIR //not working on cloudrun
echo "Mounting Cloud Filestore." //not working on cloudrun
mount --verbose -t nfs -o vers=3 -o nolock 10.67.157.122:/filestore_vol1/test/testing/ $MNT_DIR //not working
echo "Mounting completed." //not working on cloudrun
Note :- if I place node /app/index.js after echo "Mounting completed." //node app doesn't starts on cloudrun
I am attaching my code URL here.
My Docker file:
FROM node:slim
# Install system dependencies
RUN apt-get update -y && apt-get install -y \
tini \
nfs-common \
procps \
&& apt-get clean
# Set working directory
WORKDIR /app
# Set fallback mount directory
ENV MNT_DIR /app2
# Copy package.json to the working directory
COPY package*.json ./
# Copy all codes to the working directory
COPY . .
# Ensure the script is executable
RUN chmod +x /app/run.sh
# Use tini to manage zombie processes and signal forwarding
ENV TINI_VERSION v0.19.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini
ENTRYPOINT ["/tini", "--"]
ENV PORT=8080
EXPOSE 8080
EXPOSE 2049
# Pass the startup script as arguments to tini
CMD ["/app/run.sh"]
# My run.sh script file
#!/bin/bash
set -eo pipefail
node /app/index.js
# Create mount directory for service.
mkdir -p $MNT_DIR
chmod 775 $MNT_DIR
echo "Mounting Cloud Filestore."
mount --verbose -t nfs -o vers=3 -o nolock 10.x.x.122:/filestore_vol1/test/testing/ $MNT_DIR
echo "Mounting completed."
# Exit immediately when one of the background processes terminate.
wait -n
#main goal is to mount cloud run with filestore and start my node app
I also spent 2 days on that. In my case, one dependency was missing in the container. Try this line instead
RUN apt-get update -y && apt-get install -y \
tini \
nfs-common \
netbase \
procps \
&& apt-get clean
Netbase solved my issue. Let me know if it's also your case!

Running a process with nobody user with gosu

I am trying to run a process with nobody user in Linux, currently this is being run as a root user but since this process doesn't require the root access so I want to use nobody with gosu. The problem is even after activating the nobody user and running the process with that, when I do " ps aux" it shows that all processes are being run by root. Do I need to do something more after activating the nobody user to make it possible to run the process. The process I am trying to run with nobody is rails s -b 0.0.0.0
Below is my dockerfile
FROM ruby:3.0.1
EXPOSE $PORT
WORKDIR /srv
COPY Gemfile Gemfile.lock /srv/
COPY . /srv
RUN apt-get update -qq && apt-get install -y build-essential iproute2 libpq-dev nodejs && apt-
get clean && bundle install --no-cache
#activating the nobody user account
RUN chsh -s /bin/bash nobody
RUN set -eux; \
apt-get install -y gosu; \
rm -rf /var/lib/apt/lists/*; \
gosu nobody true
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["server"]
Here is the docker-entrypoint.sh
#!/bin/sh
export BASH_SHELL=$(cat /etc/shells | grep /bash)
export ASH_SHELL=$(cat /etc/shells | grep /ash)
#Setting available Shell to $SHELL_PROFILE
if [ -n "$BASH_SHELL" ];
then
SHELL_PROFILE=$BASH_SHELL
elif [ -n "$ASH_SHELL" ];
then
SHELL_PROFILE=$ASH_SHELL
else
SHELL_PROFILE=sh
fi
rm -f tmp/pids/puma.5070.pid tmp/pids/server.pid
XRAY_ADDRESS="$(ip route | grep default | cut -d ' ' -f 3):2000"
export AWS_XRAY_DAEMON_ADDRESS=$XRAY_ADDRESS
echo "export AWS_XRAY_DAEMON_ADDRESS=$XRAY_ADDRESS" >> /root/.bashrc
case "$*" in
shell)
exec $SHELL_PROFILE
;;
server)
# gosu command to run rails s -b 0.0.0.0 process as nobody user
gosu nobody:nogroup bundle exec rails s -b 0.0.0.0
;;
*)
exec $#
;;
esac
Don't bother installing gosu or another tool; just set your Docker image to run as the nobody user (or some other non-root user). Do this at the very end of your Dockerfile, where you otherwise declare the CMD.
# Don't install gosu or "activate a user"; but instead
USER nobody
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["rails", "server", "-b", "0.0.0.0"]
In turn, that means you can remove the gosu invocation from the entrypoint script. I might remove most of it and trim it down to
#!/bin/sh
# Clean up stale pid files
rm -f tmp/pids/*.pid
# (Should this environment variable be set via `docker run`?)
export AWS_XRAY_DAEMON_ADDRESS="$(ip route | grep default | cut -d ' ' -f 3):2000"
# Run whatever the provided command was, in a Bundler context
exec bundle exec "$#"
If you need an interactive shell to debug the image, you can docker run --rm -it the-image bash which works on many images (provided they (a) honor CMD and (b) have bash installed); you don't need a special shell artificial command and you don't need to detect what's installed in the (fixed) image.

Installing nvm on Elastic Beanstalk

I'm trying to install nvm on my Elastic Beanstalk instance because our rails application requires node 6.9.5, currently node 4 exists on the instance. I'm running the commands
01_node_install:
command: "sudo yum install make glibc-devel gcc patch openssl-devel c++"
02_node_install:
command: "curl https://raw.githubusercontent.com/creationix/nvm/v0.16.1/install.sh | sh"
03_node_install:
command: "source ~/.bash_profile"
04_node_install:
command: "nvm install 6.9.5"
05_node_install:
command: "nvm alias default 6.9.5"
I am getting an error of
=> Profile not found. Tried ~/.bashrc, ~/.bash_profile, ~/.zshrc, and ~/.profile.
=> Create one of them and run this script again
OR
=> Append the following lines to the correct file yourself:
export NVM_DIR="/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm
=> Close and reopen your terminal to start using nvm
What's strange to me is that it's saying the file ~/.bash_profile does not exist but I can ssh into the instance and see it. I tried to echo the lines into the file but got similar errors.
Any help would be appreciated!
I noticed that this question has gotten at least some viewers over the past bit. I was able to do this with the following commands.
000_dd:
command: echo “noswap”#dd if=/dev/zero of=/swapfile bs=1M count=3072
001_mkswap:
command: echo “noswap”#mkswap /swapfile
002_swapon:
command: echo “noswap”#swapon /swapfile
01-install-nvm:
command: curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.6/install.sh | bash
02-setup-bashrc:
command: |
cat << EOF >> /etc/bashrc
export NVM_DIR="/.nvm"
[ -s "\$NVM_DIR/nvm.sh" ] && \. "\$NVM_DIR/nvm.sh" # This loads nvm
[ -s "\$NVM_DIR/bash_completion" ] && \. "\$NVM_DIR/bash_completion" # This loads nvm bash_completion
EOF
03-install-node:
command: source /etc/bashrc && nvm install 6.9.5
04-set-node-default:
command: source /etc/bashrc && nvm alias default 6.9.5
05-set-node-default:
command: source /etc/bashrc && ln -sf $(nvm which 6.9.5) /usr/bin/node
Probably some poorly named commands but I hope this helps some people!

Docker Bash prompt does not display color output

I use the command docker run --rm -it govim bash -l to run Docker images, but it does not display color output.
If I source ~/.bash_profile or run bash -l again, output will then correctly be output with color.
Bash Prompt Image
My bash_profile and bash_prompt files.
The OP SolomonT reports that docker run with env do work:
docker run --rm -it -e "TERM=xterm-256color" govim bash -l
And Fernando Correia adds in the comments:
To get both color support and make tmux work, I combined both examples:
docker exec -it my-container env TERM=xterm-256color script -q -c "/bin/bash" /dev/null
As chepner commented (earlier answer), .bash_profile is sourced (itis an interactive shell), since bash_prompt is called by .bash_profile.
But docker issue 9299 illustrates that TERM doesn't seem to be set right away, forcing the users to open another bash with:
docker exec -ti test env TERM=xterm-256color bash -l
You have similar color issues with issue 8755.
To illustrate/reproduce the problem:
docker exec -ti $CONTAINER_NAME tty
not a tty
The current workaround is :
docker exec -ti `your_container_id` script -q -c "/bin/bash" /dev/null
Both are supposing you have a running container first, which might not be convenient here.
Based on VonC's answer I adding the following to my Dockerfile (which allows me to run the container without typing the environment variables on the command line every time):
ENV TERM xterm-256color
#... more stuff
CMD ["bash", "-l"]
And sure enough it works with:
docker run -it my-image:tag
For tmux to work with color, in my ~/.tmux.conf I need:
set -g default-terminal "screen-256color"
and for UTF-8 support in tmux, in my ~/.bashrc:
alias tmux='tmux -u'
My Dockerfile:
FROM fedora:26
ENV TERM xterm-256color
RUN dnf upgrade -y && \
dnf install golang tmux git vim -y && \
mkdir -p /app/go/{bin,pkg,src} && \
echo 'export GOPATH=/app/go' >> $HOME/.bashrc && \
echo 'export PATH=$PATH:$GOPATH/bin' >> $HOME/.bashrc && \
mkdir -p ~/.vim/autoload ~/.vim/bundle && \
curl -LSso ~/.vim/autoload/pathogen.vim \
https://tpo.pe/pathogen.vim && \
git clone https://github.com/farazdagi/vim-go-ide.git \
~/.vim_go_runtime && \
bash ~/.vim_go_runtime/bin/install && \
echo "alias govim='vim -u ~/.vimrc.go'" >> ~/.bashrc && \
echo "alias tmux='tmux -u'" >> ~/.bashrc && \
echo 'set -g default-terminal "screen-256color"' >> ~/.tmux.conf
CMD ["bash", "-l"]
The Dockerfile builds an image based off Fedora 26, updates it, installs a few packages (Git, Vim, golang and tmux), installs the pathogen plugin for Vim, then it installs a Git repository from here vim-go-ide and finally does a few tweaks to a few configuration files to get color and UTF-8 working fine. You just need to add persistent storage, probably mounted under /app/go.
If you have an image with all the development tools already installed, just make a Dockerfile with ENV statement and add the commands to modify the configuration files in a RUN statement without the installation commands and use your base image in the FROM statement. I prefer this solution because I'm lazy and (besides the initial setup) it saves typing when you want to run the image.
Using Vim and plugins within tmux
Adding -t is working for me:
docker exec -t vendor/bin/phpunit
Adding to VonC's answer, I made this Bash function:
drun() { # start container with the specified entrypoint and colour terminal
if [[ $# -lt 2 ]]; then
echo "drun needs 2+ arguments: image entrypoint" >&2
return
fi
docker run -ti -e "TERM=xterm-256color" "$#"
}
I think this is something that you'd have to implement manually. My container has Python, so here's how I print in color using a single line:
Example Docker file:
FROM django:python3
RUN python -c "print('\033[90m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[91m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[92m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[93m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[94m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[95m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[96m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[97m HELLO_WORLD \033[0m')"
RUN python -c "print('\033[98m HELLO_WORLD \033[0m')"
Standard terminal:
You need to add the following line to your Dockerfile:
RUN echo PS1="'"'\[\e]0;\u#\h: \w\a\]${debian_chroot:+($debian_chroot)}\ \033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '"'" >> /app/.bashrc
Change the /app/.bashrc to where your .bashrc file is in the docker.
If you want ls command to have colors too add this line:
RUN echo alias ls="'"'ls --color=auto'"'" >> /app/.bashrc

Why doesn't the cron service in Dockerfile run?

While searching for this issue I found that: cron -f should start the service.
So I have:
RUN apt-get install -qq -y git cron
Next I have:
CMD cron -f && crontab -l > pullCron && echo "* * * * * git -C ${HOMEDIR} pull" >> pullCron && crontab pullCron && rm pullCron
My dockerfile deploys without errors but the cron doesn't run. What can I do to start the cron service with an added line?
PS:
I know that the git function in my cron should actually be a hook, but for me (and probably for others) this is about learning how to set crons with Docker :-)
PPS:
Complete Dockerfile (UPDATED):
RUN apt-get update && apt-get upgrade -y
RUN mkdir -p /var/log/supervisor
RUN apt-get install -qq -y nginx git supervisor cron wget
RUN echo "daemon off;" >> /etc/nginx/nginx.conf
RUN wget -O ./supervisord.conf https://raw.githubusercontent.com/..../supervisord.conf
RUN mv ./supervisord.conf /etc/supervisor/conf.d/supervisord.conf
RUN apt-get install software-properties-common -y && apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0x5a16e7281be7a449 && add-apt-repository 'deb http://dl.hhvm.com/ubuntu utopic main' && apt-get update && apt-get install hhvm -y
RUN cd ${HOMEDIR} && git clone ${GITDIR} && mv ./tybalt/* ./ && rm -r ./tybalt && git init
RUN echo "* * * * * 'cd ${HOMEDIR} && /usr/bin/git pull origin master'" >> pullCron && crontab pullCron && rm pullCron
EXPOSE 80
CMD ["/usr/bin/supervisord"]
PPPS:
Supervisord.conf:
[supervisord]
autostart=true
autorestart=true
nodaemon=true
[program:nginx]
command=/usr/sbin/nginx -c /etc/nginx/nginx.conf
[program:cron]
command = cron -f -L 15
autostart=true
autorestart=true
Having started crond with supervisor, your cron jobs should be executed. Here are the troubleshooting steps you can take to make sure cron is running
Is the cron daemon running in the container? Login to the container and run ps a | grep cron to find out. Use docker exec -ti CONTAINERID /bin/bash to login to the container.
Is supervisord running?
In my setup for instance, the following supervisor configuration works without a problem. The image is ubuntu:14.04. I have CMD ["/usr/bin/supervisord"] in the Dockerfile.
[supervisord]
nodaemon=true
[program:crond]
command = /usr/sbin/cron
user = root
autostart = true
Try another simple cron job to findout whether the problem is your cron entry or the cron daemon. Add this when logged in to the container with crontab -e :
* * * * * echo "hi there" >> /tmp/test
Check the container logs for any further information on cron:
docker logs CONTAINERID | grep -i cron
These are just a few troubleshooting tips you can follow.
Cron is not running because only the last CMD overrides the first one (as #xuhdev said). It's documented here : https://docs.docker.com/reference/builder/#cmd.
There can only be one CMD instruction in a Dockerfile. If you list more than one CMD then only the last CMD will take effect.
If you want to have nginx and cron running in the same container, you will need to use some kind of supervisor (like supervisord or others) that will be the pid 1 process of your container and manage the chield processes. I think this project should help : https://github.com/nbraquart/docker-nginx-php5-cron (it seems to do what you're trying to achieve).
Depending on what you're cron is here for, there would be other solution to that — like building a new image for each commit or each tags, etc...
I've used this with CentOS and it works:
CMD service crond start ; tail -f /var/log/cron
The rest of my Dockerfile just yum installs cronie and touches the /var/log/cron file so it will be there when the CMD runs.
On centos 7 this works for me
[program:cron]
command=/usr/sbin/crond -n -s
user = root
autostart = true
stderr_logfile=/var/log/cron.err.log
stdout_logfile=/var/log/cron.log
-n is for foreground
-s is to log to stdout and stderr
In my case, it turns out I needed to run cron start at run time. I can't put it in my Dockerfile nor docker-compose.yml, so I ended up placing in the Makefile I use for deploy.
Something like:
task-name:
# docker-compose down && docker-compose build && docker-compose up -d
docker exec CONTAINERNAME /bin/bash -c cron start

Resources