Bad Substitution Error Installing NVM within Debian-based Docker image - linux

I am trying to install nvm on my Docker image. I originally thought that this Docker image was built on Ubuntu, but it is actually built on Debian. I am installing bash to curl NVM, and subsequently install node, but I get a bad substitution error:
Here's my Dockerfile:
FROM docker
RUN apk add --update bash \
&& touch /root/.bashrc \
&& curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.2/install.sh | bash \
&& source /root/.bashrc \
&& nvm install node \
&& npm install
I think the following error has to do with the line && source /root/.bashrc \
=> Downloading nvm as script to '/root/.nvm'
0
=> Appending source string to /root/.bashrc
=> Close and reopen your terminal to start using nvm
/bin/sh: /root/.nvm/nvm.sh: line 107: syntax error: bad substitution
ERROR: Service 'docker' failed to build: The command '/bin/sh -c apk add --update bash && touch /root/.bashrc && curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.2/install.sh | bash && source /root/.bashrc && nvm install node && npm install' returned a non-zero code: 2
Do you see what is causing this bad substitution error, and is there a more simple way to install nvm on a Debian based Docker image? Thanks for any help.

Docker image is based out of Alpine Linux. Alpine Linux uses the default shell as sh. The error is because of the sh vs bash incompatibilities.
Unfortunately, NVM home page has instructions about Alpine Linux, but quite discouraging:
nvm on Alpine Linux
After some changes, the final version that made nvm work with Alpine:
FROM docker
RUN apk add --update bash coreutils ncurses tar gzip nodejs \
&& touch ~/.bashrc \
&& curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.2/install.sh | sh \
&& LINE=$(cat /root/.nvm/nvm.sh | grep -in '{BASH_SOURCE\[0\]}' | awk -F: '{print $1}') \
&& sed -i "${LINE}s/BASH_SOURCE\[0\]\}/BASH_SOURCE\}\$\{0\}/" /root/.nvm/nvm.sh \
&& source ~/.bashrc \
&& nvm ls \
&& nvm install node \
&& nvm use --delete-prefix v6.3.1 \
&& npm install
A little inconvenience being, you need to use the nvm use --delete-prefix v6.3.1 every time you need to work with it.
I suggest to try #BMitch's updated answer as well.

FROM docker bases your image on the "Docker in Docker" Alpine image. Unless you have a special use case that requires Docker in Docker, this probably isn't the base image you want.
If you want a node image, consider using the premade node image. This is based on Debian jessie.
If you need to base your node install another version of Debian or Ubuntu, you can pick from multiple versions of those images, e.g. FROM debian:jessie.
Edit: it's pretty easy to add Docker to another image. Here are my Dockerfile entries for a Debian based image (appuser is a user added elsewhere that the container would normally run as, hence the Docker group addition):
ARG DOCKER_GID=999
USER root
RUN curl -sSL https://get.docker.com/ | sh
RUN groupmod -g ${DOCKER_GID} docker && \
usermod -aG docker appuser

Related

Unzip not handling utf-8 in Node Alpine Docker image: how to set correct locale?

With this zip file, this Node script successfully outputs the files:
const child_process = require('child_process')
const util = require('util')
const exec = util.promisify(child_process.exec)
exec(`unzip -Z1 metamorpR.zip`).then(zip_contents => {
if (zip_contents.stderr) {
throw new Error(`unzip error: ${zip_contents.stderr}`)
}
console.log(zip_contents.stdout)
})
metamorpR.z5
Варианты Прохождения.txt
Интерактивная Литература.pdf
But when I run the script from within Docker, it doesn't.
Using this Dockerfile:
FROM node:16-alpine
RUN apk add --no-cache unzip
COPY . .
ENTRYPOINT ["node", "unzip.js"]
Build and run (substitute in your container image name):
docker build .
docker run --rm 1dc072
Output:
metamorpR.z5
??????? ????????.txt
???????????? ??????????.pdf
I think this means the locales aren't set correctly within the Docker image? Any ideas how to fix this?
TL;DR
unzip on alpine doesn't appear to support locales. unzip on debian doesn't appear to support locales either. unzip on ubuntu supports using locales (however there exists no official node ubuntu image).
On ubuntu:
FROM ubuntu:18.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends \
locales \
unzip && \
apt-get clean
RUN sed -i -e 's/# ru_RU.UTF-8 UTF-8/ru_RU.UTF-8 UTF-8/' /etc/locale.gen && \
locale-gen && \
update-locale LANG=ru_RU.UTF-8 LC_ALL=ru_RU.UTF-8 && \
ldconfig
ENV LANG=ru_RU.UTF-8
COPY metamorpR.zip /metamorpR.zip
CMD ["unzip", "-l", "metamorpR.zip"]
... there are no issues in the unzip file name output:
... however the same build FROM node:16-bullseye won't produce the same results:
You could apply this patch during the build, then generate the locales, however unzip doesn't appear to use the locales:
FROM node:16-alpine
RUN apk add --no-cache unzip wget
RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub && \
wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.34-r0/glibc-2.34-r0.apk && \
wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.34-r0/glibc-bin-2.34-r0.apk && \
wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.34-r0/glibc-i18n-2.34-r0.apk && \
apk add glibc-2.34-r0.apk glibc-bin-2.34-r0.apk glibc-i18n-2.34-r0.apk && \
rm /glibc-2.34-r0.apk /glibc-bin-2.34-r0.apk /glibc-i18n-2.34-r0.apk && \
/usr/glibc-compat/bin/localedef -i ru_RU -f UTF-8 ru_RU.UTF-8
ENV LANG=ru_RU.UTF-8
COPY metamorpR.zip /metamorpR.zip
CMD ["unzip", "-l", "metamorpR.zip"]
Thanks to #masseyb's answer, I was able to get it working with this Dockerfile, which basically just installs Node manually into an Ubuntu image. The main downside is the image is twice the size, but it's comparatively simple so that's an acceptable downside to me.
FROM ubuntu:20.04
RUN apt-get update && \
apt install -y curl locales unzip && \
curl -fsSL https://deb.nodesource.com/setup_16.x | bash - && \
apt install -y nodejs && \
rm -rf /var/lib/apt/lists/* && \
localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.UTF-8
COPY . .
ENTRYPOINT ["node", "unzip.js"]
Apparently some versions of unzip that is available from Ubuntu repositories can handle automatic decoding of filenames if you specify the -a switch.

python3: command not found

I have a dockerfile in which I have specified the entrypint as shell script named run-services.sh
Contents of the shell script are as follows:
apache2ctl start
echo "Started apache2ctl..."
python3 mock_ta.py
Now when I deploy this service at my local machine I get an error saying
python3: command not found
I removed entrypoint and went inside the container and executed the which python3 command and I can see that python3 is installed at /usr/bin/python3.
Ideally it should run the python script if python is installed, right? Any idea why this happens?
============================================================
Edit:Added Dockerfile
FROM php:7.1-apache
# Utilities
RUN apt-get update && \
apt-get -y install apt-transport-https git curl vim --no-install-recommends && \
rm -r /var/lib/apt/lists/*
# SimpleSAMLphp
ARG SIMPLESAMLPHP_VERSION=1.15.2
RUN curl -s -L -o /tmp/simplesamlphp.tar.gz https://github.com/simplesamlphp/simplesamlphp/releases/download/v$SIMPLESAMLPHP_VERSION/simplesamlphp-$SIMPLESAMLPHP_VERSION.tar.gz && \
tar xzf /tmp/simplesamlphp.tar.gz -C /tmp && \
rm -f /tmp/simplesamlphp.tar.gz && \
mv /tmp/simplesamlphp-* /var/www/simplesamlphp && \
touch /var/www/simplesamlphp/modules/exampleauth/enable
COPY config/simplesamlphp/config.php /var/www/simplesamlphp/config
COPY config/simplesamlphp/authsources.php /var/www/simplesamlphp/config
COPY config/simplesamlphp/saml20-sp-remote.php /var/www/simplesamlphp/metadata
COPY config/simplesamlphp/server.crt /var/www/simplesamlphp/cert/
COPY config/simplesamlphp/server.pem /var/www/simplesamlphp/cert/
# Apache
COPY config/apache/ports.conf /etc/apache2
COPY config/apache/simplesamlphp.conf /etc/apache2/sites-available
COPY config/apache/cert.crt /etc/ssl/cert/cert.crt
COPY config/apache/private.key /etc/ssl/private/private.key
RUN echo "ServerName localhost" >> /etc/apache2/apache2.conf && \
a2enmod ssl && \
a2dissite 000-default.conf default-ssl.conf && \
a2ensite simplesamlphp.conf
COPY config/run-services.sh /var/www/simplesamlphp/config/run-services.sh
ENTRYPOINT ["/var/www/simplesamlphp/config/run-services.sh"]
# Set work dir
WORKDIR /var/www/simplesamlphp
# General setup
EXPOSE 8080 8443
Thanks #David
With your help I was able to figure out that the python3 image that was present inside container was not accessible indeed.
So I had to install python3 and pip packages with the help of following command
RUN apt update -y && apt upgrade -y && apt install -y python3 && apt install -y python3-pip

using a docker app to make a new directory in an external hard drive

I am using a docker container to execute a python script located at my host machine. The script should make a new directory at a target location.
When the target location is located under $HOME or $HOME/*, everything works. However, when I want to create a directory at /media/my_name/external_drive, the terminal says that PermissionError: [Errno 13] Permission denied: '/media/my_name'
Here is the code I run
sudo docker-compose run --rm --user="$(id -u):$(id -g)" main process_all.py
Here is docker-compose.yml:
version: '2.3'
services:
main:
build: .
volumes:
- .:/app
- /etc/localtime:/etc/localtime:ro
environment:
- PYTHONIOENCODING=utf_8
init: true
network_mode: host
Here is the dockerfile
FROM ubuntu:16.04
# Install some basic utilities
RUN apt-get update && apt-get install -y \
curl \
ca-certificates \
sudo \
git \
bzip2 \
axel \
&& rm -rf /var/lib/apt/lists/*
# Create a working directory
RUN mkdir /app
WORKDIR /app
# Create a non-root user and switch to it
RUN adduser --disabled-password --gecos '' --shell /bin/bash user \
&& chown -R user:user /app
RUN echo "user ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/90-user
USER user
# All users can use /home/user as their home directory
ENV HOME=/home/user
RUN chmod 777 /home/user
# Install Miniconda
RUN curl -so ~/miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-4.4.10-Linux-x86_64.sh \
&& chmod +x ~/miniconda.sh \
&& ~/miniconda.sh -b -p ~/miniconda \
&& rm ~/miniconda.sh
ENV PATH=/home/user/miniconda/bin:$PATH
# Create a Python 3.6 environment
RUN /home/user/miniconda/bin/conda install conda-build \
&& /home/user/miniconda/bin/conda create -y --name py36 python=3.6.4 \
&& /home/user/miniconda/bin/conda clean -ya
ENV CONDA_DEFAULT_ENV=py36
ENV CONDA_PREFIX=/home/user/miniconda/envs/$CONDA_DEFAULT_ENV
ENV PATH=$CONDA_PREFIX/bin:$PATH
# Ensure conda version is at least 4.4.11
# (because of this issue: https://github.com/conda/conda/issues/6811)
ENV CONDA_AUTO_UPDATE_CONDA=false
RUN conda install -y "conda>=4.4.11" && conda clean -ya
# Install FFmpeg
RUN conda install --no-update-deps -y -c conda-forge ffmpeg=3.2.4 \
&& conda clean -ya
# Install NumPy
RUN conda install --no-update-deps -y numpy=1.13.3 \
&& conda clean -ya
# Install build tools
RUN sudo apt-get update \
&& sudo apt-get install -y build-essential gfortran libncurses5-dev \
&& sudo rm -rf /var/lib/apt/lists/*
# Build and install CDF
RUN cd /tmp \
&& curl -O https://spdf.sci.gsfc.nasa.gov/pub/software/cdf/dist/cdf36_4/linux/cdf36_4-dist-all.tar.gz \
&& tar xzf cdf36_4-dist-all.tar.gz \
&& cd cdf36_4-dist \
&& make OS=linux ENV=gnu CURSES=yes FORTRAN=no UCOPTIONS=-O2 SHARED=yes all \
&& sudo make INSTALLDIR=/usr/local/cdf install
# Install other dependencies from pip
COPY requirements.txt .
RUN pip install -r requirements.txt
# Create empty SpacePy config (suppresses an annoying warning message)
RUN mkdir /home/user/.spacepy && echo "[spacepy]" > /home/user/.spacepy/spacepy.rc
# Copy scripts into the image
COPY --chown=user:user . /app
# Set the default command to python3
CMD ["python3"]
Untested, going by memory but I would debug the issue with an interactive version of your container.
Something like:
sudo docker run -t -i --rm --user="$(id -u):$(id -g)" main /bin/bash
You'll get a bash shell. Then you can debug it by
cd /media
ls -l
What I think you'll find is that the drive is probably not mounted. Or, the user doesn't have permission to access it.
With regards to mounts, either pass it through from the host or create a volume mount. I'm a little bit unsure about what you can do there because since I last used docker many changes around mounting and volume drivers were introduced. But the documentation on the docker website is pretty good. So experiment.
This is the cmd line reference for docker: https://docs.docker.com/engine/reference/run/
The key is to use the -t -i parameters to make it interactive.

How to run environment initialization shell script from Dockerfile

I am trying to build an API wrapped in a docker image that serves Openvino model. How do I run the "setupvars.sh" from Dockerfile itself so that my application can access it?
I have tried running the script using RUN. For ex: RUN /bin/bash setupvars.sh
or RUN ./setupvars.sh . However, none of them work and I get ModelNotFoundError: no module named openvino
RUN $INSTALL_DIR/install_dependencies/install_openvino_dependencies.sh
RUN cd /opt/intel/openvino/deployment_tools/model_optimizer/install_prerequisites && sudo ./install_prerequisites_tf.sh
COPY . /app
WORKDIR /app
RUN apt autoremove -y && \
rm -rf /openvino /var/lib/apt/lists/*
RUN /bin/bash -c "source $INSTALL_DIR/bin/setupvars.sh"
RUN echo "source $INSTALL_DIR/bin/setupvars.sh" >> /root/.bashrc
CMD ["/bin/bash"]
RUN python3 -m pip install opencv-python
RUN python3 test.py
I want OpenVino accessible to my gunicorn application that will serve the model in a docker image
Next commands works for me.
ARG OPENVINO_DIR=/opt/intel/computer_vision_sdk
# Unzip the OpenVINO installer
RUN cd ${APP_DIR} && tar -xvzf l_openvino_toolkit*
# installing OpenVINO dependencies
RUN cd ${APP_DIR}/l_openvino_toolkit* && \
./install_cv_sdk_dependencies.sh
# installing OpenVINO itself
RUN cd ${APP_DIR}/l_openvino_toolkit* && \
sed -i 's/decline/accept/g' silent.cfg && \
./install.sh --silent silent.cfg
# Setup the OpenVINO environment
RUN /bin/bash -c "source ${OPENVINO_DIR}/bin/setupvars.sh"
You need to re-run it every time you start the container, because those variables are only for the session.
Option 1:
Run your application something like this:
CMD /bin/bash -c "source /opt/intel/openvino/bin/setupvars.sh && python test.py"
Option 2 (not tested):
Add the source command to your .bashrc so it will be run every time on startup
# Assuming running as root
RUN echo "/bin/bash -c 'source /opt/intel/openvino/bin/setupvars.sh'" >> ~root/.bashrc
CMD python test.py
For the rest of the Dockerfile, there is a guide here (also not tested, and it doesn't cover the above):
https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_docker_linux.html
As mentioned in the two previous answers, the setupvars.sh script sets the environment variables required by OpenVINO.
But rather than running this every time, you can add the variables to your Dockerfile. While writing your Dockerfile run:
CMD /bin/bash -c "source /opt/intel/openvino/bin/setupvars.sh && printenv"
This will give you the values that the environment variables are set to. You might also want to run printenv without setting the OpenVINO variables:
CMD /bin/bash printenv
Comparing the two outputs will let you figure out exactly what the setupvars.sh script is setting.
Once you know the values set by the script, you can set these as part of the Dockerfile using the ENV instruction. I wouldn't copy this because it's likely to be specific to your setup, but in my case, this ended up looking like:
ENV PATH=/opt/intel/openvino/deployment_tools/model_optimizer:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV LD_LIBRARY_PATH=/opt/intel/openvino/opencv/lib:/opt/intel/openvino/deployment_tools/ngraph/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/tbb/lib::/opt/intel/openvino/deployment_tools/inference_engine/external/hddl/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/omp/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/gna/lib:/opt/intel/openvino/deployment_tools/inference_engine/external/mkltiny_lnx/lib:/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64
ENV INTEL_CVSDK_DIR=/opt/intel/openvino
ENV OpenCV_DIR=/opt/intel/openvino/opencv/cmake
ENV TBB_DIR=/opt/intel/openvino/deployment_tools/inference_engine/external/tbb/cmake
# The next one will be whatever your working directory is
ENV PWD=/workspace
ENV InferenceEngine_DIR=/opt/intel/openvino/deployment_tools/inference_engine/share
ENV ngraph_DIR=/opt/intel/openvino/deployment_tools/ngraph/cmake
ENV SHLVL=1
ENV PYTHONPATH=/opt/intel/openvino/python/python3.6:/opt/intel/openvino/python/python3:/opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/accuracy_checker:/opt/intel/openvino/deployment_tools/model_optimizer
ENV HDDL_INSTALL_DIR=/opt/intel/openvino/deployment_tools/inference_engine/external/hddl
ENV _=/usr/bin/printenv

Attempting to install nvm on aws ubuntu instance

I am using this command.
nvm - curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash
This is the error I get:
No command 'nvm' found, did you mean:
Command 'pvm' from package 'pvm' (universe)
etc.
My .bashrc looks like this at the bottom:
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
source ~/nvm/nvm.sh
And my .profile looks like this:
# set PATH so it includes user's private bin directories
[[ -s $HOME/.nvm/nvm.sh ]] && . $HOME/.nvm/nvm.sh
source ~/.nvm/nvm.sh
node -v
v6.10.2
What is the problem here?
All help is appreciated. I can provide more information if needed.
The command you want to run is:
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash
you cannot run nvm if you haven't install it and is not in the ubuntu repositories.
You need to have curl installed so might need to install it:
sudo apt-get install curl
Curl downloads the install.sh script and pipe it to bash to run it.
Remember to source your .bashrc afte running the script:
source ~/.bashrc

Resources