yarn doesn't create node_modules folder in docker container - node.js

I am trying to build an application using dockerfile and docker_compose. Everything seems ok until it comes to yarn command, which is not creating node_module and not installing dependencies.
docker-compose file:
version: '2'
services:
mdisassistant:
env_file: .env
build:
context: .
args:
- APP_NAME=${APP_NAME}
- APP_USER=app
ports:
- ${APP_PORT}:${APP_PORT}
Dockerfile:
FROM node:10.15.2-alpine
ARG APP_NAME
ARG APP_USER
RUN adduser -D -s /bin/false -h /home/$APP_USER $APP_USER $APP_USER
ENV HOME=/home/$APP_USER
WORKDIR $HOME/$APP_NAME
COPY . .
RUN chown $APP_USER:$APP_USER -R $HOME
USER $APP_USER
RUN whoami
RUN yarn install
RUN ls -la
RUN pwd
RUN yarn build
CMD ["node", "./dist/index.js"]
Output:
Building mdisassistant
Step 1/15 : FROM node:10.15.2-alpine
---> 072459fe4d8a
Step 2/15 : ARG APP_NAME
---> Using cache
---> 4c30b08b312d
Step 3/15 : ARG APP_USER
---> Using cache
---> 9631ef748cd7
Step 4/15 : RUN adduser -D -s /bin/false -h /home/$APP_USER $APP_USER $APP_USER
---> Using cache
---> f5d045ca5282
Step 5/15 : ENV HOME=/home/$APP_USER
---> Using cache
---> 32c4f9457b9e
Step 6/15 : WORKDIR $HOME/$APP_NAME
---> Using cache
---> a8d71a9f563f
Step 7/15 : COPY . .
---> d4ef17f02a9f
Step 8/15 : RUN chown $APP_USER:$APP_USER -R $HOME
---> Running in f6c194316e12
Removing intermediate container f6c194316e12
---> f6742a0c10df
Step 9/15 : USER $APP_USER
---> Running in ec22ed655aa5
Removing intermediate container ec22ed655aa5
---> af800732027d
Step 10/15 : RUN whoami
---> Running in ba9fa81a95a3
app
Removing intermediate container ba9fa81a95a3
---> ebf0f6a4f8a7
Step 11/15 : RUN yarn install
---> Running in 4d5e76dd1508
yarn install v1.13.0
[1/4] Resolving packages...
[2/4] Fetching packages...
Removing intermediate container 4d5e76dd1508
---> 1785eec9829e
Step 12/15 : RUN ls -la
---> Running in b0f3f1b1e5fc
total 92
drwxr-sr-x 1 app app 4096 Mar 21 02:52 .
drwxr-sr-x 1 app app 4096 Mar 21 02:52 ..
-rw-r--r-- 1 app app 1610 Mar 21 02:05 .dockerignore
-rw-r--r-- 1 app app 62 Mar 21 00:56 .env.example
drwxr-xr-x 1 app app 4096 Mar 21 02:52 .git
-rw-r--r-- 1 app app 1610 Mar 20 23:56 .gitignore
-rw-r--r-- 1 app app 333 Mar 21 02:52 Dockerfile
-rw-r--r-- 1 app app 1060 Mar 20 23:56 LICENSE
-rw-r--r-- 1 app app 681 Mar 20 23:56 README.md
-rw-r--r-- 1 app app 280 Mar 21 02:35 docker-compose.yml
-rw-r--r-- 1 app app 614 Mar 21 01:57 package.json
drwxr-xr-x 1 app app 4096 Mar 21 01:05 src
-rw-r--r-- 1 app app 44193 Mar 21 01:20 yarn.lock
Removing intermediate container b0f3f1b1e5fc
---> eb6fbee4548f
Step 13/15 : RUN pwd
---> Running in 92b3d6d20201
/home/app/mdisassistant
Removing intermediate container 92b3d6d20201
---> 853d9879da99
Step 14/15 : RUN yarn build
---> Running in 205ef8079386
yarn run v1.13.0
$ ncc build src/index.js -o dist -m
/bin/sh: ncc: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: Service 'mdisassistant' failed to build: The command '/bin/sh -c yarn build' returned a non-zero code: 1
How to fix this problem?

Can you try without user creation
FROM node:10.15.2-alpine
WORKDIR /app
COPY package.json .
RUN whoami
RUN yarn install
RUN ls -la
RUN pwd
COPY . .
RUN yarn build
CMD ["node", "./dist/index.js"]

The problem was in the version of the docker and OS that I am used to building the docker container.
I was running docker-compose with docker version 18.x and OS was Deepin 15.11
I switched to EleebtaryOS version 5.1.2 Hera with installed docker version 19.03.6 and now everything works.
I don't know why this happens in Deepin OS.

Related

chown not working when coping a file in a dockerfile

I'm running docker engine on windows and am trying to add my own file to the image. Problem is that when I copy the file its ownership is always root:root but it needs to be heartbeat:heartbeat (exisitng user on image). Mounting a single file with the -v parameter und docker run doesn't seam to be possible on windows atm. Thats why I tried to create my own image with a docker file:
FROM docker.elastic.co/beats/heartbeat:7.16.3
USER root
COPY --chown=heartbeat:heartbeat yml/heartbeat.yml /usr/share/heartbeat/heartbeat.yml
RUN chown -R heartbeat:heartbeat /usr/share/heartbeat
The --chown parameter behind the coping does nothing. It is still root when I check and the RUN chown command results in a error. Here the output:
docker image build ./ -t custom/heartbeat:7.16.3
Sending build context to Docker daemon 10.75kB
Step 1/4 : FROM docker.elastic.co/beats/heartbeat:7.16.3
---> b64ad4b42006
Step 2/4 : USER root
---> Using cache
---> 922a9121e51b
Step 3/4 : COPY --chown=heartbeat:heartbeat yml/heartbeat.yml /usr/share/heartbeat/heartbeat.yml
---> Using cache
---> f30eb4934dca
Step 4/4 : RUN chown -R heartbeat:heartbeat /usr/share/heartbeat
---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (windows/amd64) and no specific platform was requested
---> Running in 2ae3bfdd5422
The command '/bin/sh -c chown -R heartbeat:heartbeat /usr/share/heartbeat' returned a non-zero code: 4294967295: failed to shutdown container: container 2ae3bfdd5422e81461a14896db0908e4cd67af1a6f99c629abff1e588f62fc32 encountered an error during hcsshim::System::waitBackground: failure in a Windows system call: The virtual machine or container with the specified identifier is not running. (0xc0370110): subsequent terminate failed container 2ae3bfdd5422e81461a14896db0908e4cd67af1a6f99c629abff1e588f62fc32 encountered an error during hcsshim::System::waitBackground: failure in a Windows system call: The virtual machine or container with the specified identifier is not running. (0xc0370110)
All help is welcome...
Running with --platform:
PS C:\SynteticMonitoring> docker image build ./ -t custom/heartbeat:7.16.3
Sending build context to Docker daemon 9.728kB
Step 1/4 : FROM --platform=linux/amd64 docker.elastic.co/beats/heartbeat:7.16.3
---> b64ad4b42006
Step 2/4 : USER root
---> Using cache
---> 922a9121e51b
Step 3/4 : COPY --chown=heartbeat:heartbeat yml/heartbeat.yml /usr/share/heartbeat/heartbeat.yml
---> Using cache
---> f30eb4934dca
Step 4/4 : RUN chmod +r /usr/share/heartbeat/heartbeat.yml
---> Using cache
---> e9a075d2ab53
Successfully built e9a075d2ab53
Successfully tagged custom/heartbeat:7.16.3
PS C:\SynteticMonitoring> docker run --interactive --tty --entrypoint /bin/sh custom/heartbeat:7.16.3
sh-4.2# ls -l
total 106916
-rw-r--r-- 1 root root 13675 Jan 7 00:47 LICENSE.txt
-rw-r--r-- 1 root root 1964303 Jan 7 00:47 NOTICE.txt
-rw-r--r-- 1 root root 851 Jan 7 00:47 README.md
drwxrwxr-x 2 root root 4096 Jan 7 00:48 data
-rw-r--r-- 1 root root 374197 Jan 7 00:47 fields.yml
-rwxr-xr-x 1 root root 107027952 Jan 7 00:47 heartbeat
-rw-r--r-- 1 root root 69196 Jan 7 00:47 heartbeat.reference.yml
-rw-rw-rw- 1 root root 1631 Jan 26 06:49 heartbeat.yml
drwxr-xr-x 2 root root 4096 Jan 7 00:47 kibana
drwxrwxr-x 2 root root 4096 Jan 7 00:48 logs
drwxr-xr-x 2 root root 4096 Jan 7 00:47 monitors.d
sh-4.2# pwd
/usr/share/heartbeat
You can't chown of a file to a user that does not exist. It seems that the heartbeat user and group do not exist in your base image.
That's why the COPY --chown does nothing and you get files owned by root.
You can fix this by creating the user before COPYing. To do this, add a line before your COPY statement, such as:
RUN addgroup heartbeat && adduser -S -H heartbeat -G heartbeat
If you don't have addgroup and adduser in your base image, try alternative:
RUN useradd -rUM -s /usr/sbin/nologin heartbeat
This will create the group and user heartbeat and then chown will be able to successfully change the ownership.
According to Dockerfile documentation:
The optional --platform flag can be used to specify the platform of the image in case FROM references a multi-platform image. For example, linux/amd64, linux/arm64, or windows/amd64. By default, the target platform of the build request is used.
I suggest try something like:
FROM [--platform=<platform>] <image> [AS <name>]
FROM --platform=linux/amd64 docker.elastic.co/beats/heartbeat:7.16.3

Execing docker image entrypoint, which is a compiled go app, fails with "not found"

I have built a small Go app and done local testing of it on my Linux VM.
I'm now trying to build a prototype Docker image for it and test running the image. The Dockerfile structure is pretty simple. I base it on Alpine, copy the executable to the root directory and my entrypoint is running the executable.
It fails with "not found".
Now for more details.
Here is the Dockerfile, with some information elided:
FROM <registry>/<namespace>/alpine-base:3.12.3
COPY target/dist/linux-amd64/<appname> /
EXPOSE 8080
RUN echo hello
RUN ls -ltd .
RUN ls -lt
RUN whoami
#ENTRYPOINT ["./<appname>"]
ENTRYPOINT ./<appname>
This is approximately what I do when I build the image:
chmod 777 target/dist/linux-amd64/<appname>
docker build --no-cache -f Dockerfile -t <registry>/<namespace>/<appname>:dev-latest .
This is the output of that:
Sending build context to Docker daemon 14.48MB
Step 1/8 : FROM <registry>/<namespace>/alpine-base:3.12.3
---> d7eec24f3d29
Step 2/8 : COPY target/dist/linux-amd64/<appname> /
---> e056bbe44bd6
Step 3/8 : EXPOSE 8080
---> Running in 921cc1fe8804
Removing intermediate container 921cc1fe8804
---> 00b30c5a2770
Step 4/8 : RUN echo hello
---> Running in 9fb08d924d3c
hello
Removing intermediate container 9fb08d924d3c
---> 6788feafae4b
Step 5/8 : RUN ls -ltd .
---> Running in 78e6d4aea09f
drwxr-xr-x 1 root root 4096 Jan 10 23:02 .
Removing intermediate container 78e6d4aea09f
---> 711f3d247efe
Step 6/8 : RUN ls -lt
---> Running in 32e703a9d480
total 14200
drwxr-xr-x 5 root root 340 Jan 10 23:02 dev
drwxr-xr-x 1 root root 4096 Jan 10 23:02 etc
dr-xr-xr-x 324 root root 0 Jan 10 23:02 proc
dr-xr-xr-x 13 root root 0 Jan 10 23:02 sys
-rwxrwxrwx 1 root root 14480384 Jan 10 22:39 <appname>
drwxr-xr-x 1 root root 4096 Jan 12 2021 home
drwxr-xr-x 1 root root 4096 Jan 12 2021 opt
drwxr-xr-x 2 root root 4096 Dec 16 2020 bin
drwxr-xr-x 2 root root 4096 Dec 16 2020 sbin
drwxr-xr-x 1 root root 4096 Dec 16 2020 lib
drwxr-xr-x 5 root root 4096 Dec 16 2020 media
drwxr-xr-x 2 root root 4096 Dec 16 2020 mnt
drwx------ 2 root root 4096 Dec 16 2020 root
drwxr-xr-x 2 root root 4096 Dec 16 2020 run
drwxr-xr-x 2 root root 4096 Dec 16 2020 srv
drwxrwxrwt 2 root root 4096 Dec 16 2020 tmp
drwxr-xr-x 1 root root 4096 Dec 16 2020 usr
drwxr-xr-x 1 root root 4096 Dec 16 2020 var
Removing intermediate container 32e703a9d480
---> 68871e80b517
Step 7/8 : RUN whoami
---> Running in 40b2460bc349
kube
Removing intermediate container 40b2460bc349
---> 4cf57c0b5f10
Step 8/8 : ENTRYPOINT ./<appname>
---> Running in 3c57717800ab
Removing intermediate container 3c57717800ab
---> eaafc953da46
Successfully built eaafc953da46
Successfully tagged <registry>/<namespace>/<appname>:dev-latest
And this is what I run to test it:
docker rm <appname>-1
docker run -P --name=<appname>-1 -d -t <registry>/<namespace>/<appname>:dev-latest
docker logs <appname>-1
And this is the output:
docker rm <appname>-1
<appname>-1
docker run -P --name=<appname>-1 -d -t <registry>/<namespace>/<appname>:dev-latest
66bb4756783b3ef64d9a4b0d8b7227184ba3b5a3fde25ea0d19b9523285d76b7
docker logs <appname>-1
/bin/sh: ./<appname>: not found
It says "not found". I don't understand that. I showed the contents of the root directory. The file is clearly there. Is this error saying that some OTHER file is not found, like if it thought it was a shell script and the shebang pointed to a shell that doesn't exist?
Update:
So the one tiny little detail that I realized I didn't mention in the original post is that disabling CGO is not going to be possible. The entire reason for this app is to link with a C library and call functions in it, so I have to use Cgo.
What I conclude from these helpful comments and other threads like Go-compiled binary won't run in an alpine docker container on Ubuntu host , is that my "workaround" of changing to an ubuntu base image is actually the only reasonable solution.
If disabling cgo is not an option you can pass "-static" parameter to the linker.
Example:
package main
/*
#include <stdio.h>
void test_puts() {
puts("puts() called");
}
*/
import "C"
func main() {
C.test_puts()
}
Run:
go build --ldflags '-extldflags "-static"'

Why Docker COPY doesn't change file permissions? (--chmod)

Given this Dockerfile:
FROM docker.io/alpine
RUN mkdir test
# RUN umask 0022
COPY README /test/README
COPY --chmod=777 README /test/README-777
COPY --chmod=755 README /test/README-755
COPY FORALL /test/FORALL
COPY --chmod=777 FORALL /test/FORALL-777
COPY --chmod=755 FORALL /test/FORALL-755
RUN ls -la /test
I'd expect to have the read, write, execute permissions be set accordingly by Docker during the build process (docker build ./).
But the last command returns
total 8
drwxr-xr-x 1 root root 4096 Jun 9 19:20 .
drwxr-xr-x 1 root root 4096 Jun 9 19:20 ..
-rwxrwxrwx 1 root root 0 Jun 9 19:19 FORALL
-rwxrwxrwx 1 root root 0 Jun 9 19:19 FORALL-755
-rwxrwxrwx 1 root root 0 Jun 9 19:19 FORALL-777
-rw-rw-r-- 1 root root 0 Jun 9 19:19 README
-rw-rw-r-- 1 root root 0 Jun 9 19:19 README-755
-rw-rw-r-- 1 root root 0 Jun 9 19:19 README-777
No file permission was changed, and no error was raised.
Why doesn't it work?
How to fix this?
I figured out:
the flag --chmod is a new feature from Docker Buildkit, so it is necessary to run the build enabling it via:
DOCKER_BUILDKIT=1 docker build ./
However, it is really not clear why Docker swallows the --chmod option without any error or warn about the non-existing option 😕.
This is fixed in 20.10.6 (pull request, tracking issue):
$ cat df.chmod
FROM busybox as base
RUN touch /test
FROM busybox as release
COPY --from=base --chmod=777 /test /test-777
COPY --from=base --chmod=555 /test /test-555
CMD ls -l /test*
$ DOCKER_BUILDKIT=0 docker build -t test-chmod-classic -f df.chmod .
Sending build context to Docker daemon 22.02kB
Step 1/6 : FROM busybox as base
---> a9d583973f65
Step 2/6 : RUN touch /test
---> Running in ed48f45a5dca
Removing intermediate container ed48f45a5dca
---> 5606d2d23861
Step 3/6 : FROM busybox as release
---> a9d583973f65
Step 4/6 : COPY --from=base --chmod=777 /test /test-777
the --chmod option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled
And if the build is run with buildkit, the expected result occurs:
$ DOCKER_BUILDKIT=1 docker build -t test-chmod-buildkit -f df.chmod .
[+] Building 1.0s (8/8) FINISHED
=> [internal] load build definition from df.chmod 0.0s
=> => transferring dockerfile: 214B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 49B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 0.0s
=> CACHED [base 1/2] FROM docker.io/library/busybox 0.0s
=> [base 2/2] RUN touch /test 0.6s
=> [release 2/3] COPY --from=base --chmod=777 /test /test-777 0.1s
=> [release 3/3] COPY --from=base --chmod=555 /test /test-555 0.1s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:a4df92175046e36a72a769f9c7b297bc04a825708c5f6ca5873428b55c340036 0.0s
=> => naming to docker.io/library/test-chmod-buildkit 0.0s
$ docker run --rm test-chmod-buildkit
-r-xr-xr-x 1 root root 0 Jun 10 13:00 /test-555
-rwxrwxrwx 1 root root 0 Jun 10 13:00 /test-777

docker-compose gives mongo error about not owning the data folder

Based on the following files:
Dockerfile
FROM node:8-alpine
RUN apk add --no-cache ffmpeg
RUN apk add --no-cache git
RUN apk add --no-cache tar
WORKDIR /app/
COPY package*.json /app/
COPY bower.json /app/
RUN npm i
RUN npm i -g bower
RUN bower install --allow-root
COPY . .
EXPOSE 8080
CMD ["npm", "start"]
docker-compose-yml
version: '3.1'
services:
node:
container_name: nodetube
build:
context: .
dockerfile: Dockerfile
ports:
- "49161:3000"
volumes:
- .:/app/
- /app/node_modules
- ./upload:/app/upload
- ./uploads:/app/uploads
environment:
- REDIS_HOST=redis
- MONGODB_DOCKER_URI=mongodb://nodetube-mongo:27017/nodetube
depends_on:
- redis
- mongo
command: npm start
networks:
- nodetube-network
mongo:
container_name: nodetube-mongo
image: mongo:3.6
volumes:
- ./data/db:/data/db
ports:
- "27011:27017"
networks:
- nodetube-network
redis:
container_name: nodetube-redis
image: redis
networks:
- nodetube-network
networks:
nodetube-network:
driver: bridge
.dockerignore
.*
docker-compose.yml
*.md
node_modules
npm-debug.log
Running $ docker-compose up --build gives me an error:
nodetube-mongo | 2020-01-06T19:33:08.815+0000 I STORAGE [initandlisten] exception in initAndListen: IllegalOperation: Attempted to create a lock file on a read-only directory: /data/db, terminating
I'm not a Docker expert, how can I get this to work? Thanks
On my local machine if I run $ ls -al /data
I receive:
total 0
drwxr-xr-x 4 root wheel 128 May 13 2018 .
drwxr-xr-x 33 root wheel 1056 Apr 11 2019 ..
drwxrwxrwx 432 anthony wheel 13824 Jan 6 12:21 db
drwxr-xr-x 385 anthony wheel 12320 May 13 2018 db2
anthony at Anthonys-MacBook-Pro in /data/db
$ ls -al
total 17565016
drwxrwxrwx 432 anthony wheel 13824 Jan 6 12:21 .
drwxr-xr-x 4 root wheel 128 May 13 2018 ..
-rw--w--w- 1 anthony wheel 48 May 13 2018 WiredTiger
-rw--w--w- 1 anthony wheel 21 May 13 2018 WiredTiger.lock
-rw--w--w- 1 anthony wheel 1088 Jan 6 12:21 WiredTiger.turtle
-rw--w--w- 1 anthony wheel 1216512 Jan 6 12:21 WiredTiger.wt
-rw--w--w- 1 anthony wheel 4096 Jan 6 12:20 WiredTigerLAS.wt

Node not found on Jenkins docker instance

I am running Jenkins as a docker container, and have installed the NodeJS plugin and followed thoroughly the setup instructions. When I try to run a script using node, I get the following error:
/tmp/jenkins9123978873441132802.sh: line 1: node: not found
Build step 'Execute shell' marked build as failure
Finished: FAILURE
I checked the docker volume, the node bin is where it should be and is executable is there and it works fine when I run it from my host server:
user#server:/data/jenkins/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/latest/bin$ ./node --version
v9.2.0
I modified my build script to explore a bit further the problem:
echo $PATH
cd /var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/latest/bin
ls -all
./node --version
node --version
npm --version
and look how strange this is:
Building in workspace /var/jenkins_home/workspace/release
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Done
Adding all registry entries
copy managed file [Main config] to file:/var/jenkins_home/workspace/release#tmp/config69012336710357692tmp
[release] $ /bin/sh -xe /tmp/jenkins6243047436861395796.sh
+ echo /var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/latest/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin
/var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/latest/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin
+ cd /var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/latest/bin
+ ls -all
total 34112
drwxr-xr-x 2 jenkins jenkins 4096 Nov 20 16:16 .
drwxr-xr-x 6 jenkins jenkins 4096 Nov 20 16:16 ..
-rwxrwxrwx 1 jenkins jenkins 34921762 Nov 14 20:33 node
lrwxrwxrwx 1 jenkins jenkins 38 Nov 20 16:16 npm -> ../lib/node_modules/npm/bin/npm-cli.js
lrwxrwxrwx 1 jenkins jenkins 38 Nov 20 16:16 npx -> ../lib/node_modules/npm/bin/npx-cli.js
+ ./node --version
/tmp/jenkins6243047436861395796.sh: line 1: ./node: not found
Build step 'Execute shell' marked build as failure
Finished: FAILURE
The node executable is present, and it's executable (+x). The path is correctly set, but the build still fails.
This is because the path to the node binary
/data/jenkins/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/latest/bin
Does not exist on your shell path.
You should edit Jenkins' variables to adjust your PATH.

Resources