Equivalent builds don't behave the same - node.js

if I build an image like this:
docker build -f app/Dockerfile -t test .
it builds just fine, but the following docker-compose.yml (which I believe should be equivalent):
app:
container_name: test
build:
context: .
dockerfile: app/Dockerfile
fails to build. if I run docker-compose with --verbose I see:
compose.cli.verbose_proxy.proxy_callable: docker build <- (pull=False,
stream=True, nocache=False, tag=u'prj_app', buildargs=None,
rm=True, forcerm=False, path='/Users/ekkis/dev/prj',
dockerfile='./app/Dockerfile')
which would seem pretty much what I expect. specifically the error I'm getting is this (sorry to include as code but quotes make a mess of things):
npm WARN app#1.0.0 No description
npm WARN app#1.0.0 No repository field.
npm ERR! Linux 4.9.12-moby
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "/tmp/inc"
npm ERR! node v7.7.3
npm ERR! npm v4.1.2
npm ERR! path /usr/src/app/node_modules/inc
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall access
npm ERR! enoent ENOENT: no such file or directory, access '/usr/src/app/node_modules/inc'
npm ERR! enoent ENOENT: no such file or directory, access '/usr/src/app/node_modules/inc'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! Please include the following file with any support request:
npm ERR! /usr/src/app/npm-debug.log
as you can see from the above, the Dockerfile performs an npm install /tmp/inc, which from the output I see succeeded since it prints a tree of modules installed. /tmp/inc is created by the Dockerfile command shown below as successful:
Step 8/11 : ADD inc /tmp/inc
---> Using cache
---> fce607bb13ca
and I note that the app/npm-debug.log was not written to, nor is there a log file in the directory where I ran docker compose.
what could be the issue here? why does it build ok manually but not with docker-compose?
* Addendum I *
in my continuing efforts to figure it out, I added a directory listing before the Dockerfile executes the npm install...
Step 9/13 : RUN ls -alF
---> Running in ac0c62b7da93
total 28
drwxr-xr-x 3 root root 4096 Mar 19 07:01 ./
drwxr-xr-x 4 root root 4096 Mar 19 07:01 ../
-rw-r--r-- 1 root root 722 Mar 19 05:48 config.json
-rw-r--r-- 1 root root 3786 Mar 19 05:48 index.js
drwxr-xr-x 119 root root 4096 Mar 17 18:57 node_modules/
-rw-r--r-- 1 root root 2910 Mar 17 20:51 npm-debug.log
-rw-r--r-- 1 root root 416 Mar 19 06:39 package.json
...and, most interestingly, there is already a node_modules directory there!! that means the problem is that docker-compose is not honouring the .dockerignore file (which prevents copying **/node_modules into the context)
so is there some special voodoo I have to perform for the build to generate the context properly?
* Addendum II *
here's my Dockerfile:
FROM node:latest
EXPOSE 8000
ENV PORT 8000
RUN apt-get update
RUN groupadd -r tst && useradd -m -r -g tst tst
WORKDIR /usr/src/app
ADD app /usr/src/app
ADD inc /tmp/inc
RUN npm install --production
RUN npm install /tmp/inc
RUN rm -rf /tmp/inc
CMD ["npm", "start"]

My guess is you don't use --unsafe-perm npm key. Try change npm install to:
RUN npm install --production --unsafe-perm

Related

npm ERR! code EACCES, npm install --global yarn met error

Mac mini (M1, 2020)
Monterey
Brownie v1.17.2
nodejs v16.13.4
I am learning solidity according to reference(https://www.youtube.com/watch?v=M576WGiDBdQ&t=25510s).
Node.JS install is fine
when I tried this
npm install --global yarn
the terminal give the error information
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /usr/local/lib/node_modules/yarn
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/yarn'
npm ERR! [Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/yarn'] {
npm ERR! errno: -13,
npm ERR! code: 'EACCES',
npm ERR! syscall: 'mkdir',
npm ERR! path: '/usr/local/lib/node_modules/yarn'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/liwei/.npm/_logs/2022-01-08T12_13_37_765Z-debug.log
(base) liwei#liweideMac-mini-2 com~apple~CloudDocs %
I checked the owner of the file ,the owner is root ,should i use sudo?
(base) liwei#liweideMac-mini-2 com~apple~CloudDocs % ls -l /usr/local/lib/node_modules/npm
total 64
-rw-r--r-- 1 root wheel 9742 10 14 08:49 LICENSE
-rw-r--r-- 1 root wheel 4190 10 19 06:36 README.md
drwxr-xr-x 9 root wheel 288 12 1 19:46 bin
drwxr-xr-x 4 root wheel 128 12 1 19:46 docs
-rw-r--r-- 1 root wheel 145 11 23 14:00 index.js
drwxr-xr-x 77 root wheel 2464 12 1 19:46 lib
drwxr-xr-x 5 root wheel 160 12 1 19:46 man
drwxr-xr-x 188 root wheel 6016 12 1 19:46 node_modules
-rw-r--r-- 1 root wheel 6007 12 1 19:13 package.json
drwxr-xr-x 3 root wheel 96 12 1 19:46 tap-snapshots
(base) liwei#liweideMac-mini-2 com~apple~CloudDocs %
find this thread (Error: EACCES: permission denied, access '/usr/local/lib/node_modules'),changed the owner , stll one file don't change.
(base) liwei#liweideMac-mini-2 com~apple~CloudDocs % sudo chown -R liwei: /usr/local/lib/node_modules
(base) liwei#liweideMac-mini-2 com~apple~CloudDocs % ls -la /usr/local/lib/node_modules
total 0
drwxr-xr-x 4 liwei wheel 128 1 8 11:51 .
drwxr-xr-x 4 root wheel 128 1 8 11:51 ..
drwxr-xr-x 7 liwei wheel 224 12 1 19:46 corepack
drwxr-xr-x 13 liwei wheel 416 12 1 19:46 npm
what is this 2 dots is?
reference answer is here
(Error: EACCES: permission denied, access '/usr/local/lib/node_modules')
This works for me very quick.
To minimize the chance of permissions errors, you can configure npm to use a different directory. In this example, you will create and use a hidden directory in your home directory.
Back up your computer. On the command line, in your home directory, create a directory for global installations:
mkdir ~/.npm-global
Configure npm to use the new directory path:
npm config set prefix '~/.npm-global'
In your preferred text editor, open or create a
~/.profile
file and add this line:
export PATH=~/.npm-global/bin:$PATH
On the command line, update your system variables:
source ~/.profile
To test your new configuration, install a package globally without using sudo
I had the same problem but I could solve it after seeing a youtube video
use this command:
sudo npm install --global yarn

Trying to update NPM in Docker: code EISDIR

I'm trying to update NPM with ["npm","install","-g","npm"] but it says:
Step 4/12 : RUN ["npm", "install", "-g", "npm"]
---> Running in 708a95e71771
npm ERR! code EISDIR
npm ERR! syscall copyfile
npm ERR! path /usr/local/lib/node_modules/npm
npm ERR! dest /usr/local/lib/node_modules/.npm-i9nnxROI
npm ERR! errno -21
npm ERR! EISDIR: illegal operation on a directory, copyfile '/usr/local/lib/node_modules/npm' -> '/usr/local/lib/node_modules/.npm-i9nnxROI'
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-12-31T13_38_29_907Z-debug.log
The command 'npm install -g npm' returned a non-zero code: 235
My dockerfile:
FROM node
WORKDIR /root
COPY package*.json ./
RUN ["npm", "install", "-g", "npm"]
RUN ["npm", "audit", "fix"]
RUN ["npm", "install"]
COPY ./*.js ./
COPY static ./static
COPY views ./views
COPY conf.d/* /etc/dashboard/
EXPOSE 80
CMD ["node", "index.js"]
So it looks like /usr/local/lib/node_modules/.npm-i9nnxROI or /usr/local/lib/node_modules/npm is a directory and not a file but I think it should be a directory.
On my system (Linux 5.10.2-2-MANJARO #1 SMP PREEMPT x86_64 GNU/Linux) it works but on my server (Linux 5.4.0-58-generic #64-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux) it doesn't.
From node
installs latest node , which has npm version 7+ . Its not stable has some issues
use
FROM node:lts

Docker image creation throwing "no such file or directory, access '/node_modules/sails'" when dockerizing a sails.js app

Below is the Dockerfile
FROM node:latest
RUN npm install -g sails#0.12.13
ADD . / ./
RUN npm install
EXPOSE 80
CMD (sails lift)
Image creation fails with following log:
ending build context to Docker daemon 70.03MB
Step 1/6 : FROM node:latest
---> 60bea5b86079
Step 2/6 : RUN npm install -g sails#0.12.13
---> Using cache
---> 3f3c7fcdb090
Step 3/6 : ADD . / ./
---> Using cache
---> 78700b41cf26
Step 4/6 : RUN (npm install)
---> Running in d49423611a77
npm info it worked if it ends with ok
npm info using npm#5.3.0
npm info using node#v8.4.0
npm info lifecycle ecs-notification#0.0.0~preinstall: ecs-notification#0.0.0
npm WARN checkPermissions Missing write access to /node_modules/sails
npm ERR! path /node_modules/sails
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall access
npm ERR! enoent ENOENT: no such file or directory, access '/node_modules/sails'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2017-09-08T13_00_06_956Z-debug.log
The command '/bin/sh -c (npm install)' returned a non-zero code: 254
Even if I try to create the directory, or change the permission to 777 using:
RUN (mkdir -p /node_modules/sails; chmod 777 /node_modules/sails)
it still fails with the same error:
Sending build context to Docker daemon 70.03MB
Step 1/7 : FROM node:latest
---> 60bea5b86079
Step 2/7 : RUN npm install -g sails#0.12.13
---> Using cache
---> 3f3c7fcdb090
Step 3/7 : RUN (mkdir -p /node_modules/sails; chmod 777 /node_modules/sails)
---> Using cache
---> c7f1784c24c8
Step 4/7 : ADD . / ./
---> Using cache
---> 334017659dde
Step 5/7 : RUN (npm install)
---> Running in 833e3ef6a010
npm info it worked if it ends with ok
npm info using npm#5.3.0
npm info using node#v8.4.0
npm info lifecycle ecs-notification#0.0.0~preinstall: ecs-notification#0.0.0
npm WARN checkPermissions Missing write access to /node_modules/sails
npm ERR! path /node_modules/sails
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall access
npm ERR! enoent ENOENT: no such file or directory, access '/node_modules/sails'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2017-09-08T13_13_55_258Z-debug.log
The command '/bin/sh -c (npm install)' returned a non-zero code: 254
Docker version: 17.06.2-ce-mac27 (19124)
Any pointers around how I can debug this?
Change your Dockerfile to below
FROM node:latest
RUN npm install -g sails#0.12.13
WORKDIR /usr/app
COPY package.json ./
RUN npm install
COPY . ./
EXPOSE 80
CMD sails lift
You should only copy package.json first and the do npm install and then copy code. Also your ADD statement was wrong. There was an extra space
Next make sure you have .dockerignore which ignores the node_modules. Else node_modules will be overwritten by the copy operation

Docker running test with nodejs and volume

I'm learning docker and I'm having some troubles with the volume in a development environment with Nodejs.
I having the following simple dockerfile that aims to start my unit tests from a NodeJS parent image:
FROM node:4-onbuild
VOLUME ["/usr/src/app"]
CMD [ "npm", "test" ]
I'm running my container this way:
docker run -v /C/Users/myUser/dockertest:/usr/src/app notest
But I keep receiving the following error:
npm info it worked if it ends with ok
npm info using npm#2.15.9
npm info using node#v4.5.0
npm ERR! Linux 4.4.15-moby
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "test"
npm ERR! node v4.5.0
npm ERR! npm v2.15.9
npm ERR! path /usr/src/app/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/usr/src/app/package.json'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! Please include the following file with any support request:
npm ERR! /usr/src/app/npm-debug.log
I don't understand why I'm not able to find the package.json file inside of the /usr/src/app folder inside of my container.
The fact is that if I'm using the following docker file :
FROM node:4-onbuild
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
CMD [ "npm", "test" ]
My tests will be running in the container, but only after building step (so I have to build each time I make a modification...) . I need to play these tests any time I want, by launching a docker command (restart ? exec ?) quickly
Thanks for your help
You want to COPY everything build relevant and link your codebase on containerstart.
FROM node:4.4
# Enviroment variables
ENV HOMEDIR /data
RUN mkdir -p ${HOMEDIR}
WORKDIR ${HOMEDIR}
# install all dependencies
COPY package.json ./
RUN npm install
# add node content initially
COPY . .
CMD ["npm", "test"]
And then link it on start:
version: '2'
services:
myApp:
build:
context: .
dockerfile: Dockerfile
image: "myApp"
volumes:
- .:/data
You should now be able to change your codebase without restarting/rebuilding your container.

Performing a npm install via Docker on a windows host

I'm trying to create a docker dev tools container for a devlopement environment on a windows host via docker toolbox but I have some trouble running the npm install command.
It worked fine on a linux host but on the windows host I got the following error :
npm ERR! Linux 4.1.13-boot2docker
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v5.5.0
npm ERR! npm v3.3.12
npm ERR! path /var/www/site/.npm/gulp/3.9.0/package.tgz.e87c24357cd6065ee71ce44c6f23673b
npm ERR! code ETXTBSY
npm ERR! errno -26
npm ERR! syscall rename
npm ERR! ETXTBSY: text file is busy, rename '/var/www/site/.npm/gulp/3.9.0/package.tgz.e87c24357cd6065ee71ce44c6f23673b' -> '/var/www/site/.npm/gulp/3.9.0/package.tgz'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Linux 4.1.13-boot2docker
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v5.5.0
npm ERR! npm v3.3.12
npm ERR! path npm-debug.log.39d944b679d410e5293d6721cbc8287a
npm ERR! code ETXTBSY
npm ERR! errno -26
npm ERR! syscall rename
npm ERR! ETXTBSY: text file is busy, rename 'npm-debug.log.39d944b679d410e5293d6721cbc8287a' -> 'npm-debug.log'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR! /var/www/site/npm-debug.log
Here is my Dockerfile :
FROM node:latest
RUN apt-get update
RUN apt-get install vim -y
RUN useradd -ms /bin/bash node
RUN echo "fs.inotify.max_user_watches=100000" > /etc/sysctl.conf
ADD . /var/www/site
RUN chown -R node:node /var/www/site
RUN chown -R node:node /usr/local/lib/node_modules
RUN chown -R node:node /usr/local/bin
USER node
ENV HOME /var/www/site
WORKDIR /var/www/site
RUN npm install -g bower
RUN npm install --global gulp -y
EXPOSE 80 8080 35729
In Docker quickstart terminal, I use the following commands :
Building the image (works fine)
docker build -t dev_tools .
Building the container (works fine)
docker run --name=dev_tools_container -t --rm -v "//c/Users/Public/site:/var/www/site" --net=host dev_tools
Trying to install npm dependencies (shoots the error):
docker exec -it dev_tools_container npm install
Thank you for your time !
Instead of
RUN npm install --global gulp -y
use
RUN sudo npm install --global gulp -y
You try to install gulp as a global package from user node (not superuser).
Or install gulp before switch user to node.
USER node
RUN npm install --global gulp -y
EDIT:
boot2docker is based on VirtualBox. Virtualbox does not allow symlinks on shared folders for security reasons.
To enable symlinks You must set VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME to 1. (Here is link to description how to do it on Vargrant: Symbolic links and synced folders in Vagrant)
VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME 1
Replace VM_NAME and SHARE_NAME and restart VirtualBox.
Another solution is add --no-bin-link to npm:
RUN npm install -g bower --no-bin-link
RUN npm install --global gulp -y --no-bin-link
EDIT 2
By default Windows 7 security policy does not allow creating symlinks as it's a potential security threat. If user is not in Administrators group run secpol.msc and navigate to Local Policies-User Rights Assignments and add your user to Create symbolic links.
If your user belongs to Administrators group then start VirtualBox with Run as Administrator.
You can mount node_modules as a volume, so it will be a Linux filesystem inside the Docker container. Add this to your Dockerfile:
VOLUME /var/www/site/node_modules
You will see the directory at C:Users/Public/site/node_modules because it is necessary for a mount point, but you will not see any contents unless you are inside the container.

Resources