I run a Docker build, using a Dockerfile to build an image. But I get an error on npm install:
Dockerfile :
# Download Runnable-web Repo
RUN eval $(ssh-agent) > /dev/null && ssh-add /.ssh/id_rsa && git clone git#github.com:CodeNow/runnable-web.git
WORKDIR runnable-web
RUN npm install
RUN bower install --allow-root
Error:
Step 5 : RUN npm install
---> Running in 3fefdf5af71d
npm ERR! install Couldn't read dependencies
npm ERR! Error: ENOENT, open '/runnable-web/package.json'
If you need help, you may report this log at:
<http://github.com/isaacs/npm/issues>
or email it to:
<npm-#googlegroups.com>
Linux 3.13.0-24-generic
"/usr/local/bin/node" "/usr/local/bin/npm" "install"
I tried it in Directory C:\ which is a shared driver.It throws the same error with yours.
Then I move Dockerfile to a sub directory like C:\Intel and cd it to run docker build ., it works
So I guess Dockerfile should not be placed in shared dirver root directory.
Related
I have a local package that I packed into an npm package using npm pack.
I then install this package to some nodejs apps that are external to this local package using npm install path/to/package.tgz. This works like a charm, but now I want to dockerize the apps and I can't manage to install the local package.
My structure looks like this:
-my_package
|-package.tgz
-app1
|-app1.js
|-Dockerfile
-app2
|-app2.js
|-Dockerfile
My Dockerfile looks like this:
FROM node:16
WORKDIR /app
# Copy and download dependencies
COPY package.json .
RUN npm install
# Copy the source files into the image
COPY . .
EXPOSE 4002
CMD ["npm", "start"]
And the error I'm getting when running docker build . -t js/app1 is:
npm WARN tarball tarball data for #my_package#file:/package.tgz (sha512-u9tY/j1VOzO1y1RpcCgYteDOEsh7TaSMYwmR2Rs7hoJopE11qa1XcnrrMKNx1/H/aHsZ3Gr0bOMx1SygYTf/rg==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for #my_package#file:/package.tgz (sha512-u9tY/j1VOzO1y1RpcCgYteDOEsh7TaSMYwmR2Rs7hoJopE11qa1XcnrrMKNx1/H/aHsZ3Gr0bOMx1SygYTf/rg==) seems to be corrupted. Trying again.
npm notice
npm notice New major version of npm available! 8.19.2 -> 9.2.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.2.0>
npm notice Run `npm install -g npm#9.2.0` to update!
npm notice
npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /my_package/package.tgz
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/my_package/package.tgz'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-12-09T04_54_04_868Z-debug-0.log
The command '/bin/sh -c npm ci' returned a non-zero code: 254
Some times I get null instead of the sha512 stuff, otherwise the error is the same.
Is there anything I can do to fix this?
Thanks!
The documentation says:
COPY obeys the following rules:
The path must be inside the context of the build; you cannot
COPY ../something /something, because the first step of a docker build
is to send the context directory (and subdirectories) to the docker
daemon.
The easiest fix is to paste the file(s) you need into the same folder that your Dockerfile is in.
I am trying to create a task in our Azure pipeline to validate our javascript.
We have a node container which performs an npm install when spun up:
node:
image: node:12-alpine
user: "node"
working_dir: /home/node/app
environment:
- NODE_ENV=development
volumes:
- ./:/home/node/app
expose:
- "8081"
command: "npm install"
To perform my task I have created a make command in the Makefile:
js-check: ## Run Jshint
docker-compose run node npm install && npm run jshint
Which I then call in the build job as follows:
- script: make js-check
displayName: 'Run JSHint'
Locally when I call the make js-check it performs the install, followed by the jshint which outputs 0 vulnerabilities found. However when the pipeline reaches this task remotely it fails claiming missing write access to /home/node/app
npm WARN checkPermissions Missing write access to /home/node/app
npm ERR! code EACCES
npm ERR! syscall access
npm ERR! path /home/node/app
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, access '/home/node/app'
npm ERR! [Error: EACCES: permission denied, access '/home/node/app'] {
npm ERR! errno: -13,
npm ERR! code: 'EACCES',
npm ERR! syscall: 'access',
npm ERR! path: '/home/node/app'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
Your Makefile runs two commands; the shell interprets the && marker before it gets to Docker. That command is equivalent to:
js-check: ## Run Jshint
docker-compose run node npm install
npm run jshint # (without Docker)
It looks like your environment already has Node installed. You need to resolve the permissions issues (generally the CI system will check out source trees as a user that can run commands), and then you can use this native Node:
js-install: package.json
npm install
js-check: js-install
npm run jshint
This has the advantage of only depending on normal Javascript development tools; you don't need the extra docker-compose.yml file or administrator privileges just to run your unit tests.
If you really need to run this in Docker, you can either run this as two separate commands, or make the single container command be a shell that can interpret the && itself:
js-install1: package.json docker-compose.yml
docker-compose run node npm install
js-check1: js-install1
docker-compose run node npm run jshint
js-check2: package.json docker-compose.yml
docker-compose run node \
sh -c "npm install && npm run jshint"
Above error is because /home/node directory is owned by the node user in the default node image. The /app directory is created and owned by root. See this open issue about above error for more information.
I reproduced the same error with your compose file. When i changed the user to root. The error was gone.
So you can try changing the user to root instead of node in your compose file.
node:
image: node:12-alpine
user: "root"
working_dir: /home/node/app
As David pointed out, you also need to change your makefile to docker-compose run node sh -c "npm install && npm run jshint". if you want to run the commands in docker.
Another workaround is to build and run your container from a dockerfile instead of the compose file. See below simple example dockerfile.
from node:12-alpine
ENV NODE_ENV=development
RUN mkdir -p /home/node/app
RUN chown -R node:node /home/node/app
USER node
WORKDIR /home/node/app
COPY . ./
RUN npm install
CMD [ "npm", "run", "jshint" ]
Then change the Makefile like example:
js-check: ## Run Jshint
docker build --tag nodejshint:1.0 . && docker run --detach --name jshintContainer nodejshint:1.0
I'm building a Docker container for my Node.js + Vue application.
Since I have a global css library in another repository I have added this line in my package.json file:
"lib-css": "git+ssh://git#git.lib.com:9922/username/lib-css.git#development",
That way when I run npm install I install also my CSS library. The problem is that on my local env it asks for my password and I can insert it, but in the Docker build the process fails with the following error:
Step 7/10 : RUN npm install
---> Running in db10ca83586d
npm WARN deprecated babel-preset-es2015#6.24.1: 🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!
npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://git#git.lib.com:9922/username/lib-css.git
npm ERR!
npm ERR! Host key verification failed.
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR!
npm ERR! exited with error code: 128
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2017-12-11T08_49_11_152Z-debug.log
This is my current Dockerfile:
FROM node:carbon
WORKDIR /usr/src/app
RUN mkdir -p /root/.ssh
COPY .secrets /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh && chmod 600 /root/.ssh/*
# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm#5+)
COPY package*.json ./
RUN npm install
# If you are building your code for production
# RUN npm install --only=production
# Bundle app source
COPY . .
EXPOSE 8081
CMD [ "npm", "run dev" ]
~
My .secrets file contains my private key associated to the repository.
How can I make this works?
I'm trying to create a docker dev tools container for a devlopement environment on a windows host via docker toolbox but I have some trouble running the npm install command.
It worked fine on a linux host but on the windows host I got the following error :
npm ERR! Linux 4.1.13-boot2docker
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v5.5.0
npm ERR! npm v3.3.12
npm ERR! path /var/www/site/.npm/gulp/3.9.0/package.tgz.e87c24357cd6065ee71ce44c6f23673b
npm ERR! code ETXTBSY
npm ERR! errno -26
npm ERR! syscall rename
npm ERR! ETXTBSY: text file is busy, rename '/var/www/site/.npm/gulp/3.9.0/package.tgz.e87c24357cd6065ee71ce44c6f23673b' -> '/var/www/site/.npm/gulp/3.9.0/package.tgz'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Linux 4.1.13-boot2docker
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v5.5.0
npm ERR! npm v3.3.12
npm ERR! path npm-debug.log.39d944b679d410e5293d6721cbc8287a
npm ERR! code ETXTBSY
npm ERR! errno -26
npm ERR! syscall rename
npm ERR! ETXTBSY: text file is busy, rename 'npm-debug.log.39d944b679d410e5293d6721cbc8287a' -> 'npm-debug.log'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR! /var/www/site/npm-debug.log
Here is my Dockerfile :
FROM node:latest
RUN apt-get update
RUN apt-get install vim -y
RUN useradd -ms /bin/bash node
RUN echo "fs.inotify.max_user_watches=100000" > /etc/sysctl.conf
ADD . /var/www/site
RUN chown -R node:node /var/www/site
RUN chown -R node:node /usr/local/lib/node_modules
RUN chown -R node:node /usr/local/bin
USER node
ENV HOME /var/www/site
WORKDIR /var/www/site
RUN npm install -g bower
RUN npm install --global gulp -y
EXPOSE 80 8080 35729
In Docker quickstart terminal, I use the following commands :
Building the image (works fine)
docker build -t dev_tools .
Building the container (works fine)
docker run --name=dev_tools_container -t --rm -v "//c/Users/Public/site:/var/www/site" --net=host dev_tools
Trying to install npm dependencies (shoots the error):
docker exec -it dev_tools_container npm install
Thank you for your time !
Instead of
RUN npm install --global gulp -y
use
RUN sudo npm install --global gulp -y
You try to install gulp as a global package from user node (not superuser).
Or install gulp before switch user to node.
USER node
RUN npm install --global gulp -y
EDIT:
boot2docker is based on VirtualBox. Virtualbox does not allow symlinks on shared folders for security reasons.
To enable symlinks You must set VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME to 1. (Here is link to description how to do it on Vargrant: Symbolic links and synced folders in Vagrant)
VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME 1
Replace VM_NAME and SHARE_NAME and restart VirtualBox.
Another solution is add --no-bin-link to npm:
RUN npm install -g bower --no-bin-link
RUN npm install --global gulp -y --no-bin-link
EDIT 2
By default Windows 7 security policy does not allow creating symlinks as it's a potential security threat. If user is not in Administrators group run secpol.msc and navigate to Local Policies-User Rights Assignments and add your user to Create symbolic links.
If your user belongs to Administrators group then start VirtualBox with Run as Administrator.
You can mount node_modules as a volume, so it will be a Linux filesystem inside the Docker container. Add this to your Dockerfile:
VOLUME /var/www/site/node_modules
You will see the directory at C:Users/Public/site/node_modules because it is necessary for a mount point, but you will not see any contents unless you are inside the container.
I am in process of setting up the the deployment of a Node.js Service with Docker.
The Dockerfile I have is pieced together from various examples from around the net.
The directory for the Dockerfile includes:
Dockerfile
id_rsa
start.sh
This is the Dockerfile:
FROM ubuntu:13.10
# make sure apt is up to date
RUN apt-get update
# install npm, git, ssh, curl
RUN apt-get install -y npm git git-core ssh curl
RUN mkdir /nodejs && curl http://nodejs.org/dist/v0.10.31/node-v0.10.31-linux-x64.tar.gz | tar xvzf - -C /nodejs --strip-components=1
# Fixes empty home
ENV PATH $PATH:/nodejs/bin
ENV HOME /root
# SSH SETUP
RUN mkdir -p /root/.ssh
ADD id_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN echo "IdentityFile /root/.ssh/id_rsa" >> /root/.ssh/ssh_config
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts
ADD start.sh /tmp/
RUN chmod +x /tmp/start.sh
CMD ./tmp/start.sh
After the set-up is complete, start.sh runs and I experience problems with a private NPM dependency that the private Node.js service has. This is what start.sh is doing:
cd /tmp
# try to remove the repo if it already exists
rm -rf MediaFX; true
git clone https://<username>:<password>#github.com/company/ExampleRepo.git
cd RepoName
node --version
ls
npm install
NODE_ENV=test DEBUG=* PORT=3000 node server.js
In package.json for ExampleRepo, there is one private module that we import like this:
"dependencies": {
"scribe": "git+ssh://git#github.com:Company/PrivateDep.git"
},
When npm install gets to this repo, it outputs these logs:
npm ERR! git clone git#github.com:InboxAppCo/scribe.git Cloning into bare repository '/root/.npm/_git-remotes/git-github-com-InboxAppCo-scribe-git-abae334a'...
npm ERR! git clone git#github.com:InboxAppCo/scribe.git
npm ERR! git clone git#github.com:InboxAppCo/scribe.git Warning: Permanently added the RSA host key for IP address '192.30.252.130' to the list of known hosts.
npm ERR! git clone git#github.com:InboxAppCo/scribe.git Permission denied (publickey).
npm ERR! git clone git#github.com:InboxAppCo/scribe.git fatal: Could not read from remote repository.
npm ERR! git clone git#github.com:InboxAppCo/scribe.git
npm ERR! git clone git#github.com:InboxAppCo/scribe.git Please make sure you have the correct access rights
npm ERR! git clone git#github.com:InboxAppCo/scribe.git and the repository exists.
npm ERR! Error: `git "clone" "--mirror" "git#github.com:InboxAppCo/scribe.git" "/root/.npm/_git-remotes/git-github-com-InboxAppCo-scribe-git-abae334a"` failed with 128
npm ERR! at ChildProcess.cpclosed (/usr/share/npm/lib/utils/exec.js:59:20)
npm ERR! at ChildProcess.EventEmitter.emit (events.js:98:17)
npm ERR! at Process.ChildProcess._handle.onexit (child_process.js:789:12)
npm ERR! If you need help, you may report this log at:
npm ERR! <http://bugs.debian.org/npm>
npm ERR! or use
npm ERR! reportbug --attach /tmp/MediaFX/npm-debug.log npm
npm ERR! System Linux 3.16.4-tinycore64
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! cwd /tmp/MediaFX
npm ERR! node -v v0.10.15
npm ERR! npm -v 1.2.18
I thought that since the git clone of the private Node service works fine, any of its private NPM dependencies would install smoothly.
I am fairly positive that my SSH set up is flawed (and that it didn't manifest its self while git cloning the private parents repo) because I added username and password to the link. However, I am unsure and would appreciate some guidance on how to do this correctly.
git clone https://<username>:<password>#github.com/company/ExampleRepo.git
Works, because you are passing the username and password and doing it over https
"dependencies": {
"scribe": "git+ssh://git#github.com:Company/PrivateDep.git"
},
Fails, because you are connecting directly over ssh and Docker doesn't do any ssh agent forwarding from the host machine.
Unfortunatly it dosen't look like npm supports any url format to send username and password like your clone line: https://docs.npmjs.com/files/package.json#git-urls-as-dependencies
You'd have to add your ssh keys to the docker container ( Not Reccomended )
Or do something funky like share you SSH_SOCKET from the host like:
https://gist.github.com/d11wtq/8699521
Here's an approach I'm going to try implementing this evening:
docker create --build-arg TOKEN <my priv token> <dockerFile>
maybe declare the arg in the docker file?
ARG TOKEN
then in the script have npm install use that TOKEN in dependencies
"privModule": "git+https://${TOKEN}:x-oauth-basic#github.com/<githubID>/<privateModule>.git"
and if that doesn't work, somehow replace that var in the package.json (with sed) or have npm use an environment var.