Following my Dockerfile
FROM node:lts-alpine as node
ARG STAGE='dev'
ARG STAGEPATH='/dev'
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN $(npm bin)/ng build --configuration $STAGE --base-href=${STAGEPATH}/konnektor/
I execute manually
docker build --progress=plain --no-cache -t konnektor .
And get the following Error
#14 [node 6/6] RUN $(npm bin)/ng build --configuration dev --base-href=/dev/konnektor/
#14 sha256:0d47aa4d98557f141c02bf395f4e2f6dd49e9cebe6dddad4ea4e5126da5016e7
#14 10.72 /bin/sh: Unknown: not found
#14 ERROR: executor failed running [/bin/sh -c $(npm bin)/ng build --configuration $STAGE --base-href=${STAGEPATH}/konnektor/]: exit code: 127
------> [node 6/6] RUN $(npm bin)/ng build --configuration dev --base-href=/dev/konnektor/:
The build was working yesterday morning. I figured out, that the parentimage "node:lts-alpine" got an update a few hours ago. So i strongly think this is the problem. My problem now is, how can i get my build up and running again? The last Image was overwritten by this build and docker.io does not give older digest-hashes to pull older images.
I tried to get older Image-Versions without success on docker.io.
I saw, that other alpine-containers were updated on the same time.
For others, maybe facing the same problem, here my fix:
I was able to fix my setup by hardwiring the Digest of the Dockerimage to the version before.
The problem was, finding the digest hash-values of the images. They are not to find in docker.io (only the most recent hash).
But fortunately, there is an repo keeping track of the Versions:
https://github.com/docker-library/repo-info/blob/master/repos/node/remote/lts-alpine.md
Following the git-history of lts-alpine.md gave me the older image-hash. Accordingly i changed
FROM node:lts-alpine as node
to
FROM node#sha256:fda98168118e5a8f4269efca4101ee51dd5c75c0fe56d8eb6fad80455c2f5827 as node
Related
I am trying to move my tests to my docker image build stage but it seems to ignore my test build at all and just skip it when I build the image.
What can be the problem?
# Base build
FROM node:16.13 AS build
RUN mkdir /app
WORKDIR /app
ADD package*.json ./
RUN npm i -g npm#6.14.15
RUN npm i --production
COPY . .
RUN npm run build
# Clean
RUN npm prune --production
RUN npm install -g node-prune
RUN node-prune
# Test
FROM build AS test
RUN npm i -g npm#6.14.15
RUN npm i -D jest typescript
RUN npm i -D ts-jest #types/jest
RUN npm i -D #shelf/jest-mongodb
RUN npx jest
# Release
FROM node:16.13-alpine
RUN mkdir /app
WORKDIR /app
COPY --from=build /app/dist ./dist/
COPY --from=build /app/node_modules ./node_modules/
COPY --from=build /app/package.json ./
CMD npm start
Above you can see my Dockerfile where I am preparing the build, then plan to have tests and after that, I am making my release image.
I've already been playing around with that for hours; I cleared cache, made tweaks with the order in the file) but it didn't help. It keeps ignoring my test build.
Any hint of that and in general on my Dockerfile is welcomed
Docker internally has two different systems to build images. Newer Dockers default to a newer system called BuildKit. One of the major differences is the way the two systems handle multi-stage builds: the older system just runs all of the build stages until the end, but BuildKit figures out that the final stage COPY --from=build but doesn't use anything from the test stage, and just skips it.
I wouldn't run these tests in a Dockerfile (since it doesn't produce a runnable artifact) or in Docker at all. I'd probably run them in my host-based development environment, before building a Docker image:
# On the host
npm install # devDependencies include all test dependencies
npm run tests # locally
# repeat until the tests pass (without involving a container)
# Build the image _after_ the tests pass
docker build -t myapp .
docker build also has a --target option to name a specific stage so you could tell Docker to "build the test image", but you'd just immediately delete it. This is where I don't think Docker quite makes sense as a test runner.
docker build -t delete-me --target test
docker build -t my-app .
docker rmi delete-me
Your Dockerfile also installs a MongoDB test helper. If the test depends on a live database rather than a mock implementation, the other thing to be aware of here is that code in a Dockerfile can never connect to other containers. You have to run this test from somewhere else in this case.
I am trying some something out with gitlab-runner image,
FROM gitlab/gitlab-runner:alpine
WORKDIR /app
COPY . /app
RUN apk add yarn && yarn install
RUN yarn --version # this layer prints 1.16.0
RUN ng build --prod
EXPOSE 3000
CMD ["yarn", "run", "start"]
above is the docker file I have created
docker build -t runner:1 .
I was able to build the image successfully
docker run -p 3000:3000 runner:1
but when I try to run the container it gives me below error
`*FATAL: Command yarn not found.*`
not sure about the behavior, if it is able to install yarn (apk add yarn) in base images and install the dependencies using yarn install then how it is not able to find the yarn command while running the container? Where I am going wrong.
Also at which directory yarn is installed in the alpine ?
I know it is not an efficient docker file, but I am trying to run the container first before optimizing it.
It outputs the version. It means the yarn is installed already. You could find the path the same as you find the version.
RUN which yarn
Step 6/10 : RUN which yarn
---> Running in 0f633b81f2ed
/usr/bin/yarn
We can see the /usr/bin/ has added to PATH.
Step 7/11 : RUN echo $PATH
---> Running in fc3f40b6bfd9
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
But I couldn't figure out why isn't reading yarn from the PATH.
So, we have set the PATH explicitly in our Dockerfile.
ENV PATH=${PATH}
But still, the issue persists. Now we have to separate yarn and commands as ENTRYPOINT and CMD respectively in the Dockerfile.
ENTRYPOINT ["yarn"]
CMD ["run", "start"]
Updated Dockerfile
FROM gitlab/gitlab-runner:alpine
ENV PATH=${PATH}
WORKDIR /app
COPY . /app
RUN apk add yarn && yarn install
RUN yarn --version # this layer prints 1.16.0
RUN ng build --prod
EXPOSE 3000
ENTRYPOINT ["yarn"]
CMD ["run", "start"]
---
$ docker run -p 3000:3000 harik8/yarn:latest
yarn run v1.16.0
error Couldn't find a package.json file in "/app"
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
The behaviours of the base image look unusual. It'd be better to go through it.
To build your app you shouldn't use gitlab-runner image, but a 'node' one.
Gilab-runner image is for running a gitlab agent which can be connected to docker engine and spawn the node container in which you will execute your build, in your case a docker image build.
To use gilab you need to prepare a gitlab-ci file where you will define what steps and which 'services' you need to do your build.
Tl;dr: change base image to node:latest and as a entirely separate work setup gitlab runner.
However if your aim is to have your application to extend gitlab runner, try docker multistage builds.
First, use node:latest image to build your app, and then copy the build output into gitlab-runner.
Runtime images such as gitlab-runner are stripped from build tools like yarn or npm, that's why your image fails. Main goal is to keep runtime images as small as possible and sdk's are not needed and sometimes dangerous when it comes to production-level work.
I have created my Dockerfile with attention to use the docker cache system. I'm adding the package.json and package-lock.json files into a clean container, and then run npm ci command. I expect it to be used from the cache if the package.json and package-lock.json were not changed, but it keeps running.
Am I missing something? Is there something wrong with my Dockerfile?
FROM node:13.10.1-stretch as base
ADD package.json /app/package.json
ADD package-lock.json /app/package-lock.json
WORKDIR /app
RUN npm ci --unsafe-perm
The output I get:
Step 1/12 : FROM node:13.10.1-stretch as base
---> 7aef30ae6655
Step 2/12 : ADD package.json /app/package.json
---> ce655a3453f2
Step 3/12 : ADD package-lock.json /app/package-lock.json
---> 797cda1e10b2
Step 4/12 : WORKDIR /app
Removing intermediate container 5b6929b80ad6
---> 80c2aac903c5
Step 5/12 : RUN npm ci --unsafe-perm
---> Running in 7732a8aca146
> fsevents#1.2.12 install /app/node_modules/webpack-dev-server/node_modules/fsevents
> node-gyp rebuild
make: Entering directory '/app/node_modules/webpack-dev-server/node_modules/fsevents/build'
SOLINK_MODULE(target) Release/obj.target/.node
COPY Release/.node
make: Leaving directory '/app/node_modules/webpack-dev-server/node_modules/fsevents/build'
[...]
Assuming that you don't run other commands before package.json was added, your Dockerfile is fine and the layer adding your package file is cached. For example, we build a simple Dockerfile that just adds your package.json config file:
FROM node:13.10.1-stretch as base
ADD package.json /app/package.json
First run:
$ docker build -t so-example .
Sending build context to Docker daemon 3.072kB
Step 1/2 : FROM node:13.10.1-stretch as base
13.10.1-stretch: Pulling from library/node
[...]
Status: Downloaded newer image for node:13.10.1-stretch
---> 7aef30ae6655
Step 2/2 : ADD package.json /app/package.json
---> a7bb80c06ecb
Successfully built a7bb80c06ecb
Successfully tagged so-example:latest
Second run
$ docker build -t so-example .
Sending build context to Docker daemon 3.072kB
Step 1/2 : FROM node:13.10.1-stretch as base
---> 7aef30ae6655
Step 2/2 : ADD package.json /app/package.json
---> Using cache
---> a7bb80c06ecb
Successfully built a7bb80c06ecb
Successfully tagged so-example:latest
As you can see, the caching works. Could you please verify this with such a minium example? Most of the time caching breaks because of a suboptimal ordner. Please check the following:
Execute the build command twice. Caching could only work after the first run
Make sure, that no other steps you may havent posted here were executed in your Dockerfile that invalidates the cache
Are there any cleanup commands running? Something like docker prune or docker image prune (or manual list/delete images on older versions) would delete your image
Check/post your calls how you build the image
I have a Node / TypeScript application which I'm trying to run from Docker.
yarn tsc works fine locally, but does not work in the context of the Dockerfile. I think the issue is regarding which directory the command is being run from with the Dockerfile, but I'm not sure how to fix it.
How can I make sure tsc can see the tsconfig.json file?
Dockerfile
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN yarn
# Copy source files.
COPY . .
# Run tsc.
RUN yarn prepare
# Run app.
CMD [ "yarn", "start" ]
package.json
"scripts": {
"prepare": "yarn tsc",
"tsc": "tsc -p .",
"dev": "ts-node-dev --respawn --transpileOnly server.ts",
"start": "./node_modules/nodemon/bin/nodemon.js ./build/server.js",
},
Error output
docker build --tag app ./
Sending build context to Docker daemon 114.1MB
Step 1/7 : FROM node:10
---> e05cbde47b8f
Step 2/7 : WORKDIR /usr/src/app
---> Using cache
---> faaea91b16ae
Step 3/7 : COPY package*.json ./
---> 64310f50355d
Step 4/7 : RUN yarn
---> Running in be8aed305980
yarn install v1.16.0
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents#1.2.9: The platform "linux" is incompatible with this module.
info "fsevents#1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
$ yarn tsc
yarn run v1.16.0
$ tsc -p ./
error TS5057: Cannot find a tsconfig.json file at the specified directory: './'.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command '/bin/sh -c yarn' returned a non-zero code: 1
As David said,
Just running yarn is equivalent to yarn install, which runs a "prepare" script.
You could change your script to copy the application code to the given directory before running yarn command, if you do not need for yarn command to run first.
FROM node:10
WORKDIR /usr/src/app
# Copy source files.
COPY . .
RUN yarn
# Run app.
CMD [ "yarn", "start" ]
If you look carefully at your docker build output, you'll notice that the last Dockerfile instruction that gets run is the
Step 4/7 : RUN yarn
In particular, yarn treats a couple of script names as special. Just running yarn is equivalent to yarn install, which runs a "prepare" script. In your case the "prepare" script runs tsc; but, this is happening during the "install dependencies" phase of the Dockerfile, and your application code isn't there yet.
You can work around this by using yarn's --ignore-scripts option. A number of other options make sense here in the context of building a Docker image. (You could use a multi-stage build to build a final image with only --production dependencies and not the tsc compiler or other build-time tools, for example.)
FROM node:10
WORKDIR /usr/src/app
COPY package.json yarn.lock ./
RUN yarn install --ignore-scripts --frozen-lockfile --non-interactive
COPY . .
RUN yarn prepare
CMD ["node", "./build/server.js"]
Yarn version: 0.21.2
Nodejs version: 6.9 and 4.7
When running yarn locally it works
When running npm install it works
When running yarn install Dockerfile (docker build .) it fails with: error Couldn't find a package.json file in "/root/.cache/yarn/npm-readable-stream-2.2.2-a9e6fec3c7dda85f8bb1b3ba7028604556fc825e"
I have absolutely no idea why.
Step 16 : RUN yarn install
---> Running in 917c2b1b57fb
yarn install v0.21.2
[1/4] Resolving packages...
[2/4] Fetching packages...
error Couldn't find a package.json file in "/root/.cache/yarn/npm-readable-stream-2.2.2-a9e6fec3c7dda85f8bb1b3ba7028604556fc825e"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command '/bin/sh -c yarn install' returned a non-zero code: 1
Yarn is installed this way in the Dockerfile
RUN curl -o- -L https://yarnpkg.com/install.sh | bash -s -- --version 0.21.2
ENV PATH /root/.yarn/bin:$PATH
When you build a docker image, no files are copied automatically in the docker image despite that are part of the docker build context . (depends also of your .dockerignore file configuration if you have any). To add files from the docker context to your docker image you can do it explicitly with running commands like ADD or COPY.
Below an example of dockerfile:
WORKDIR /app
COPY ["yarn.lock", "package.json", "./"]
# Install app dependencies
RUN yarn install --check-files --non-interactive --ignore-optional --frozen-lockfile