Bitbucket Pipelines from Docker Image has Missing NPM Modules - node.js

Question

Question
What is wrong with my Dockerfile or bitbucket-pipelines.yml? Why are modules missing from the bitbucket pipelines environment?
Error
When I try to npm run build my Vue2 project with webpack using Bitbucket Pipelines, I get errors regarding missing modules.
From Logs
npm run build
> people-is#1.0.0 build /opt/atlassian/pipelines/agent/build
> node build/build.js
module.js:549
throw err;
^
Error: Cannot find module 'cli-spinners'
Files
Here are the files for configuration.
Dockerfile - builds cportwine/people-is
FROM node:8.10.0
RUN npm install
RUN npm install -g firebase-tools
CMD [ "npm", "run", "build" ]
bitbucket-pipelines.yml
image:
name: cportwine/people-is
pipelines:
default:
- step:
script:
- npm run build
package.json
{
"name": "people-is",
"version": "1.0.0",
"description": "A Vue.js project",
"author": "cportwine",
"private": true,
"scripts": {
"dev": "node build/dev-server.js",
"start": "node build/dev-server.js",
"build": "node build/build.js",
"deploy": "firebase deploy --token $FIREBASE_TOKEN"
},
"dependencies": {
"rxjs": "^5.5.8",
"uuid": "^3.2.1",
"vue": "^2.5.16",
"vue-json-excel": "^0.1.9",
"vue-router": "^2.8.1",
"vue-rx": "^5.0.0",
"vuefire": "^1.4.5",
"vuetify": "^0.15.2"
},
"devDependencies": {
"autoprefixer": "^7.2.6",
"babel-core": "^6.22.1",
"babel-loader": "^7.1.4",
"babel-plugin-transform-runtime": "^6.22.0",
"babel-preset-env": "^1.6.1",
"babel-preset-stage-2": "^6.22.0",
"babel-register": "^6.22.0",
"chalk": "^2.3.2",
"connect-history-api-fallback": "^1.5.0",
"copy-webpack-plugin": "^4.5.1",
"css-loader": "^0.28.11",
"cssnano": "^3.10.0",
"eslint": "^4.19.1",
"eslint-config-standard": "^11.0.0",
"eslint-friendly-formatter": "^3.0.0",
"eslint-loader": "^1.9.0",
"eslint-plugin-html": "^4.0.2",
"eslint-plugin-promise": "^3.7.0",
"eslint-plugin-standard": "^3.0.1",
"eventsource-polyfill": "^0.9.6",
"express": "^4.16.3",
"extract-text-webpack-plugin": "^2.0.0",
"file-loader": "^0.11.1",
"firebase": "^4.12.0",
"firebase-tools": "^3.17.7",
"friendly-errors-webpack-plugin": "^1.1.3",
"html-webpack-plugin": "^2.28.0",
"http-proxy-middleware": "^0.17.3",
"opn": "^5.3.0",
"optimize-css-assets-webpack-plugin": "^2.0.0",
"ora": "^1.4.0",
"rimraf": "^2.6.0",
"semver": "^5.5.0",
"shelljs": "^0.7.6",
"url-loader": "^0.5.8",
"vue-loader": "^13.7.1",
"vue-style-loader": "^3.1.2",
"vue-template-compiler": "^2.5.16",
"vuex": "^2.5.0",
"webpack": "^2.6.1",
"webpack-bundle-analyzer": "^2.11.1",
"webpack-dev-middleware": "^1.12.2",
"webpack-hot-middleware": "^2.21.2",
"webpack-merge": "^4.1.2"
},
"engines": {
"node": ">=8.10.0",
"npm": ">= 5.6.0"
},
"browserslist": [
"> 1%",
"last 2 versions",
"not ie <= 8"
],
"main": "index.js",
"repository": {
"type": "git",
"url": "git+https://chaddportwine#bitbucket.org/jahnelgroup/people-is.git"
},
"keywords": [],
"license": "ISC",
"homepage": "https://bitbucket.org/jahnelgroup/people-is#readme"
}
What I see
When I ls the node_modules folder in both environments, they do not match. Modules are missing from bitbucket pipelines.
local folder
people-is/node_modules
...
chalk
char-spinner
chardet
check-types
chokidar
chownr
cipher-base
circular-json
cjson
clap
class-utils
clean-css
cli-boxes
cli-cursor
cli-spinners
cli-table
cli-table2
cli-width
cliui
...
bitbucket folder
/opt/atlassian/pipelines/agent/build/node_modules
Woah, missing modules!
...
chalk
cli-cursor
co
...
What I have tried
I added a command to the bitbucket-pipelines.yml to npm install before I build.
bitbucket-pipelines.yml
image:
name: cportwine/people-is
pipelines:
default:
- step:
script:
- npm install
- npm run build
This adds some additional modules (like cli-spinners from the error) to /opt/atlassian/pipelines/agent/build/node_modules.
bitbucket folder
/opt/atlassian/pipelines/agent/build/node_modules
...
chalk
char-spinner
chardet
check-types
chokidar
chownr
cipher-base
circular-json
cjson
clap
class-utils
clean-css
cli-boxes
cli-cursor
cli-spinners
cli-table
cli-table2
cli-width
cliui
clone
clone-response
co
...
However, the build command still fails, due to a different missing module.
Error
> people-is#1.0.0 build /opt/atlassian/pipelines/agent/build
> node build/build.js
module.js:549
throw err;
^
Error: Cannot find module './_safeGet'

Answer 1

Solutions
I can now build the app, but I don't know why!
1 - Simplify the Dockerfile
I removed all the npm commands. Maybe the npm install commands were redundant? There was no advantage using the Docker Image to pre-install npm packages.
2 - Remove Node_Modules before install
Using the bitbucket-pipelines.yml, remove the node_modules folder, and then perform npm install -g npm and npm install and npm install -g firebase-tools.
File Changes
bitbucket-pipelines.yml (added lines)
image:
name: cportwine/people-is
pipelines:
default:
- step:
script:
- rm -r node_modules <---- remove
- npm install -g npm <---- install
- npm install <---- install
- npm install -g firebase-tools <---- install
- npm run build
Dockerfile (lines removed)
FROM node:8.10.0
<---- remove
CMD [ "npm", "run", "build" ]
Answer ?
I'm not sure why moving all the npm install stuff into the bitbucket.pipelines.yml solved my issue building the project. I thought Docker would enable me to define my environment, e.g., install a version of node/npm and firebase. And pipelines would "run" that.
If someone could explain what I am missing, I would accept that answer.

Answer 2

Answer
I received support from the Atlassian Team
Leave npm install -g firebase in the docker image.
Move npm install from the docker image to the
bitbucket-pipelines.yml file.
Reason
The node_modules folder was listed in .gitignore
tl;dr
My mistake - I forgot about .gitignore and how that affects the node_modules folder in source control, e.g., Bitbucket Pipelines.
I was looking at my local node_modules folder and building locally which worked.
However
The node_modules in source control, by design, is not in-sync with my local folder because it's included in the .gitignore file.
So
It was necessary for me to rm node_modules and npm install using the bitbucket-pipelines.yml. Now, BitPipes finds the modules I have installed locally.
This is sort of the point of maintaining the package.json, but I got confused.