I have a monorepo project created with lerna.
I am trying to create a CI flow to test my repository installation and build actions on a remote environment.
After hitting lerna bootstrap to install all packages dependencies, I am using lerna run build --stream on the root folder which invokes lerna run build for each sub package I have in my repository.
This flow works great on my local machine but fails to work using travis-ci build.
Conclusions so far:
Issue is probably not related to travis-ci service nor to operating system for the reason same behaviour occur on both Ubuntu VM instance on GCP im trying to run the flow on, and on travis-ci server which is configured as a macOS machine.
Neither of bootstrap flags --no-ci, --force-local changes anything.
Navigating to ~/sqpoc/node_modules/#superquery/components/dist suggesting that build files are properly created, meaning issue is probably related to internal lerna symlink flow which fails on the stated environments.
Output:
$ lerna run build --stream
lerna notice cli v4.0.0
lerna info Executing command in 4 packages: "yarn run build"
#superquery/components: $ node ./scripts/build.ts
#superquery/greeter: $ tsc -p tsconfig.release.json && webpack
#superquery/superquery: $ react-scripts build
#superquery/superquery: Creating an optimized production build...
#superquery/greeter: asset bundle.js 39.4 KiB [compared for emit] (name: main)
#superquery/greeter: runtime modules 670 bytes 3 modules
#superquery/greeter: cacheable modules 11.5 KiB
#superquery/greeter: ./build/src/main.js 701 bytes [built] [code generated]
#superquery/greeter: ./node_modules/tslib/tslib.es6.js 10.8 KiB [built] [code generated]
#superquery/greeter: webpack 5.37.0 compiled successfully in 3703 ms
#superquery/server: $ tsc -p tsconfig.release.json
#superquery/superquery: Failed to compile.
#superquery/superquery: ./src/App.tsx
#superquery/superquery: Cannot find module: '#superquery/components'. Make sure this package is installed.
#superquery/superquery: You can install this package by running: yarn add #superquery/components.
#superquery/superquery: error Command failed with exit code 1.
#superquery/superquery: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
lerna ERR! yarn run build exited 1 in '#superquery/superquery'
error Command failed with exit code 1.
Expected output:
$ lerna run build --stream
lerna notice cli v4.0.0
lerna info Executing command in 4 packages: "yarn run build"
#superquery/components: $ node ./scripts/build.ts
#superquery/greeter: $ tsc -p tsconfig.release.json && webpack
#superquery/greeter: asset bundle.js 39.4 KiB [compared for emit] (name: main)
#superquery/greeter: runtime modules 670 bytes 3 modules
#superquery/greeter: cacheable modules 11.5 KiB
#superquery/greeter: ./build/src/main.js 701 bytes [built] [code generated]
#superquery/greeter: ./node_modules/tslib/tslib.es6.js 10.8 KiB [built] [code generated]
#superquery/greeter: webpack 5.37.0 compiled successfully in 1521 ms
#superquery/server: $ tsc -p tsconfig.release.json
#superquery/superquery: $ react-scripts build
#superquery/superquery: Creating an optimized production build...
#superquery/superquery: Compiled successfully.
#superquery/superquery: File sizes after gzip:
#superquery/superquery: 119.52 KB build/static/js/2.ea8fed9a.chunk.js
#superquery/superquery: 4.79 KB build/static/js/main.7a35accd.chunk.js
#superquery/superquery: 1.41 KB build/static/js/3.80a7969b.chunk.js
#superquery/superquery: 1.17 KB build/static/js/runtime-main.d48b464c.js
#superquery/superquery: 574 B build/static/css/main.9d5b29c0.chunk.css
#superquery/superquery: The project was built assuming it is hosted at /.
#superquery/superquery: You can control this with the homepage field in your package.json.
#superquery/superquery: The build folder is ready to be deployed.
#superquery/superquery: You may serve it with a static server:
#superquery/superquery: npm install -g serve
#superquery/superquery: serve -s build
#superquery/superquery: Find out more about deployment here:
#superquery/superquery: https://cra.link/deployment
lerna success run Ran npm script 'build' in 4 packages in 35.9s:
lerna success - #superquery/components
lerna success - #superquery/greeter
lerna success - #superquery/server
lerna success - #superquery/superquery
✨ Done in 37.28s.
package.json file:
{
"name": "superquery",
"version": "1.0.0",
"author": "***************",
"repository": "https://github.com/doitintl/sqpoc.git",
"main": "index.js",
"license": "MIT",
"private": true,
"workspaces": {
"packages": [
"packages/*"
]
},
"scripts": {
"start": "lerna run start --stream",
"test": "lerna run test --stream",
"test:ci": "lerna run test:ci --stream",
"watch": "lerna run watch --stream",
"storybook": "lerna run storybook --stream",
"production": "lerna run production --stream",
"build": "lerna run build --stream",
"deploy": "lerna run deploy --stream",
"bootstrap": "lerna run rimraf && lerna bootstrap",
"rimraf": "rm -rf node_modules",
"installApp": "yarn rimraf && yarn install && yarn bootstrap",
"dev": "lerna run dev",
"dev-logs": "lerna run dev --parallel",
"dev-module": "lerna run dev --scope",
"e2e": "nightwatch -e chrome,firefox",
"e2e-test": "concurrently -k --success first \"npm run start\" \"npm run e2e\""
},
"devDependencies": {
"chromedriver": "^90.0.0",
"geckodriver": "^1.22.3",
"lerna": "^4.0.0",
"selenium-server": "^3.141.59"
}
}
lerna.json file:
{
"useWorkspaces": true,
"npmClient": "yarn",
"version": "1.0.0",
"packages": [
"packages/superquery",
"packages/components"
]
}
Technology stack used:
node 14.17.0
npm 6.14.13
nvm 0.37.2
lerna 4.0.0
yarn 1.22.10
Tried to follow suggestion from below links (and more), it was either naive or irelevant to my issue.
Lerna bootstrap does not link local dependencies?
https://github.com/lerna/lerna/issues/1444
Solved after updating lerna.json file:
{
"useWorkspaces": true,
"npmClient": "yarn",
"version": "1.0.0"
}
Related
I'm using parcel-bundler for sass on my projects, I've always used npm start instead of npm run build and it has always worked for me. But this time, when I try to deploy my project on Vercel, it failed and it says "Error: Command "npm run build" exited with 127"?
I already tried setting the CI Environment Variable to false Using Vercel CLI but it's still giving me the same result.
package.json
{
"name": "portfolio",
"version": "1.0.0",
"description": "",
"source": "src/index.html",
"scripts": {
"start": "parcel src/index.html",
"build": "parcel build src/index.html"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"sass": "^1.52.2"
}
}
[21:07:21.022] Cloning completed: 371.994ms
[21:07:21.103] Installing build runtime...
[21:07:22.998] Build runtime installed: 1.894s
[21:07:23.868] No Build Cache available
[21:07:24.021] Installing dependencies...
[21:07:25.307]
[21:07:25.308] added 17 packages in 1s
[21:07:25.308]
[21:07:25.308] 2 packages are looking for funding
[21:07:25.308] run `npm fund` for details
[21:07:25.510] Detected `package-lock.json` generated by npm 7...
[21:07:25.510] Running "npm run build"
[21:07:25.783]
[21:07:25.784] > portfolio#1.0.0 build
[21:07:25.784] > parcel build src/index.html
[21:07:25.784]
[21:07:25.789] sh: parcel: command not found
[21:07:25.798] Error: Command "npm run build" exited with 127
You are using Parcel but it's not listed in your devDependencies. My guess is that you have it installed globally. Try adding it as a local devDependency:
npm install --save-dev parcel
On an inherited project I have, I am trying to get the build command to build a version other than Production.
I have attempted to change the alias in the script section in package.json to pass in extra variables such as --dev and --configuration=dev to no avail.
The project has these json data files:
env.dev
env.development
env.production
with the package.json has this build alias build:dev which I run npm run build:dev:
"scripts": {
"start": "NODE_ENV=dev && react-scripts start",
…
"build:dev": "npm run build --dev --configuration=dev && react-scripts build"
}
This works and builds, but for production only which I verify when I view the resultant files.
If I remove the file env.production from the directory and run the build command, it fails with:
Creating an optimized production build...
Failed to compile.
Module not found: Error: Can't resolve 'polyfills' in 'C:\Work\MyProj\WebSiteName\src'
which just informs me that it can alias polyfills found in the env.production file for the location NODE_PATH=src/.
Thoughts?
you need to set the env. variable like you do in "start" before calling the build command.
"build:dev": "NODE_ENV=dev npm run build --dev --configuration=dev && react-scripts build"
I have created a npm module(let call it ModuleA) and defined clean script in its package.json file like below:
"scripts": {
"test": "nyc mocha tests/ --opts mocha.opts",
"build": "babel -d dist/ src/",
"prepublish": "yarn run clean && yarn run build",
"postinstall": "yarn run clean && yarn run build",
"clean": "rimraf ./dist"
},
I use rimraf to remove the dist directory. This dependency is defined in devDependencies as "rimraf": "^2.6.1". It works fine on this project. But in one of my other project (let call it ModuleB) which has a dependency on this module, the yarn install doesn't work and I get below error:
$ rimraf ./dist
sh: 1: rimraf: not found
this error happens when npm/yarn is building the ModuleB. I have checked that rimraf exist in node_modules/.bin directory in ModuleB. It works fine if I install rimraf globally. I wonder how I can make the npm/yarn to use rimraf from node_modules/.bin/rimraf?
BTW, I also put the rimraf in devDependencies in ModuleB.
I tried to update the script in ModuleA to use rimraf from node_modules/.bin/rimraf as below:
"clean": "node_modules/.bin/rimraf ./dist"
it works fine on ModuleA. But I got below error when run yarn install on ModuleB:
$ node_modules/.bin/rimraf ./dist
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
sh: node_modules/.bin/rimraf: No such file or directory
error Command failed with exit code 127.
See those issues:
https://github.com/yarnpkg/yarn/issues/721 (postinstall hook doesn't appear to be run #721)
https://github.com/yarnpkg/yarn/issues/853 (preinstall and postinstall are not run #853)
https://github.com/yarnpkg/yarn/issues/614 (Bug: issue with postinstall scripts in git-hooks package)
Now make sure that yarn clean and npm clean works as expected.
For example this will not work if you didn't install rimraf globally:
$ rimraf ./dist
But this should work:
$ ./node_modules/.bin/rimraf ./dist
Test those commands first:
npm run clean
and:
yarn run clean
to use the "clean": "rimraf ./dist" script defined in package.json.
Note that ./node_modules/.bin is added to your PATH when you run package.json scripts.
Try running the commands from the simple ones where you run the script directly without npm or yarn, then test with both npmand yarn, and then finally try to narrow down the problem with postinstall hooks.
If you want to increase the memory limit in node, the option to be passed will be:
node --max-old-space-size=4096 yourFile.js
But in my scenario I am using yarn, my package.json looks like this:
{
"name": "myapp",
"productName": "myapp",
"version": "1.0.0",
"main": "app/main.js",
...
"scripts": {
"package-win": "npm run build && build --win --x64",
"package-mac": "npm run build && build --mac",
...
},
...
I execute yarn package-win on a windows machine to invoke electron-builder for my electron app built with react.js. But I get always npm ERR! code ELIFECYCLE Due to out of memory. In my mac also got the FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory Error but still got the packages generated (how? I don't know) when invoking yarn package-mac.
I search a lot on how to use the --max-old-space-size=4096 option, but I haven't found anything that works.
I tried "package-win": "node --max-old-space-size=4096 npm run build && build --win --x64", But the path have problems locating npm. Even if I use which npm, still which is not recognized.
Thanks for helping.
Answering to myself: Using this options in package-win is wrong. As package-win executes the build task, then:
"build": "npm run build-main && npm run build-renderer",
...
"build-main": "cross-env NODE_ENV=production node --max_old_space_size=6144 --optimize_for_size --stack_size=6144 --trace-warnings -r babel-register ./node_modules/webpack/bin/webpack --config webpack.config.main.prod.js --progress --profile --colors",
"build-renderer": "cross-env NODE_ENV=production node --max_old_space_size=6144 --optimize_for_size --stack_size=6144 --trace-warnings -r babel-register ./node_modules/webpack/bin/webpack --config webpack.config.renderer.prod.js --progress --profile --colors",
That way it works!
You don't have to use option in package-win.
The way you should use-
"build-ts-prod": "node --max-old-space-size=1024
./node_modules/typescript/bin/tsc --skipLibCheck --diagnostics",
I'm using Bluemix DevOps Pipelines to deploy a Node.js app to Bluemix as a Cloud Foundry app. It is using the sdk-for-nodejs buildback (although I haven't specified that, it's detecting the package.json file).
It takes a long time to stage the app. Looking at the logs, this is because it appears to be running npm install. However, I have set up my pipeline so that npm install is called in an earlier stage, and the build artifact that is to be deployed already contains the installed node_modules, so if I have to restage it should be quick, as it shouldn't have to install all the modules again.
This is the command that my pipeline is using to deploy:
cf push "${CF_APP}" -c "npm run start:prod"
My Procfile also contains the same command:
web: npm run start:prod
node: npm run start:prod
(I've tried it both with and without the second line. I think the logs from below were when it contained both lines.)
And indeed towards the end of my deployment logs it says
App mcp-server was started using this command `npm run start:prod`
That script in my package.json file does not run npm install:
"scripts": {
...
"start:prod": "cross-env NODE_ENV=production node index.js",
The logs that make it look like it's re-installing the modules are:
Staging...
-----> IBM SDK for Node.js Buildpack v3.11-20170303-1144
-----> Creating runtime environment
NPM_CONFIG_PRODUCTION=true
NODE_ENV=production
NODE_MODULES_CACHE=true
-----> Installing binaries
engines.npm (package.json): unspecified (use default)
Resolving node version >=4 via 'node-version-resolver'
Installing IBM SDK for Node.js (4.8.0) from cache
Using default npm version: 2.15.11
-----> Restoring cache
Loading 2 from cacheDirectories (default):
- node_modules (exists - skipping)
- bower_components (not cached - skipping)
-----> Building dependencies
Rebuilding any native modules
> radium#0.18.2 postinstall /tmp/app/node_modules/radium
> dtrace-provider#0.8.1 install /tmp/app/node_modules/dtrace-provider
> pre-commit#1.2.2 install /tmp/app/node_modules/pre-commit
> spawn-sync#1.0.15 postinstall /tmp/app/node_modules/spawn-sync
> node postinstall
Completed: 0.1% (0.1mb / 78.2mb)
...
Completed: 99.9% (78.2mb / 78.2mb)
inside extract, run complete
Completed: 0.0% (0.0mb / 78.2mb)
Completed: 0.1% (0.0mb / 78.2mb)
Completed: 2.6% (2.0mb / 78.2mb)
Completed: 8.3% (6.5mb / 78.2mb)
Completed: 43.1% (33.7mb / 78.2mb)
Completed: 44.3% (34.7mb / 78.2mb)
Completed: 45.5% (35.6mb / 78.2mb)
Completed: 64.2% (50.2mb / 78.2mb)
Completed: 65.0% (50.8mb / 78.2mb)
Completed: 66.5% (52.1mb / 78.2mb)
Completed: 71.4% (55.8mb / 78.2mb)
> node-gyp rebuild
make: Entering directory `/tmp/app/node_modules/nodejieba/build'
...
> spawn-sync#1.0.15 postinstall /tmp/app/node_modules/spawn-sync
> npm run clean && npm run build
> myapp#2.0.0 clean /tmp/app
> myapp#2.0.0 build /tmp/app
> cross-env NODE_ENV=production webpack --config webpack.config.js
Of particular note are the lines:
Completed: 0.1% (0.1mb / 78.2mb)
...
Completed: 99.9% (78.2mb / 78.2mb)
where I've omitted well over 100 lines where it seems to be installing something very slowly.
And also:
> npm run clean && npm run build
> myapp#2.0.0 clean /tmp/app
> myapp#2.0.0 build /tmp/app
> cross-env NODE_ENV=production webpack --config webpack.config.js
which are scripts from my app that I really don't want to be running at deployment time.
I've already tried to tell it to run npm run start:prod - how can I stop it from running these install steps?
Looking at the buildpack code, I don't see a flag to disable npm install.
https://github.com/cloudfoundry/nodejs-buildpack/blob/master/lib/dependencies.sh#L80
Option 1
Create a package_nodependencies.json file with no dependencies listed. After your build steps runs the npm install using your regular package.json, delete the package.json and rename package_nodependencies.json to package.json.
rm package.json && mv package_nodependencies.json package.json
npm install on cf push should now be quick. Be sure that you don't have node_modules in your .cfignore file
Option 2
Move all your dependencies to devDependencies in package.json. During your build, run npm install without PRODUCTION
NODE_ENV=development npm install