I'm getting a memory error when trying to deploy a Node.js app to heroku. It doesn't seem to get past the installation section:
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Node.js app detected
remote:
remote: -----> Creating runtime environment
remote:
remote: NPM_CONFIG_LOGLEVEL=error
remote: NODE_ENV=production
remote: NODE_MODULES_CACHE=true
remote: NODE_VERBOSE=false
remote:
remote: -----> Installing binaries
remote: engines.node (package.json): unspecified
remote: engines.npm (package.json): unspecified (use default)
remote:
remote: Resolving node version 12.x...
remote: Downloading and installing node 12.13.1...
remote: Using default npm version: 6.12.1
remote:
remote: -----> Restoring cache
remote: Cached directories were not restored due to a change in version of node, npm, yarn or stack
remote: Module installation may take longer for this build
remote:
remote: -----> Installing dependencies
remote: Installing node modules (package.json + package-lock)
remote: npm ERR! Maximum call stack size exceeded
remote:
remote: npm ERR! A complete log of this run can be found in:
remote: npm ERR! /tmp/npmcache.TGwfg/_logs/2019-11-23T11_15_42_537Z-debug.log
I've tried turning the module cache off, which did nothing. I've also tried turning more verbose npm logging, but all that tells me is that the installation hasn't finished.
The application isn't particularly big, just an express application. There must be some dependency that has a large stack of pre/post install commands... I guess the only way to find out which one it is by uninstalling one at a time and redeploying :(
But before I go down that route, I thought I'd ask around to see if anyone else has experienced this before. I'm only using a free dyno at the moment to test if Heroku would be suitable.
I had this issue and #Rashomon's solution worked, but I had to type it thusly "engines": {"node": "14.16.1", "npm": "6.14.12"}, note the double quotes on everything.
To check your versions, use npm version and node -v in command line
Turns out that using yarn will help me avoid this problem.
rm package-lock.json
yarn
git add -A
git commit -m "chore: use yarn"
git push heroku master
If I decide to upgrade the service then I ask Heroku, directly, why I experienced this. Until then, this is an OK solution,
Related
I made one tiny text change to an old(er) Node application which works as intended in my local development system, including the npm ci command, but it fails to deploy to Heroku with this error:
npm ERR! The `npm ci` command can only install with an existing package-lock.json or
remote npm-shrinkwrap.json with lockfileVersion >= 1
but I have a package-lock.json with:
"lockfileVersion": 2,
This application used to deploy just fine but it's been a year of two since the last deploy so I'm guessing this is some underlying issue with the Heroku deployment, perhaps with the updated build-packs. The application is currently using heroku-18 but it's trying to upgrade to heroku-22 in the same deploy.
I've tried different versions of Node and NPM though I can't go above Node 12 without making significant updates to this legacy application...which I'd r e a a a l l y like to avoid.
Any ideas on how to tickle this away?
Full Error
remote: Building source:
remote:
remote: -----> Building on the Heroku-22 stack
remote: -----> Using buildpack: heroku/nodejs
remote: -----> Node.js app detected
remote:
remote: -----> Creating runtime environment
remote:
remote: NPM_CONFIG_LOGLEVEL=error
remote: NODE_VERBOSE=false
remote: NODE_ENV=production
remote: NODE_MODULES_CACHE=true
remote:
remote: -----> Installing binaries
remote: engines.node (package.json): 15.3.0
remote: engines.npm (package.json): 7.0.14
remote:
remote: Resolving node version 15.3.0...
remote: Downloading and installing node 15.3.0...
remote: npm 7.0.14 already installed with node
remote:
remote: -----> Restoring cache
remote: Cached directories were not restored due to a change in version of node, npm, yarn or stack
remote: Module installation may take longer for this build
remote:
remote: -----> Prebuild
remote: Running heroku-prebuild
remote:
remote: > snazzywiz#2.1.1 heroku-prebuild
remote: > npm --global install pushstate-server#3.0.1; npm --global install #angular/cli
remote:
remote:
remote: added 44 packages, and audited 44 packages in 3s
remote:
remote: 10 vulnerabilities (5 low, 3 moderate, 2 high)
remote:
remote: To address issues that do not require attention, run:
remote: npm audit fix
remote:
remote: To address all issues, run:
remote: npm audit fix --force
remote:
remote: Run `npm audit` for details.
remote:
remote: added 209 packages, and audited 209 packages in 8s
remote:
remote: 25 packages are looking for funding
remote: run `npm fund` for details
remote:
remote: found 0 vulnerabilities
remote: npm notice
remote: npm notice New major version of npm available! 7.0.14 -> 8.19.2
remote: npm notice Changelog: <https://github.com/npm/cli/releases/tag/v8.19.2>
remote: npm notice Run `npm install -g npm#8.19.2` to update!
remote: npm notice
remote:
remote: -----> Installing dependencies
remote: Installing node modules
remote: npm ERR! The `npm ci` command can only install with an existing package-lock.json or
remote: npm ERR! npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm#5 or
remote: npm ERR! later to generate a package-lock.json file, then try again.
remote:
remote: npm ERR! A complete log of this run can be found in:
remote: npm ERR! /tmp/npmcache.ULXke/_logs/2022-09-17T19_15_54_600Z-debug.log
remote:
remote: -----> Build failed
Note: There are several SO questions with the same "The npm ci command can only..." error message but triggered from different scenarios but none from just a deploy to Heroku.
I am willing to install a custom build of libvips with sharp on heroku. I use buildpacks that run without errors, but when it comes to node-gyp compilation, it fails here saying it cannot access /app/vendor/vips/lib/libvips-cpp.so. When I run the build without installing sharp and then connect with heroku run bash, I can see that the file is at this location.
Why node-gyp cannot find it then ?
My buildpacks
=== my-site Buildpack URLs
1. heroku-community/apt
2. https://github.com/lechinoix/heroku-buildpack-vips
3. heroku/nodejs
The error
remote: -----> vips (heroku-20 stack) app detected
remote: -----> Vendoring binaries
remote: Fetching /tmp/codon/tmp/buildpacks/2ef659928923e1b5ae4bf85b886e53d084c6a3d4/build/heroku-20.tar.gz
remote: Changing permissions on libvips folder
remote: -----> Configuring build environment
remote: -----> Building runtime environment
remote: -----> Node.js app detected
remote:
remote: -----> Creating runtime environment
remote:
remote: NPM_CONFIG_LOGLEVEL=error
remote: NODE_VERBOSE=false
remote: NODE_ENV=production
remote: NODE_MODULES_CACHE=false
remote:
remote: -----> Installing binaries
remote: engines.node (package.json): unspecified
remote: engines.npm (package.json): unspecified (use default)
remote:
remote: Resolving node version 14.x...
remote: Downloading and installing node 14.18.1...
remote: Using default npm version: 6.14.15
remote:
remote: -----> Restoring cache
remote: Caching has been disabled because NODE_MODULES_CACHE=false
remote:
remote: -----> Installing dependencies
remote: Installing node modules (package.json)
remote:
remote: > sharp#0.28.1 install /tmp/build_1f4c5c03/node_modules/sharp
remote: > (node install/libvips && node install/dll-copy && prebuild-install) || (node-gyp rebuild && node install/dll-copy)
remote:
remote: sharp: Detected globally-installed libvips v8.10.6
remote: sharp: Building from source via node-gyp
remote: readelf: Error: '/app/vendor/vips/lib/libvips-cpp.so': No such file
remote: gyp: Call to 'if readelf -Ws "$(PKG_CONFIG_PATH="/tmp/build_1f4c5c03/.apt/usr/lib/x86_64-linux-gnu/pkgconfig:/tmp/build_1f4c5c03/.apt/usr/lib/i386-linux-gnu/pkgconfig:/tmp/build_1f4c5c03/.apt/usr/lib/pkgconfig::/tmp/build_1f4c5c03/vendor/vips/lib/pkgconfig:/tmp/build_1f4c5c03/.apt/usr/lib/x86_64-linux-gnu/pkgconfig:/tmp/build_1f4c5c03/.apt/usr/lib/i386-linux-gnu/pkgconfig:/tmp/build_1f4c5c03/.apt/usr/lib/pkgconfig::/usr/local/lib/pkgconfig:/usr/lib/pkgconfig" pkg-config --variable libdir vips-cpp)/libvips-cpp.so" | c++filt | grep -qF __cxx11;then echo "1";else echo "0";fi' returned exit status 0 while in binding.gyp. while trying to load binding.gyp
Thank you so much for your help !
Finally found that heroku is creating the whole /app folder in a directory /tmp/build_1f4c5c03 that also contains the freshly installed vendors. In my .pc files, the package is located in /app so it cannot be found. I used a heroku-prebuild npm script to sed inplace the paths :
TMP_DIR=$(pwd)
for file in $(grep -rl /app/vendor ${TMP_DIR}/vendor/vips/lib/pkgconfig)
do
sed -i "s+/app/vendor+${TMP_DIR}/vendor+g" $file
done
Not perfectly sure it is the best way to do...
I encountered the error mentioned in the Title.
I have referred to a few relevant Heroku articles and tried their suggested changes including specifying the exact version of Node.js, Yarn, or npm that I'm using locally , but that still didn't work.
Here's the relevant sections extracted from my package.json:
"engines": {
"node": "14.18.0",
"npm": "6.14.15",
"yarn": "1.19.1"
},
"ember": {
"edition": "octane"
}
Here's the deployment log:
Enumerating objects: 1134, done.
Counting objects: 100% (1134/1134), done.
Delta compression using up to 4 threads
Compressing objects: 100% (1027/1027), done.
Writing objects: 100% (1134/1134), 309.32 KiB | 2.64 MiB/s, done.
Total 1134 (delta 581), reused 0 (delta 0), pack-reused 0
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Building on the Heroku-20 stack
remote: -----> Using buildpack: https://codon-buildpacks.s3.amazonaws.com/buildpacks/heroku/emberjs.tgz
remote: -----> emberjs app detected
remote: -----> Setting NPM_CONFIG_PRODUCTION to false to install ember-cli toolchain
remote: -----> Fetching buildpack heroku/nodejs-v98
remote: -----> Node.js detected
remote:
remote: -----> Creating runtime environment
remote:
remote: NPM_CONFIG_PRODUCTION=false
remote: NPM_CONFIG_LOGLEVEL=error
remote: NODE_VERBOSE=false
remote: NODE_ENV=production
remote: NODE_MODULES_CACHE=true
remote:
remote: -----> Installing binaries
remote: engines.node (package.json): 14.18.0
remote: engines.npm (package.json): 6.14.15
remote:
remote: Downloading and installing node 14.18.0...
remote: Unable to download node 14.18.0; does it exist?
remote:
remote: -----> Build failed
remote:
remote: We're sorry this build is failing! You can troubleshoot common issues here:
remote: https://devcenter.heroku.com/articles/troubleshooting-node-deploys
remote:
remote: If you're stuck, please submit a ticket so we can help:
remote: https://help.heroku.com/
remote:
remote: Love,
remote: Heroku
remote:
remote: -----> Failed trying to compile heroku/nodejs-v98
remote: ! Push rejected, failed to compile emberjs app.
remote:
remote: ! Push failed
remote: !
According to this Heroku page, versions 12.x, 14.x, and 16.x are supported.
When such version numbers are literally specified in package.json file, during the Heroku build process, those will be resolved to the latest patch version, e.g. 12.22.7, 14.18.1, and 16.13.0 respectively as of the time of writing.
Somehow Heroku was unable to download the latest version of every major series.
In my original question, I used version 14.18.0 in my local and that version was specified in package.json. It appears that Heroku was also not able to download that version.
I double checked this Heroku Help page and noticed version 12.16.3 is given in the example when illustrating how to compare Node and npm versions. I updated my package.json to use 12.16.3 and retried the the deployment and it was successful.
I tried to search but unfortunately I couldn't find any Heroku Help page that specifies the exact Node.js version numbers that they support.
I'm deploying my NodeJs app to Heroku. No issues running the app with nodemon locally.
However when I'm trying to push to Heroku Master, seems like "sharp" package is causing an issue:
remote: -----> Installing dependencies
remote: Installing node modules (package.json + package-lock)
remote:
remote: > sharp#0.22.0 install
/tmp/build_4cfeda2379ea362c72ac083b57638cce/node_modules/sharp
remote: > (node install/libvips && node install/dll-copy &&
prebuild-install) || (node-gyp rebuild && node install/dll-copy)
remote:
remote: info sharp Downloading
https://github.com/lovell/sharp-libvips/releases/download/v8.7.4/libvips-8.7.4-linux-x64.tar.gz
remote:
/tmp/build_4cfeda2379ea362c72ac083b57638cce/node_modules/sharp/install/libvips.js:86
remote: throw new Error(Status
${response.statusCode});
remote: ^
remote:
remote: Error: Status 403
remote: at
/tmp/build_4cfeda2379ea362c72ac083b57638cce/node_modules/sharp/install/libvips.js:86:17
remote: at f
(/tmp/build_4cfeda2379ea362c72ac083b57638cce/node_modules/once/once.js:25:25)
remote: at ClientRequest.protocol.request.res
(/tmp/build_4cfeda2379ea362c72ac083b57638cce/node_modules/simple-get/index.js:63:5)
remote: at Object.onceWrapper (events.js:277:13)
remote: at ClientRequest.emit (events.js:189:13)
remote: at HTTPParser.parserOnIncomingClient [as
onIncoming] (_http_client.js:556:21)
remote: at HTTPParser.parserOnHeadersComplete
(_http_common.js:109:17)
remote: at TLSSocket.socketOnData (_http_client.js:442:20)
remote: at TLSSocket.emit (events.js:189:13)
remote: at addChunk (_stream_readable.js:284:12)
remote: make: Entering directory
'/tmp/build_4cfeda2379ea362c72ac083b57638cce/node_modules/sharp/build'
remote: TOUCH Release/obj.target/libvips-cpp.stamp
remote: CXX(target) Release/obj.target/sharp/src/common.o
remote: ../src/common.cc:25:10: fatal error: vips/vips8: No
such file or directory
remote: #include
remote: ^~~~~~~~~~~~
remote: compilation terminated.
remote: sharp.target.mk:128: recipe for target
'Release/obj.target/sharp/src/common.o' failed
remote: make: *** [Release/obj.target/sharp/src/common.o] Error
1
remote: make: Leaving directory
'/tmp/build_4cfeda2379ea362c72ac083b57638cce/node_modules/sharp/build'
Update:
Try removing sharp from package.json and add it back again using npm install sharp works this time... I still don't know why.
remote: -----> Installing dependencies
remote: Installing node modules (package.json + package-lock)
remote:
remote: > sharp#0.22.0 install
/tmp/build_5c8d3b01594e0b86f9d3e0d372534414/node_modules/sharp
remote: > (node install/libvips && node install/dll-copy &&
prebuild-install) || (node-gyp rebuild && node install/dll-copy)
remote:
remote: info sharp Downloading
https://github.com/lovell/sharp-libvips/releases/download/v8.7.4/libvips-8.7.4-linux-x64.tar.gz
remote: added 57 packages from 92 contributors and audited
20393 packages in 10.683s
remote: found 62 low severity vulnerabilities
remote: run npm audit fix to fix them, or npm audit for
details
remote:
remote: -----> Build
remote:
remote: -----> Caching build
remote: - node_modules
remote:
remote: -----> Pruning devDependencies
remote: removed 606 packages and audited 497 packages in 8.181s
remote: found 0 vulnerabilities
remote:
remote:
remote: -----> Build succeeded!
There are many people with that problem, it's related to libvips as dependency not sharp by itself.
On github in this issue-comment the problem is explained:
The pre-built binaries of sharp are only guaranteed to work with the pre-built binaries of libvips.
If the download of a pre-built binary of libvips fails then sharp has to fall back to attempting to build itself from source.
http://sharp.pixelplumbing.com/en/stable/install/#pre-compiled-libvips-binaries provides information about serving the libvips binaries from a local URL.
The reason why the download is failing might be the cache, or a corrupted download like reported in the above linked issue on github:
I had to manually remove the ~/.npm/_libvips/libvips-8.7.0-linux-x64.tar.gz and try again. Seems to be downloaded a corrupted file.
Here is an interesting discussion concerning the same problem related to gatsby, a solution seems not yet existing even the issue was closed (before any comment was made): https://github.com/gatsbyjs/gatsby/issues/1754
However if the problem is only related to the download for some reason it's possible to deploy libvips locally like explained in the manual:
Pre-compiled libvips binaries
This module will attempt to download a pre-compiled bundle of libvips
and its dependencies on Linux and Windows machines under either of
these conditions:
If a global installation of libvips that meets the minimum version
requirement cannot be found;
If SHARP_IGNORE_GLOBAL_LIBVIPS environment variable is set.
SHARP_IGNORE_GLOBAL_LIBVIPS=1 npm install sharp
Should you need to manually download and inspect these files,
you can do so via https://github.com/lovell/sharp-libvips/releases
Should you wish to install these from your own location, set the
sharp_dist_base_url npm config option, e.g.
npm config set sharp_dist_base_url "https://hostname/path/"
npm install sharp
or set the SHARP_DIST_BASE_URL environment variable, e.g.
SHARP_DIST_BASE_URL="https://hostname/path/" npm install sharp
to use https://hostname/path/libvips-x.y.z-platform.tar.gz.
So this is a bit background, I'm aware that this can not really be seen as an answer with solution :/
If someone reaches here and didn't found an answer - my issue was that my project was using an old node version (8.x)
The solution was to install an older version of sharp - The latest one that support 8.x - "sharp": "0.23.4"
After updating the node server to version 6.11.1 in response to the recent security update email from Heroku, I cannot git push my local master changes any more. The server and app restarted fine after the update.
I have tried deleted and reinstalled all node_modules, committing the changes and the app runs without errors locally but I get the same build failure when trying to deploy the master. The only change I've made to the package.json file is adding the npm version which was previously unspecified.
From the logs the server build fails when installing node modules:
remote: -----> Building dependencies
remote: Installing node modules (package.json)
remote: npm ERR! Cannot read property '0' of undefined
Is force pushing the best/a good solution?
This is the full logs:
Counting objects: 18, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (18/18), done.
Writing objects: 100% (18/18), 8.80 KiB | 0 bytes/s, done.
Total 18 (delta 13), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Node.js app detected
remote:
remote: -----> Creating runtime environment
remote:
remote: NPM_CONFIG_LOGLEVEL=error
remote: NPM_CONFIG_PRODUCTION=true
remote: NODE_VERBOSE=false
remote: NODE_ENV=production
remote: NODE_MODULES_CACHE=true
remote:
remote: -----> Installing binaries
remote: engines.node (package.json): 6.11.1
remote: engines.npm (package.json): 5.3.0
remote:
remote: Downloading and installing node 6.11.1...
remote: Downloading and installing npm 5.3.0 (replacing version 3.10.10)...
remote:
remote: -----> Restoring cache
remote: Loading 2 from cacheDirectories (default):
remote: - node_modules
remote: - bower_components (not cached - skipping)
remote:
remote: -----> Building dependencies
remote: Installing node modules (package.json)
remote: npm ERR! Cannot read property '0' of undefined
remote:
remote: npm ERR! A complete log of this run can be found in:
remote: npm ERR! /app/.npm/_logs/2017-07-20T22_54_19_003Z-debug.log
remote:
remote: -----> Build failed
remote:
remote: We're sorry this build is failing! You can troubleshoot common issues here:
remote: https://devcenter.heroku.com/articles/troubleshooting-node-deploys
remote:
remote: If you're stuck, please submit a ticket so we can help:
remote: https://help.heroku.com/
remote:
remote: Love,
remote: Heroku
remote:
remote: ! Push rejected, failed to compile Node.js app.
remote:
remote: ! Push failed
remote: Verifying deploy...
remote:
remote: ! Push rejected to app-name.
remote:
To https://git.heroku.com/app-name.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/app-name.git'
Thanks.
I got the same error message after updating to node 6.11.1 and npm 5.3.0. heroku support suggested that I run these commands:
heroku plugins:install heroku-repo
heroku repo:purge_cache -a APPNAME
git commit --allow-empty -m "Purge cache"
git push heroku master
... where "APPNAME" was the heroku identifier for my app.
That resolved the error and I did not have to revert to an earlier version of npm. I've asked for an explanation but they haven't responded yet (just asked a few minutes ago).
UPDATE 2017-07-27: heroku support responded and said, "We're still working on the root cause for this but it seems to be fairly intermittent." Now that I have installed the plugin (see above), I am running these steps to update my app:
heroku repo:purge_cache -a APPNAME
git push heroku master
That's been working reliably for me.