This question already has answers here:
Avoid npm refresh after every deployment on Heroku
(4 answers)
Closed 8 years ago.
Heroku is great. But every time I deploy, Heroku seems to like to redownload and rebuild all the packages. With socket.io and mailparser this is taking around 3 minutes.
Is there a way to speed up the deployment process? Is there a way to tell Heroku that it can cache these items? Or can I upload prebuilt node_modules?
It seems like as of today Heroku is finally caching the node_modules folder!
-----> Deleting 6 files matching .slugignore patterns.
-----> Node.js app detected
-----> Requested node range: 0.10.x
-----> Resolved node version: 0.10.22
-----> Downloading and installing node
-----> Restoring node_modules from cache
-----> Installing dependencies
-----> Pruning unused dependencies
-----> Caching node_modules directory for future builds
-----> Cleaning up node-gyp and npm artifacts
Build time is like 3 seconds for me now.
One thing I did to speed up process was to add .slugignore file to the main folder and add all the files and folders I did not want to run the app.
Sample content of .slugignore file:
working
mockups
*.psd
*.pdf
I'm running into the same problem.
Some discussion here about caching the node_modules folder: https://github.com/heroku/heroku-buildpack-nodejs/pull/37
Another idea: https://github.com/heroku/heroku-buildpack-nodejs/issues/25
I'm thinking about a few solutions right now.
Check in node_modules in a separate branch: The core Node.js maintainers actually recommend checking in the node_modules folder into source control (for apps, not libs). I don't like this. A way to get around it though might be to have a separate production branch with a different .gitignore file that doesn't ignore node_modules. When you want to deploy, just do a rebase from your master and node_modules will be checked in. At least this keeps your master branch free from dependencies.
Add a preinstall script to package.json to download compressed dependency zip: You could also add a pre-push git hook to bundle up your dependencies and upload them to S3. This would probably be too slow though.
Modify the heroku-buildpack-nodejs: Integrate the outstanding pull request with node_modules caching:
heroku config:set BUILDPACK_URL=https://github.com/opdemand/buildpack-nodejs.git
I had the same question (see Avoid npm refresh after every deployment on Heroku).
Heroku forces a download/build/etc. sequence because they need to start an app with a 'blank slate': to clean previous undeleted files, when they move your app to another server, when you assign new web dynos, etc.
The issue is clearly with native packages, and recompilation. For all js-only packages, I commit them with my project, and remove them from package.json. It gains a few seconds, but not that much.
I should definitely be possible to pre-compile and commit native modules (I successfully run wkhtml2pdf on Heroku, for instance, with a binary compiled for linux-amd64), if you get access to a Linux box (or VM) with the same configuration - as of today, Linux [...] 2.6.32-350-ec2 #57-Ubuntu SMP [...] x86_64 GNU/Linux.
Though I would not recommend it as a definitive solution, since it is likely to break some day - It does not seem to me that heroku guarantees the platform an app runs onto.
Seems like there has recently been progress at the heroku-buildpack-nodejs.
Once the pull request is merged, you can add
heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs
to your heroku environment variables.
For now, David Dollar's forked repository is available at
https://github.com/ddollar/heroku-buildpack-nodejs
With this as your BUILDPACK_URL it should cache the npm modules.
I tried it with node.js 0.10.5a, npm version: 1.3.5 and npm_modules in .gitignore. Tt seems to work fine so far!
Check out this branch of the new Heroku Node.js buildpack, now in beta, which supports node_modules caching between builds:
https://github.com/heroku/heroku-buildpack-nodejs/tree/diet
To use it:
heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs#diet -a my-node-app
git commit -am "fakeout" --allow-empty
git push heroku
Related
I am working on client side application.
We are using the following technologies:
git, node, ember, grunt, sass and other components
Once I cloned the application from git server every time I have to do make tooling to download all the necessary node, sass and bower components and it will take 200MB of data will be downloaded and time consumed.
Is there any solution with out downloading the node modules the app has to run by reusing the Already downloaded modules with out make tooling.
Yes, you can include the node modules in your git repository as well. You'll still have to download them (there's no way around fetching the modules somehow). I'm guessing in your .gitignore you have a line that looks like node_modules. If you remove that line, the modules will be included in your git repository and will be included when you do a git clone.
Be aware that there are a couple of drawbacks to this method:
Git repo size will increase significantly
Modules that have to be compiled likely won't work on other machines, especially different OS's (developing on a Mac, deploying on Linux, for example).
I've got a Node app that I'm deploying to Heroku. Their docs say it's best practice to check in your node_modules directory (i.e. don't gitignore it), for faster deploys and for dev/prod parity.
In my package.json, I've got loads of devDependencies (mostly Grunt plugins and all their deps) and a few regular production dependencies like Express. Heroku only needs the production deps. I'd rather not check in all my dev deps, because they come to about 50MB.
Is there some way to have a separate folder for you dev deps, e.g. node_modules_dev? If this was possible, then I could just add node_modules_dev to my .gitignore, and check in the regular production node_modules directory as per Heroku's advice.
Is there any way to do this? Or can you think of another way to do what I'm trying to do?
I use a CI server to build, test, and deploy my files — so I was looking for a similar solution that would prevent me from needing to deploy extra dependencies and/or re-build on Heroku.
After all my tests run, I run npm prune --production, which removes devDependencies from node_modules, and then I push the result to Heroku.
No extra files go to the server, and deployment times are much faster since Heroku avoids having to build all the binaries usually found in Gulp/Grunt plugins.
If you don't mind checking them in anyways, and your only concern is the resulting slug size (i.e.: not your git repo size, or transfer of that repo to Heroku), then simply add the relevant node_modules to .slugignore.
Docs: Ignoring files with .slugignore.
I have a Node.js app which I'm attempting to deploy to Heroku. However, Heroku seems to think that it's a Ruby app, likely because it has a Gemfile (we have a private gem containing some custom scripts and the like, as well as the Heroku gem itself for command-line control). I added Gemfile and Gemfile.lock to the .slugignore (discussed in Heroku's Slug Compiler article), but the app is still detected as a Ruby app.
$ git push heroku master
Total 0 (delta 0), reused 0 (delta 0)
-----> Heroku receiving push
-----> Ruby app detected
-----> Installing dependencies using Bundler version 1.2.0.pre
Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
The bundle fails, because our custom gem is a private repo, and I never get any further. I'm unable to find any way to force Heroku to treat the app as a Node.js app and use the Procfile to start the web app. Does anyone have and idea how I can do this?
Well crap, I must not have looked far enough. This was already asked on Stack Overflow and the answer was to use the custom Node.js build pack, a la:
heroku config:add BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs.git
I am managing a dependency at Github which I use as a dependency in my project. I placed the dependecy as a tarball link (viz. https://github.com/username/dependecy/tarball/master) into the package.json and it is working fine as expected locally. When I update the package at Github I can run a npm install and all dependencies including the Github tarballs get updated. However it is not the case at Heroku. tarball-linked dependencies does not get updated. Any ideas?
I had the similar problem. My App had the dependency with caret versioning like this:
"dependency": "^0.6",
So every time the dependency got its patch version updated, I wanted Heroku to have the updated dependency without any commits/pushes to my App. For that I set just in case
heroku config:set NODE_MODULES_CACHE=false
And when the new patch version became available I did the manual redeploy of the same App that was already deployed from Heroku Dashboard.
Can you try to do the same for your case? Possibly this will help you.
Since no one has answered this yet I will share what I have learned. The trick is getting heroku to think the tarball is different or new so that it downloads it again. As #celalo suggested you can remove it or change the path, commit, push, change it back, commit and push. This is messy but it works.
What I ended up doing was making a master1 branch. I keep the branch in sync with master and then alternate the tarball url between master and master1 when I need it to update.
Is it possible to switch a Heroku app on the Cedar stack from one language to another?
In this particular instance I am trying to migrate an app from PHP to NodeJS, which is being detected as a NodeJS app after performing a git push:
-----> Heroku receiving push
-----> Node.js app detected
-----> Fetching Node.js binaries
-----> Vendoring node 0.4.7
-----> Installing dependencies with npm 1.0.94
Dependencies installed
-----> Discovering process types
Procfile declares types -> web
-----> Compiled slug size is 5.0MB
-----> Launching... done, v7
... however it then crashes with:
Error: No such file or directory - node main.js
Trying to run the Node REPL also suggests the binary doesn't exist:
> heroku run node
Running node attached to terminal... up, run.1
sh: node: not found
Is there any way to reinitialise a Cedar stack app, without creating a new instance?
Create a new instance application instance. It is probably not a good idea to try to morph one into the other. DNS will update for you automatically as that's handled by the routing mesh, and you'll be much happier and better off for it.
I (although a slightly different use case) migrated an application from Bamboo to Cedar and the whole process took me less than five minutes. The only downtime was a couple of seconds while I relocated the custom domain setup on the application.
I know it's an old question but I ran into this problem changing my app from PHP to NodeJs. The first time I run the git push heroku master I got an error saying that it couldn't use PHP buildpack.
So I went in my app Settings tab in Heroku dashboard and remove the PHP buildpack from the Buildpacks list.
Finally whe I ran git push heroku master, Heroku auto detected the NodeJS buildpack.