I am managing a dependency at Github which I use as a dependency in my project. I placed the dependecy as a tarball link (viz. https://github.com/username/dependecy/tarball/master) into the package.json and it is working fine as expected locally. When I update the package at Github I can run a npm install and all dependencies including the Github tarballs get updated. However it is not the case at Heroku. tarball-linked dependencies does not get updated. Any ideas?
I had the similar problem. My App had the dependency with caret versioning like this:
"dependency": "^0.6",
So every time the dependency got its patch version updated, I wanted Heroku to have the updated dependency without any commits/pushes to my App. For that I set just in case
heroku config:set NODE_MODULES_CACHE=false
And when the new patch version became available I did the manual redeploy of the same App that was already deployed from Heroku Dashboard.
Can you try to do the same for your case? Possibly this will help you.
Since no one has answered this yet I will share what I have learned. The trick is getting heroku to think the tarball is different or new so that it downloads it again. As #celalo suggested you can remove it or change the path, commit, push, change it back, commit and push. This is messy but it works.
What I ended up doing was making a master1 branch. I keep the branch in sync with master and then alternate the tarball url between master and master1 when I need it to update.
Related
Pardon the title, it's terrible.
Situation:
I pull a github repo that represents a website. I build it locally for development. It's running on localhost.
It uses webpack, and has a series of node modules loaded.
I need to edit one of those modules and see the results on that website.
I then need to push those changes to the repo that builds to the npm module.
Ideally I do the edits on the npm module as a local pull of that repo, and the local website is referencing those changes and webpack watch is rebuilding as I go along, then when all good, I can do a pull request to that npm module.
What do I use to do this? Is their a common workflow or term for this so I can figure out how?
Thank you.
Note: I attempted to use npm link,
I.e. in the repo "custom-npm" I typed "npm link"
Then in the website repo, in the node_modules/custom-npm folder I typed "npm link custom-npm"
error is: "custom-npm" is not in this registry
"custom-npm" is a github private package. Not sure if that does something here unintended...
We have a node web project hosted off of Heroku that contains a library hosted on GitHub. We often make changes to the GitHub library.
When we push the updated node web project to Heroku for deployment, it doesn't check if the library on GitHub has been updated.
We're wondering if there is a command that clears the cache/node_modules folder on Heroku to resolve this?
Cheers,
Peter
I found the solution after searching for a while. You have to disable the cache via config and also purge the existing one for old builds. Also I recommend deleting the package-lock.json to clear any issues.
https://devcenter.heroku.com/articles/nodejs-support#cache-behavior
https://help.heroku.com/18PI5RSY/how-do-i-clear-the-build-cache
I have a custom npm module that I am working on, and it has a GitHub repo. I'm also working on a project that uses the custom module. When working on the larger project, it is nice to use npm link so I can make changes to the module and see them right away in the main project.
To deploy to staging or production, I use shrinkwrap and shrinkpack so I can do an npm install after every deploy (some of the dependencies need binaries, and dev systems aren't the same as production systems, so they do need to be installed and not just kept in source control). Edit: I'm crossing this out as the answer below technically solves my issue, even though it doesn't solve for this particular point, but that wasn't as important as the rest of it.
Of course, since the module is linked to my main project and not listed in package.json, a deploy and install misses it entirely. I can go ahead and list it in package.json and have it point to the appropriate GitHub repo, but then every time I need to test a change in the main project I would have to commit and push those changes, then update the main project, kill and restart the app...that would get tiresome pretty quickly.
I guess I need something like the opposite of "devDependencies"; something where I can have it not install the module on dev, but do install it from GitHub when doing npm install on staging or production. Other than remembering to manually change package.json every time I need to go back and forth, is there a better way to do this?
you can specify a github repository as your package to install, in your package.json file:
{
dependencies: {
"my-library": "githubusername/my-library"
}
}
this will work in your production environment.
in your development environment, use "npm link".
from within the "my-library" folder, run npm link directly. that will tell npm on your local box that "my-library" is avaialable as a link.
now, in your project that uses "my-library", run npm link my-library. this will create a symlink to your local development version of "my-library", allowing you to change code in that repository and have it work in your other project that needs it.
once you are ready to push to production, push "my-library" to your github repository, and then you can npm install on your servers, like normal.
This question already has answers here:
Avoid npm refresh after every deployment on Heroku
(4 answers)
Closed 8 years ago.
Heroku is great. But every time I deploy, Heroku seems to like to redownload and rebuild all the packages. With socket.io and mailparser this is taking around 3 minutes.
Is there a way to speed up the deployment process? Is there a way to tell Heroku that it can cache these items? Or can I upload prebuilt node_modules?
It seems like as of today Heroku is finally caching the node_modules folder!
-----> Deleting 6 files matching .slugignore patterns.
-----> Node.js app detected
-----> Requested node range: 0.10.x
-----> Resolved node version: 0.10.22
-----> Downloading and installing node
-----> Restoring node_modules from cache
-----> Installing dependencies
-----> Pruning unused dependencies
-----> Caching node_modules directory for future builds
-----> Cleaning up node-gyp and npm artifacts
Build time is like 3 seconds for me now.
One thing I did to speed up process was to add .slugignore file to the main folder and add all the files and folders I did not want to run the app.
Sample content of .slugignore file:
working
mockups
*.psd
*.pdf
I'm running into the same problem.
Some discussion here about caching the node_modules folder: https://github.com/heroku/heroku-buildpack-nodejs/pull/37
Another idea: https://github.com/heroku/heroku-buildpack-nodejs/issues/25
I'm thinking about a few solutions right now.
Check in node_modules in a separate branch: The core Node.js maintainers actually recommend checking in the node_modules folder into source control (for apps, not libs). I don't like this. A way to get around it though might be to have a separate production branch with a different .gitignore file that doesn't ignore node_modules. When you want to deploy, just do a rebase from your master and node_modules will be checked in. At least this keeps your master branch free from dependencies.
Add a preinstall script to package.json to download compressed dependency zip: You could also add a pre-push git hook to bundle up your dependencies and upload them to S3. This would probably be too slow though.
Modify the heroku-buildpack-nodejs: Integrate the outstanding pull request with node_modules caching:
heroku config:set BUILDPACK_URL=https://github.com/opdemand/buildpack-nodejs.git
I had the same question (see Avoid npm refresh after every deployment on Heroku).
Heroku forces a download/build/etc. sequence because they need to start an app with a 'blank slate': to clean previous undeleted files, when they move your app to another server, when you assign new web dynos, etc.
The issue is clearly with native packages, and recompilation. For all js-only packages, I commit them with my project, and remove them from package.json. It gains a few seconds, but not that much.
I should definitely be possible to pre-compile and commit native modules (I successfully run wkhtml2pdf on Heroku, for instance, with a binary compiled for linux-amd64), if you get access to a Linux box (or VM) with the same configuration - as of today, Linux [...] 2.6.32-350-ec2 #57-Ubuntu SMP [...] x86_64 GNU/Linux.
Though I would not recommend it as a definitive solution, since it is likely to break some day - It does not seem to me that heroku guarantees the platform an app runs onto.
Seems like there has recently been progress at the heroku-buildpack-nodejs.
Once the pull request is merged, you can add
heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs
to your heroku environment variables.
For now, David Dollar's forked repository is available at
https://github.com/ddollar/heroku-buildpack-nodejs
With this as your BUILDPACK_URL it should cache the npm modules.
I tried it with node.js 0.10.5a, npm version: 1.3.5 and npm_modules in .gitignore. Tt seems to work fine so far!
Check out this branch of the new Heroku Node.js buildpack, now in beta, which supports node_modules caching between builds:
https://github.com/heroku/heroku-buildpack-nodejs/tree/diet
To use it:
heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs#diet -a my-node-app
git commit -am "fakeout" --allow-empty
git push heroku
I'm using npm link as described here
http://npmjs.org/doc/link.html
Locally everything works perfectly. When I deploy to Heroku I get the error message
Error: Cannot find module '...'
How can I get this working with Heroku?
I wish there were an elegant solution to this (it would make my life a hell of a lot easier). Your custom package is symlinked into node_modules by npm link, but git doesn't follow symbolic links nowadays. So when you git push to Heroku, there's no way to make your custom packages go along for the ride.
Note, however, that from my experiments, Heroku will honor any node_modules you do push in, instead of trying to install them from the network. It just runs npm install --production, essentially. Perhaps a hard link directly to the development source of your package would do the trick, but I'm not sure whether Git would play nicely with that. Use at your own risk!
EDIT: If you want to know exactly what Heroku does, it's all open source.
The ideal situation would be to get the packages, if they're open source, onto NPM itself. Which is pretty painless and automatic.
If you are hosting your private module on GitHub (or BitBucket), you can add the git repo as a dependency in your package.json.
"dependencies": {
// ... all your deps
"my_private_module": "git+ssh://git#github.com:my-username/my-private-module.git"
}
You will, however, need to grant privileges to Heroku to read your repo (assuming it's private -- which is the crux of the issue). Check out this answer for a detailed set of instructions showing how to do so with Github. This might help for Bitbucket.
I've found that the build time increases when doing this. Worth it for my needs.