I use npm link myDependency within myPackage so that I can work on both at the same time (vs. publishing myDependency after every change and then updating myPackage to test it.)
I would like to be able to use Travis CI with myPackage but, as one could expect (I actually forgot, but it became quite obvious after attempting to build): running npm install on Travis won't manage to install the linked package.
So, what can I do? I saw someone suggest using a Docker container but that feels like a lot of infrastructure and I'm not experienced with Docker. Another option I thought of was adding a pre-install script to clone the dev branch of myDependency repo into the node_modules folder.
First though, I'm sure I'm not the first person who ever worked on two packages alongside each other, so there must be some consensus on how this should be done.
I solved this by replicating my development set-up on Travis.
The key to solving this involves giving Travis a way to access GitHub. To do this, log in to GitHub and go to the Personal access tokens page.
You’ll see a form with a field for the name of your token and what permissions you want to grant access to. Fill in a name such as "Travis CI Pull Repo" and select the "repo" section. None of the others are necessary, so no need to give access to them. At the bottom of the page is a green button "Generate token."
Back in your code editor, create a .travis.yml file in the root directory of your project if you don’t have one already. We will be using a RubyGem for the next step, so if you don’t have Ruby gems installed, you’ll need to download it. You can check if you have it installed by running gem -v in the terminal.
If you do, run the following in the terminal to install the Travis RubyGem:
gem install travis
Next, in the terminal, make sure you’re working in your project’s root directory, and use the Travis gem to add the access token to your .travis.yml file:
travis encrypt GH_TOKEN="token-from-github-goes-here" --add
If you were successful, your .travis.yml file should have a bunch of random text an encrypted token saved:
env:
global:
secure: "lots-of-seemingly-random-characters"
That’s it! Travis should now be able to pull (and push if that’s what you’re into) to your GitHub repository.
Obviously your .travis.yml file can vary greatly from mine, but at it’s most basic, I set up .travis.yml like this:
language: node_js
node_js:
- '6'
cache:
directories:
- node_modules
install:
- npm install
script:
- npm run lint
- npm run test
env:
global:
secure: "lots-of-seemingly-random-characters"
To add the cloning and linking of the dependency, add a before_install section with the following commands:
before_install:
- git config credential.helper "store --file=.git/credentials"
- echo "https://${GH_TOKEN}:#github.com" > .git/credentials
- cd ..
- git clone https://github.com/my-name/my-dependency.git my-dependency
- cd my-dependency
- npm install
- npm link
- cd ../my-main-project
What is this actually doing?
We configure Git to use our saved access token.
We go one directory up and clone the repository into a new folder with the same name as the repository.
We go into the repository and install its dependencies.
We create a global NPM link.
Finally, we return to the main project (the one we are running Travis on). Note that this name must match the repository’s name on GitHub as that’s the name Travis will use.
In addition, we’ll need to actually use the link created above, so in the install section add the following line:
install:
- npm install
- npm link my-dependency
Make sure you put npm link after npm install as by default npm install will obliterate any links (a very annoying bug for those of us who use npm link).
Related
As a team practice, I would like to force my teammate to use yarn install/ run but not npm install/ run.
Is it possible to force a package.json 's dependency be installed only via yarn install or package.json's script be run only via yarn run?
If it cannot be done, can I at least get a warning when using npm install?
Again, this is only to align the team practice, so that reduces the possibility of error/ problem produced during dev/ops. Thanks
One of the way I can think of is CI can set a rules to detect if there is new file package-lock.json file being created, make the build fail. The developer will then realized he made a mistake since his build failed.
Alternatively you can also rely on husky pre-commit hook, which essentially runs a command to check if package-lock.json existed everytime developer trying to run git commit.
I am building a simple node app and using Bootstrap to style my frontend. I want to deploy the app to Heroku by loading it from the GitHub repository. I don't want to put any compiled CSS/JS files in the repository, which means that they need to be compiled on Heroku after the source is pulled from GitHub.
The problem: Bootstrap's default package.json has its build dependencies in devDependencies, so the dependencies will not be installed on Heroku, which runs npm install in production mode.
Specifically, the problem comes up in my postinstall script, which consists of cd node_modules/bootstrap && npm install && ../.bin/grunt dist. My own npm install command does not end up installing anything because the overarching npm install --production is ignoring Bootstrap's devDependencies.
What is the best workaround to get Bootstrap's dependencies to be installed locally? I'm open to changing my workflow as long as (1) I don't have to put compiled files in my source repository and (2) I can still run the main npm install in production mode.
EDIT: Unless someone can think of a better solution, I think my options at this point are as follows:
Build a custom Github fork of Bootstrap that has its dependencies in dependencies instead of devDependencies
Build a custom Heroku buildpack that somehow leaves Bootstrap's dependencies intact
Abandon principle and compile Bootstrap locally, and put the compiled files in my source
Just use Heroku in development mode (obviously not a good idea)
I think I'm going to try for option 1, and go for option 3 if that fails. I'd still be interested in hearing any other ideas that people come up with.
I would look into using postinstall. The official documentation is on Heroku here.
Since you're doing some build steps at deploy time, you need some devDependencies (bootstrap sources in your case), but heroku will only install in production mode, like you said.
When connected with the heroku toolbelt client, run :
heroku config:set NPM_CONFIG_PRODUCTION=false
You'll have your devDependencies and you'll be able to build.
You only have to do that once (no need to do that each time you deploy).
PS: I don't know about your project, but maybe you need to put the NODE_ENV to production, here is the command:
heroku config:set NODE_ENV=production
You don't need yo put back the NPM_CONFIG_PRODUCTION flag back to true, this is the purpose of making the build on heroku.
But if you prefer, here is the workflow you could use (you said you were opened to change workflow):
ignore dist folder from your source repo (it should already be ignored)
make dist folder a git repo & add the git remote of heroku in repo of the dist folder
cd dist
git init
git remote add origin git#heroku.com:project-name.git
git pull origin master
make sure your build routine doesn't remove this /dist/.git folder - you can use the following glob (if you're using rimraf or something like it with grunt/gulp or whatever) : ['/dist/**/*','/dist/!.git/**/*']
This way, your workflow will be:
build (with your tool)
cd dist
git add .
git commit -m "new version bundled"
git push origin master there, you are in dist folder where origin is the remote for your heroku
You might do the same kind of things with github pages ...
https://github.com/tejas-manohar/itnerary-civic-hacking -- I cloned this git repo down locally. Installed grunt-cli globally, and ran npm run nss (script written by author of original server template -- look in package.json, not complex) and faced numerous errors at the traceur install + git cloning. I've included the trail from terminal window in the pastebin linked below. The npm debug/error log mentioned is not present at the suggested location. npm install alone does not help the situation and returns nothing back. NodeJS v0.10.29 is installed via Node Version Manager (NVM) on OS X 10.9.3.
http://pastebin.com/UJFL3k2E
I'm not the most adept with some of the technologies discussed here. Would someone please attempt to walk me through (a) solution(s) and/or steps to get better clues? All assistance is appreciated.
There's a couple of problems going on which are causing the problems you have seen. The first of which is the command that is run when you run npm run nss, which fails for you with this:
rm: ../../app/static/js/vendor/traceur.js: No such file or directory
This is because, well, the file simply does not exist (yet). If you look at the commands which are run when you run npm run nss, you'll see that the remove file is followed by the copy file of traceur.js to that very location. So I would guess that whoever wrote the script intended for this to be run after it was initially setup, and the remove/copy would be done once the file was initially copied there. Therefore, you must first copy the traceur.js file to app/static/js/vendor before you can successfully execute npm run nss.
But then the next problem, the file should be copied from tools/traceur-compiler/bin doesn't exist either. This is because the traceur-compiler project has recently deleted this (compiled) file from their project, which you can see via this commit: https://github.com/google/traceur-compiler/commit/429c3d850dcb7636320e81fd782c61a06de0fbf1
So you need to regenerate this file, which you can do by (from the itnerary-civic-hacking root directory):
cd tools/traceur-compiler
make bin/traceur.js
cp bin/traceur.js ../../app/static/js/vendor
This will get you in the same state as if you successfully ran npm run nss.
You won't be able to re-run the npm run nss command, but you really shouldn't need to since its more of a setup command than anything (it creates a directory, clones a github repo, etc). If you wanted to update the traceur-compiler in the future (though you may never have to do this) then I would recommend running the following steps (from the itnerary-civic-hacking root directory):
cd tools/traceur-compiler
git pull origin master
npm install
make bin/traceur.js
cp bin/traceur.js ../../app/static/js/vendor
When I push my code to OpenShift, it looks like it's installing my devDependencies which takes forever. I would really love to set it up so it will only install the dependencies (by running with the --production flag). Is there any way to do this?
You can tell npm to install using the --production flag by setting the NPM_CONFIG_PRODUCTION environment variable to "true".
Here is an example that should work for existing applications:
rhc env set NPM_CONFIG_PRODUCTION="true"
Or, you can set this variable as a part of your initial app-create step:
rhc app create myapplication nodejs-0.10 NPM_CONFIG_PRODUCTION="true"
Found a way to specify it in source instead of during app creation.
The benefit (for me) over an env var is that it applies to all ways to launch the app, including a "Launch on OpenShift" button.
Create an .openshift/action_hooks/pre_build file:
#!/bin/bash
# This makes npm not install devDependencies.
echo 'Enabling npm production'
echo 'production = true' >> $OPENSHIFT_REPO_DIR/.npmrc
That's it! I've tested and it does affect npm for this build, and the .npmrc disappears if you remove this hook in the future.
(Obviously I could also achieve this by simply adding an .npmrc to my repo, but do not want to affect people checking out the source and running npm install, only how it works on OpenShift.)
It looks like the only solution is to update the cartridge itself. The npm install command is located in the cartridge's bin/control folder. Meanwhile, it's been fixed in the originating github repo at wshearn/openshift-origin-cartridge-nodejs so you can just install from github rather than using the Quickstart.
Create a .npmrc file where the node_modules folder is located.
Open it with your text-editor and add this to it:
production = true
P.S. no semicolons or any other characters
This will ensure that devDependencies are not installed on the OPENSHIFT server
I'm using grunt and also grunt plugins like grunt-contrib-copy, grunt-contrib-mincss (that listed as npm dependencies for my application).
Also I don't commit npm_modules folder and public folder, where all generated files are. And I can't figure out how to build my app (I have grunt build command) after deploy and setup my server (it's already looking for public folder).
I saw some stuff like grunt-heroku-deploy, but it seems me a bad idea to commit before upload. Maybe there are some gentle decisions... Any thoughts?
npm has a support for a postinstall step (among many others) that might be just what you're looking for.
The node.js heroku buildpack runs this command when you push to heroku to resolve build dependencies:
$ npm install --production
https://devcenter.heroku.com/articles/nodejs-support#build-behavior
If you take a look at the npm documentation, you can setup a series of scripts to run either before or after anyone runs npm install for your package. It's configured in the scripts property of package.json. The scripts property allows to run custom scripts (including grunt) when certain things happen in a package's lifecycle.
For example, to echo some text and run the grunt command whenever anyone (including Heroku) runs npm install, add this to your package.json:
{
...
"scripts": {
"postinstall": "echo postinstall time; ./node_modules/grunt-cli/bin/grunt <your task name>"
},
...
}
https://npmjs.org/doc/scripts.html
Important caveats:
You might have to change the path to the grunt binary in the postinstall script, check the error output if the grunt command doesn't execute.
grunt and grunt-cli must be listed as a dependency in your package.json so it gets installed by Heroku. Listing them under devDependencies is not sufficient since Heroku won't install those. Also, note that Heroku won't install it as a global package so to execute it on Heroku you're going to have to use a relative path (as it is configured above).
If this doesn't work (you'll probably need to fiddle with the relative paths a bit), then you might want to consider writing your own custom buildpack for Heroku.
Update
As of 0.4, the grunt package no longer contains the grunt binary, which is now part of the grunt-cli package. The answer has been updated to reflect this.
This looks like it will largely be solved when the Heroku Platorm API slug and release features make it into the mainline. At that point, you can build your code locally (or on a ci server), package it up and send it to heroku via an API call and release it from there.
This is still in the beta period and was only announced on December 19, 2013.
https://devcenter.heroku.com/articles/platform-api-deploying-slugs
I was never super happy with how many people seemed ok with checking in your generated code into git or the NPM postinstall hook. :(
Plus from a philosophical stance, doing a build during a release is simply another potential failure point.
Just for fun: Since that's not finalized yet, here's a bash script I threw together you can use for the time being to build your code on a deployment branch, commit it, deploy it to heroku and then remove the deployment branch. (I really am not a fan of bash deployment scripts, so I'm really looking forward to the platform API additions)
#!/bin/bash
set -e
# Delete current deploy branch
git branch -D deploy
# Create new deploy branch based on master
git checkout -b deploy
# Grunt comands to build our site
grunt build:production
# the dist/ directory is in my .gitignore, so forcibly add it
git add -f dist/
git commit -m "Deploying to Heroku"
# Push it up to heroku, the -f ensures that heroku won't complain
git push heroku -f deploy:master
# Switch it back to master
git checkout master
Grunt (et al.) is a build tool, not (really) something you should be packaging up and running on production. A different approach would be to use Grunt to prepare your project locally (or better on a CI server) before only pushing the built files to Heroku. As already mentioned Heroku will do an npm install on your app after its pushed which should be enough on its own to finally prepare your app.
I have it set up so that the Grunt derived/built Heroku app lives in a totally separate Git repo to my main app source code repo. So that when I do a grunt deploy it optimises and copies the relevant files to the Heroku repo, tidies it up (git add -A etc.) and then git push heroku master (or whatever).
It seems like a cleaner separation of concerns if your live servers are only responsible for running a pre-built app package.
YMMV of course, and the accepted answer above is totally valid too ... especially on a well understood and stable live environment like Heroku.
Heroku buildpack works fine for me. Great stuff.
To get this working with grunt 4.0 I followed the instructions here https://discussion.heroku.com/t/grunt-on-heroku/98/2 . The only change I had to make was to remove the path to grunt as using unix style slashes would make it fail in windows and vice versa. Luckily you don't even need to specify the path as NPM will look for grunt in the node_modules/.bin folder https://npmjs.org/doc/scripts.html#path.
make sure you have both grunt and grunt-cli installed locally in your package.json even if grunt tells you to install the cli globally: $: npm i -S grunt grunt-cli
add a postinstall step to your package.json that looks like this: "postinstall": "grunt prod"
The npm postinstall step is probably your best option, since you can invoke grunt from there. But you should also check out a custom buildpack, such as heroku-buildpack-nodejs-grunt.
This post is Rails-specific but I don't see why you couldn't use it with any back-end framework and just swap the Ruby buildpack with whatever you're using.
The solution is basically to use multi buildpacks, and have the Node/Grunt buildpack run grunt build for you right on Heroku.
Significantly, this solution does not have you check build artifacts into version control. (Yay!!!)
http://www.angularonrails.com/deploy-angular-rails-single-page-application-heroku/