How to define a gulp task that initiates watchify and gulp-stylus? - node.js

I am trying to create a composite gulp task which runs 2 separate watch tasks at the same time:
gulp.task('watch-css', watchCssTask); // uses gulp-stylus
gulp.task('watch-js', watchJsTask); // uses watchify (browserify)
What is the correct way to achieve the following with gulp?
gulp.task('watch', watchCssAndJsTask);
For instance, with vanilla npm tasks I would just do this:
"scripts": {
"watch-css": ...
"watch-js": ...
"watch": "npm run watch-css && npm run watch-js",
}


The following seems to work:
gulp.task('watch', [ 'watch-css', 'watch-js' ]);

Related

Run two tasks/scripts that both "hog the terminal" [duplicate]

In my package.json I have these two scripts:
"scripts": {
"start-watch": "nodemon run-babel index.js",
"wp-server": "webpack-dev-server",
}
I have to run these 2 scripts in parallel everytime I start developing in Node.js. The first thing I thought of was adding a third script like this:
"dev": "npm run start-watch && npm run wp-server"
... but that will wait for start-watch to finish before running wp-server.
How can I run these in parallel? Please keep in mind that I need to see the output of these commands. Also, if your solution involves a build tool, I'd rather use gulp instead of grunt because I already use it in another project.

Use a package called concurrently.
npm i concurrently --save-dev
Then setup your npm run dev task as so:
"dev": "concurrently --kill-others \"npm run start-watch\" \"npm run wp-server\""

If you're using an UNIX-like environment, just use & as the separator:
"dev": "npm run start-watch & npm run wp-server"
Otherwise if you're interested on a cross-platform solution, you could use npm-run-all module:
"dev": "npm-run-all --parallel start-watch wp-server"

From windows cmd you can use start:
"dev": "start npm run start-watch && start npm run wp-server"
Every command launched this way starts in its own window.

You should use npm-run-all (or concurrently, parallelshell), because it has more control over starting and killing commands. The operators &, | are bad ideas because you'll need to manually stop it after all tests are finished.
This is an example for protractor testing through npm:
scripts: {
"webdriver-start": "./node_modules/protractor/bin/webdriver-manager update && ./node_modules/protractor/bin/webdriver-manager start",
"protractor": "./node_modules/protractor/bin/protractor ./tests/protractor.conf.js",
"http-server": "./node_modules/http-server/bin/http-server -a localhost -p 8000",
"test": "npm-run-all -p -r webdriver-start http-server protractor"
}
-p = Run commands in parallel.
-r = Kill all commands when one of them finishes with an exit code of zero.
Running npm run test will start Selenium driver, start http server (to serve you files) and run protractor tests. Once all tests are finished, it will close the http server and the selenium driver.

I've checked almost all solutions from above and only with npm-run-all I was able to solve all problems. Main advantage over all other solution is an ability to run script with arguments.
{
"test:static-server": "cross-env NODE_ENV=test node server/testsServer.js",
"test:jest": "cross-env NODE_ENV=test jest",
"test": "run-p test:static-server \"test:jest -- {*}\" --",
"test:coverage": "npm run test -- --coverage",
"test:watch": "npm run test -- --watchAll",
}
Note run-p is shortcut for npm-run-all --parallel
This allows me to run command with arguments like npm run test:watch -- Something.
EDIT:
There is one more useful option for npm-run-all:
-r, --race - - - - - - - Set the flag to kill all tasks when a task
finished with zero. This option is valid only
with 'parallel' option.
Add -r to your npm-run-all script to kill all processes when one finished with code 0. This is especially useful when you run a HTTP server and another script that use the server.
"test": "run-p -r test:static-server \"test:jest -- {*}\" --",

I have a crossplatform solution without any additional modules. I was looking for something like a try catch block I could use both in the cmd.exe and in the bash.
The solution is command1 || command2 which seems to work in both enviroments same. So the solution for the OP is:
"scripts": {
"start-watch": "nodemon run-babel index.js",
"wp-server": "webpack-dev-server",
// first command is for the cmd.exe, second one is for the bash
"dev": "(start npm run start-watch && start npm run wp-server) || (npm run start-watch & npm run wp-server)",
"start": "npm run dev"
}
Then simple npm start (and npm run dev) will work on all platforms!

If you replace the double ampersand with a single ampersand, the scripts will run concurrently.

How about forking
Another option to run multiple Node scripts is with a single Node script, which can fork many others. Forking is supported natively in Node, so it adds no dependencies and is cross-platform.
Minimal example
This would just run the scripts as-is and assume they're located in the parent script's directory.
// fork-minimal.js - run with: node fork-minimal.js
const childProcess = require('child_process');
let scripts = ['some-script.js', 'some-other-script.js'];
scripts.forEach(script => childProcess.fork(script));
Verbose example
This would run the scripts with arguments and configured by the many available options.
// fork-verbose.js - run with: node fork-verbose.js
const childProcess = require('child_process');
let scripts = [
{
path: 'some-script.js',
args: ['-some_arg', '/some_other_arg'],
options: {cwd: './', env: {NODE_ENV: 'development'}}
},
{
path: 'some-other-script.js',
args: ['-another_arg', '/yet_other_arg'],
options: {cwd: '/some/where/else', env: {NODE_ENV: 'development'}}
}
];
let runningScripts= [];
scripts.forEach(script => {
let runningScript = childProcess.fork(script.path, script.args, script.options);
// Optionally attach event listeners to the script
runningScript.on('close', () => console.log('Time to die...'))
runningScripts.push(runningScript); // Keep a reference to the script for later use
});
Communicating with forked scripts
Forking also has the added benefit that the parent script can receive events from the forked child processes as well as send back. A common example is for the parent script to kill its forked children.
runningScripts.forEach(runningScript => runningScript.kill());
For more available events and methods see the ChildProcess documentation

npm-run-all --parallel task1 task2
edit:
You need to have npm-run-all installed beforehand. Also check this page for other usage scenarios.

Quick Solution
In this case, I'd say the best bet If this script is for a private module intended to run only on *nix-based machines, you can use the control operator for forking processes, which looks like this: &
An example of doing this in a partial package.json file:
{
"name": "npm-scripts-forking-example",
"scripts": {
"bundle": "watchify -vd -p browserify-hmr index.js -o bundle.js",
"serve": "http-server -c 1 -a localhost",
"serve-bundle": "npm run bundle & npm run serve &"
}
You'd then execute them both in parallel via npm run serve-bundle. You can enhance the scripts to output the pids of the forked process to a file like so:
"serve-bundle": "npm run bundle & echo \"$!\" > build/bundle.pid && npm run serve & echo \"$!\" > build/serve.pid && npm run open-browser",
Google something like bash control operator for forking to learn more on how it works. I've also provided some further context regarding leveraging Unix techniques in Node projects below:
Further Context RE: Unix Tools & Node.js
If you're not on Windows, Unix tools/techniques often work well to achieve something with Node scripts because:
Much of Node.js lovingly imitates Unix principles
You're on *nix (incl. OS X) and NPM is using a shell anyway
Modules for system tasks in Nodeland are also often abstractions or approximations of Unix tools, from fs to streams.

step by step guide to run multiple parallel scripts with npm.
install npm-run-all package globally
npm i -g npm-run-all
Now install and save this package within project where your package.json exists
npm i npm-run-all --save-dev
Now modify scripts in package.json file this way
"scripts": {
"server": "live-server index.html",
"watch": "node-sass scss/style.scss --watch",
"all": "npm-run-all --parallel server watch"
},
now run this command
npm run all
more detail about this package in given link
npm-run-all

with installing npm install concurrently
"scripts": {
"start:build": "tsc -w",
"start:run": "nodemon build/index.js",
"start": "concurrently npm:start:*"
},

Use concurrently to run the commands in parallel with a shared output stream. To make it easy to tell which output is from which process, use the shortened command form, such as npm:wp-server. This causes concurrently to prefix each output line with its command name.
In package.json, your scripts section will look like this:
"scripts": {
"start": "concurrently \"npm:start-watch\" \"npm:wp-server\"",
"start-watch": "nodemon run-babel index.js",
"wp-server": "webpack-dev-server"
}

npm install npm-run-all --save-dev
package.json:
"scripts": {
"start-watch": "...",
"wp-server": "...",
"dev": "npm-run-all --parallel start-watch wp-server"
}
More info: https://github.com/mysticatea/npm-run-all/blob/master/docs/npm-run-all.md

In a package.json in the parent folder:
"dev": "(cd api && start npm run start) & (cd ../client && start npm run start)"
this work in windows

Just add this npm script to the package.json file in the root folder.
{
...
"scripts": {
...
"start": "react-scripts start", // or whatever else depends on your project
"dev": "(cd server && npm run start) & (cd ../client && npm run start)"
}
}

... but that will wait for start-watch to finish before running wp-server.
For that to work, you will have to use start on your command. Others have already illustrated but this is how it will work, your code below:
"dev": "npm run start-watch && npm run wp-server"
Should be :
"dev": " start npm run start-watch && start npm run wp-server"
What this will do is, it will open a separate instance for each command and process them concurrently, which shouldn't be an issue as far as your initial issue is concerned. Why do I say so? It's because these instances both open automatically while you run only 1 statement, which is your initial goal.

I ran into problems with & and |, which exit statuses and error throwing, respectively.
Other solutions want to run any task with a given name, like npm-run-all, which wasn't my use case.
So I created npm-run-parallel that runs npm scripts asynchronously and reports back when they're done.
So, for your scripts, it'd be:
npm-run-parallel wp-server start-watch

My solution is similar to Piittis', though I had some problems using Windows. So I had to validate for win32.
const { spawn } = require("child_process");
function logData(data) {
console.info(`stdout: ${data}`);
}
function runProcess(target) {
let command = "npm";
if (process.platform === "win32") {
command = "npm.cmd"; // I shit you not
}
const myProcess = spawn(command, ["run", target]); // npm run server
myProcess.stdout.on("data", logData);
myProcess.stderr.on("data", logData);
}
(() => {
runProcess("server"); // package json script
runProcess("client");
})();

This worked for me
{
"start-express": "tsc && nodemon dist/server/server.js",
"start-react": "react-scripts start",
"start-both": "npm -p -r run start-react && -p -r npm run start-express"
}
Both client and server are written in typescript.
The React app is created with create-react-app with the typescript template and is in the default src directory.
Express is in the server directory and the entry file is server.js
typescript code and transpiled into js and is put in the dist directory .
checkout my project for more info: https://github.com/nickjohngray/staticbackeditor
UPDATE:
calling npm run dev, to start things off
{"server": "tsc-watch --onSuccess \"node ./dist/server/index.js\"",
"start-server-dev": "npm run build-server-dev && node src/server/index.js",
"client": "webpack-dev-server --mode development --devtool inline-source-map --hot",
"dev": "concurrently \"npm run build-server-dev\" \"npm run server\" \"npm run client\""}

You can also use pre and post as prefixes on your specific script.
"scripts": {
"predev": "nodemon run-babel index.js &",
"dev": "webpack-dev-server"
}
And then run:
npm run dev

In my case I have two projects, one was UI and the other was API, and both have their own script in their respective package.json files.
So, here is what I did.
npm run --prefix react start& npm run --prefix express start&

Simple node script to get you going without too much hassle. Using readline to combine outputs so the lines don't get mangled.
const { spawn } = require('child_process');
const readline = require('readline');
[
spawn('npm', ['run', 'start-watch']),
spawn('npm', ['run', 'wp-server'])
].forEach(child => {
readline.createInterface({
input: child.stdout
}).on('line', console.log);
readline.createInterface({
input: child.stderr,
}).on('line', console.log);
});

I have been using npm-run-all for some time, but I never got along with it, because the output of the command in watch mode doesn't work well together. For example, if I start create-react-app and jest in watch mode, I will only be able to see the output from the last command I ran. So most of the time, I was running all my commands manually...
This is why, I implement my own lib, run-screen. It still very young project (from yesterday :p ) but it might be worth to look at it, in your case it would be:
run-screen "npm run start-watch" "npm run wp-server"
Then you press the numeric key 1 to see the output of wp-server and press 0 to see the output of start-watch.

A simple and native way for Windows CMD
"start /b npm run bg-task1 && start /b npm run bg-task2 && npm run main-task"
(start /b means start in the background)

I think the best way is to use npm-run-all as below:
1- npm install -g npm-run-all <--- will be installed globally
2- npm-run-all --parallel server client

How about a good old fashioned Makefile?
This allows you a lot of control including how you manage subshells, dependencies between scripts etc.
# run both scripts
start: server client
# start server and use & to background it
server:
npm run serve &
# start the client
client:
npm start
call this Makefile and then you can just type
make start to start everything up. Because the server command is actually running in a child process of the start command when you ctrl-C the server command will also stop - unlike if you just backgrounded it yourself at the shell.
Make also gives you command line completion, at least on the shell i'm using. Bonus - the first command will always run so you can actually just type make on it's own here.
I always throw a makefile into my projects, just so I can quickly scan later all the common commands and parameters for each project as I flip between them.

Using just shell scripting, on Linux.
"scripts": {
"cmd": "{ trap 'trap \" \" TERM; kill 0; wait' INT TERM; } && blocking1 & blocking2 & wait"
}
npm run cmd
and then
^C will kill children and wait for clean exit.

As you may need to add more and more to this scripts it will become messy and harder to use. What if you need some conditions to check, variables to use? So I suggest you to look at google/zx that allows to use js to create scripts.
Simple usage:
install zx: npm i -g zx
add package.json commands (optional, you can move everything to scripts):
"scripts": {
"dev": "zx ./scripts/dev.mjs", // run script
"build:dev": "tsc -w", // compile in watch mode
"build": "tsc", // compile
"start": "node dist/index.js", // run
"start:dev": "nodemon dist/index.js", // run in watch mode
},
create dev.mjs script file:
#!/usr/bin/env zx
await $`yarn build`; // prebuild if dist is empty
await Promise.all([$`yarn start:dev`, $`yarn build:dev`]); // run in parallel
Now every time you want to start a dev server you just run yarn dev or npm run dev.
It will first compile ts->js and then run typescrpt compiler and server in watch mode in parallel. When you change your ts file->it's will be recompiled by tsc->nodemon will restart the server.
Advanced programmatic usage
Load env variables, compile ts in watch mode and rerun server from dist on changes (dev.mjs):
#!/usr/bin/env zx
import nodemon from "nodemon";
import dotenv from "dotenv";
import path from "path";
import { fileURLToPath } from "url";
// load env variables
loadEnvVariables("../env/.env");
await Promise.all([
// compile in watch mode (will recompile on changes in .ts files)
$`tsc -w`,
// wait for tsc to compile for first time and rerun server on any changes (tsc emited .js files)
sleep(4000).then(() =>
nodemon({
script: "dist/index.js",
})
),
]);
function sleep(ms) {
return new Promise((resolve) => {
setTimeout(resolve, ms);
});
}
function getDirname() {
return path.dirname(fileURLToPath(import.meta.url));
}
function loadEnvVariables(relativePath) {
const { error, parsed } = dotenv.config({
path: path.join(getDirname(), relativePath),
});
if (error) {
throw error;
}
return parsed;
}

Run npm script with nodemon command

I am testing GraphQL Server from Apollo and what to integrate nodemon to it. Here's sample file structure :
build/
src/
server.js
Here my npm scripts looks like this
"scripts": {
"start": "babel --presets es2015,stage-2 server.js -d build/ && node build/server.js",
"dev": "nodemon server.js" // Sample code here
}
What npm run start would do is to convert ES6 code into build/server.js using babel and execute it. This would start the server correctly.
What I want is to watch for changes in server.js or in src/ and restart the server if changes occur. Here I want to execute npm run start command if any changes occur. What is the correct 'nodemon' command to my need. Its better if I could use npm run dev like command to start development using nodemon.

you can use gulpjs to watch any changes in specific folders and then command it to do something.
With your sample, you want to convert the code to es6 too. so it also need gulp-bable and . you can include babel-preset-stage-2 if you want. So you can simply put the code below in gulpfile.js
gulp.task('build-es2015', () => {
return gulp.src('server.js')
.pipe(babel({
presets: ['es2015']
}))
.pipe(gulp.dest('build'));
});
gulp.task('watch', () => {
gulp.watch(['./app/*.js'], ['build-es2015'])
})
Basically, the task 'watch' will keep watching the specific files. When they are saved then it will to execute the task 'build-es2015' to convert to es6.
And then nodemon, it needs gulp-nodemon then you can do on gulpfile.js
gulp.task('server', () => {
nodemon({
script: 'build/server.js',
ext: 'js',
ignore: [
'server.js',
'node_modules/**',
'test/**',
'build/**'
]
})
.on('restart', () => { console.log(`Server restarted!`) })
})
The above will keep watching build/server.js'. whenever it is changed, nodemon will automatically restart the server.
And the last piece for gulpfile.js
gulp.task('dev', ['server', 'watch'])
Include the tasks that need to be executed for gulp command.
$ gulp dev
or with npm command
"scripts": {
"start": "gulp dev"
}
so you can npm run start as well.
And dont forget to require all packages in gulpfile.js
const gulp = require('gulp')
const babel = require('gulp-babel')
const nodemon = require('gulp-nodemon')

Passing variables from NPM Scripts to Webpack

I have a production build with Webpack that uses node's process.env to set environment variables:
webpack.prod.babel.js:
const DefinePlugin = new webpack.DefinePlugin({
'process.env': {
NODE_ENV: JSON.stringify('production'),
API_URL: JSON.stringify('https://myprodurl.com'),
},
});
packge.json:
"scripts: {
"build:prod": "webpack"
}
It's working fine, but I need something different.
I need to set the production url as variable in the NPM Script.
So, instead of this:
npm run build:prod
I need this:
npm run build:prod --URL https://myprodurl.com

How about defining your environment variable in the command line, like:
URL=https://myprodurl.com npm run build:prod
I tested this with a simple script and was able to print out the URL.
"scripts": {
"test": "./myTest.js"
},
myTest.js:
#!/usr/local/bin/node
'use strict'
console.log(process.env.URL);
console.log('Done!');
then:
$ URL=https://whatever.com npm run test
> my-test#1.0.0 test /Test/my-test
> ./myTest.js
https://whatever.com
Done!
EDIT: As mentioned by #RyanZim, see the following for Windows: https://github.com/kentcdodds/cross-env
(disclaimer: I don't use Windows and have never tried this lib)

How grunt command works?

What command is actually executed?
Like using npm start with following included in package.json
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node index.js"
}
npm start runs node index.js behind the scenes,
Similarly, what does grunt run (behind the scenes)?

When you call the Grunt task runner, it runs whatever Grunt plugins you have specified in your Gruntfile in the order you specify.
A Grunt plugin consists of a single task file. That file is essentially nothing but a Node.js script that carries out the task in question. It has access to the settings passed to the plugin and can use Grunt's file API to access the filesystem, but otherwise it's just a Node.js script.
It's not hard to write a Grunt plugin and if you're interested in learning more about Grunt it's a good way to become more familiar with it. I personally have written several static site generators as Grunt plugins and it was very useful. Grunt task file ie. gruntfile.js looks something like this,
module.exports = function(grunt) {
// Project configuration.
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
uglify: {
options: {
banner: '/*! <%= pkg.name %> <%= grunt.template.today("yyyy-mm-dd") %> */\n'
},
build: {
src: 'src/<%= pkg.name %>.js',
dest: 'build/<%= pkg.name %>.min.js'
}
}
});
// Load the plugin that provides the "uglify" task.
grunt.loadNpmTasks('grunt-contrib-uglify');
// Default task(s).
grunt.registerTask('default', ['uglify']);
};
when you run command grunt uglify basically it runs the task defined under uglify.You can find more in their get started guide here

grunt command executes Gruntfile.js, it does this through node, however it passes grunt configurations and functions to the plugins used inside Gruntfile. thats why you write
module.exports = function(grunt) {
// grunt parameter is passed from grunt-cli
// which contains grunt functions and utilities
// that you can use to configure the tasks etc.
});

How to deploy node that uses Gulp to heroku

I'm using gulp and also gulp plugins like gulp-minify-css, gulp-uglify etc (that listed as npm dependencies for my application).
Also I don't commit npm_modules folder and public folder, where all generated files are. And I can't figure out how to build my app (I have gulp build command) after deploy and setup my server (it's already looking for public folder).
It seems me a bad idea to commit before upload. Maybe there are some gentle decisions... Any thoughts?
Forked from: How to deploy node app that uses grunt to heroku

I was able to get this to work by adding this into my "package.json" file:
"scripts": {
"start": "node app",
"postinstall": "gulp default"
}
The postinstall script is run after the build pack. Check this for more information. The only annoying thing is that all of your dependencies have to live under "dependencies" instead of having separate "devDependencies"
I didn't need to do anything else with buildpacks or configuration. This seems like the simplest way to do it.
I wrote about the process I used here

You can do it!
There were a few key measures that helped me along the way:
heroku config:set NODE_ENV=production - to set your environment to 'production'
heroku config:set BUILDPACK_URL=https://github.com/krry/heroku-buildpack-nodejs-gulp-bower - to enable a customised Heroku buildpack. I incorporated elements of a few to make one that worked for me.
A gulp task entitled heroku:production that performs the build tasks that need to happen on the heroku server when NODE_ENV===production. Here's mine:
var gulp = require('gulp')
var runSeq = require('run-sequence')
gulp.task('heroku:production', function(){
runSeq('clean', 'build', 'minify')
})
clean, build, and minify are, of course separate gulp tasks that do the magic gulpage
If your application lives in /app.js, either:
(A) make a Procfile in the project root that contains only: web: node app.js, or
(B) add a start script to your package.json:
"name": "gulp-node-app-name",
"version": "10.0.4",
"scripts": {
"start": "node app.js"
},
And like #Zero21xxx says, put your gulp modules in your normal dependencies list in package.json, not in the devDependencies, which get overlooked by the buildpack, which runs npm install --production

The easiest way I found was:
Setup gulp on package.json scripts area:
"scripts": {
"build": "gulp",
"start": "node app.js"
}
Heroku will run build before starting the app.
Include gulp on dependencies instead of devDevependencies, otherwise Heroku won't be able to find it.
There is more relevant info about it on Heroku Dev Center: Best Practices for Node.js Development

How to deploy to Heroku (or Azure) with git-push
// gulpfile.js
var gulp = require('gulp');
var del = require('del');
var push = require('git-push');
var argv = require('minimist')(process.argv.slice(2));
gulp.task('clean', del.bind(null, ['build/*', '!build/.git'], {dot: true}));
gulp.task('build', ['clean'], function() {
// TODO: Build website from source files into the `./build` folder
});
gulp.task('deploy', function(cb) {
var remote = argv.production ?
{name: 'production', url: 'https://github.com/<org>/site.com', branch: 'gh-pages'},
{name: 'test', url: 'https://github.com/<org>/test.site.com', branch: 'gh-pages'};
push('./build', remote, cb);
});
Then
$ gulp build --release
$ gulp deploy --production
See also
https://github.com/koistya/git-push
https://github.com/kriasoft/react-starter-kit (tools/deploy.js)

There's a specific startup script that Heroku provides;
"scripts": {
"start": "nodemon app.js",
"heroku-postbuild": "gulp"
}
note that in your gulpfile.js (gulpfile.babel.js if you es6-ifed your gulp build process), you should have a task name default which will be automatically run after the dependencies are installed via Heroku.
https://devcenter.heroku.com/articles/nodejs-support#heroku-specific-build-steps

Heroku finds that there is a gulpfile in your project and expects there to be a heroku:production task (in the gulpfile). So all you need to do is register a task that matches that name:
gulp.task("heroku:production", function(){
console.log('hello'); // the task does not need to do anything.
});
This is enough for heroku to not reject your app.

I had to take a slightly different to get this working because I'm using browsersync:
package.json
"scripts": {
"start": "gulp serve"
}
gulp.js
gulp.task('serve', function() {
browserSync({
server: {
baseDir: './'
},
port: process.env.PORT || 5000
});
gulp.watch(['*.html', 'css/*.css', 'js/*.js', 'views/*.html', 'template/*.html', './*.html'], {cwd: 'app'}, reload);
});
Setting the port to be environment port is important to prevent error when deploying in Heroku. I did not need to set a postinstall script.

It's possible to piggyback any command you want over the top of npm install. Much like the linked question in your post, you can add an install directive in scripts within package.json that will run after all the node deps have been installed that does the build.
Your main issue will be sorting out the correct relative paths for everything.
{
...
scripts:{
install: "YOUR GULP BUILD COMMAND"
}
...
}

Resources