I know the pipe function in NodeJs. A readable stream calls the pipe function with the first argument being the writable stream. Something like:
readable.pipe(fs.createWriteStream('file.txt'));
This will pipe all the output to file.txt. But I have not understood this in context of gulp.
What does a call to a pipe function like:
gulp.src('./assets/styles/**/*.scss')
.pipe(sass());
mean? Here is the full snippet:
var gulp = require('gulp');
var gutil = require('gulp-util');
// require sass
var sass = require('gulp-ruby-sass');
gulp.task('sass', function () {
gulp.src('./assets/styles/**/*.scss')
.pipe(sass())
.pipe(gulp.dest('./assets/styles'));
});
gulp.task('default', ['sass']);
I understand the dest part. But do not understand the pipe(sass()) part. What does it do? What stream does each of these functions return?
Note: I have taken the sample example from a blog
The pipe in gulp is the exact same as the pipe in Node.
This flow streams the sources files from .src() and creates a new stream a pipes it through the sass plugin - the sass plugin will then process all sass files into css and concat them to the destination path as a new stream.
Related
I use an NPM library to parse markdown to HTML like this:
var Markdown = require('markdown-to-html').Markdown;
var md = new Markdown();
...
md.render('./test', opts, function(err) {
md.pipe(process.stdout)
});
This outputs the result to my terminal as intended.
However, I need the result inside the execution of my node program. I thought about writing the output stream to file and then reading it in at a later time but I can't figure out a way to write the output to a file instead.
I tried to play around var file = fs.createWriteStream('./test.html'); but the node.js streams rather give me headaches than results.
I've also looked into the library's repo and Markdown inherits from Readable via util like this:
var util = require('util');
var Readable = require('stream').Readable;
util.inherits(Markdown, Readable);
Any resources or advice would be highly appreciated. (I would also take another library for parsing the markdown, but this gave me the best results so far)
Actually creating a writable file-stream and piping the markdown to this stream should work just fine. Try it with:
const writeStream = fs.createWriteStream('./output.html');
md.render('./test', opts, function(err) {
md.pipe(writeStream)
});
// in case of errors you should handle them
writeStream.on('error', function (err) {
console.log(err);
});
I am attempting to compile react (.jsx), coffeescript (.coffee), and vanilla javascript (.js) files using gulp to package all of the resulting .js files into one file, app.js, which gets loaded into my index.html page. I am generating a stream for each compilation type and using merge-stream to collect the contents of the 3 feeder streams into a single stream, which I am passing to gulp-concat to create app.js.
I am getting an exception from gulp-concat, index.js, line 39, letting me know that 'file' is not a function. Here is my entire gulpfile.js, the reference to gulp-concat is near the bottom of this section.
var browserify = require('browserify');
var coffee = require('gulp-coffee');
var concat = require('gulp-concat');
var gulp = require('gulp');
var gutil = require('gulp-util');
var mergeStream = require('merge-stream');
var reactify = require('reactify');
var sass = require('gulp-sass');
var source = require('vinyl-source-stream');
gulp.task('javascript', function(){
// convert .jsx files to .js, collecting them in a stream
var b = browserify();
b.transform(reactify); // use the reactify transform
b.add('./jsx-transforms.js');
jsxStream = b.bundle();
if (gutil.isStream(jsxStream)) {
gutil.log("jsxStream is a stream");
} else {gulp-concatgulp
gutil.log("jsxStream is not a stream");
}
merged = mergeStream(jsxStream);
if (gutil.isStream(merged)) {
gutil.log("merged is a stream");
} else {
gutil.log("merged is not a stream");
}
// collect all .js files in a stream
jsStream = gulp.src(['./client/**/*.js','./common/**/*.js']);
if (gutil.isStream(jsStream)) {
gutil.log("jsStream is a stream");
} else {
gutil.log("jsStream is not a stream");
}
merged.add(jsStream);
// compile all .coffee file to .js, collect in a stream
coffeeStream = gulp.src(['./client/**/*.coffee','./common/**/*.coffee'])
.pipe(coffee({bare: true}).on('error', gutil.log));
if (gutil.isStream(coffeeStream)) {
gutil.log("coffeeStream is a stream");
} else {
gutil.log("coffeeStream is not a stream");
}
merged.add(coffeeStream);
// concatenate all of the .js files into ./build/app.js
merged
.pipe(concat('app.js'))
.pipe(gulp.dest('./build'));
});
gulp.task('styles', function() {
gulp.src('./client/assets/stylesheets/**/*.scss')
.pipe(sass().on('error', sass.logError))
.pipe(concat('app.css'))
.pipe(gulp.dest('./build'));
});
gulp.task('default', ['javascript', 'styles']);
I have used gulp-concat before, but never hit this problem before.
Gulp streams are a very particular sort of stream: they are Node streams in object mode containing vinyl file objects. If your stream comes from somewhere other than gulp.src(), such as from the browserify API, then you will have to first convert the stream into the sort that gulp can deal with.
There are two steps you need to take. First, convert your browserify bundle stream into a stream containing a vinyl file object with vinyl-source-stream (which you have required but not used).
var source = require('vinyl-source-stream');
...
var jsxStream = b.bundle()
.pipe(source('bundle.js'));
Now there is another catch. Vinyl streams may be in one of two modes: streaming mode or buffer mode. Vinyl-source-stream gives you a stream in streaming mode. Many Gulp plugins, including gulp-concat, only support buffer mode. The fix for this is easy: use vinyl-buffer.
var source = require('vinyl-source-stream');
var buffer = require('vinyl-buffer');
...
var jsxStream = b.bundle()
.pipe(source('bundle.js'))
.pipe(buffer());
Now you have something you can merge with your other streams and pipe to gulp-concat. For more detail, see this recipe.
I'm using gulp to convert SCSS into CSS code with the gulp-sass plugin. This is all working fine, but I also want to use gulp to receive input (SCSS code) from a Unix pipe (i.e. read process.stdin) and consume this and stream the output to process.stdout.
From reading around process.stdin is a ReadableStream and vinyl seems like it could wrap stdin and then be used onwards in a gulp task, e.g.
gulp.task('stdin-sass', function () {
process.stdin.setEncoding('utf8');
var file = new File({contents: process.stdin, path: './test.scss'});
file.pipe(convert_sass_to_css())
.pipe(gulp.dest('.'));
});
However, when I do this I get an error:
TypeError: file.isNull is not a function
This makes me think that stdin is somehow special, but the official documentation for node.js states that it is a true ReadableStream.
So I got this to work by processing process.stdin and writing to process.stdout:
var buffer = require('vinyl-buffer');
var source = require('vinyl-source-stream');
var through = require('through2');
gulp.task('stdio-sass', function () {
process.stdin.setEncoding('utf8');
process.stdin.pipe(source('input.scss'))
.pipe(buffer())
.pipe(convert_sass_to_css())
.pipe(stdout_stream());
});
var stdout_stream = function () {
process.stdout.setEncoding('utf8');
return through.obj(function (file, enc, complete) {
process.stdout.write(file.contents.toString());
this.push(file);
complete();
});
};
I've got the following sample code which attempts to pipe a stream to gulp.dest():
var gulp = require('gulp');
var stream = require('stream');
var readable = new stream.Readable;
readable.push('Hello, world!');
readable.push(null);
readable
.pipe(gulp.dest('./test.txt'));
This code produces the following error:
path.js:146
throw new TypeError('Arguments to path.resolve must be strings');
^
TypeError: Arguments to path.resolve must be strings
at Object.win32.resolve (path.js:146:13)
at DestroyableTransform.saveFile [as _transform] (C:\paylocity\expense\node_modules\gulp\node_modules\vinyl-fs\lib\dest\index.js:36:26)
at DestroyableTransform.Transform._read (C:\paylocity\expense\node_modules\gulp\node_modules\vinyl-fs\node_modules\through2\node_modules\readable-stream\lib\_stream_transform.js:184:10)
at DestroyableTransform.Transform._write (C:\paylocity\expense\node_modules\gulp\node_modules\vinyl-fs\node_modules\through2\node_modules\readable-stream\lib\_stream_transform.js:172:12)
at doWrite (C:\paylocity\expense\node_modules\gulp\node_modules\vinyl-fs\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:237:10)
at writeOrBuffer (C:\paylocity\expense\node_modules\gulp\node_modules\vinyl-fs\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:227:5)
at DestroyableTransform.Writable.write (C:\paylocity\expense\node_modules\gulp\node_modules\vinyl-fs\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:194:11)
at Readable.ondata (_stream_readable.js:540:20)
at Readable.emit (events.js:107:17)
at Readable.read (_stream_readable.js:373:10)
I can't, however, see what exactly is wrong with the code. If I replace gulp.dest() with process.stdout then it works and the gulp.dest() works within the context of other calls. What is a viable way of doing this?
Gulp works with Vinyl streams:
var gulp = require('gulp'),
stream = require('stream'),
source = require('vinyl-source-stream');
var readable = new stream.Readable;
readable.push('Hello, world!');
readable.push(null);
readable
.pipe(source('test.txt'))
.pipe(gulp.dest('.'));
A Gulp stream normally begins with some file or files as source, so you need to wrap that readable stream into a Vinyl stream allowing Gulp and any gulp-plugin getting info from it like filename (obviously faked), avoiding that error of yours.
So, Gulp streams are streams of files, just check the source...
I have the following simplified gulp task:
gulp.src(...)
.pipe(stuff())
.pipe(moreStuff())
.pipe(imagemin())
.pipe(yetMoreStuff());
I only want the imagemin stream to be called when the file path contains "xyz", but I want the other three streams to always be called.
Called gulp.src() in another place is not appropriate—this example is massively simplified, and duplicating everything would be messy as hell.
So far, I've got this far:
var through = require('through2');
gulp.src(...)
.pipe(stuff())
.pipe(moreStuff())
.pipe(through.obj(function (file, enc, cb) {
console.log(file.path.indexOf('hero') !== -1);
// file has a pipe method but what do I do?!
}))
.pipe(yetMoreStuff());
Doesn't do anything. I don't know vinyl / streams well enough to be able to do this by myself :(
How do I do this?
It sounds like gulp-filter might be what you're looking for.
var Filter = require('gulp-filter');
var filter = Filter(['**xyz**']);
gulp.src(...)
.pipe(stuff())
.pipe(moreStuff())
.pipe(filter)
.pipe(imagemin())
.pipe(filter.restore())
.pipe(yetMoreStuff());