I've been trying to figure this out by myself, but had no success yet. I don't even know how to start researching for this (though I've tried some Google searchs already, to no avail), so I decided to ask this question here.
Is it possible to return multiple Vinyl files from a Through2 Object Stream?
My use case is this: I receive an HTML file via stream. I want to isolate two different sections of the files (using jQuery) and return them in two separate HTML files. I can do it with a single section (and a single resulting HTML file), but I have absolutely no idea on how I would do generate two different files.
Can anyone give me a hand here?
Thanks in advance.
The basic approach is something like this:
Create as many output files from your input file as you need using the clone() function.
Modify the .contents property of each file depending on what you want to do. Don't forget that this is a Buffer, not a String.
Modify the .path property of each file so your files don't overwrite each other. This is an absolute path so use something like path.parse() and path.join() to make things easier.
Call this.push() from within the through2 transform function for every file you have created.
Here's a quick example that splits a file test.txt into two equally large files test1.txt and test2.txt:
var gulp = require('gulp');
var through = require('through2').obj;
var path = require('path');
gulp.task('default', function () {
return gulp.src('test.txt')
.pipe(through(function(file, enc, cb) {
var c = file.contents.toString();
var f = path.parse(file.path);
var file1 = file.clone();
var file2 = file.clone();
file1.contents = new Buffer(c.substring(0, c.length / 2));
file2.contents = new Buffer(c.substring(c.length / 2));
file1.path = path.join(f.dir, f.name + '1' + f.ext);
file2.path = path.join(f.dir, f.name + '2' + f.ext);
this.push(file1);
this.push(file2);
cb();
}))
.pipe(gulp.dest('out'));
});
Related
I need to create many .json files for the system i am trying to develop. To do this, i ran a for loop over the file names i needed, then used fs.writeFileSync('filename.json', [data]).
However, when trying to open these later, and when I try to find them in the project folder, I cannot find them anywhere.
I have tried writing in a name that was less complex and should have appeared in the same directory as my script, however that was fruitless as well. To my understanding, even if my file name wasn't what I expected it to be, I should get at least something, somewhere, however I end up with nothing changed.
My current code looks like this:
function addEmptyDeparture(date) {
fs.readFileSync(
__dirname + '/reservations/templates/wkend_dep_template.json',
(err, data) => {
if (err) throw err
fs.writeFileSync(
getDepartureFileName(date),
data
)
}
)
}
function getDepartureFileName(date){
return __dirname + '/reservations/' +
date.getFullYear() +
'/departing/' +
(date.getMonth() + 1).toString().padStart(2, "0") +
date.getDate().toString().padStart(2, "0") +
'.json'
}
Where data is the JSON object returned from fs.readFileSync() and is immediately written into fs.writeFileSync(). I don't think I need to stringify this, since it's already a JSON object, but I may be wrong.
The only reason I think it's not working at all (as opposed to simply not showing up in my project) is that, in a later part of the code, we have this:
fs.readFileSync(
getDepartureFileName(date)
)
.toString()
which is where I get an error for not having a file by that name.
It is also worth noting that date is a valid date object, as I was able to test that part in a fiddle.
Is there something I'm misunderstanding in the effects of fs.writeFile(), or is this not the best way to write .json files for use on a server?
You probably are forgetting to stringify the data:
fs.writeFileSync('x.json', JSON.stringify({id: 1}))
I have tried to create similar case using a demo with writeFileSync() creating different files and adding json data to these ,using a for loop. In my case it works . Each time a new file name is created . Here is my GitHub for the same :-
var fs = require('fs');
// Use readFileSync() method
// Store the result (return value) of this
// method in a variable named readMe
for(let i=0 ; i < 4 ; i++) {
var readMe = JSON.stringify({"data" : i});
fs.writeFileSync('writeMe' + i + '.json', readMe, "utf8");
}
// Store the content and read from
// readMe.txt to a file WriteMe.txt
Let me know if this what you have been trying at your end.
I use browserify in order to be able to use require. To use fs functions with browserify i need to transform it with brfs but as far as I understood this results in only being able to input static strings as parameters inside my fs function. I want to be able to use variables for this.
I want to search for xml files in a specific directory and read them. Either by searching via text field or showing all of their data at once. In order to do this I need fs and browserify in order to require it.
const FS = require('fs')
function lookForRoom() {
let files = getFileNames()
findSearchedRoom(files)
}
function getFileNames() {
return FS.readdirSync('../data/')
}
function findSearchedRoom(files) {
const SEARCH_FIELD_ID = 'room'
let searchText = document.getElementById(SEARCH_FIELD_ID).value
files.forEach((file) => {
const SEARCHTEXT_FOUND = file.includes(searchText.toLowerCase())
if (SEARCHTEXT_FOUND) loadXML(file)
})
}
function loadXML(file) {
const XML2JS = require('xml2js')
let parser = new XML2JS.Parser()
let data = FS.readFile('../data/' + file)
console.dir(data);
}
module.exports = { lookForRoom: lookForRoom }
I want to be able to read contents out of a directory containing xml files.
Current status is that I can only do so when I provide a constant string to the fs function
The brfs README contains this gotcha:
Since brfs evaluates your source code statically, you can't use dynamic expressions that need to be evaluated at run time.
So, basically, you can't use brfs in the way you were hoping.
I want to be able to read contents out of a directory containing xml files
If by "a directory" you mean "any random directory, the name of which is determined by some form input", then that's not going to work. Browsers don't have direct access to directory contents, either locally or on a server.
You're not saying where that directory exists. If it's local (on the machine the browser is running on): I don't think there are standardized API's to do that, at all.
If it's on the server, then you need to implement an HTTP server that will accept a directory-/filename from some clientside code, and retrieve the file contents that way.
I have the following simplified gulp task:
gulp.src(...)
.pipe(stuff())
.pipe(moreStuff())
.pipe(imagemin())
.pipe(yetMoreStuff());
I only want the imagemin stream to be called when the file path contains "xyz", but I want the other three streams to always be called.
Called gulp.src() in another place is not appropriate—this example is massively simplified, and duplicating everything would be messy as hell.
So far, I've got this far:
var through = require('through2');
gulp.src(...)
.pipe(stuff())
.pipe(moreStuff())
.pipe(through.obj(function (file, enc, cb) {
console.log(file.path.indexOf('hero') !== -1);
// file has a pipe method but what do I do?!
}))
.pipe(yetMoreStuff());
Doesn't do anything. I don't know vinyl / streams well enough to be able to do this by myself :(
How do I do this?
It sounds like gulp-filter might be what you're looking for.
var Filter = require('gulp-filter');
var filter = Filter(['**xyz**']);
gulp.src(...)
.pipe(stuff())
.pipe(moreStuff())
.pipe(filter)
.pipe(imagemin())
.pipe(filter.restore())
.pipe(yetMoreStuff());
Let's say I want to replace the version number in a bunch of files, many of which live in subdirectories. I will pipe the files through gulp-replace to run the regex-replace function; but I will ultimately want to overwrite all the original files.
The task might look something like this:
gulp.src([
'./bower.json',
'./package.json',
'./docs/content/data.yml',
/* ...and so on... */
])
.pipe(replace(/* ...replacement... */))
.pipe(gulp.dest(/* I DONT KNOW */);
So how can I end it so that each src file just overwrites itself, at its original location? Is there something I can pass to gulp.dest() that will do this?
I can think of two solutions:
Add an option for base to your gulp.src like so:
gulp.src([...files...], {base: './'}).pipe(...)...
This will tell gulp to preserve the entire relative path. Then pass './' into gulp.dest() to overwrite the original files. (Note: this is untested, you should make sure you have a backup in case it doesn't work.)
Use functions. Gulp's just JavaScript, so you can do this:
[...files...].forEach(function(file) {
var path = require('path');
gulp.src(file).pipe(rename(...)).pipe(gulp.dest(path.dirname(file)));
}
If you need to run these asynchronously, the first will be much easier, as you'll need to use something like event-stream.merge and map the streams into an array. It would look like
var es = require('event-stream');
...
var streams = [...files...].map(function(file) {
// the same function from above, with a return
return gulp.src(file) ...
};
return es.merge.apply(es, streams);
Tell gulp to write to the base directory of the file in question, just like so:
.pipe(
gulp.dest(function(data){
console.log("Writing to directory: " + data.base);
return data.base;
})
)
(The data argument is a vinyl file object)
The advantage of this approach is that if your have files from multiple sources each nested at different levels of the file structure, this approach allows you to overwrite each file correctly. (As apposed to set one base directory in the upstream of your pipe chain)
if you are using gulp-rename, here's another workaround:
var rename = require('gulp-rename');
...
function copyFile(source, target){
gulp.src(source)
.pipe(rename(target))
.pipe(gulp.dest("./"));
}
copyFile("src/js/app.js","dist/js/app.js");
and if you want source and target to be absolute paths,
var rename = require('gulp-rename');
...
function copyFile(source, target){
gulp.src(source.replace(__dirname,"."))
.pipe(rename(target.replace(__dirname,".")))
.pipe(gulp.dest("./"));
}
copyFile("/Users/me/Documents/Sites/app/src/js/app.js","/Users/me/Documents/Sites/app/dist/js/app.js");
I am not sure why people complicate it but by just starting your Destination path with "./" does the job.
Say path is 'dist/css' Then you would use it like this
.pipe(gulp.dest("./dist/css"));
That's it, I use this approach on everyone of my projects.
I want to use factor-bundle to find common dependencies for my browserify entry points and save them out into a single common bundle:
https://www.npmjs.org/package/factor-bundle
The factor-bundle documentation makes it seem very easy to do on the command line, but I want to do it programmatically and I'm struggling to get my head around it.
My current script is this (I'm using reactify to transform react's jsx files too):
var browserify = require('browserify');
var factor = require('factor-bundle')
var glob = require('glob');
glob('static/js/'/**/*.{js,jsx}', function (err, files) {
var bundle = browserify({
debug: true
});
files.forEach(function(f) {
bundle.add('./' + f);
});
bundle.transform(require('reactify'));
// factor-bundle code goes here?
var dest = fs.createWriteStream('./static/js/build/common.js');
var stream = bundle.bundle().pipe(dest);
});
I'm trying to figure out how to use factor-bundle as a plugin, and specify the desired output file for each of the input files (ie each entry in files)
This answer is pretty late, so it's likely you've either already found a solution or a work around for this question. I'm answering this as it's quite similar to my question.
I was able to get this working by using factor-bundle as a browserify plugin. I haven't tested your specific code, but the pattern should be the same:
var fs = require('fs'),
browserify = require('browserify'),
factor = require('factor-bundle');
var bundle = browserify({
entries: ['x.js', 'y.js', 'z.js'],
debug: true
});
// Group common dependencies
// -o outputs the entry files without the common dependencies
bundle.plugin('factor-bundle', {
o: ['./static/js/build/x.js',
'./static/js/build/y.js',
'./static/js/build/z.js']
});
// Create Write Stream
var dest = fs.createWriteStream('./static/js/build/common.js');
// Bundle
var stream = bundle.bundle().pipe(dest);
The factor-bundle plugin takes output options o which need to have the same indexes as the entry files.
Unfortunately, I haven't figured out how to do anything else with these files after this point because I can't seem to access factor-bundle's stream event. So for minification etc, it might need to be done also via a browserify plugin.
I have created grunt-reactify to allow you to have a bundle file for a JSX file, in order to make it easier to work with modular React components.
All what you have to do is to specify a parent destination folder and the source files:
grunt.initConfig({
reactify: {
'tmp': 'test/**/*.jsx'
},
})