Using a gulp task output inside another task - node.js

Question

I have two gulp tasks as following:
gulp.task("merge-json", () => {
return gulp
.src(
[
src_folder + "/modules/**/*.json",
src_folder + "/organisms/**/*.json",
src_folder + "/pages/**/*.json",
],
{
since: gulp.lastRun("merge-json"),
}
)
.pipe(plumber())
.pipe(merge({fileName: "data.json"}))
.pipe(gulp.dest(src_folder + "/datas/dist/"))
.pipe(browserSync.stream());
});
gulp.task("nunjucks", () => {
return gulp
.src([src_folder + "pages/**/*.njk"], {
base: src_folder + "pages",
since: gulp.lastRun("nunjucks"),
})
.pipe(plumber())
.pipe(data(() => JSON.parse(fs.readFileSync(src_folder + "datas/dist/data.json"))))
.pipe(nunjucks({path: src_folder}))
.pipe(gulp.dest(dist_folder))
.pipe(browserSync.stream());
});
The first task uses merge-json plugin to merge all the json files into a single one called data.json, and then this data.json will be used by the gulp-nunjucks-render plugin to generate html pages.
The problem I have here is that a file called data.json is generated in my project src, which will only be used for the nunjucks plugin.
My question is, Isn't there any way to use the output of the merge-json (data.json) plugin directly inside the nunjucks plugin?

Answer 1

This is not necessarily an ideal solution, but I will leave it here nonetheless.
Using map-stream, introduce a new stage in the pipeline that intercepts the contents of the merged JSON file buffer and store it into a shared variable.
const map = require('map-stream');
// a variable to store the merged JSON string
let mergedJsonString;
gulp.task("merge-json", () => {
return gulp
.src(
[
src_folder + "/modules/**/*.json",
src_folder + "/organisms/**/*.json",
src_folder + "/pages/**/*.json",
],
{
since: gulp.lastRun("merge-json"),
}
)
.pipe(plumber())
.pipe(merge({fileName: "data.json"}))
// only one file should ever be passed - data.json
.pipe(map((file, callback) => {
// capture the file buffer as a string
mergedJsonString = file.contents.toString()
// pass on the file to the next pipe, unmodified
callback(null, file)
}))
// can optionally be removed if not necessary to save to a file anymore
.pipe(gulp.dest(src_folder + "/datas/dist/"))
.pipe(browserSync.stream());
});
Now the nunjucks task will need to be modified to use a different data source and specify a gulp task dependency. The data source could instead be conditionally generated to use an old cached version if the merge-json task is not run prior; e.g. JSON.parse(typeof mergedJsonString !== "undefined" ? mergedJsonString : fs.readFileSync(src_folder + "datas/dist/data.json"))
// optionally use gulp.series to create a task dependency on "merge-json", ensuring it runs prior to the "nunjucks" task
gulp.task("nunjucks", gulp.series(["merge-json"], () => {
return gulp
.src([src_folder + "pages/**/*.njk"], {
base: src_folder + "pages",
since: gulp.lastRun("nunjucks"),
})
.pipe(plumber())
// source the JSON from a variable in memory; or if undefined, maybe change the implementation to use an old cached "/data/dists/data.json" file
.pipe(data(() => JSON.parse(mergedJsonString)))
.pipe(nunjucks({path: src_folder}))
.pipe(gulp.dest(dist_folder))
.pipe(browserSync.stream());
});
Might need a few minor tweaks for your specific use case, but this does meet the use case, even if it is hacky.