I am trying to send a pdfkit generated pdf file as input to pdflib for merging. I am using async function. My project is being developed using sails Js version:"^1.2.3", "node": "^12.16", my pdf-kit version is: "^0.11.0", "pdf-lib": "^1.9.0",
This is the code:
const textbytes=fs.readFileSync(textfile);
var bytes1 = new Uint8Array(textbytes);
const textdoc = await PDFDocumentFactory.load(bytes1)
The error i am getting is:
UnhandledPromiseRejectionWarning: Error: Failed to parse PDF document (line:0 col:0 offset=0): No PDF header found
Please help me with this issue.
You really don't need this line.
var bytes1 = new Uint8Array(textbytes);
By just reading the file and sending textbytes in the parameters is more than enough.
I use this function to merge an array of pdfBytes to make one big PDF file:
async function mergePdfs(pdfsToMerge)
{
const mergedPdf = await pdf.PDFDocument.create();
for (const pdfCopyDoc of pdfsToMerge)
{
const pdfDoc = await pdf.PDFDocument.load(pdfCopyDoc);
const copiedPages = await mergedPdf.copyPages(pdfDoc, pdfDoc.getPageIndices());
copiedPages.forEach((page) => {
mergedPdf.addPage(page);
});
}
const mergedPdfFile = await mergedPdf.save();
return mergedPdfFile;
};
So basically after you add the function mergePdfs(pdfsToMerge)
You can just use it like this:
const textbytes = fs.readFileSync(textfile);
const textdoc = await PDFDocumentFactory.load(bytes1)
let finalPdf = await mergePdfs(textdoc);
I have this async function in a file called index.main.js which has a variable , 'targetFiles' which I would like to export to another file which is index.js. Problem is I can't find a way to export the value of this particular variable without getting "undefined" as a result.
I have tried implementing promise, callback , export default function, and doing countless hours of research to no avail.
//this code is in index.main.js
var targetFiles = "";
async function listFilesInDepth()
{
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucketName = 'probizmy';
const [files] = await storage.bucket(bucketName).getFiles();
console.log('List Of Files Available:');
files.forEach(file =>
{
targetFiles = file.name; //this is the variable to export
console.log(`-----`+file.name);
});
return targetFiles;
}
module.exports = {
fn : targetFiles
}
trying to export the value to index.js is either empty or "undefined"
//this is the code in index.js
const a = require('./index.main');
console.log(a.fn); //undefined or empty
The expected value that should be the output is the value of targetFiles. Let's say if targetFiles is abc12345.JSON in the async function,the console.log in index.js should be of that value.
I'm hoping someone could give me some insight on how I could overcome this issue. Thank you in advance :)
Following solution might help you, but not sure for your use case. (Not using module-exports):
You can use request-context package to achieve same functionality.
What is package does is, you can set the value(data) against a key and then access the same in the following code execution within the same context.
Run npm install request-context
In your main app.js (server file), register the request-context as a middleware.
const contextService = require("request-context");
const app = express();
...
// Multiple contexts are supported, but below we are setting for per request.
app.use(contextService.middleware("request"));
...
And then in your index.main.js, once targetFiles is ready, set the targetFiles into request context.
const contextService = require("request-context");
...
files.forEach(file =>
{
targetFiles = file.name; //this is the variable to export
console.log(`-----`+file.name);
});
// Here `Request` is the namespace(context), targetFileKey is the key and targetFiles is the value.
contextService.set("request:targetFileKey", targetFiles);
return targetFiles;
}
...
And in the same request, where you wanna use targetFile, you can do the following:
index.js (Can be any file where you need targetFiles after being set):
const contextService = require("request-context");
...
// Reading from same namespace request to which we had set earlier
const targetFiles = contextService.get("request:targetFileKey");
...
Please note:
You will be able to access targetFiles in the same request you set. That means, request-context we configured in app.js is per API request, meaning, in every API request, you have to set it before reading.
Please let me know if the above solution doesn't fit for you.
This a repeat question (not yet answered) but I have revised and tightened up the code. And, I have included the specific example. I am sorry to keep beating this drum, but I need help.
This is a Node API. I need to read and write JSON data. I am using the Node core module 'fs', not the npm package by the same name (or fs-extra). I have extracted the particular area of concern onto a standalone module that is shown here:
'use strict';
/*==================================================
This service GETs the list of ids to the json data files
to be processed, from a json file with the id 'ids.json'.
It returns and exports idsList (an array holding the ids of the json data files)
It also calls putIdsCleared to clear the 'ids.json' file for the next batch of processing
==================================================*/
// node modules
const fs = require('fs');
const config = require('config');
const scheme = config.get('json.scheme')
const jsonPath = config.get('json.path');
const url = `${scheme}${jsonPath}/`;
const idsID = 'ids.json';
const uri = `${url}${idsID}`;
let idsList = [];
const getList = async (uri) => {
await fs.readFile(uri, 'utf8', (err, data) => {
if (err) {
return(console.log( new Error(err.message) ));
}
return jsonData = JSON.parse(data);
})
}
// The idea is to get the empty array written back to 'ids.json' before returning to 'process.js'
const clearList = async (uri) => {
let data = JSON.stringify({'ids': []});
await fs.writeFile(uri, data, (err) => {
if (err) {
return (console.log( new Error(err.message) ));
}
return;
})
}
getList(uri);
clearList(uri)
console.log('end of idsList',idsList);
module.exports = idsList;
Here is the console output from the execution of the module:
Error: ENOENT: no such file or directory, open 'File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json'
at ReadFileContext.fs.readFile [as callback]
(/Users/doug5solas/sandbox/libertyMutual/server/.playground/ids.js:24:33)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:235:13)
Error: ENOENT: no such file or directory, open 'File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json'
at fs.writeFile
(/Users/doug5solas/sandbox/libertyMutual/server/.playground/ids.js:36:34)
at fs.js:1167:7
at FSReqWrap.oncomplete (fs.js:141:20)
I am being told there is no such file or directory. However I can copy the uri (as shown in the error message)
File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json
into the search bar of my browser and this is what is returned to me:
{
"ids": [
"5sM5YLnnNMN_1540338527220.json",
"5sM5YLnnNMN_1540389571029.json",
"6tN6ZMooONO_1540389269289.json"
]
}
This result is the expected result. I do not "get" why I can get the data manually but I cannot get it programmatically, using the same uri. What am I missing? Help appreciated.
Your File URI is in the wrong format.
It shouldn't contain the File:// protocol (that's a browser-specific thing).
I'd imagine you want C://Users/doug5solas/sandbox/libertyMutual/server/api/ids.json.
I solved the problem by going to readFileSync. I don't like it but it works and it is only one read.
Using gulp 3.9.1
I am attempting to return a bunch of files and perform a task that requires a var to be passed between two pipes.
I'm using node uuid to create a v3 UUID for each file path to
ultimately end up with a uuid for each page. I'm grabbing the file path with gulp-print.
I want to store that uuid value as a var. In the next pipe Im using
gulp-inject-string to write it into the page during the build.
Help: Either I need help getting the file path inside the gulp-inject-string pipe or I need to pass the var between the two different pipes. If I globally set a var with a default value outside the src it gets passed easily to the pipe(inject).
Super simplified code below:
// test code
var gulp = require('gulp');
var print = require('gulp-print');
var inject = require('gulp-inject-string');
var reload = browserSync.reload;
const uuidv3 = require('uuid/v3');
var uuid;
gulp.task('uuid', function() {
return gulp.src('**/*.html'])
// create uuid
.pipe(print(function(filepath) {
uuid = uuidv3(filepath, uuidv3.URL);
return "compiled: " + filepath + ' uuid: ' + uuid;
}))
// need to to add UUIDv3 to each page
.pipe(inject.before('</head>', '<meta name="dc.identifier" content="' + uuid + '">'))
.pipe(gulp.dest('/prod/./'))
.pipe(reload({ stream: true }));
});
It's worth noting that I need a cross platform way to get the file path starting in the root of the project and including forward slashes. The gulp(print) does this perfectly starting at the root of the project and ignoring anything upstream from that point. The format of the path is important because it's one half of the equation in creating the uuid and the uuid's must match on Mac or PC platforms.
examples:
/index.html
/dir1/file.html
/dir1/dir2/dir3/file.html
var gulp = require('gulp');
var print = require('gulp-print');
var inject = require('gulp-inject-string');
const uuidv3 = require('uuid/v3');
var tap = require('gulp-tap');
// you can declare here
var uuid;
gulp.task('pages', function() {
// or you can declare here
var uuid;
return gulp.src('**/*.html')
// bunch of stuff happens here involving templating/minifying
// create uuid
.pipe(print(function(filepath) {
// then set it here and use it further below
// it will be available
uuid = uuidv3(filepath, uuidv3.URL);
return "compiled: " + filepath + ' uuid: ' + uuid;
}))
// need to to add UUIDv3 to each page
//.pipe(inject.before('</head>', '<meta name="dc.identifier" content="' + uuid + '">\n'))
.pipe(tap(function(file, t) {
return t.through(inject.before('</head>', '<meta name="dc.identifier" content="' + uuid + '">\n');
})
.pipe(gulp.dest('/prod/./'))
.pipe(reload({stream:true}));
});
You are just creating a variable at a higher scope that you can set and refer to later. If you need a bunch of them create an array with filepath as an index. But I would try it first as just a simple value.
I solved the problem. It was an amateur mistake. I returned the statement where the var was set so the var was essentially killed. Updated code that allows the var to pass through the pipes.
var gulp = require('gulp');
var print = require('gulp-print');
var replace = require('gulp-replace');
const uuidv3 = require('uuid/v3');
var uuid;
gulp.task('build', function() {
return gulp.src('**/*.html')
// get a cross-platform filepath and create a uuid
.pipe(print(function(filepath) {
uuid = uuidv3(filepath, uuidv3.URL);
}))
// inject uuid
.pipe(replace('dc.identifier" content=""', function() {
return 'dc.identifier" content="' + uuid + '"';
}))
.pipe(gulp.dest('/prod/./'));
});
The var uuid passes through the pipes just fine now. This code creates a UUID based on a cross-platform file path and injects it into an empty dc.identifier meta tag.
How do I require all files in a folder in node.js?
need something like:
files.forEach(function (v,k){
// require routes
require('./routes/'+v);
}};
When require is given the path of a folder, it'll look for an index.js file in that folder; if there is one, it uses that, and if there isn't, it fails.
It would probably make most sense (if you have control over the folder) to create an index.js file and then assign all the "modules" and then simply require that.
yourfile.js
var routes = require("./routes");
index.js
exports.something = require("./routes/something.js");
exports.others = require("./routes/others.js");
If you don't know the filenames you should write some kind of loader.
Working example of a loader:
var normalizedPath = require("path").join(__dirname, "routes");
require("fs").readdirSync(normalizedPath).forEach(function(file) {
require("./routes/" + file);
});
// Continue application logic here
I recommend using glob to accomplish that task.
var glob = require( 'glob' )
, path = require( 'path' );
glob.sync( './routes/**/*.js' ).forEach( function( file ) {
require( path.resolve( file ) );
});
Base on #tbranyen's solution, I create an index.js file that load arbitrary javascripts under current folder as part of the exports.
// Load `*.js` under current directory as properties
// i.e., `User.js` will become `exports['User']` or `exports.User`
require('fs').readdirSync(__dirname + '/').forEach(function(file) {
if (file.match(/\.js$/) !== null && file !== 'index.js') {
var name = file.replace('.js', '');
exports[name] = require('./' + file);
}
});
Then you can require this directory from any where else.
Another option is to use the package require-dir which let's you do the following. It supports recursion as well.
var requireDir = require('require-dir');
var dir = requireDir('./path/to/dir');
I have a folder /fields full of files with a single class each, ex:
fields/Text.js -> Test class
fields/Checkbox.js -> Checkbox class
Drop this in fields/index.js to export each class:
var collectExports, fs, path,
__hasProp = {}.hasOwnProperty;
fs = require('fs');
path = require('path');
collectExports = function(file) {
var func, include, _results;
if (path.extname(file) === '.js' && file !== 'index.js') {
include = require('./' + file);
_results = [];
for (func in include) {
if (!__hasProp.call(include, func)) continue;
_results.push(exports[func] = include[func]);
}
return _results;
}
};
fs.readdirSync('./fields/').forEach(collectExports);
This makes the modules act more like they would in Python:
var text = new Fields.Text()
var checkbox = new Fields.Checkbox()
One more option is require-dir-all combining features from most popular packages.
Most popular require-dir does not have options to filter the files/dirs and does not have map function (see below), but uses small trick to find module's current path.
Second by popularity require-all has regexp filtering and preprocessing, but lacks relative path, so you need to use __dirname (this has pros and contras) like:
var libs = require('require-all')(__dirname + '/lib');
Mentioned here require-index is quite minimalistic.
With map you may do some preprocessing, like create objects and pass config values (assuming modules below exports constructors):
// Store config for each module in config object properties
// with property names corresponding to module names
var config = {
module1: { value: 'config1' },
module2: { value: 'config2' }
};
// Require all files in modules subdirectory
var modules = require('require-dir-all')(
'modules', // Directory to require
{ // Options
// function to be post-processed over exported object for each require'd module
map: function(reqModule) {
// create new object with corresponding config passed to constructor
reqModule.exports = new reqModule.exports( config[reqModule.name] );
}
}
);
// Now `modules` object holds not exported constructors,
// but objects constructed using values provided in `config`.
I know this question is 5+ years old, and the given answers are good, but I wanted something a bit more powerful for express, so i created the express-map2 package for npm. I was going to name it simply express-map, however the people at yahoo already have a package with that name, so i had to rename my package.
1. basic usage:
app.js (or whatever you call it)
var app = require('express'); // 1. include express
app.set('controllers',__dirname+'/controllers/');// 2. set path to your controllers.
require('express-map2')(app); // 3. patch map() into express
app.map({
'GET /':'test',
'GET /foo':'middleware.foo,test',
'GET /bar':'middleware.bar,test'// seperate your handlers with a comma.
});
controller usage:
//single function
module.exports = function(req,res){
};
//export an object with multiple functions.
module.exports = {
foo: function(req,res){
},
bar: function(req,res){
}
};
2. advanced usage, with prefixes:
app.map('/api/v1/books',{
'GET /': 'books.list', // GET /api/v1/books
'GET /:id': 'books.loadOne', // GET /api/v1/books/5
'DELETE /:id': 'books.delete', // DELETE /api/v1/books/5
'PUT /:id': 'books.update', // PUT /api/v1/books/5
'POST /': 'books.create' // POST /api/v1/books
});
As you can see, this saves a ton of time and makes the routing of your application dead simple to write, maintain, and understand. it supports all of the http verbs that express supports, as well as the special .all() method.
npm package: https://www.npmjs.com/package/express-map2
github repo: https://github.com/r3wt/express-map
Expanding on this glob solution. Do this if you want to import all modules from a directory into index.js and then import that index.js in another part of the application. Note that template literals aren't supported by the highlighting engine used by stackoverflow so the code might look strange here.
const glob = require("glob");
let allOfThem = {};
glob.sync(`${__dirname}/*.js`).forEach((file) => {
/* see note about this in example below */
allOfThem = { ...allOfThem, ...require(file) };
});
module.exports = allOfThem;
Full Example
Directory structure
globExample/example.js
globExample/foobars/index.js
globExample/foobars/unexpected.js
globExample/foobars/barit.js
globExample/foobars/fooit.js
globExample/example.js
const { foo, bar, keepit } = require('./foobars/index');
const longStyle = require('./foobars/index');
console.log(foo()); // foo ran
console.log(bar()); // bar ran
console.log(keepit()); // keepit ran unexpected
console.log(longStyle.foo()); // foo ran
console.log(longStyle.bar()); // bar ran
console.log(longStyle.keepit()); // keepit ran unexpected
globExample/foobars/index.js
const glob = require("glob");
/*
Note the following style also works with multiple exports per file (barit.js example)
but will overwrite if you have 2 exports with the same
name (unexpected.js and barit.js have a keepit function) in the files being imported. As a result, this method is best used when
your exporting one module per file and use the filename to easily identify what is in it.
Also Note: This ignores itself (index.js) by default to prevent infinite loop.
*/
let allOfThem = {};
glob.sync(`${__dirname}/*.js`).forEach((file) => {
allOfThem = { ...allOfThem, ...require(file) };
});
module.exports = allOfThem;
globExample/foobars/unexpected.js
exports.keepit = () => 'keepit ran unexpected';
globExample/foobars/barit.js
exports.bar = () => 'bar run';
exports.keepit = () => 'keepit ran';
globExample/foobars/fooit.js
exports.foo = () => 'foo ran';
From inside project with glob installed, run node example.js
$ node example.js
foo ran
bar run
keepit ran unexpected
foo ran
bar run
keepit ran unexpected
One module that I have been using for this exact use case is require-all.
It recursively requires all files in a given directory and its sub directories as long they don't match the excludeDirs property.
It also allows specifying a file filter and how to derive the keys of the returned hash from the filenames.
Require all files from routes folder and apply as middleware. No external modules needed.
// require
const { readdirSync } = require("fs");
// apply as middleware
readdirSync("./routes").map((r) => app.use("/api", require("./routes/" + r)));
I'm using node modules copy-to module to create a single file to require all the files in our NodeJS-based system.
The code for our utility file looks like this:
/**
* Module dependencies.
*/
var copy = require('copy-to');
copy(require('./module1'))
.and(require('./module2'))
.and(require('./module3'))
.to(module.exports);
In all of the files, most functions are written as exports, like so:
exports.function1 = function () { // function contents };
exports.function2 = function () { // function contents };
exports.function3 = function () { // function contents };
So, then to use any function from a file, you just call:
var utility = require('./utility');
var response = utility.function2(); // or whatever the name of the function is
Can use : https://www.npmjs.com/package/require-file-directory
Require selected files with name only or all files.
No need of absoulute path.
Easy to understand and use.
Using this function you can require a whole dir.
const GetAllModules = ( dirname ) => {
if ( dirname ) {
let dirItems = require( "fs" ).readdirSync( dirname );
return dirItems.reduce( ( acc, value, index ) => {
if ( PATH.extname( value ) == ".js" && value.toLowerCase() != "index.js" ) {
let moduleName = value.replace( /.js/g, '' );
acc[ moduleName ] = require( `${dirname}/${moduleName}` );
}
return acc;
}, {} );
}
}
// calling this function.
let dirModules = GetAllModules(__dirname);
Create an index.js file in your folder with this code :
const fs = require('fs')
const files = fs.readdirSync('./routes')
for (const file of files) {
require('./'+file)
}
And after that you can simply load all the folder with require("./routes")
If you include all files of *.js in directory example ("app/lib/*.js"):
In directory app/lib
example.js:
module.exports = function (example) { }
example-2.js:
module.exports = function (example2) { }
In directory app create index.js
index.js:
module.exports = require('./app/lib');