I am really new to node.js. I need to read .json files from a directory and then add them to an array and return it. I am able to read each file separately by passing the address:
const fs = require("fs");
fs.readFile("./fashion/customer.json", "utf8", (err, jsonString) => {
if (err) {
console.log("Error reading file from disk:", err);
return;
}
try {
const customer = JSON.parse(jsonString);
console.log("Customer address is:", customer.address); // => "Customer address is: Infinity Loop Drive"
} catch (err) {
console.log("Error parsing JSON string:", err);
}
});
But the same fashion folder has multiple json files. I want to add these files to an array and then return it. I tried using readdirSync but that just returned the file names. Is it possible to add json files to an array and return it?
Basically I require an array of this format:
Array[{contents of json file1}, {contents of json file2}, .....]
Any help is appreciated!
Here is a simple solution to your question:
const fs = require("fs");
const jsonFolder = './fashion'
var customerDataArray = []
fs.readdirSync(jsonFolder).forEach(file => {
let fileData = JSON.parse(fs.readFileSync(jsonFolder+'/'+file))
customerDataArray.push(fileData)
});
console.log(customerDataArray)
readdirSync returns an array with all the file names or objects in the directory. You can use forEach to iterate through every item in the array, which will be the file names in this scenario. To read the contents of each file, use readFileSync and specify the path to the file as the name of the directory plus the name of the file. The data is returned as a buffer and needs to be parsed using JSON.parse(), and then it is pushed to the customerDataArray.
I hope this answers your question!
Related
I'm wanting to bulk rename files within a folder based on a JSON file that I have with the following format:
{
"1": {
"Filename": "Background-1",
"New Filename": "Background-1#4"
},
"2": {
"Filename": "Background-2",
"New Filename": "Background-2#6"
},
The original Filenames are within a folder structure such as
Background
--Background-1
--Background-2
Other Folder
--Another-Filename
--Another-Filename-2
And so on and so forth. I want to copy the files with the new names, while retaining the name of the folder they're in, over to a new folder.
So far I've tried using fs and klaw-sync to read the filenames, traverse through directories, etc, but it seems wildly inefficient to run through each key and then run through each folder recurisvely to find a matching file, then rename it and copy. There's over 180 files and ~15 folders.
Any idea how I can approach this better, or any suggestions/examples I could use?
Here's what I've got so far.
Thanks.
// Require Node's File System module
const fs = require('fs');
var path = require('path');
var klawSync = require('klaw-sync');
// Read the JSON file
fs.readFile(__dirname + '/rename_config.json', function (error, data) {
if (error) {
console.log(error);
return;
}
const obj = JSON.parse(data);
// Iterate over the object
Object.keys(obj).forEach(key => {
// Create an empty variable to be accesible in the closure
var paths;
// The directory that you want to explore
var directoryToExplore = path.join(__dirname, '../art');
try {
paths = klawSync(directoryToExplore);
} catch (err) {
console.error(err);
}
//console.log(paths);
//traverse through paths to find an equal name
//find the path of that equivalent name, then rename to new directory
});
});
I'm trying to update a huge text document by deleting text that is dynamically received from an array. I cannot use readFileSync because the file is way too large so I have to stream it. The problem im encountering is the function deletes everything instead of only deleting what's in the array. Perhaps im not understanding how to properly delete something from a stream. How can this be done?
largeFile_example.txt
test_domain_1
test_domain_2
test_domain_3
test_domain_4
test_domain_5
test_domain_6
test_domain_7
test_domain_8
test_domain_9
test_domain_10
stream.js
const es = require('event-stream');
const fs = require('fs');
//array of domains to delete
var domains = ['test_domain_2','test_domain_6','test_domain_8'];
//loop
domains.forEach(function(domain){
//domain to delete
var dom_to_delete = domain;
//stream
var s = fs
.createReadStream('largeFile_example.txt')
.pipe(es.split())
.pipe(
es
.mapSync(function(line) {
//check if found in text
if(line === dom_to_delete){
//delete
var newValue = dom_to_delete.replace(line, '');
fs.createWriteStream('largeFile_example.txt', newValue, 'utf-8');
}
})
.on('error', function(err) {
console.log('Error while reading file.', err);
})
.on('end', function() {
//...do something
}),
);
})
You can simply use readline interface with the streams and you can read line by line. When you encounter any domain from the array just don't add it.
You can use for-of with async/await
const fs = require('fs');
const readline = require('readline');
async function processLine() {
const fileStream = fs.createReadStream('yourfile');
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity
});
// Note: crlfDelay recognize all instances of CR LF
// ('\r\n') in file as a single line break.
for await (const line of rl) {
// each line will be here as domain
// create a write stream and append it to the file
// line by line using { flag: a }
}
}
processLine();
To delete the domains from the existing file, you need to follow these steps:
Need to read the file as a stream.
Replace the text you don't want with the '' using regex or replace method.
add the updated content to the temp file or a new file.
There is no way you can read from one point and update the same line. I mean I am not aware of such a technique in Node.js(will be happy to know that). So that's why you need to create a new file and once updated remove the old file.
Maybe you can add some more value to how you code it as I am not sure why you want to do that. If your file is not large you can do that in-place, but your case is different.
I have two folders, input and output folder with many text files in the below format. How do I read all the files from the input folder,run the regex and write all the updated files to another output folder?I am using nodejs.
Input: $.Carpool[0].NoofSeats], [$.Carpool[1].NoofSeats]
So after replace with regex the updated text file should be:
Regex: str = str.replace(/\.[A-Z]/g, (m0) => m0.toLowerCase());
Output: [$.carpool[0].noOfSeats], [$.carpool[1].noOfSeats]
So far I got to reading files from the directory:
const fs= require("fs");
let directory = "Input" // Desktop/Input
let files = fs.readdirSync(directory)
console.log(files);
You want to loop through the files, assuming if the contents are a text file in UTF-8 format here is an example.
You use fs.readFile to read a specific file after listing directory.
Then use fs.writeFile to write a new file with contents.
I use /directory/${f} for the new file directory path and ${f} for filename that was opened.
const fs = require("fs");
// Directory
let directory = "/";
// Files
let files = fs.readdirSync(directory);
// Loop through the files
files.forEach(f => {
// Read the contents in UTF-8 format
fs.readFile(f, 'utf8', function(err, contents) {
if (err) { console.log(err); }
// Output Contents
console.log(contents);
// Perform regex here
contents = contents.replace(/\.[A-Z]/g, (m0) => m0.toLowerCase());
// Write new file to path /new, with contents
fs.writeFile(`/directory/${f}`, contents, function(err) {
if (err) {
// Error writing
return console.log(err);
}
console.log("The file was saved!");
});
});
});
Is it possible to look for files of given extension by the containing string provided?
What should my approach be? For example the input is txt and hello, and the output will be list of all files containing the string hello with extension txt.
You would write this in your terminal: node main.js hello
For a given directory it will search inside all subdirectories and all files for a text file with hello
Here is the code:
const { readdirSync, readFileSync, lstatSync } = require('fs');
const path = require('path');
const getDir = source => {
const results = readdirSync(source);
results.forEach(function (result) {
if (lstatSync(path.join(source, result))
.isFile()) {
if (readFileSync(path.join(source, result))
.includes(argument) && path.extname(result)
.toLowerCase() === extension) {
console.log("Your string is is in file: ", result)
}
}
else if (lstatSync(path.join(source, result))
.isDirectory()) {
getDir(path.join(source, result));
}
});
}
const dir = process.cwd();
const extension = '.txt'; //You can change the extension type here
let argument = process.argv[2];
getDir(dir);
You can use FileSystem to get the contents of a folder with the readdir method, this returns you an array of strings.
Let's say your working directory is a src file, in which there is a files folder that you want to read. Your script would look something like this:
const fs = require('fs');
var files = fs.readdir('./files', (err, filenames) => {
if (err) throw err;
return filenames;
})
You can then separate your string using string methods. Let's say you want to have two variables, fileNameand fileExt
You would use the String.split(separator) method, and use the . character as separator.
var files = // fs snippet above
for (file in files) {
let fileComponents = file.split('.');
let fileName = fileComponents[0];
let fileExt = fileComponents[1];
// You can run your code on the name, and extension of your file here.
}
This will not work for files containing multiple dots in their name. You will need extra work on the array to make sure fileName contains every string concatenated, separated by a . up until the final index of your fileComponents string
How we can check uploaded file extension in sails js?
I tried on skipper and multer but have no result.
any suggestion?
You should use saveAs options for each file before saving.
var md5 = require('md5');
module.exports = {
testUpload:function(req,res){
// setting allowed file types
var allowedTypes = ['image/jpeg', 'image/png'];
// skipper default upload directory .tmp/uploads/
var allowedDir = "../../assets/images";
// don not define dirname , use default path
req.file("uploadFiles").upload({
saveAs:function(file, cb) {
var d = new Date();
var extension = file.filename.split('.').pop();
// generating unique filename with extension
var uuid=md5(d.getMilliseconds())+"."+ extension;
// seperate allowed and disallowed file types
if(allowedTypes.indexOf(file.headers['content-type']) === -1) {
// save as disallowed files default upload path
cb(null,uuid);
}else{
// save as allowed files
cb(null,allowedDir+"/"+uuid);
}
}
},function whenDone(err,files){
return res.json({
files:files,
err:err
});
});
}
}
Just get uploaded files array and check last chunk of string after dot.
req.file('file').upload({
maxBytes: 2000000,
dirname: 'uploadFolder'
}, function (error, files) {
if (error) return sails.log.error(error);
// You have files array, so you can do this
files[0].fd.split('.').pop(); // You get extension
}
What is going on here? When upload is finished you will get array of files with their filenames. You can get data from that array and see where this file is located (full path).
The last thing is splitting string by dots and get last item from the array with pop() method.