Getting error while reading json file using node.js - node.js

I am getting the following error while reading the json file using Node.js. I am explaining my code below.
SyntaxError: Unexpected token # in JSON at position 0
at JSON.parse (<anonymous>)
My json file is given below.
test.json:
#PATH:/test/
#DEVICES:div1
#TYPE:p1
{
name:'Raj',
address: {
city:'bbsr'
}
}
This json file has some # included strings . Here I need to remove those # included string from this file. I am explaining my code below.
fs.readdirSync(`${process.env['root_dir']}/uploads/${fileNameSplit[0]}`).forEach(f => {
console.log('files', f);
let rawdata = fs.readFileSync(`${process.env['root_dir']}/uploads/${fileNameSplit[0]}/${f}`);
let parseData = JSON.parse(rawdata);
console.log(parseData);
});
Here I am trying to read the code first but getting the above error. My need is to remove those # included lines from the json file and then read all the data and convert the removed lines to object like const obj ={PATH:'/test/',DEVICES:'div1',TYPE:p1}. Here I am using node.js fs module to achive this.

As you said, you need to remove those # lines from the JSON file. You need to code this yourself. To help with that, read the file into a string and not a Buffer by providing a charset to readFileSync.
const text = fs.readFileSync(path, 'utf8');
console.log(text);
const arr = raw.split("\n");
const noComments = arr.filter(x => x[0] !== "#"));
const filtered = noComments.join("\n");
const data = JSON.parse(filtered);
console.log(data);

Related

Cannot read cyrillic symbols from a .csv file

I need to read some .csv file, get data in .json format and work with it.
I'm using npm package convert-csv-to-json. As a result - cyrillic symbols aren't displaying properly:
const csvToJson = require('convert-csv-to-json');
let json = csvToJson.fieldDelimiter(',').getJsonFromCsv("input.csv");
console.log(json);
Result:
If I try to decode file:
const csvToJson = require('convert-csv-to-json');
let json = csvToJson.asciiEncoding().fieldDelimiter(',').getJsonFromCsv("input.csv");
console.log(json);
result is:
When I open a .csv file using AkelPad or notepad++ - it displays as it has to, and detected format is Win 1251 (ANSI - кириллица).
Is there a way to read a file with properly encoding, or to decode a result string?
Try using UTF-8 encoding instead of ASCII.
As a result, change
let json = csvToJson.asciiEncoding().fieldDelimiter(',').getJsonFromCsv("input.csv");
to
let json = csvToJson.utf8Encoding().fieldDelimiter(',').getJsonFromCsv("input.csv");
This is a code to solve the problem:
const fs = require('fs');
var iconv = require('iconv-lite');
const Papa = require('papaparse');
// read csv file and get buffer
const buffer = fs.readFileSync("input.csv");
// parse buffer to string with encoding
let dataString = iconv.decode(buffer, 'win1251');
// parse string to array of objects
let config = {
header: true
};
const parsedOutput = Papa.parse(dataString, config);
console.log('parsedOutput: ', parsedOutput);

Reading file using Node.js "Invalid Encoding" Error

I am creating an application with Node.js and I am trying to read a file called "datalog.txt." I use the "append" function to write to the file:
//Appends buffer data to a given file
function append(filename, buffer) {
let fd = fs.openSync(filename, 'a+');
fs.writeSync(fd, str2ab(buffer));
fs.closeSync(fd);
}
//Converts string to buffer
function str2ab(str) {
var buf = new ArrayBuffer(str.length*2); // 2 bytes for each char
var bufView = new Uint16Array(buf);
for (var i=0, strLen=str.length; i < strLen; i++) {
bufView[i] = str.charCodeAt(i);
}
return buf;
}
append("datalog.txt","12345");
This seems to work great. However, now I want to use fs.readFileSync to read from the file. I tried using this:
const data = fs.readFileSync('datalog.txt', 'utf16le');
I changed the encoding parameter to all of the encoding types listed in the Node documentation, but all of them resulted in this error:
TypeError: Argument at index 2 is invalid: Invalid encoding
All I want to be able to do is be able to read the data from "datalog.txt." Any help would be greatly appreciated!
NOTE: Once I can read the data of the file, I want to be able to get a list of all the lines of the file.
Encoding and type are an object:
const data = fs.readFileSync('datalog.txt', {encoding:'utf16le'});
Okay, after a few hours of troubleshooting a looking at the docs I figured out a way to do this.
try {
// get metadata on the file (we need the file size)
let fileData = fs.statSync("datalog.txt");
// create ArrayBuffer to hold the file contents
let dataBuffer = new ArrayBuffer(fileData["size"]);
// read the contents of the file into the ArrayBuffer
fs.readSync(fs.openSync("datalog.txt", 'r'), dataBuffer, 0, fileData["size"], 0);
// convert the ArrayBuffer into a string
let data = String.fromCharCode.apply(null, new Uint16Array(dataBuffer));
// split the contents into lines
let dataLines = data.split(/\r?\n/);
// print out each line
dataLines.forEach((line) => {
console.log(line);
});
} catch (err) {
console.error(err);
}
Hope it helps someone else with the same problem!
This works for me:
index.js
const fs = require('fs');
// Write
fs.writeFileSync('./customfile.txt', 'Content_For_Writing');
// Read
const file_content = fs.readFileSync('./customfile.txt', {encoding:'utf8'}).toString();
console.log(file_content);
node index.js
Output:
Content_For_Writing
Process finished with exit code 0

How do I export the data generated by this code to a CSV file in puppeteer?

I need to export the data generated by this code into a CSV file. I am new to node.js/puppeteer so I am struggling on generating a CSV file.
I understand I can use the fs write function and tried adding this to the end of my code to no avail:
const fs = require('fs');
const csv = await page.$$eval('.product_desc_txt', function(products){
// Iterate over product descriptions
let csvLines = products.map(function(product){
// Inside of each product find product SKU and its price
let productId = document.querySelector(".custom-body-copy").innerText.trim();
let productPrice = document.querySelector("span[data-wishlist-linkfee]").innerText.trim();
// Fomrat them as a csv line
return `${productId};${productPrice}`;
});
// Join all lines into one file
return csvLines.join("\n");
});
fs.writeFileSync("test.csv", csv)
});
You've got csv with data from puppeteer, but don't use it. Just write the data to file:
fs.writeFileSync("test.csv", csv);
Also writing to file this
'${productId};${productPrice}'
won't work, there are no such variables at that place and even if there were, the correct way to format variables into a string is with backticks:
`${productId};${productPrice}`

JSON.parse is not a function in node js

I'm new in node js and I'm trying to parse a JSON. This is what I've done so far:
const JSON = require("nodemon/lib/utils");
...
someFunction() {
let sens = [];
// sensor is the result of fs.readFileSync call and file is a json array
sens = JSON.parse(sensors);
It throws me:
JSON.parse is not a function.
How can I solve it?
Just comment out the very first line. JSON is built in already.
//const JSON = require("nodemon/lib/utils");
JSON is kind of a global object. It doesn't need to be required. So, just remove the line where JSON is required and it'll work fine.

Why am I getting a NOENT using Node core module 'fs'

This a repeat question (not yet answered) but I have revised and tightened up the code. And, I have included the specific example. I am sorry to keep beating this drum, but I need help.
This is a Node API. I need to read and write JSON data. I am using the Node core module 'fs', not the npm package by the same name (or fs-extra). I have extracted the particular area of concern onto a standalone module that is shown here:
'use strict';
/*==================================================
This service GETs the list of ids to the json data files
to be processed, from a json file with the id 'ids.json'.
It returns and exports idsList (an array holding the ids of the json data files)
It also calls putIdsCleared to clear the 'ids.json' file for the next batch of processing
==================================================*/
// node modules
const fs = require('fs');
const config = require('config');
const scheme = config.get('json.scheme')
const jsonPath = config.get('json.path');
const url = `${scheme}${jsonPath}/`;
const idsID = 'ids.json';
const uri = `${url}${idsID}`;
let idsList = [];
const getList = async (uri) => {
await fs.readFile(uri, 'utf8', (err, data) => {
if (err) {
return(console.log( new Error(err.message) ));
}
return jsonData = JSON.parse(data);
})
}
// The idea is to get the empty array written back to 'ids.json' before returning to 'process.js'
const clearList = async (uri) => {
let data = JSON.stringify({'ids': []});
await fs.writeFile(uri, data, (err) => {
if (err) {
return (console.log( new Error(err.message) ));
}
return;
})
}
getList(uri);
clearList(uri)
console.log('end of idsList',idsList);
module.exports = idsList;
Here is the console output from the execution of the module:
Error: ENOENT: no such file or directory, open 'File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json'
at ReadFileContext.fs.readFile [as callback]
(/Users/doug5solas/sandbox/libertyMutual/server/.playground/ids.js:24:33)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:235:13)
Error: ENOENT: no such file or directory, open 'File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json'
at fs.writeFile
(/Users/doug5solas/sandbox/libertyMutual/server/.playground/ids.js:36:34)
at fs.js:1167:7
at FSReqWrap.oncomplete (fs.js:141:20)
I am being told there is no such file or directory. However I can copy the uri (as shown in the error message)
File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json
into the search bar of my browser and this is what is returned to me:
{
"ids": [
"5sM5YLnnNMN_1540338527220.json",
"5sM5YLnnNMN_1540389571029.json",
"6tN6ZMooONO_1540389269289.json"
]
}
This result is the expected result. I do not "get" why I can get the data manually but I cannot get it programmatically, using the same uri. What am I missing? Help appreciated.
Your File URI is in the wrong format.
It shouldn't contain the File:// protocol (that's a browser-specific thing).
I'd imagine you want C://Users/doug5solas/sandbox/libertyMutual/server/api/ids.json.
I solved the problem by going to readFileSync. I don't like it but it works and it is only one read.

Resources