I was trying to use neo4j at backend. First I want to import csv to neo4j. (first tried to see how many lines csv file has)
But having problem, the code is following
var neo4j = require('neo4j-driver').v1;
var driver = neo4j.driver("bolt://localhost", neo4j.auth.basic("neo4j", "neo4j"));
function createGraphDataBase(csvfilepath)
{
var session = driver.session();
return session
.run( 'LOAD CSV FROM {csvfilepath} AS line RETURN count(*)',
{csvfilepath}
)
.then(result => {
session.close();
console.log(' %d lines in csv.file', result);
return result;
})
.catch(error => {
session.close();
console.log(error);
return error;
});
}
the "csvfilepath" is the path of csv file, it is as follows.
'/Users/.../Documents/Project/.../test/spots.csv';
is there something wrong with giving path like this?
I am calling that function on other module as
var api = require('./neo4j.js');
const csvFile = path.join(__dirname,csvFileName);
api.createGraphDataBase(csvFile);
I am having error as
Error: Connection was closed by server
....
I am new to these, please help!
The URL that you specify in a LOAD CSV clause must be a legal URL.
As stated in this guide:
Make sure to use the right URLs esp. file URLs.+ On OSX and Unix use
file:///path/to/data.csv, on Windows, please use
file:c:/path/to/data.csv
In your case, csvfilepath needs to specify the file:/// protocol (since you seem to be running on OSX) for a local file. Based on your example, the value should be something like this:
'file:///Users/.../Documents/Project/.../test/spots.csv'
Related
I'm using WebMScore to render audio of music scores (it's a fork of MuseScore that runs in the browser or node).
I can successfully load my own, local .sf2 or .sf3 files, however
Trying to load an .sfz soundfont throws error 15424120. (And error.message is simply 'undefined'.)
Unlike .sf2 and .sf3, which contain the sounds and instructions in a single file, the .sfz format is just a text instruction file that refers to a separate folder of samples.
The reason I need the .sfz is that I need to be able to edit the .sfz file textually and programatically without an intervening Soundfont generator.
Is there a way to use .sfz's? Do I need to specify Zerberus (the Musescore .sfz player)? Do I need a different file structure? Please see below.
My environment is node js, with the following test case and file structure:
File Structure
Project Folder
app.js
testScore.mscz
mySFZ.sfz
samples
one.wav
two.wav
etc.wav
Test Case (Works with .sf3 , errors with .sfz)
const WebMscore = require('webmscore');
const fs = require('fs');
// free example scores available at https://musescore.com/openscore/scores
const name = 'testScore.mscz';
const exportedPrefix = 'exported';
const filedata = fs.readFileSync(`./${name}`);
WebMscore.ready.then(async () => {
const score = await WebMscore.load('mscz', filedata, [], false);
await score.setSoundFont(fs.readFileSync('./mySFZ.sfz'));
try { fs.writeFileSync(`./${exportedPrefix}.mp3`, await score.saveAudio('mp3')); }
catch (err) { console.log(err) }
score.destroy();
});
I'm using NodeJS on a VM. One part of it serves up pages, and another part is an API. I've run into a problem, where fs.createReadStream attempts to access a different path than what is being passed into the function. I made a small test server to see if it was something else in the server affecting path usage, for whatever reason, but it's happening on my test server as well. First, here's the code:
const fs = require('fs');
const path = require('path');
const csv = require('csv-parser');
const readCSV = (filename) => {
console.log('READ CSV GOT ' + filename); // show me what you got
return new Promise((resolve, reject) => {
const arr = [];
fs.createReadStream(filename)
.pipe(csv())
.on('data', row => {
arr.push(row);
})
.on('error', err => {
console.log(err);
})
.on('end', () => {
resolve(arr);
});
}
}
// tried this:
// const dir = path.relative(
// path.join('path', 'to', 'this', 'file),
// path.join('path', 'to', 'CONTENT.csv')
// );
// tried a literal relative path:
// const dir = '../data/CONTENT.csv';
// tried a literal absolute path:
// const dir = '/repo/directory/server/data/CONTENT.csv';
// tried an absolute path:
const dir = path.join(__dirname, 'data', 'CONTENT.csv');
const content = readCSV(dir)
.then(result => {console.log(result[0]);})
.catch(err => {console.log(err);});
...but any way I slice it, I get the following output:
READCSV GOT /repo/directory/server/data/CONTENT.csv
throw er; // Unhandled 'error' event
^
Error: ENOENT: no such file or directory, open '/repo/directory/data/CONTENT.csv'
i.e., is fs.createReadStream somehow stripping out the directory of the server, for some reason? I suppose I could hard code the directory into the call to createReadStream, maybe? I just want to know why this is happening.
Some extra: I'm stuck on node v8.11, can't go any higher. On the server itself, I believe I'm using older function(param) {...} instead of arrow functions -- but the behavior is exactly the same.
Please help!!
Code is perfect working.
I think you file CONTENT.csv should be in data folder like "/repo/directory/data/CONTENT.csv".
I'm answering my own question, because I found an answer, I'm not entirely sure why it's working, and at least it's interesting. To the best of my estimation, it's got something to do with the call stack, and where NodeJS identifies as the origin of the function call. I've got my server set up in an MVC pattern so my main app.js is in the root dir, and the function that's being called is in /controllers folder, and I've been trying to do relative paths from that folder -- I'm still not sure why absolute paths didn't work.
The call stack goes:
app.js:
app.use('/somepath', endpointRouter);
...then in endpointRouter.js:
router.get('/request/file', endpointController.getFile);
...then finally in endpointController.js:
const readCSV = filename => {
//the code I shared
}
exports.getFile = (req, res, next) => {
// code that calls readCSV(filename)
}
...and I believe that because Node views the chain as originating from app.js, it then treats all relative paths as relative to app.js, in my root folder. Basically when I switched to the super unintuitive single-dot-relative path: './data/CONTENT.csv', it worked with no issue.
I am using this code in my electron app to connect to an sftp server where I need to collect some data. I have no problem listing the files in the /out folder, but it fails to get the sftp file with 'deined permission' error. Ideally I would like to be able get() file and access the text data within directly in the function without storing to a file.
let Client = require('ssh2-sftp-client');
let sftp = new Client();
var root = '/out';
var today = new Date();
var mon = ((today.getMonth()+1) < 10)? "0" + (today.getMonth()+1) : (today.getMonth()+1);
var date = (today.getDate() < 10)? "0" + today.getDate() : today.getDate();
var fileDate = mon + date;
sftp.connect({
host: '<server-address>',
port: 2222,
username: 'XXXXXXXX',
password: 'xxxxxxxx',
privateKey: fs.readFileSync(path.join(__dirname, '../rsa/<file-name-here>.pem'))
})
.then(() => {
return sftp.list(root, 'SN5M' + fileDate);
})
.then((fileInfo) => {
if (fileInfo) {
var filePath = root + '/' + fileInfo[fileInfo.length - 1].name;
return sftp.get(filePath).then((file) => {
console.log(file);
event.returnValue = file;
sftp.end();
})
.catch((err) => {
console.log('File get error', err);
event.returnValue = err;
sftp.end();
});
}
})
.catch((err) => {
console.log('File info error', err);
event.returnValue = err;
sftp.end();
});
Try this and see if it works or not
'get' returns (String|Stream|Buffer).
let dst = fs.createWriteStream('/local/file/path/data.txt');
sftp.get(filePath,dst)
Refer https://www.npmjs.com/package/ssh2-sftp-client#orga0dfcd5
Looking at your code, you have two problems.
If you call get() with only 1 argument, it returns a buffer, not a file. To get the file, just do
client.get(sourceFilePath, localFilePath)
and the file will be saved locally as localFilePath. Both arguments are strings and need to be full paths i.e. include the filename, not just the directory. The filename for the second argument can be different from the first. However, if all you want is to retrieve the file, you are better off using fastGet() rather than get(). The get() method is good for when you want to do something in the code with the data e.g. a buffer or write stream piping/processing. The fastGet() method is faster than get() as it does the transfer using concurrent processes, but does not permit use of buffers or streams for further processing.
The error message you are seeing is either due to the way you are calling get() or it is an indication you don't have permission to read the file your trying to access (as the user your connected with). Easiest way to check this is to use the openSSH sftp program (available on Linux, mac and windows) and the key your using (use the -i switch) to try and download the file. If it fails with a permission error, then you know it is a permission error and not a problem with your code or ssh2-sftp-client module.
EDIT: I just noticed you are also using both a password and a key file. You don't need both - either one will work, but you don't need to use both. I tend to use a keyfile when possible as it avoids having to have a password stored somewhere. Make sure not to add a passphrase to your key. Alternatively, you can use something like the dotenv module and store your credentials and other config in a .env file which you do not check into version control.
This a repeat question (not yet answered) but I have revised and tightened up the code. And, I have included the specific example. I am sorry to keep beating this drum, but I need help.
This is a Node API. I need to read and write JSON data. I am using the Node core module 'fs', not the npm package by the same name (or fs-extra). I have extracted the particular area of concern onto a standalone module that is shown here:
'use strict';
/*==================================================
This service GETs the list of ids to the json data files
to be processed, from a json file with the id 'ids.json'.
It returns and exports idsList (an array holding the ids of the json data files)
It also calls putIdsCleared to clear the 'ids.json' file for the next batch of processing
==================================================*/
// node modules
const fs = require('fs');
const config = require('config');
const scheme = config.get('json.scheme')
const jsonPath = config.get('json.path');
const url = `${scheme}${jsonPath}/`;
const idsID = 'ids.json';
const uri = `${url}${idsID}`;
let idsList = [];
const getList = async (uri) => {
await fs.readFile(uri, 'utf8', (err, data) => {
if (err) {
return(console.log( new Error(err.message) ));
}
return jsonData = JSON.parse(data);
})
}
// The idea is to get the empty array written back to 'ids.json' before returning to 'process.js'
const clearList = async (uri) => {
let data = JSON.stringify({'ids': []});
await fs.writeFile(uri, data, (err) => {
if (err) {
return (console.log( new Error(err.message) ));
}
return;
})
}
getList(uri);
clearList(uri)
console.log('end of idsList',idsList);
module.exports = idsList;
Here is the console output from the execution of the module:
Error: ENOENT: no such file or directory, open 'File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json'
at ReadFileContext.fs.readFile [as callback]
(/Users/doug5solas/sandbox/libertyMutual/server/.playground/ids.js:24:33)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:235:13)
Error: ENOENT: no such file or directory, open 'File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json'
at fs.writeFile
(/Users/doug5solas/sandbox/libertyMutual/server/.playground/ids.js:36:34)
at fs.js:1167:7
at FSReqWrap.oncomplete (fs.js:141:20)
I am being told there is no such file or directory. However I can copy the uri (as shown in the error message)
File:///Users/doug5solas/sandbox/libertyMutual/server/api/ids.json
into the search bar of my browser and this is what is returned to me:
{
"ids": [
"5sM5YLnnNMN_1540338527220.json",
"5sM5YLnnNMN_1540389571029.json",
"6tN6ZMooONO_1540389269289.json"
]
}
This result is the expected result. I do not "get" why I can get the data manually but I cannot get it programmatically, using the same uri. What am I missing? Help appreciated.
Your File URI is in the wrong format.
It shouldn't contain the File:// protocol (that's a browser-specific thing).
I'd imagine you want C://Users/doug5solas/sandbox/libertyMutual/server/api/ids.json.
I solved the problem by going to readFileSync. I don't like it but it works and it is only one read.
Am a new user of MonogoDB n Nodejs. Building a simple product catalogue. Trying to import CSV file contents to mongodb. I got Syntax error:: Unexpected identifier error: Is there any way to resolve this?
I am curious to know about using ya-csv package available on Git. Can anyone help?
Try as below:
npm install ya-csv //install package
//Put this code in your .js file
var csv = require("ya-csv");
var validRows = [];
var reader = csv.createCsvFileReader("/csv/file.csv", {columnsFromHeader:true,'separator': ',' , encoding: "utf8"});
reader.addListener('data', function(data) { //if rows are in correct format then it will push into array
if(data){
validRows.push(data);
}
});
reader.addListener('end', function(){ //this method will be called after reading all csv rows
validRows.forEach(function(data){
//console.log(data.name); //get your data like this
});
});
reader.addListener('error', function(err){
console.log(err); //if any error occurred
});
For further detail please follow this link
https://github.com/koles/ya-csv