Random error when piping twice http request - node.js

I have this code:
var file = fs.createWriteStream(_filename, _saveFile);
var request = http.request(options);
var decrypt = Crypto.createDecipheriv('aes-128-cbc', key, iv);
var _saveFile = function(response) {
res.pipe(decrypt).pipe(file);
res.on("end", function() {
file.close();
});
};
file.on("error", function(err) {
console.log("Error while writing file", err);
});
It works ok. But randomly I'm receiving this error:
{ [Error: EBADF, write] errno: 9, code: 'EBADF' }
I read this is because the file is being written after it was closed. So, the question I have is: Am I doing something wrong? Is there any way to do decrypt.on("end")instead of res.on("end")?
MORE INFORMATION
In the same res.on("end") I have a q.defer().resolve call, because this method is downloading several .ts files and then I need to turn them into a MP4.
As #mscdex said, there is no need to close the file, however, if the promise is resolved before the file has been actually finished then, I may be working with a corrupted file. Like this:
Download 1.ts
Download 2.ts
Download 3.ts
Here all the promises are saved in a an array and then I use q.spread to wait for all of them
File 1.ts writted
File 1.ts resolve promise
File 2.ts writted
File 2.ts resolve promise
File 3.ts resolve promise
q.spread knows all the promises were resolved so it starts working with the .ts files
File 3.ts writted
From this point on, I'll be working with a wrong/incomplete file.

You don't need to explicitly call file.close() since the file stream will be closed automatically when upstream ends.
If you need to wait until the file is completely written, listen for the finish event on file:
file.on('finish', function() {
// resolve promise or do whatever
});

Related

How to read the contents of a file onto the console using Nodejs

I am going back to learning js after many years off and all i want to do is read the contents of a file onto the console using Nodejs. I found the sample code. Nice and simple. I have spent over an hour trying to figure out why it will not find the file. This is sample right off the documentation and i made it exactly like the example to debug it. The absolute only difference is the name joe is replaced with my user folder.
const fs = require('fs')
fs.readFile('/Users/gendi/test.txt', 'utf8' , (err, data) => {
if (err) {
console.error(err)
return
}
console.log(data)
})
It runs fine except it will not find test.text. no matter what. I receive the following error and no matter how i format the file path. Nothing.
C:\Users\gendi>node readfile.js
[Error: ENOENT: no such file or directory, open 'C:\Users\gendi\test.txt'] {
errno: -4058,
code: 'ENOENT',
syscall: 'open',
path: 'C:\\Users\\gendi\\pcsSnipe\\test.txt'
}
You can also only pass in the file path as 'test.txt' and the exact same results come up. on the first part of the error msg the path looks formatted correctly but on the last line of the error msg it is not? Its been years.. so i know i am missing something really simple. I assure that file is there!! Thank you in advance and forgive my ineptness.
The fs module requires an exact path to the file you'd like to read. A simple fix to this would be to add __dirname which will return the directory path along with the path of your file.
// Define modules
const fs = require('fs');
const path = require('path');
// Read the file
fs.readFile(path.join(__dirname, '/Users/gendi/test.txt'), 'utf8' , (err, data) => {
if (err) // If FS returned an error
return console.error(err); // Log the error and return
console.log(data); // If the reading was successful, log the data
});
It works if you remove the file extention, '.txt' . Idk why it make a differnce. maybe the "." is throwing it off but it doesn't matter in this respect. Thank you

No such file or directory when piping from request in Node

I need to request a .zip file from a URL then pass the contents to AdmZip
When attempting to pipe the output of the request library:
const zipFilePath = path.join(batchPath, this.zipFile.filename);
const out = fs.createWriteStream(zipFilePath);
const req = request.get(this.zipFile.url);
req.pipe(out);
req.on('end', function() {
console.log("I should be here, but I'm not");
});
I receive:
Error: ENOENT: no such file or directory, open
'C:\Users\Brandon\work\keystone4-projects\html-email\batch-content\5996588a3bc30010502bfa9e\test.zip'
What am I doing wrong?
Edit:
I added:
if (!fs.existsSync(batchPath)) {
fs.mkdirSync(batchPath);
}
before attempting to pipe the output and my function completed successfully.
Typically when you receive this error when writing a file, it means that the path leading up to the file being written does not exist.

Electron app createWriteStream throwing ENOENT error

I'm trying to download files to the filesystem in an electron app. My code, in the main thread, looks like this:
const dir = `${__dirname}/media`;
if (!fs.existsSync(dir)){
fs.mkdirSync(dir);
}
const file = fs.createWriteStream(`${dir}/${name}`);
file.on("open", function() {
const request = http.get(url, function(response) {
response.pipe(file);
response.on('end', function() {
file.close();
...
});
});
request.on('error', function(err) {
...
});
});
This works when running in development using electron . But after I build it with electron-builder, I get the error in an alert:
Uncaught Exception:
Error: ENOENT, media/uploads_2016_02_BASF_Holistic_Program.jpg not found in /Users/nicholasstephan/Desktop/XXXXXXX/dist/Mac/XXXXXX.app/Contents/Resources/app.asar
at notFoundError (ELECTRON_ASAR.js:109:19)
at Object.module.(anonymous function) [as open] (ELECTRON_ASAR.js:209:16)
at WriteStream.open (fs.js:1890:6)
at new WriteStream (fs.js:1876:10)
at Object.fs.createWriteStream (fs.js:1831:10)
at next (/Users/nicholasstephan/Desktop/XXXXXXXX/dist/Mac/XXXXXXXX.app/Contents/Resources/app.asar/media.js:19:18)
at /Users/nicholasstephan/Desktop/XXXXXXXX/dist/Mac/XXXXXXXX.app/Contents/Resources/app.asar/media.js:52:4
...
where the media.js, ln 19, being referred to is the const file = fs.createWriteStream(${dir}/${name}); line in the code.
I've tried the solutions offered in about a dozen other similar stackoverflow answers, but none have fixed the problem.
What's going on here?
Thanks.
The built Electron app uses the Asar format. Asar is an archive format (it's really just one big file) though in Electron you are able to read from it as if it were a standard directory.
I presume (though I have not seen it explicitly documented) that it is not possible to write to an Asar with the fs functions. In any case there are almost certainly more appropriate locations to write data.
Try writing to a different path. Electron provides a number of useful paths using app.getPath(name) so you could for example write to the userData directory which holds configuration files for your app.

error ENOENT,open '/tmp/45e85388793de' in nodejs

I am trying to save project and its file in GridFS. I want to save project first and using "_id" of project as metadata for file I want to save file. When i tried so i am getting ENOENT, open '/tmp/45e85388793de' error. here is my code
newProject.save(function (err,project) {
if (err) {
console.log('save error', err);
}
console.log("project added");
var id=poject._id;
var filepath = req.files.file.path;
var filename = req.files.file.name;
var writestream = gfs.createWriteStream({ filename: filename, metadata:id });
console.log(filepath);
fs.createReadStream(filepath)
.on('end', function() {
})
.on('error', function(err) {
console.log("error encountered"+err);//ENOENT,open error
})
.pipe(writestream);
});
Why i am getting this error and how to resolve it?
ENOENT in this context means "No such file or directory." It means the filepath you are trying to read with createReadStream does not exist.
I think you are getting this error since :
Your file is saved in a temporary location.
When you are inside the callback function your file is removed from that location and you are getting "No such file" error. Path and other variables still exists as part of js and that's why you are able to print them in console.
Solution:
Above(Outside) callback function move your file to some other permanent location using:
fs.rename(req.files.file.path, "./someKnownPath/filename");
Keep note of that location. In your callback function use the new location as path and try saving the file in gridfs. Once the file is saved you may delete it file from that location(/someKnownPath/filename).
This error was occuring for me as well. And the reason was temp directory was not in place. After I created manually and gave a try, it worked.
Now I have shifted to creating directory on the fly through node.js itself.

Unable to read a saved file in heroku

I am using NodeJS on heroku.
I read a file from another server and save it into my application in the /temp directory.
Next, I read the same file to pass it on to my client.
The code that saves the file and then subsequently reads it is:
http.request(options, function (pdfResponse) {
var filename = Math.random().toString(36).slice(2) + '.pdf',
filepath = nodePath.join(process.cwd(),'temp/' + filename);
pdfResponse.on('end', function () {
fs.readFile(filepath, function (err, contents) {
//Stuff to do after reading
});
});
//Read the response and save it directly into a file
pdfResponse.pipe(fs.createWriteStream(filepath));
});
This works well on my localhost.
However, when deployed to heroku, I get the following error:
events.js:72
throw er; // Unhandled 'error' event
Error: ENOENT, open '/app/temp/nvks0626yjf0qkt9.pdf'
Process exited with status 8
State changed from up to crashed
I am using process.cwd() to ensure that the path is correctly used. But even then it did not help. As per the heroku documentation, I am free to create files in the applications directory, which I am doing. But I can't figure out why this is failing to read the file...
The error you describe there is consistent with /app/temp/ not existing. You need to create it before you start writing in it. The idea is:
var fs = require("fs");
var path = require("path");
var temp_dir = path.join(process.cwd(), 'temp/');
if (!fs.existsSync(temp_dir))
fs.mkdirSync(temp_dir);
I've used the sync version of the calls for illustrations purposes only. This code should be part of the start up code for your app (instead of being called for each request) and how you should structure it depends on your specific application.

Resources