Im wondering if is posible to know what is the file name of an incomming binary request.
This is my situation I have this code that handles the file upload
router.route('/:filename')
.put(function(req,res){
var uuid = guid();
var fileExtension = req.params.filename.substring(req.params.filename.lastIndexOf("."));
if(!fs.existsSync('../files')){
fs.mkdirSync('../files')
}
var newFile = fs.createWriteStream('../files/'+uuid+fileExtension);
req.pipe(newFile);
req.on('end',function(end){
console.log("Finished")
res.send(uuid+fileExtension)
})
})
as you can see now ,I need the file name specified in the URL('/:filename'). My question is: If it is possible to take that attribute from the resquest stream, instead the url or a form key?
If you use multer middleware you can access the uploaded filename like so
var multer = require('multer')
var upload = multer()
router.route('/:filename')
.put(upload.single('fileField'),function(req,res){
var fileName = req.file.originalname
var uuid = guid();
var fileExtension = req.params.filename.substring(req.params.filename.lastIndexOf("."));
if(!fs.existsSync('../files')){
fs.mkdirSync('../files')
}
var newFile = fs.createWriteStream('../files/'+uuid+fileExtension);
req.pipe(newFile);
req.on('end',function(end){
console.log("Finished")
res.send(uuid+fileExtension)
})
})
You'll need to inspect the Content-Disposition header of the request and parse the file name information out if processing the HTTP request manually.
However, I'd recommend you look at some of the existing file upload middlewares, no point in reinventing the wheel
busboy
multer
formidable
multiparty
pez
Related
At my endpoint in my NodeJS server, after retrieving an audio file stored as a Buffer in MongoDB, I want to represent it with a URL (much like how you do with URL.createObjectURL(blob) in the frontend on the browser). I then plan to res.render() the URL in HTML through Handlebars on the client, so that the user can click on it to download it:
<a href={{url}}>Click me to download the file!</a>
In the NodeJs server, I have converted the MongoDB Buffer into a JavaScript ArrayBuffer through:
var buffer = Buffer.from(recordingFiles[0].blobFile);
var arrayBuffer = Uint8Array.from(buffer).buffer;
I am unsure where to proceed from here. I seen solutions using fs or res.download(), but they don't seem applicable to my situation. Thanks in advance for any help!
Hopefully this can help.
var blob = new Blob(BUFFER, {type: "audio mime type"});
var link = document.createElement('a');
link.href = window.URL.createObjectURL(blob);
var fileName = reportName;
link.download = fileName;
link.click();
Do you always need to preload the audio file onto the page?
If not, then I would advise you to add a separate endpoint to download the file on demand. The frontend link can send a get request to the endpoint and download the file only if the user clicked it.
Otherwise you'd always be downloading the buffer behind the scenes, even if the user didn't intend to download it. This is especially problematic on slow connections.
Frontend:
<a href={{`${baseUrl}/download/${audioId}`}}>Click me to download the file!</a>
Backend:
const stream = require('stream');
app.get('/download/:audioId', function (request, response) {
// Retrieve the tag from our URL path
const audioId = request.params.audioId;
const fileData; // TODO: Get file buffer from mongo.
const fileContents = Buffer.from(fileData, 'base64');
const readStream = new stream.PassThrough();
readStream.end(fileContents);
response.set('Content-disposition', 'attachment; filename=' + fileName);
response.set('Content-Type', '<your MIME type here>');
readStream.pipe(response);
});
A list of relevant MIME types can be found here.
I've set up a small node js BE app, built with express and fastCsv module on top of it. The desired outcome would be to be able to download a csv file to the client side, without storing it anywhere inside the server, since the data is generated depending on user criteria.
So far I've been able to get somewhere it it, Im using streams, since that csv file could be pretty large depending on the user selection. Im pretty sure something is missing inside the code bellow:
const fs = require('fs');
const fastCsv = require('fast-csv');
.....
(inside api request)
.....
router.get('/', async(req, res) => {
const gatheredData ...
const filename = 'sometest.csv'
res.writeHead(200, {
'Content-Type': 'text/csv',
'Content-Disposition': 'attachment; filename=' + filename
})
const csvDataStream = fastCsv.write(data, {headers: true}).pipe(res)
})
The above code 'works' in some way as it does deliver back the response, but not the actual file, but the contents of the csv file, which I can view in the preview tab as a response. To sum up, Im trying to stream in that data, into a csv and push it to download file to client, and not store it on the server. Any tips or pointers are very much appreciated.
Here's what worked for me after created a CSV file on the server using the fast-csv package. You need to specify the full, absolute directory path where the output CSV file was created:
const csv = require("fast-csv");
const csvDir = "abs/path/to/csv/dir";
const filename = "my-data.csv";
const csvOutput = `${csvDir}/${filename}`;
console.log(`csvOutput: ${csvOutput}`); // full path
/*
CREATE YOUR csvOutput FILE USING 'fast-csv' HERE
*/
res.type("text/csv");
res.header("Content-Disposition", `attachment; filename="${filename}"`);
res.header("Content-Type", "text/csv");
res.sendFile(filename, { root: csvDir });
You need to make sure to change the response content-type and headers to "text/csv", and try enclosing the filename=... part in double-quotes, like in the above example.
I am sending audio data stored as a blob to my backend (node/express). When I save the file as .wav and attempt to use in the SpeechRecogition package in python it throws an error saying the "file does not start with RIFF id". So how can I add the headers to my blob file before I save it so that it is a correctly formatted .wav file? I can provide the code if necessary.
node.js file
var multer = require('multer');
var fs = require('fs'); //use the file system so we can save files
var uniqid = require('uniqid');
var spawn = require('child_process').spawn;
const storage = multer.memoryStorage()
var upload = multer({ storage: storage });
router.post('/api/test', upload.single('upl'), function (req, res) {
console.log(req.file);
console.log(req.file.buffer);
var id = uniqid();
fs.writeFileSync(id+".wav", Buffer.from(new Uint8Array(req.file.buffer))); //write file to server as .wav file
const scriptPath = 'handleAudio.py'
const process = spawn('python3', [__dirname+"/../"+scriptPath, "/home/bitnami/projects/sample/"+id+".wav", req.file.originalname, 'True']); //throws error about header in .wav
});
Also I had this same example working with a php endpoint that just saved the blob to a file with .wav extension and the python file accepted it. What could be different in the move_uploaded_file in php and what I am doing above with node?
Every .wav file needs a header specified by the WAVE file format, available here. While it's fine for you to build the header yourself, it's much easier to just use a proper lib to do the work for you.
One example is node-wav, which has a nice API to write WAVE files from raw PCM data (what you have at the moment). Example code is provided by the node-wav documentation.
I want to have persistent memory (store the user's progress) in a .json file in %AppData%. I tried doing this according to this post, but it doesn't work. For testing purposes I'm only working with storing one object.
The code below doesn't work at all. If I use fs.open(filePath, "w", function(err, data) { ... instead of readFile(..., it does create a json file in %AppData%, but then it doesn't write anything to it, it's always 0 bytes.
var nw = require('nw.gui');
var fs = require('fs');
var path = require('path');
var file = "userdata.json";
var filePath = path.join(nw.App.dataPath, file);
console.log(filePath); // <- This shows correct path in Application Data.
fs.readFile(filePath ,function (err, data) {
var idVar = "1";
var json = JSON.parse(data);
json.push("id :" + idVar);
fs.writeFile(filePath, JSON.stringify(json));
});
If anyone has any idea where I'm messing this up, I'd be grateful..
EDIT:
Solved, thanks to kailniris.
I was simply trying to parse an empty file
There is no json in the file you try to read. Before parsing data check if the file is empty. If it is then create an empty json, push the new data into it then write it to the file else parse the json in the file.
The request documentation talks about importing cookies from a file with the following example:
var FileCookieStore = require('tough-cookie-filestore');
// NOTE - currently the 'cookies.json' file must already exist!
var j = request.jar(new FileCookieStore('cookies.json'));
request = request.defaults({ jar : j })
request('http://www.google.com', function() {
request('http://images.google.com')
})
However, as noted in the comment, it expects cookies.json to already exist. The question is, if I have an exsting jar with cookies in it, how can I export it to JSON?
I am not sure to understand what you mean by "if I have an exsting jar with cookies in it", but here is how I manage persistent cookies with nodejs.
To avoid errors with FileCookieStore, I add a piece of code to create the json file if it does not exist. The file can be empty, as long as it exists:
if(!fs.existsSync(cookiepath)){
fs.closeSync(fs.openSync(cookiepath, 'w'));
}
Now, if you look closely at the FileCookieStore code, you will see it calls the saveToFile method anytime there is a change in the cookies. It means that by passing a FileCookieStore object to the request module (using the jar option as the request documentation explains), the json file will always reflect the state of the cookies.
Here is a complete example:
var FileCookieStore = require('tough-cookie-filestore');
var request = require('request');
var fs = require("fs");
var cookiepath = "cookies.json";
// create the json file if it does not exist
if(!fs.existsSync(cookiepath)){
fs.closeSync(fs.openSync(cookiepath, 'w'));
}
// use the FileCookieStore with the request package
var jar = request.jar(new FileCookieStore(cookiepath));
request = request.defaults({ jar : jar });
// do whatever you want
request('http://www.google.com', function() {
request('http://images.google.com')
});
// the cookies in 'jar' corresponds to the cookies in cookies.json
console.log(jar);
To start anew, simply delete the cookipath file.