I am sending audio data stored as a blob to my backend (node/express). When I save the file as .wav and attempt to use in the SpeechRecogition package in python it throws an error saying the "file does not start with RIFF id". So how can I add the headers to my blob file before I save it so that it is a correctly formatted .wav file? I can provide the code if necessary.
node.js file
var multer = require('multer');
var fs = require('fs'); //use the file system so we can save files
var uniqid = require('uniqid');
var spawn = require('child_process').spawn;
const storage = multer.memoryStorage()
var upload = multer({ storage: storage });
router.post('/api/test', upload.single('upl'), function (req, res) {
console.log(req.file);
console.log(req.file.buffer);
var id = uniqid();
fs.writeFileSync(id+".wav", Buffer.from(new Uint8Array(req.file.buffer))); //write file to server as .wav file
const scriptPath = 'handleAudio.py'
const process = spawn('python3', [__dirname+"/../"+scriptPath, "/home/bitnami/projects/sample/"+id+".wav", req.file.originalname, 'True']); //throws error about header in .wav
});
Also I had this same example working with a php endpoint that just saved the blob to a file with .wav extension and the python file accepted it. What could be different in the move_uploaded_file in php and what I am doing above with node?
Every .wav file needs a header specified by the WAVE file format, available here. While it's fine for you to build the header yourself, it's much easier to just use a proper lib to do the work for you.
One example is node-wav, which has a nice API to write WAVE files from raw PCM data (what you have at the moment). Example code is provided by the node-wav documentation.
Related
I have an express service that's taking a pdf file from my front-end and saving it to an s3 bucket. I'm running into issues trying to take the file and create a stream so that I can then pass that to the s3 upload function. I'm trying to avoid writing the file to disc so I don't think I can use fs.createReadStream() but I can't seem to find an alternative way to do it..
router.post('/upload', upload.single('my-pdf'), async (req, res, next) {
const file = req.file;
// Needs a file path not an actual file
const stream = fs.createReadStream(file);
return s3.upload(file).promise();
}
Any help or advice on how to get around this would be greatly appreciated.
Assuming that req.file.<name_of_upload_field> is a buffer holding the file contents, you can convert that to a readable stream via
var str = new stream.PassThrough();
str.end(req.file.<name_of_upload_field>);
return s3.upload(str).promise();
I've set up a small node js BE app, built with express and fastCsv module on top of it. The desired outcome would be to be able to download a csv file to the client side, without storing it anywhere inside the server, since the data is generated depending on user criteria.
So far I've been able to get somewhere it it, Im using streams, since that csv file could be pretty large depending on the user selection. Im pretty sure something is missing inside the code bellow:
const fs = require('fs');
const fastCsv = require('fast-csv');
.....
(inside api request)
.....
router.get('/', async(req, res) => {
const gatheredData ...
const filename = 'sometest.csv'
res.writeHead(200, {
'Content-Type': 'text/csv',
'Content-Disposition': 'attachment; filename=' + filename
})
const csvDataStream = fastCsv.write(data, {headers: true}).pipe(res)
})
The above code 'works' in some way as it does deliver back the response, but not the actual file, but the contents of the csv file, which I can view in the preview tab as a response. To sum up, Im trying to stream in that data, into a csv and push it to download file to client, and not store it on the server. Any tips or pointers are very much appreciated.
Here's what worked for me after created a CSV file on the server using the fast-csv package. You need to specify the full, absolute directory path where the output CSV file was created:
const csv = require("fast-csv");
const csvDir = "abs/path/to/csv/dir";
const filename = "my-data.csv";
const csvOutput = `${csvDir}/${filename}`;
console.log(`csvOutput: ${csvOutput}`); // full path
/*
CREATE YOUR csvOutput FILE USING 'fast-csv' HERE
*/
res.type("text/csv");
res.header("Content-Disposition", `attachment; filename="${filename}"`);
res.header("Content-Type", "text/csv");
res.sendFile(filename, { root: csvDir });
You need to make sure to change the response content-type and headers to "text/csv", and try enclosing the filename=... part in double-quotes, like in the above example.
I'm trying to analyze a file I'll be uploading from react, I need to know if it can be uploaded based on several factors.
I found https://github.com/TooTallNate/node-wav
It works great on nodejs and I'm trying to use it on react. The sample creates a readable stream and pipes it to the wav reader.
var fs = require('fs');
var wav = require('wav');
var file = fs.createReadStream('track01.wav');
var reader = new wav.Reader();
// the "format" event gets emitted at the end of the WAVE header
reader.on('format', function (format) {
//Format of the file
console.log(format);
});
file.pipe(reader);
Using FilePond controller I'm able to get a base64 string of the file. But I can't figure out how to pass it to the reader
this is what I have so far on ReactJS:
var reader = new wav.Reader();
reader.on('format', function (format) {
//Format of file
console.log('format', format);
});
const buffer = new Buffer(base64String, 'base64')
const readable = new Readable()
readable._read = () => { }
readable.push(buffer)
readable.push(null)
readable.pipe(reader)
But I get Error: bad "chunk id": expected "RIFF" or "RIFX", got "u+Zj"
Since this file works on NodeJS with the same lib is obvious I'm doing something wrong.
EDIT:
this was a problem with my Base64 string, this method works if anyone needs to analyze a wav on the frontend
Im wondering if is posible to know what is the file name of an incomming binary request.
This is my situation I have this code that handles the file upload
router.route('/:filename')
.put(function(req,res){
var uuid = guid();
var fileExtension = req.params.filename.substring(req.params.filename.lastIndexOf("."));
if(!fs.existsSync('../files')){
fs.mkdirSync('../files')
}
var newFile = fs.createWriteStream('../files/'+uuid+fileExtension);
req.pipe(newFile);
req.on('end',function(end){
console.log("Finished")
res.send(uuid+fileExtension)
})
})
as you can see now ,I need the file name specified in the URL('/:filename'). My question is: If it is possible to take that attribute from the resquest stream, instead the url or a form key?
If you use multer middleware you can access the uploaded filename like so
var multer = require('multer')
var upload = multer()
router.route('/:filename')
.put(upload.single('fileField'),function(req,res){
var fileName = req.file.originalname
var uuid = guid();
var fileExtension = req.params.filename.substring(req.params.filename.lastIndexOf("."));
if(!fs.existsSync('../files')){
fs.mkdirSync('../files')
}
var newFile = fs.createWriteStream('../files/'+uuid+fileExtension);
req.pipe(newFile);
req.on('end',function(end){
console.log("Finished")
res.send(uuid+fileExtension)
})
})
You'll need to inspect the Content-Disposition header of the request and parse the file name information out if processing the HTTP request manually.
However, I'd recommend you look at some of the existing file upload middlewares, no point in reinventing the wheel
busboy
multer
formidable
multiparty
pez
Node webshot is used to take a picture of an external website. The node webshot API is:
var webshot = require('webshot');
var fs = require('fs');
webshot('google.com', function(err, renderStream) {
var file = fs.createWriteStream('google.png', {encoding: 'binary'});
renderStream.on('data', function(data) {
file.write(data.toString('binary'), 'binary');
});
});
I am confused about file.write. Is the file being stored in the file object?
I want to be able to use filepickers rest API to upload the image like so:
curl -X POST -F fileUpload=#filename.txt https://www.filepicker.io/api/store/S3?key=MY_API_KEY
But I am confused as how to integrate webshot with renderStream with filepicker without saving the file to disk first. When the file is in memory I want to immediately send it to filepicker then get rid of it from memory.
Is this possible? Thanks!
I'm not sure about filepicker, but here is an example that streams the file to S3.
https://github.com/brenden/node-webshot/issues/90