Node webshot is used to take a picture of an external website. The node webshot API is:
var webshot = require('webshot');
var fs = require('fs');
webshot('google.com', function(err, renderStream) {
var file = fs.createWriteStream('google.png', {encoding: 'binary'});
renderStream.on('data', function(data) {
file.write(data.toString('binary'), 'binary');
});
});
I am confused about file.write. Is the file being stored in the file object?
I want to be able to use filepickers rest API to upload the image like so:
curl -X POST -F fileUpload=#filename.txt https://www.filepicker.io/api/store/S3?key=MY_API_KEY
But I am confused as how to integrate webshot with renderStream with filepicker without saving the file to disk first. When the file is in memory I want to immediately send it to filepicker then get rid of it from memory.
Is this possible? Thanks!
I'm not sure about filepicker, but here is an example that streams the file to S3.
https://github.com/brenden/node-webshot/issues/90
Related
At my endpoint in my NodeJS server, after retrieving an audio file stored as a Buffer in MongoDB, I want to represent it with a URL (much like how you do with URL.createObjectURL(blob) in the frontend on the browser). I then plan to res.render() the URL in HTML through Handlebars on the client, so that the user can click on it to download it:
<a href={{url}}>Click me to download the file!</a>
In the NodeJs server, I have converted the MongoDB Buffer into a JavaScript ArrayBuffer through:
var buffer = Buffer.from(recordingFiles[0].blobFile);
var arrayBuffer = Uint8Array.from(buffer).buffer;
I am unsure where to proceed from here. I seen solutions using fs or res.download(), but they don't seem applicable to my situation. Thanks in advance for any help!
Hopefully this can help.
var blob = new Blob(BUFFER, {type: "audio mime type"});
var link = document.createElement('a');
link.href = window.URL.createObjectURL(blob);
var fileName = reportName;
link.download = fileName;
link.click();
Do you always need to preload the audio file onto the page?
If not, then I would advise you to add a separate endpoint to download the file on demand. The frontend link can send a get request to the endpoint and download the file only if the user clicked it.
Otherwise you'd always be downloading the buffer behind the scenes, even if the user didn't intend to download it. This is especially problematic on slow connections.
Frontend:
<a href={{`${baseUrl}/download/${audioId}`}}>Click me to download the file!</a>
Backend:
const stream = require('stream');
app.get('/download/:audioId', function (request, response) {
// Retrieve the tag from our URL path
const audioId = request.params.audioId;
const fileData; // TODO: Get file buffer from mongo.
const fileContents = Buffer.from(fileData, 'base64');
const readStream = new stream.PassThrough();
readStream.end(fileContents);
response.set('Content-disposition', 'attachment; filename=' + fileName);
response.set('Content-Type', '<your MIME type here>');
readStream.pipe(response);
});
A list of relevant MIME types can be found here.
I've set up a small node js BE app, built with express and fastCsv module on top of it. The desired outcome would be to be able to download a csv file to the client side, without storing it anywhere inside the server, since the data is generated depending on user criteria.
So far I've been able to get somewhere it it, Im using streams, since that csv file could be pretty large depending on the user selection. Im pretty sure something is missing inside the code bellow:
const fs = require('fs');
const fastCsv = require('fast-csv');
.....
(inside api request)
.....
router.get('/', async(req, res) => {
const gatheredData ...
const filename = 'sometest.csv'
res.writeHead(200, {
'Content-Type': 'text/csv',
'Content-Disposition': 'attachment; filename=' + filename
})
const csvDataStream = fastCsv.write(data, {headers: true}).pipe(res)
})
The above code 'works' in some way as it does deliver back the response, but not the actual file, but the contents of the csv file, which I can view in the preview tab as a response. To sum up, Im trying to stream in that data, into a csv and push it to download file to client, and not store it on the server. Any tips or pointers are very much appreciated.
Here's what worked for me after created a CSV file on the server using the fast-csv package. You need to specify the full, absolute directory path where the output CSV file was created:
const csv = require("fast-csv");
const csvDir = "abs/path/to/csv/dir";
const filename = "my-data.csv";
const csvOutput = `${csvDir}/${filename}`;
console.log(`csvOutput: ${csvOutput}`); // full path
/*
CREATE YOUR csvOutput FILE USING 'fast-csv' HERE
*/
res.type("text/csv");
res.header("Content-Disposition", `attachment; filename="${filename}"`);
res.header("Content-Type", "text/csv");
res.sendFile(filename, { root: csvDir });
You need to make sure to change the response content-type and headers to "text/csv", and try enclosing the filename=... part in double-quotes, like in the above example.
I am sending audio data stored as a blob to my backend (node/express). When I save the file as .wav and attempt to use in the SpeechRecogition package in python it throws an error saying the "file does not start with RIFF id". So how can I add the headers to my blob file before I save it so that it is a correctly formatted .wav file? I can provide the code if necessary.
node.js file
var multer = require('multer');
var fs = require('fs'); //use the file system so we can save files
var uniqid = require('uniqid');
var spawn = require('child_process').spawn;
const storage = multer.memoryStorage()
var upload = multer({ storage: storage });
router.post('/api/test', upload.single('upl'), function (req, res) {
console.log(req.file);
console.log(req.file.buffer);
var id = uniqid();
fs.writeFileSync(id+".wav", Buffer.from(new Uint8Array(req.file.buffer))); //write file to server as .wav file
const scriptPath = 'handleAudio.py'
const process = spawn('python3', [__dirname+"/../"+scriptPath, "/home/bitnami/projects/sample/"+id+".wav", req.file.originalname, 'True']); //throws error about header in .wav
});
Also I had this same example working with a php endpoint that just saved the blob to a file with .wav extension and the python file accepted it. What could be different in the move_uploaded_file in php and what I am doing above with node?
Every .wav file needs a header specified by the WAVE file format, available here. While it's fine for you to build the header yourself, it's much easier to just use a proper lib to do the work for you.
One example is node-wav, which has a nice API to write WAVE files from raw PCM data (what you have at the moment). Example code is provided by the node-wav documentation.
I'm currently running Node.js by Browserify for my website.
It reads the JSON file and I get the message through MQTT.
But the problem is that it seems like writefile does not work.
(Running this as node test.js in the terminal works by the way).
What is wrong with my code?
Moreover, Is this the best way to store any user data?
Thank you so much in advance.
Here's some part of my code
var fs = require("fs");
var path = require("path");
let newFile = fs.readFileSync('/home/capstone/www/html/javascript/test.json');
function testT() { //THIS WORKS FINE
let student0 = JSON.parse(newFile);
var myJSON = JSON.stringify(student0);
client.publish("send2IR", myJSON);
response.end();
};
function write2JSON() { //PROBLEM OF THIS CODE
const content = 'Some content!'
fs.writeFileSync('/home/capstone/www/html/javascript/test.json', content)
};
document.getElementById("blink").addEventListener("click", publish);
document.getElementById("write").addEventListener("click", write2JSON);
You cann't write directly for security reasons. For other hand you can use a server as API to do the filye system tasks and in the client only trigger the events.
This post is very related with your problem:
Is it possible to write data to file using only JavaScript?
I have to make a POST request to a server that returns me this.
And I have to write a ZIP from that, how do I get the bytes from that string to generate my zip file?
you just need to create a buffer from the api response and create a zip file using that buffer.
var fs = require('fs');
var buff = new Buffer(response_from_api);
fs.writeFile("./test1.zip", buff, function(err){
//do something
});