I want to build a NodeJS server that accepts a .wav file (1Mb) sent to its single endpoint, then changes the file through AudioContext API and then sends back the response with the result?
The server shouldn't store anything, so, no database required.
How can I achieve this? (or, please correct me if don't understand how things work)
I would do this with express: https://expressjs.com/
and as middleware add express-fileuplaod: https://www.npmjs.com/package/express-fileupload
app.post('/upload', function(req, res) {
console.log(req.files.foo); // the uploaded file object
});
instead of the console.log(); you'd make a readable stream / buffer and then use it in the
AudioContext API
here is also a interesting Article explaining to use this:
https://www.russellgood.com/process-uploaded-file-web-audio-api/
Related
I have a 15GB video file that I would like to proxy through Node.js. This code works fine:
app.all('/shareRedirects/:path', cors({origin: '*'}), function (req, res) {
var path = req.params.path;
request.get(path).pipe(res);
});
The issue is that the entire file needs to download on server side before being sent to client via Express / request. What's the best way to break it into chunks?
after further investigation, it seems .pipe() does stream it in chunks.
I am using expressjs (all newest versions of express, node, and npm). I have create a route such as this:
router.post("/", function(req, res, next) {
});
This route will need to be able to have a file (image/video/docx,etc) uploaded and needs to then be stored on a cloud storage service (Google Storage). I do not want to store anything on the server that express is running on, just want to receive the file and pass it on over to Google Cloud Storage. I see there are some libraries which do this in addition to express, but I couldn't not find how to do it using just express.
I think your clients might be able to upload directly to GCS by constructing a HTML form. Basically you can create a signed url and embed it in the form and then on submit, the upload goes straight to GCS and your app doesn't need to handle it at all.
See: https://cloud.google.com/storage/docs/xml-api/post-object
I should implement an upload form
I thought of using bodyparser but I read
http://andrewkelley.me/post/do-not-use-bodyparser-with-express-js.html
so what's the way to upload a file with express using the mean stack ?
may be formidable or other modules ?
That warning is specifically against adding the express.bodyparser middleware to your entire stack as it adds express.multipart to all POST endpoints and therefore file uploads are automatically accepted at all POST endpoints. By default the framework automatically saves any uploaded files to /tmp and so unless you are cleaning them up an attacker could flood your disk with uploaded files.
If you want to avoid using additional modules, what you should do is implement express.multipart on the endpoint(s) where you want to allow file uploads. Here's what I'm talking about:
var express = require("express")
, app = express();
// middleware (no bodyparser here)
app.use(express.json());
app.use(express.urlencoded());
// average GET endpoint
app.get("/", function(req,res) {
res.send('ok');
});
// average POST endpont
app.post("/login", function(req,res) {
res.send('ok');
});
// File upload POST endpoint
app.post('/upload', express.multipart, function(req, res) {
//File upload logic here
//Make sure to delete or move the file accordingly here, otherwise files will pile up in `/tmp`
});
Note the inclusion of express.multipart in the file upload endpoint. This endpoint will now process multipart file uploads, and assuming you handle them correctly they won't be a threat.
Now, having told you all of this, Connect is moving to deprecate multipart due to this exact issue, but there don't seem to be any plans to add a stream based file upload replacement. What they instead recommend is that you use node-multiparty which uses streams to avoid ever placing a file on disk. However, there don't seem to be any good references I can find for using multiparty as a middleware without saving files though, so you'll have to contact the author of multiparty or take a closer look at the API for implementing it with Express.
I created an example that uses Express & Multer - very simple, avoids all Connect warnings
https://github.com/jonjenkins/express-upload
I want to be able to add image uploads to a site I'm working on, but I don't know how that is supposed to work with meteorjs. Is there a way to accept multipart forms on meteorjs? i.e. image uploads?
I could always rackup a small sinatra app for file uploads but I'd rather not.
You need to write a middleware to accept such requests.
WebApp.connectHandlers.stack.splice(0, 0, {
route: '/your/url/for/inputs',
handle: function(req, res, next) {
// Handle request and response
// just like you would do in node.js.
// Make sure to wrap your db calls in Fibers.
// Use next() if you change your mind
// and don't want to handle this request after all.
},
});
Looks like eventminded has a file upload package you could use.
https://www.eventedmind.com/posts/meteor-build-a-file-upload-package
I'm trying to build a quick and simple image uploading service with Node, that takes the received images and saves them to Mongo's GridFS.
GridFS get requires a Buffer object NodeJS Mongo Driver GridFS put
The question is pretty simple: how do I exactly cast/transform the received request body into a proper buffer.
My code so far (only the important pieces):
api.js
var express = require('express');
var connect = require('connect');
var app = module.exports = express.createServer();
app.configure(function(){
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(app.router);
});
var upload = require('./upload.js');
app.post('/upload', upload.upload);
upload.js
exports.upload = (function(req, res, next){
console.log("Uploading image...");
// Create buffer
// Rest of the code
}
I've tried:
var buffer = new Buffer(util.inspect(req.body),'binary');
Creates the buffer, but it has a wrong size and probably not the correct content since util.inspect is obviously not the right way to go.
And:
var buffer = new Buffer(req.body);
Result:
[Decode error - output not utf-8][Decode error - output not utf-8]
Buffer length = 0
I'm quite new to both Node and JavaScript developing in general, so probably I'm missing something quite simple, don't hesitate to point the obvious :)
Thanks!
First, remember that Express is built on top of Connect, which is the library that handles a large amount of the lower-level HTTP work, and it's where bodyParser() comes from.
The body parser middleware internally uses Formidable to parse file uploads.
Formidable's default behavior is to write uploaded files directly to disk – in other words, you don't actually have access to the uploaded file stream within your route handler. You get the values of any regular form fields (<input>s) sent along in req.body, and you get uploaded file paths via req.files, but you don't get file content.
The easy answer here is to simply read the file from disk and use that to save into Mongo, remembering to delete the temporary file when done. Of course, this introduces the unnecessary intermediate step of writing the file upload to a temporary folder and then loading to Mongo.
If you want to stream file data directly into Mongo, you have more of a challenge in front of you. You'll have to write your own middleware to parse the upload stream.
This is actually relatively easy. You can just start with the internal Connect body parser implementation—using Formidable to do the heavy lifting—and use the onPart API to pass the stream back to your route handler so that you can pass it off to the Mongo driver.