I am trying to upload a simple file using the vaadin file upload element(refer https://vaadin.com/elements/-/element/vaadin-upload)
I am using nodejs and express with multer as middle-ware on the server side.
The nodejs simplified code is as follows:
var express = require("express");
var multer = require('multer');
var app = express();
var storage = multer.diskStorage({
destination: function (req, file, callback) {
callback(null, './uploads');
},
filename: function (req, file, callback) {
callback(null, file.fieldname + '-' + Date.now());
}
});
var upload = multer({ storage : storage}).single('userPhoto');
app.get('/',function(req,res){
res.sendFile(__dirname + "/index.html");
});
app.post('/api/photo',function(req,res){
upload(req,res,function(err) {
if(err) {
return res.end("Error uploading file.");
}
res.end("File is uploaded");
});
});
I tried with an HTML element in my index.html and this code works
<form id = "uploadForm"
enctype = "multipart/form-data"
action = "/api/photo"
method = "post"
>
<input type="file" name="userPhoto" />
<input type="submit" value="Upload Image" name="submit">
</form>
But I want to use the vaadin upload element. I wrote the code
<link rel="import" href="../bower_components/vaadin-upload/vaadin-upload.html">
<vaadin-upload id="userPhoto" target="/api/photo"></vaadin-upload>
When I use this, it does not work as expected. I am able to see the vaadin element. But I am unable to upload a file. As I select the file, the server responds with the string 'Error uploading file'. I am pretty new to polymerjs and web development. Could someone please point me where I am going wrong?
In your backend code, you have to change the file parameter name from 'userPhoto' to 'file'.
In case of the native HTML form, it sends a file in the parameter declared by the name attribute of the <input type="file">.
When using the Vaadin Upload element, the files are sent in the same manner as by the HTML form. But instead of the provided name, they are sent in the parameter called "file".
Note that you have the id="userPhoto" attribute for the upload. The id is a different attribute, that does not act like a name.
Unfortunately, the file parameter name for the upload requests cannot be easily customized for now. We are planning to add more upload request customization features in future.
Related
I am having hard time finding how i can upload an image to my image classifier api and do prediction on it.
in node js i copy paste many code examples but it only saves a copy of the file to an upload folder but there is no option to add the file to an api.
I am running both server( NodeJs and flask(api)) on my localhost.
Eg.I need something similar like AWS Rekognition makes us select an image and show the response.
This is the code that I copy pasted but don't know how add the api and send the image selected on the page
var http = require('http');
var formidable = require('formidable');
var fs = require('fs');
http.createServer(function (req, res) {
if (req.url == '/fileupload') {
var form = new formidable.IncomingForm();
form.parse(req, function (err, fields, files) {
var oldpath = files.filetoupload.path;
console.log(oldpath);
console.log('Fields', fields)
console.log('Files', files)
var newpath = 'C:/Users/Pratham Nishad/Desktop/' +
files.filetoupload.name;
fs.rename(oldpath, newpath, function (err) {
if (err) throw err;
res.write('File uploaded and moved!');
res.end();
});
});
} else {
res.writeHead(200, { 'Content-Type': 'text/html' });
res.write('<form action="fileupload" method="post"
enctype="multipart/form-data">');
res.write('<input type="file" name="filetoupload"><br>');
res.write('<input type="submit">');
res.write('</form>');
return res.end();
}
}).listen(4000);
What I want- Lets say that I have an Image Classifier ready to make classify image at XYZ.com and I want to make user select an image(PROBLEM HERE) from his/her storage and process it and post the result.
All the tutorial that i watched just selects the image from one location and paste it to destination location, no information how the image classifier running at XYZ.com will get the image.
Check out multer package on npm:
https://www.npmjs.com/package/multer
As requested I'll demonstrate how I actually used this tool.
Front-end Code:
<!DOCTYPE html>
<html>
<body>
<p>Click on the "Choose File" button to upload a file:</p>
<form action="xyz.com" method="post">
<input type="file" id="myFile" name="filename">
<input type="submit">
</form>
</body>
</html>
Backend Code:
const multer = require('multer');
//Multer DiskStorage Config
const diskStorage = multer.diskStorage({
destination: 'path/to/your/desired/folder',
filename: (req, file, call_back) => {
//Prepend date to the filename
//or anything that can uniquely identify the file
//so it won't get overwritten by the new ones
call_back(null, Date.now() + '_' + file.originalname);
}
});
//Create Multer Instance
const upload = multer({ storage: diskStorage });
//Upload picture
router.post('/uploadpic', upload.single('file'), (req, res) => {
res.status(200);
});
After the file has been saved, you can use it with the image classifier.
You are really free to create your own file structure. There are no specific constraints or rules that you have to follow.
I tried to upload image file from client side to my AWS S3 storage server using multer with the support of multer-s3. However, I keep getting undefined result for my req.file call. I have been googling and check all possible causes to my problem and no luck so far. Hope someone can help me here. Thanks in advance.
Client side HTML code:
<form method="post" enctype="multipart/form-data" action="/create">
<input type="text" name="name" />
<input type="text" name="topic" />
<input type="file" name="image" />
<input type="submit" value="Upload" />
</form>
Server side code:
var express = require('express');
var http = require('http');
var app = express();
var methodOverride = require('method-override');
var bodyParser = require('body-parser');
var multer = require('multer');
var multerS3 = require('multer-s3');
var AWS = require('aws-sdk');
app.use(methodOverride());
app.use(bodyParser.urlencoded({ extended: true }));
//Create S3 client
var s3 = new AWS.S3();
//this code is copy directly from multer-s3 since I don't care about my file name shown on S3 for now
var upload = multer({
storage: multerS3({
s3: s3,
bucket: mybuckname,
metadata: function (req, file, cb) {
cb(null, {fieldName: file.fieldname});
},
key: function (req, file, cb) {
cb(null, Date.now().toString())
}
})
});
app.post('/create', upload.single('image'), function(req, res) {
var nameField = req.body.name,
topicField = req.body.topic;
console.log("File: ", req.file);
console.log("Name: ", nameField);
console.log("Topic: ", topicField);
res.sendStatus(200);
res.end();
});
I have tried the following:
Removed 'body-parser' and use 'multer' only, then even worse result (both req.body and req.file are undefined). With body-parser, at least I can guarantee req.body is working
To simplify the cause, I removed all my text field inputs from client side and leave only file upload option to make sure I am only transmitting file data, still return undefined for req.file (multer is not working at all)
I also read about the possibility that "Note that req.body might not have been fully populated yet. It depends on the order that the client transmits fields and files to the server." In my case, I am not getting the file so it doesn't matter for me now.
Use multer only without multer-s3, and tried upload.array instead of upload.single, still no luck (multer-s3 is built on top of multer, so I don't think it has anything to do with my current problem, but just to give it a try)
Skip AWS S3 and store it in local path, still undefined result
after long struggle switching between multer and busboy and nothing works. All the sudden I noticed my form is actually using a Polymer element 'is="iron-form"' which my mistake didn't type it in my original post as shown below. After removed it, my file upload is working on multer - I'm sure it will also work on busboy.
<form is="iron-form" method="post" enctype="multipart/form-data" action="/create">
Polymer elements always create problems to me in the past. But don't take me wrong, it's a great project with many great things available to design your page. But you need to really understand how to use those elements to take the advantage of them. In my case, I learn from my lesson and hope to share it with you.
This HTML form below is (among other things) uploading a file to a server using express.js and multer. It works fine, if a file is chosen, but not if the file is left out.
<form id="uploadform" method="post" action="/upload" enctype="multipart/form-data">
<p> Description: <input type="text" name="description"></p>
<p><input type="file" name="img"></p>
<p><input id="submit_upload" type="submit" value="Submit"></p>
</form>
My Node.js application includes the middleware multer.
var express = require('express');
var app = express();
var multer = require('multer');
app.use(multer(
{
dest: './public/img/',
rename: function (fieldname, filename) {
return filename + Date.now();
}
}));
When the form is submitted, I will be redirected to /upload, as it is defined in the HTML-Upload form with action="/upload".
In my Node.js script the form entries will be handled as shown below:
app.post('/upload', function (req, res) {
var form_description = req.body.description;
var form_datei = JSON.stringify(req.files.img.name);
form_datei = form_datei.substring(1, form_datei.length - 1);
// insert operations into database get placed here
res.redirect('/');
});
If I don´t select a file in the form and submit it only with inserted text, not even the text will be submitted. Instead my console and browser window display a long error message starting as you can see below:
TypeError: Cannot read property 'name' of undefined
How can I make the form submit if no file is selected?
The file upload should be optional.
Thanks for your help!
The problem is when you try to retrieve the image name in req.files.img.name. Since req.files.img will return undefined, you can't read the property there.
Check if a file was submitted before trying to use it:
app.post('/upload', function (req, res) {
var form_description = req.body.description;
if(req.files.img) {
var form_datei = JSON.stringify(req.files.img.name);
form_datei = form_datei.substring(1, form_datei.length - 1);
// insert operations into database get placed here
}
res.redirect('/');
});
I am trying to implement file uploads with node.js and the multer middleware, but it doesn't seem to work. This is my code:
var express = require('express');
var multer = require('multer');
var done = false;
var app = express();
app.use(multer( {dest:'./uploads/',
onFileUploadStart : function(file){
console.log('File recieved:');
console.log(file);
},
onFileUploadData:function (file,data){
console.log('Data recieved');
},
onParseEnd: function(req,next){
next();
}
}));
app.use(express.static(__dirname+"/public"));
app.post('/upload',require(__dirname+'/upload.js').upload);
app.listen(3000);
My form looks like this:
<html>
<body>
<form action="/upload" method="post" enctype="multipart/form-data">
<input type="file" name ="file">
<input type="submit" value="Upload selected file to server">
</form>
</body>
</html>
And upload.js looks like this:
exports.upload = function (req,res)
{
console.dir(req.files);
};
I think the problem is that my form is being submitted with "application/x-www-form-urlencoded" in the Content-Type header instead of "multipart/form-data", since this is what appears when I use Fiddler to monitor the request, but I have no idea why. Can anyone shed any light on this?
I got it working by adding an accept attribute to my tag in my html. I don't know why in some examples this is not used.
Here's the code for my entire form:
<form action="/upload" method="post" enctype="multipart/form-data">
<input type="file" name ="file" accept="application/x-zip-compressed,image/*">
<input type="submit" value="Upload selected file to server">
</form>
Check the entire project eventually: https://github.com/tutiplain/quickupload
I can see you are doing everything right. I used multer sometime back, you can look into my working implementation here. In this EJS file i have an image upload functionality and then i wrote some logic to store the file to some other location.
Make sure you have appropriate router, for example you can use router.post(..)
exports.upload= function(req,res){
// get the temporary location of the file
var tmp_path = req.files.file.path;
// set where the file should actually exists - in this case it is in the "images" directory
var target_path = '/..provide path to store photos../' + req.files.file.name;
// move the file from the temporary location to the intended location
fs.rename(tmp_path, target_path, function(err) {
if (err) throw err;
// delete the temporary file, so that the explicitly set temporary upload dir does not get filled with unwanted files
fs.unlink(tmp_path, function() {
if (err) {
throw err;
}else{
//response logic ...
};
});
});
};
You can try this. It works for me. If any issues then let me know
var multer = require('multer');
var upload = multer({ dest: './uploads' });
router.post('/register',upload.single('profileimage'),function(req,res,next){
if (req.file) {
console.log('Uploading File');
var profileImageOriginlName=req.file.originalname;
var profileImageName=req.file.name;
var profileImageMime=req.file.mimetype;
var profileImagePath=req.file.path;
var profileImageExt=req.file.extension;
var profileImageSize=req.file.size;
}
else
{
var profileImageName='noimage.png';
}
});
Using Express with Node, I can upload a file successfully and pass it to Azure storage in the following block of code.
app.get('/upload', function (req, res) {
res.send(
'<form action="/upload" method="post" enctype="multipart/form-data">' +
'<input type="file" name="snapshot" />' +
'<input type="submit" value="Upload" />' +
'</form>'
);
});
app.post('/upload', function (req, res) {
var path = req.files.snapshot.path;
var bs= azure.createBlobService();
bs.createBlockBlobFromFile('c', 'test.png', path, function (error) { });
res.send("OK");
});
This works just fine, but Express creates a temporary file and stores the image first, then I upload it to Azure from the file. This seems like an inefficient and unnecessary step in the process and I end up having to manage cleanup of the temp file directory.
I should be able to stream the file directly to Azure storage using the blobService.createBlockBlobFromStream method in the Azure SDK, but I am not familiar enough with Node or Express to understand how to access the stream data.
app.post('/upload', function (req, res) {
var stream = /// WHAT GOES HERE ?? ///
var bs= azure.createBlobService();
bs.createBlockBlobFromStream('c', 'test.png', stream, function (error) { });
res.send("OK");
});
I have found the following blog which indicates that there may be a way to do so, and certainly Express is grabbing the stream data and parsing and saving it to the file system as well. http://blog.valeryjacobs.com/index.php/streaming-media-from-url-to-blob-storage/
vjacobs code is actually downloading a file from another site and passing that stream to Azure, so I'm not sure if it can be adapted to work in my situation.
How can I access and pass the uploaded files stream directly to Azure using Node?
SOLUTION (based on discussion with #danielepolencic)
Using Multiparty(npm install multiparty), a fork of Formidable, we can access the multipart data if we disable the bodyparser() middleware from Express (see their notes on doing this for more information). Unlike Formidable, Multiparty will not stream the file to disk unless you tell it to.
app.post('/upload', function (req, res) {
var blobService = azure.createBlobService();
var form = new multiparty.Form();
form.on('part', function(part) {
if (part.filename) {
var size = part.byteCount - part.byteOffset;
var name = part.filename;
blobService.createBlockBlobFromStream('c', name, part, size, function(error) {
if (error) {
res.send({ Grrr: error });
}
});
} else {
form.handlePart(part);
}
});
form.parse(req);
res.send('OK');
});
Props to #danielepolencic for helping to find the solution to this.
As you can read from the connect middleware documentation, bodyparser automagically handles the form for you. In your particular case, it parses the incoming multipart data and store it somewhere else then exposes the saved file in a nice format (i.e. req.files).
Unfortunately, we do not need (and necessary like) black magic primarily because we want to be able to stream the incoming data to azure directly without hitting the disk (i.e. req.pipe(res)). Therefore, we can turn off bodyparser middleware and handle the incoming request ourselves. Under the hood, bodyparser uses node-formidable, so it may be a good idea to reuse it in our implementation.
var express = require('express');
var formidable = require('formidable');
var app = express();
// app.use(express.bodyParser({ uploadDir: 'temp' }));
app.get('/', function(req, res){
res.send('hello world');
});
app.get('/upload', function (req, res) {
res.send(
'<form action="/upload" method="post" enctype="multipart/form-data">' +
'<input type="file" name="snapshot" />' +
'<input type="submit" value="Upload" />' +
'</form>'
);
});
app.post('/upload', function (req, res) {
var bs = azure.createBlobService();
var form = new formidable.IncomingForm();
form.onPart = function(part){
bs.createBlockBlobFromStream('taskcontainer', 'task1', part, 11, function(error){
if(!error){
// Blob uploaded
}
});
};
form.parse(req);
res.send('OK');
});
app.listen(3000);
The core idea is that we can leverage node streams so that we don't need to load in memory the full file before we can send it to azure, but we can transfer it as it comes along. The node-formidable module supports streams, hence piping the stream to azure will achieve our objective.
You can easily test the code locally without hitting azure by replacing the post route with:
app.post('/upload', function (req, res) {
var form = new formidable.IncomingForm();
form.onPart = function(part){
part.pipe(res);
};
form.parse(req);
});
Here, we're simply piping the request from the input to the output. You can read more about bodyParser here.
There are different options for uploading binary data (e.g. images) via Azure Storage SDK for Node, not using multipart.
Based on the Buffer and Stream definitions in Node and manipulating them, these could be handled using almost all the methods for BLOB upload: createWriteStreamToBlockBlob, createBlockBlobFromStream, createBlockBlobFromText.
References could be found here: Upload a binary data from request body to Azure BLOB storage in Node.js [restify]
People having trouble with .createBlockBlobFromStream trying to implement the solutions, note that this method has been changed slightly in newer versions
Old version:
createBlockBlobFromStream(containerName, blobName, part, size, callback)
New version
createBlockBlobFromStream(containerName, blobName, part, size, options, callback)
(if you don't care about options, try an empty array) for the parameter.
Oddly enough, "options" is supposed to be optional, but for whatever reason, mine fails if I leave it out.