HTTP POST FormData from Angular2 client to Node server - node.js

I have an angular2 client and I'm trying to send a file to my NodeJS server via http using a FormData object.
upload.html
...
<input #fileinput type="file" [attr.multiple]="multiple ? true : null" (change)="uploadFile()" >
...
upload.component.ts
uploadFile() {
let files = this.fileinput.nativeElement.files;
if (files.length == 0) return;
this.formData = new FormData();
for (var i = 0; i < files.length; i++) {
console.log("Appended file: " + files[i].name + " File object:" + files[i]); // Output: Appended file: simple.txt File object:[object File]
this.formData.append(files[i].name, files[i]);
}
console.log("uploadFile formData: " + this.formData); // Output: uploadFile formData: [object FormData]
// this.formData.keys() (or any other function) results in TypeError: dataForm.keys is not a function
this.userService
.formDataUpload(this.formData)
.then((response: any) => {
console.log(response);
this.router.navigateByUrl('/');
});
}
user.service.ts
formDataUpload(formData: FormData): Promise<any> {
return this.http.post('/api/v0/formdataupload', formData)
.toPromise()
.then((response: any) => {
return response.json()
})
.catch(this.handleError);
}
server.js
app.post('/api/v0/formdataupload', function(req, res) {
console.log(req.body); // Output: {}
});
How can I access the FormData object in server.js and getthe file object that was uploaded? Also, why am I unable to use any FormData functions in upload.component.ts? All the functions throw TypeError: x is not a function.

Your question is really two questions:
How to POST files from an Angular form to a server?
How to handle POSTed files on a (Node.js) server?
I'm no Node.js expert but a quick search turned up a nice tutorial using the Node.js formidable module to handle file uploads from Node.js.
As for the Angular part, the framework itself isn't of much help as far as file uploads so it all comes down to how would we do that in plain JS?
Your code seems fine. Here is a slightly simplified, working version of it (and a Plunkr):
#Component({
selector: 'my-app',
template: `
<form [formGroup]="myForm">
<p><input type="file" formControlName="file1" (change)="uploadFile($event)"></p>
</form>
`
})
export class AppComponent {
myForm: ControlGroup;
constructor(fb: FormBuilder, private http: Http) {
this.myForm = fb.group({
file1: []
});
}
uploadFile(evt) {
const files = evt.target.files;
if (files.length > 0) {
let file;
let formData = new FormData();
for (let i = 0; i < files.length; i++) {
file = files[i];
formData.append('userfile', file, file.name);
}
this.http.post('https://httpbin.org/post', formData)
.map(resp => resp.json())
.subscribe(data => console.log('response', data));
}
}
}
Only notable difference with your code is the way I access the files property of the input file element. In your code, writing <input #fileinput> in your template doesn't automagically create a fileinput property in your component, maybe that's the problem.
Notes:
I'm using https://httpbin.org/post as the backend to debug the POST request (it returns all the fields it has received in the request).
My file field doesn't have the multiple option but I think the code would work just the same if it did.

In terms of handling files posted to the Node.js server, your issue could well be one of Content-Type. If you are using something like body-parser to handle POSTs on the nodejs server, then it will not understand forms with the content type set to be multipart/form-data, which is what FormData deals in.
Note the comment here:
[body-parser] does not handle multipart bodies, due to their complex and typically large nature. For multipart bodies, you may be interested in the following modules: busboy and connect-busboy; multiparty and connect-multiparty; formidable; multer.
So in other words, you have to user a different module to handle the multipart body that FormData sends. I can recommend formidable, in which case you're server code would look something like:
const formidable = require('formidable')
exports.createPost = (req, res, next) => {
var form = new formidable.IncomingForm();
form.parse(req, (err, fields, files) => {
console.log(fields)
res.send('NOT IMPLEMENTED: pollsController createPost');
}
}

Related

download large file using streams from nodeJs to angular app

i am struggling to get this stream downloading file works
if i try the same request using curl the stream works fine and streaming the data.
in my angular app the file is completely download before the client see the download file tab it seems like the subscribe only happened after all stream body fully downloaded
what i am trying to achive is after first chunk of data send to angular app
i want the file to start downloading.
instead after observing the network only after all file downloaded from the backend
the downLoadFile methood is called and the ui expirence is stuck
this is a minimal example of what i am trying todo
at the backend i have a genrator that genreate a huge file
and pipe the request
Node JS
const FILE = './HUGE_FILE.csv'
const lines = await fs.readFileSync(FILE, 'utf8').split('\n')
function* generator() {
for (const i of files) {
console.log(i)
yield i
}
}
app.get('/', async function (req, res) {
res.setHeader('Content-Type', 'text/csv');
res.setHeader('Content-Disposition', 'attachment; filename=\"' + 'download-' + Date.now() + '.csv\"');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Pragma', 'no-cache');
const readable = Readable.from(generator());
readable.pipe(res);
});
at the client side calling the endpoint and waiting for the resonse
Angular code
#Component({
selector: 'is-optimizer-csv',
templateUrl: './optimizer-csv.component.html',
styleUrls: ['./optimizer-csv.component.scss']
})
export class OptimizerCsvComponent implements OnInit {
private onDownloadButtonClicked() {
const data = {...this.form.get('download').value, ...(this.advertiserId ? {advertiserId: this.advertiserId} : null)};
this.loading$.next(true);
this.optimizerService
.downloadOptimizerCsvData(data)
.pipe(
take(1),
tap(_ => this.loading$.next(false))
)
.subscribe();
}
}
#Injectable({
providedIn: 'root'
})
export class OptimizerService {
constructor(private readonly http: Ht, private datePipe: DatePipe) {}
downloadOptimizerCsvData(data: any) {
this.http.get(`${environment.apiUrl}`,{
responseType: 'arraybuffer',headers:headers}
).subscribe(response => this.downLoadFile(response, "text/csv"));
}
downLoadFile(data: any, type: string) {
let blob = new Blob([data], { type: type});
let url = window.URL.createObjectURL(blob);
let pwa = window.open(url);
if (!pwa || pwa.closed || typeof pwa.closed == 'undefined') {
alert( 'Please disable your Pop-up blocker and try again.');
}
}
}
it look like angular common http client does not support streaming out of the box .
see this github issue by #prabh-62
https://github.com/angular/angular/issues/44143
from this thread the why to solve your issue is by implementing the fetch stream logic using native fetch

How to Send Image , Pdf and Few Parameters together From Angular to Node.Js Server

I am using Angular Material For getting Position of Draggable Image , getting input type file(pdf) from User and Images Stored in ./assets/emojis/. I am able to send Pdf from Angular to Node using ng2-file-upload and multer.
FrontEnd
<input (change)="onFileSelected()" ng2FileSelect [uploader]="uploader" type="file" id="file">
<ion-button class="input-group-text upload" (click)="uploader.uploadAll()" >Upload</ion-button>
Backend
app.post('/api/upload', upload.single('pdf'), function (req, res) {
console.log('File is available!');
return res.send({
success: true
})
});
Now i have image path selected by User at assets
imagePathToUpload:string = './assets/emojis/'+this.selectedEmoji
topPosition:number = this.topPos
leftPosition:number = this.leftPos
How can i send all these data together to Server , there is one way FormData but i dont understand how to use it.
Objective is Send Image , Pdf , left and top position to server. Any Advice would be Appreciated.
To send a file from Angular --> Nodejs is simple
Create upload method
FinalformData: FormData;
uploadFile(event) {
if (event.target.files && event.target.files.length > 0) {
const file = event.target.files[0];
this.FinalformData = new FormData();
this.FinalformData.append('file', file, file.name);
}
}
submit() {
// call your service and send it
this._sendfileService.sendFile(this.FinalformData)
.subscribe( res => {
if (res.success) {
console.log('file uploaded');
} else { console.log('faild uploading file'))}
})
}
all done now call your angular service with your http.post method here's and ex
sendFile(file: FormData): Observable<any> {
return this.http.post<any>(this.nodejsUrl, file);
}
to handle the file in backend look into multer https://www.npmjs.com/package/multer

no_file_data response in slack file upload

I'm using node.js to try to upload a csv file via slackAPI's upload file method. The method is post. I'm unsure how to make this possible because if I use the content argument instead of the file, I get the error:
{ ok: false, error: 'invalid_array_arg' }
If I use the file aargument, I still get the error:
{ ok: false, error: 'invalid_array_arg' }
There are multiple fault points in this code and I've tried to test each one but I'm sure I'm missing some information here. Here's the uploadFile Method that I created:
function uploadFile(file){
console.log(botToken);
axios.post('https://slack.com/api/files.upload', qs.stringify({token: botToken, file: file, channels: 'testing'}))
.then(function (response) {
var serverMessage = response.data;
console.log(serverMessage);
console.log("inside file upload function");
})
}
here's how I call the method:
var file = fs.createReadStream(__dirname + '/' + csvFilePath); // <--make sure this path is correct
console.log(__dirname + '/' + csvFilePath);
uploadFile(file);
And finally the output:
Bot has started!
C:\Users\i502153\WebstormProjects\slackAPIProject/accessLogs/CSV/1548430592860output.csv*
{ ok: false, error: 'invalid_array_arg' }
inside file upload function
What am I doing wrong and how to rectify this?
Links:
https://api.slack.com/methods/files.upload
https://www.npmjs.com/package/axios
Your solution won't work because you are attempting to take a stream object (file) and stringify it into a query string, which is just going to insert the nonsense string "[object]" into the query. It won't actually stream data to Slack.
Axios, unfortunately, doesn't work in node exactly like it does in the browser, and their docs can be a little confusing.
I would suggest an approach like this (untested):
const axios = require('axios');
const FormData = require('form-data');
function uploadFile(file) {
const form = new FormData();
form.append('token', botToken);
form.append('channels, 'testing');
form.append('file', file, 'optionalfilenamehere');
return axios.post('https://slack.com/api/files.upload', form, {
headers: form.getHeaders()
}).then(function (response) {
var serverMessage = response.data;
console.log(serverMessage);
console.log('inside file upload function');
});
}
I adapted this code from the suggestion in ticket https://github.com/axios/axios/issues/1006#issuecomment-320165427, there may be other helpful comments there as well if you run into issues. Good luck!
EDIT: For people reading this later, for a similar approach using request instead of axios, see related question Slack API (files.upload) using NodeJS.

Node.js to send images via REST API

Im struggling to find material on this
I have a rest API, written in node.js, that uses mongoDB.
I want users to be able to upload images (profile pictures) and have them saved on the server (in mongoDB).
A few questions, Ive seen it is recommended to use GridFS, is this the best solution?
How do i send these files? Ive seen res.sendFile, but again is this the best solution?
If anyone has any material they can link me I would be appreciative
thanks
You won't be able to get the file object on the server directly. To get file object on the server, use connect-multiparty middleware. This will allow you to access the file on the server.
var multipart = require('connect-multiparty');
var multipartmiddleware = multipart();
var mv = require('mv');
var path = require('path');
app.post("/URL",multipartmiddleware,function(req,res){
var uploadedImage = req.files.file;
for (var i = 0; i < uploadedImage.length; i++) {
var tempPath = uploadedImage[i].path;
var targetPath = path.join(__dirname ,"../../../img/Ads/" + i + uploadedImage[i].name);
mv(tempPath, targetPath, function (err) {
if (err) { throw err; }
});
}
})
Use file system
Generally in any database you store the image location in the data as a string that tells the application where the image is stored on the file system.
Unless your database needs to be portable as a single unit, the storing of images inside of the database as binary objects generally adds unnecessary size and complexity to your database.
-Michael Stearne
In MongoDB, use GridFS for storing files larger than 16 MB.
- Mongo Documentation
Therefore unless your images will be over 16 MB, you should either store the file on a CDN (preferable) or the server's own file system and save its URL to user's document on the database.
Local file system implementation
This method uses Busboy to parse the photo upload.
in relevant html file:
<input type="file" title="Choose a file to upload" accept="image/*" autofocus="1">
Handler function for your photo upload route in server file (you will need to fill in the variables that apply to you and require the necessary modules):
function photoUploadHandlerFunction (req, res) {
var busboy = new Busboy({ headers: req.headers })
busboy.on('file', function (fieldname, file, filename, encoding, mimetype) {
const saveToDir = path.join(__dirname, uploadsPath, user.id)
const saveToFile = path.join(saveToDir, filename)
const pathToFile = path.join(uploadsPath, user.id, filename)
const writeStream = fs.createWriteStream(saveToFile)
createDirIfNotExist(saveToDir)
.then(pipeUploadToDisk(file, writeStream))
.then(findUserAndUpdateProfilePic(user, pathToFile))
.catch((err) => {
res.writeHead(500)
res.end(`Server broke its promise ${err}`)
})
})
busboy.on('finish', function () {
res.writeHead(200, { 'Connection': 'close' })
res.end("That's all folks!")
})
return req.pipe(busboy)
}
Where the promise functions createDirIfNotExist and pipeUploadToDisk could look like this:
function createDirIfNotExist (directory, callback) {
return new Promise(function (resolve, reject) {
fs.stat(directory, function (err, stats) {
// Check if error defined and the error code is "not exists"
if (err) {
if (err.code === 'ENOENT') {
fs.mkdir(directory, (err) => {
if (err) reject(err)
resolve('made folder')
})
} else {
// just in case there was a different error:
reject(err)
}
} else {
resolve('folder already existed')
}
})
})
}
function pipeUploadToDisk (file, writeStream) {
return new Promise((resolve, reject) => {
const fileWriteStream = file.pipe(writeStream)
fileWriteStream.on('finish', function () {
resolve('file written to file system')
})
fileWriteStream.on('error', function () {
reject('write to file system failed')
})
})
}
To answer your question 'How do I send these files?', I would need to know where to (MongoDB, to the client...). If you mean to the client, you could serve the static folder where they are saved.
If you still want to learn about implementing GridFs tutorialspoint have a good tutorial
More material
Good tutorial on handling form uploads
Tutorial using the node-formidable module
If you're using the mongoose odm you can use the mongoose-crate module and send the file wherever for storage.
Also, this is a good case for shared object storage like AWS S3 or Azure blob storage. If you are running a distributed setup in something like AWS, you usually don't want to store photos on the local server.
Store the url or key name in the database that points to the S3 object. This also integrates with CloudFront CDN pretty easily.
As suggested before. MultiPart for the actual upload.

node.js - streaming upload to cloud storage (busboy, request)

I'm new to node.js. What I'm trying to do is to stream the upload of a file from web browser to a cloud storage through my node.js server.
I'm using 'express', 'request' and 'busboy' modules.
var express = require("express");
var request = require("request");
var BusBoy = require("busboy");
var router = express.Router();
router.post("/upload", function(req, res, next) {
var busboy = new BusBoy({ headers: req.headers });
var json = {};
busboy.on("file", function (fieldname, file, filename, encoding, mimetype) {
file.on("data", function(data) {
console.log(`streamed ${data.length}`);
});
file.on("end", function() {
console.log(`finished streaming ${filename}`);
});
var r = request({
url: "http://<my_cloud_storage_api_url>",
method: "POST",
headers: {
"CUSTOM-HEADER": "Hello",
},
formData: {
"upload": file
}
}, function(err, httpResponse, body) {
console.log("uploaded");
json.response = body;
});
});
busboy.on("field", function(name, val) {
console.log(`name: ${name}, value: ${value}`);
});
busboy.on("finish", function() {
res.send(json);
});
req.pipe(busboy);
});
module.exports = router;
But I keep getting the following error on the server. What am I doing wrong here? Any help is appreciated.
Error: Part terminated early due to unexpected end of multipart data
at node_modules\busboy\node_modules\dicer\lib\Dicer.js:65:36
at nextTickCallbackWith0Args (node.js:420:9)
at process._tickCallback (node.js:349:13)
I realize this question is some 7 months old, but I shall answer it here in an attempt help anyone else currently banging their head against this.
You have two options, really: Add the file size, or use something other than Request.
Note: I edited this shortly after first posting it to hopefully provide a bit more context.
Using Something Else
There are some alternatives you can use instead of Request if you don't need all the baked in features it has.
form-data can be used by itself in simple cases, or it can be used with, say, got. request uses this internally.
bhttp advertises Streams2+ support, although in my experience Streams2+ support has not been an issue for me. No built in https support, you have to specify a custom agent
got another slimmed down one. Doesn't have any special handling of form data like request does, but is trivially used with form-data or form-data2. I had trouble getting it working over a corporate proxy, though, but that's likely because I'm a networking newb.
needle seems pretty light weight, but I haven't actually tried it.
Using Request: Add the File Size
Request does not (as of writing) have any support for using transfer-encoding: chunked so to upload files with it, you need to add the file's size along with the file, which if you're uploading from a web client means that client needs to send that file size to your server in addition to the file itself.
The way I came up with to do this is to send the file metadata in its own field before the file field.
I modified your example with comments describing what I did. Note that I did not include any validation of the data received, but I recommend you do add that.
var express = require("express");
var request = require("request");
var BusBoy = require("busboy");
var router = express.Router();
router.post("/upload", function(req, res, next) {
var busboy = new BusBoy({ headers: req.headers });
var json = {};
// Use this to cache any fields which are file metadata.
var fileMetas = {};
busboy.on("file", function (fieldname, file, filename, encoding, mimetype) {
// Be sure to match this prop name here with the pattern you use to detect meta fields.
var meta = fileMetas[fieldname + '.meta'];
if (!meta) {
// Make sure to dump the file.
file.resume();
// Then, do some sort of error handling here, because you cannot upload a file
// without knowing it's length.
return;
}
file.on("data", function(data) {
console.log(`streamed ${data.length}`);
});
file.on("end", function() {
console.log(`finished streaming ${filename}`);
});
var r = request({
url: "http://<my_cloud_storage_api_url>",
method: "POST",
headers: {
"CUSTOM-HEADER": "Hello",
},
formData: {
// value + options form of a formData field.
"upload": {
value: file,
options: {
filename: meta.name,
knownLength: meta.size
}
}
}
}, function(err, httpResponse, body) {
console.log("uploaded");
json.response = body;
});
});
busboy.on("field", function(name, val) {
// Use whatever pattern you want. I used (fileFieldName + ".meta").
// Another good one might be ("meta:" + fileFieldName).
if (/\.meta$/.test(name)) {
// I send an object with { name, size, type, lastModified },
// which are just the public props pulled off a File object.
// Note: Should probably add error handling if val is somehow not parsable.
fileMetas[name] = JSON.parse(val);
console.log(`file metadata: name: ${name}, value: ${value}`);
return;
}
// Otherwise, process field as normal.
console.log(`name: ${name}, value: ${value}`);
});
busboy.on("finish", function() {
res.send(json);
});
req.pipe(busboy);
});
module.exports = router;
On the client, you need to then send the metadata on the so-named field before the file itself. This can be done by ordering an <input type="hidden"> control before the file and updating its value onchange. The order of values sent is guaranteed to follow the order of inputs in appearance. If you're building the request body yourself using FormData, you can do this by appending the appropriate metadata before appending the File.
Example with <form>
<script>
function extractFileMeta(file) {
return JSON.stringify({
size: file.size,
name: file.name,
type: file.type,
lastUpdated: file.lastUpdated
});
}
function onFileUploadChange(event) {
// change this to use arrays if using the multiple attribute on the file input.
var file = event.target.files[0];
var fileMetaInput = document.querySelector('input[name=fileUpload.meta]');
if (fileMetaInput) {
fileMetaInput.value = extractFileMeta(file);
}
}
</script>
<form action="/upload-to-cloud">
<input type="hidden" name="fileUpload.meta">
<input type="file" name="fileUpload" onchange="onFileUploadChange(event)">
</form>
Example with FormData:
function onSubmit(event) {
event.preventDefault();
var form = document.getElementById('my-upload-form');
var formData = new FormData();
var fileUpload = form.elements['fileUpload'];
var fileUploadMeta = JSON.stringify({
size: fileUpload.size,
name: fileUpload.name,
type: fileUpload.type,
lastUpdated: fileUpload.lastUpdated
});
// Append fileUploadMeta BEFORE fileUpload.
formData.append('fileUpload.meta', fileUploadMeta);
formData.append('fileUpload', fileUpload);
// Do whatever you do to POST here.
}

Resources