My app (using vue) allows users to upload files with some info about the data to my node backend. When the user submits the form, this function is triggered:
methods: {
buttonOK () {
const formData = new FormData()
formData.append('name', this.detailFirm.name)
formData.append('description', this.detailFirm.description)
formData.append('version', this.detailFirm.version)
formData.append('date', this.detailFirm.date)
formData.append('file', this.file)
for (var [key, value] of formData.entries()) {
console.log(key, value)
}
let headers = {
'Content-Type': 'multipart/form-data',
'Accept': 'multipart/form-data'
}
this.$http.put('/firmware', formData, {headers: headers})
this.visible = false
}
The log statement shows everything that it ought to, and when this request is made, the network tab in the chrome dev tools shows the post data going through, and it has all the values it should:
name: test
description: test
version: 1
date: 0555-05-05
file: (binary)
My multer middleware looks like this:
const multer = require('multer')
const mult = multer({
dest: '/firmware'
})
module.exports = function (req, res, next) {
/* --Convert multipart/form-data to useable format within express-- */
if (req.path === '/firmware') {
mult.single('file')
console.log('MULTER MIDDLEWARE')
}
next()
}
The log statement there works, leading me to believe that multer is working.
I can't seem to access this information in back end though. Here I have tried both file and formData as the file name in mult.single('').
Here is my controller function:
let firmware = {
name: req.body.name,
version: req.body.version,
description: req.body.description,
date: req.body.date,
file: req.body.file
}
firmwareRepo.create(firmware, (err, create) => {
.............
I've read some other questions, and have made a few adjustments, but I always get an empty object when I log req.body in the controller. Please advise.
https://github.com/expressjs/multer#diskstorage
Note that req.body might not have been fully populated yet. It depends on the order that the client transmits fields and files to the server.
EDIT:
Firstly, I remember I had one problem on the frontend (React), by adding headers, which are not needed (somehow by adding formdata headers u **** up everything), here is the example:
data append stuff goes here
const data = new FormData()
data.append('id', values.id)
......
return async (dispatch) => {
const respond = await fetch('/api/postdata', {
method: 'post',
headers: {
//SEE? THIS IS EMPTY
},
body: data
})
// send form to backend
dispatch(dataSend())
}
}
Second issue could be on the backend. The thing is, that you can't just simply access file info through the req.body. You need to access it through the req.file
.post('/api/post', (req, res, next)=> {
const photo = {}
const newData = {}
uploadData(req, res, (err) => {
if(err){
console.log('error')
}
else {
Object.assign(photo, {file: req.file})
Object.assign(newData, {newData: req.body})
Then pass the photo to where you want to do something with it
const addDataController = new AddDataController(req, res, next, newAdvertData, photo)
addAdvertController.postAdvert()
}
})
Basically what I did is I separated regular data with file, and passed them further to combine and conclude the form. Sorry if this won't help, you're very close anyways!
I don't know why this worked, but everything started functioning as it should when I stopped using multer as an imported middleware, like this:
module.exports = function (req, res, next) {
/* --Convert multipart/form-data to useable format within express-- */
if (req.path === '/firmware') {
mult.single('formData')
console.log('MULTER MIDDLEWARE')
}
next()
}
and instead applied it directly to the route function, like this:
router.put('/firmware', upload.single('formData'), firmware.create) // doesn't work as standalone middleware
If anyone knows why that would be the case, please let me know.
Related
Could someone show me an example of a user basic file download using Node and Angular please. I understand it like this, but this is not working:
Nodejs:
// Does it matter that it is a post and not a get?
app.post('/some-api', someData, (request, response) => {
response.download('file/path/mytext.txt');
});
Angular 2+:
this.httpclient.post<any>('.../some-api', {...data...}).subscribe(response => {
console.log(response);
// This throws an error, but even if it doesn't,
// how do I download the Nodejs `response.download(...) ?`
});
Here are possible answers, but they are so complex, could someone just give me a super basic example (basically what I have here, but a working version). The easiest solution please.
How do I download a file with Angular2
Angular download node.js response.sendFile
There you go..
Node.js Server:
const express = require("express");
const router = express.Router();
router.post("/experiment/resultML/downloadReport",downloadReport);
const downloadReport = function(req, res) {
res.sendFile(req.body.filename);
};
Component Angular:
import { saveAs } from "file-saver"
download() {
let filename = "/Path/to/your/report.pdf";
this.api.downloadReport(filename).subscribe(
data => {
saveAs(data, filename);
},
err => {
alert("Problem while downloading the file.");
console.error(err);
}
);
}
Service Angular:
public downloadReport(file): Observable<any> {
// Create url
let url = `${baseUrl}${"/experiment/resultML/downloadReport"}`;
var body = { filename: file };
return this.http.post(url, body, {
responseType: "blob",
headers: new HttpHeaders().append("Content-Type", "application/json")
});
}
I'm building an Express app using Twilio to allow a group of people to communicate via SMS without having to install an app or deal with the limitations on group texts that some phones/carriers seem to have. It's deployed via Azure, but I'm reasonably sure I'm past the configuration headaches. As an early test that I can make this work and for a bit of flavor, I am trying to set up a feature so you can text "joke" (ideally case-insensitive) and it will send a random joke from https://icanhazdadjoke.com/. If anything else is texted, for now it should basically echo it back.
I get the sense this has to do with js being asynchronous and the code moving on before the GET comes back, so I'm trying to use promises to get the code to wait, but the conditional nature is a new wrinkle for me. I've been looking for answers, but nothing seems to work. I've at least isolated the problem so the non-joke arm works correctly.
Here is the function for retrieving the joke, the console.log is outputting correctly:
const rp = require('request-promise-native');
var options = {
headers: {
'Accept': 'application/json'
}
}
function getJoke() {
rp('https://icanhazdadjoke.com/', options) //add in headers
.then(joke => {
theJoke = JSON.parse(joke).joke
console.log(theJoke)
return theJoke
});
}
}
Here is the part of my router that isn't working quite right. If I text something that isn't "joke", I get it echoed back via SMS. If I text "joke", I don't get a reply SMS, I see "undefined" in the Kudu log (from below), and then I see the log of the POST, and then afterward I see the joke from the function above having run.
smsRouter.route('/')
.post((req, res, next) => {
const twiml = new MessagingResponse();
function getMsgText(request) {
return new Promise(function(resolve, reject) {
if (req.body.Body.toLowerCase() == 'joke') {
resolve(getJoke());
}
else {
resolve('You texted: ' + req.body.Body);
}
})
}
getMsgText(req)
.then(msg => {
console.log(msg);
twiml.message(msg);
res.writeHead(200, {'Content-Type': 'text/xml'});
res.end(twiml.toString());
})
})
How can I make it so that getMsgText() waits for the getJoke() call to fully resolve before moving on to the .then?
I think this is what you're looking for.
Note that I've used async/await rather than promise chaining.
// joke.get.js
const rp = require('request-promise-native');
var options = {
headers: {
'Accept': 'application/json'
}
}
async function getJoke() {
const data = await rp('https://icanhazdadjoke.com/', options) //add in headers
return JSON.parse(data).joke;
}
// route.js
smsRouter.route('/')
.post(async (req, res, next) => {
const twiml = new MessagingResponse();
async function getMsgText(request) {
if(req.body.Body.toLowerCase() === 'joke'){
return await getJoke();
}
return `You texted: ${req.body.Body}`
}
const msg = await getMsgText(req);
twiml.message(msg);
res.status(200).send(twiml.toString());
})
async/await in JS
I am learning how to use express, and I am able to get data, but I'm having more trouble figuring out how to send data back to update the backend. Here is an example of what it looks like.
server.js
app.route('/animals')
.get(function (req, res) {
res.send({ Cat, Dog, Bear, Wolf, etc... });
})
.patch(function (req, res) {
console.log('patch is working!')
// unsure of how to get this called with react or use req here
})
react front end
componentDidMount(){
this.callApi()
.then(res => this.setState({ name: res[this.state.animal].name }) )
.catch(err => console.log(err))
}
callApi = async () => {
const response = await fetch('/animals');
const body = await response.json();
if (response.status !== 200) throw Error(body.message);
return body;
};
This works flawlessly when getting the data, so i have .get down, but I am running into walls trying to use .patch. I can't get the console.log to fire,
let alone send it data! (lets say instead of trying to get the animal name, I'm trying to update it's name.) Any ideas? Thanks ahead of time.
The fix in this case was adding JSON.stringify in addition to what RishikeshDhokare shared! I hope this can help someone else down the line.
const response = await fetch('/animals', {
method: 'PATCH',
headers: {'Content-Type':'application/json'},
body: JSON.stringify({ name: 'kitten})
});
I'm new to node.js. What I'm trying to do is to stream the upload of a file from web browser to a cloud storage through my node.js server.
I'm using 'express', 'request' and 'busboy' modules.
var express = require("express");
var request = require("request");
var BusBoy = require("busboy");
var router = express.Router();
router.post("/upload", function(req, res, next) {
var busboy = new BusBoy({ headers: req.headers });
var json = {};
busboy.on("file", function (fieldname, file, filename, encoding, mimetype) {
file.on("data", function(data) {
console.log(`streamed ${data.length}`);
});
file.on("end", function() {
console.log(`finished streaming ${filename}`);
});
var r = request({
url: "http://<my_cloud_storage_api_url>",
method: "POST",
headers: {
"CUSTOM-HEADER": "Hello",
},
formData: {
"upload": file
}
}, function(err, httpResponse, body) {
console.log("uploaded");
json.response = body;
});
});
busboy.on("field", function(name, val) {
console.log(`name: ${name}, value: ${value}`);
});
busboy.on("finish", function() {
res.send(json);
});
req.pipe(busboy);
});
module.exports = router;
But I keep getting the following error on the server. What am I doing wrong here? Any help is appreciated.
Error: Part terminated early due to unexpected end of multipart data
at node_modules\busboy\node_modules\dicer\lib\Dicer.js:65:36
at nextTickCallbackWith0Args (node.js:420:9)
at process._tickCallback (node.js:349:13)
I realize this question is some 7 months old, but I shall answer it here in an attempt help anyone else currently banging their head against this.
You have two options, really: Add the file size, or use something other than Request.
Note: I edited this shortly after first posting it to hopefully provide a bit more context.
Using Something Else
There are some alternatives you can use instead of Request if you don't need all the baked in features it has.
form-data can be used by itself in simple cases, or it can be used with, say, got. request uses this internally.
bhttp advertises Streams2+ support, although in my experience Streams2+ support has not been an issue for me. No built in https support, you have to specify a custom agent
got another slimmed down one. Doesn't have any special handling of form data like request does, but is trivially used with form-data or form-data2. I had trouble getting it working over a corporate proxy, though, but that's likely because I'm a networking newb.
needle seems pretty light weight, but I haven't actually tried it.
Using Request: Add the File Size
Request does not (as of writing) have any support for using transfer-encoding: chunked so to upload files with it, you need to add the file's size along with the file, which if you're uploading from a web client means that client needs to send that file size to your server in addition to the file itself.
The way I came up with to do this is to send the file metadata in its own field before the file field.
I modified your example with comments describing what I did. Note that I did not include any validation of the data received, but I recommend you do add that.
var express = require("express");
var request = require("request");
var BusBoy = require("busboy");
var router = express.Router();
router.post("/upload", function(req, res, next) {
var busboy = new BusBoy({ headers: req.headers });
var json = {};
// Use this to cache any fields which are file metadata.
var fileMetas = {};
busboy.on("file", function (fieldname, file, filename, encoding, mimetype) {
// Be sure to match this prop name here with the pattern you use to detect meta fields.
var meta = fileMetas[fieldname + '.meta'];
if (!meta) {
// Make sure to dump the file.
file.resume();
// Then, do some sort of error handling here, because you cannot upload a file
// without knowing it's length.
return;
}
file.on("data", function(data) {
console.log(`streamed ${data.length}`);
});
file.on("end", function() {
console.log(`finished streaming ${filename}`);
});
var r = request({
url: "http://<my_cloud_storage_api_url>",
method: "POST",
headers: {
"CUSTOM-HEADER": "Hello",
},
formData: {
// value + options form of a formData field.
"upload": {
value: file,
options: {
filename: meta.name,
knownLength: meta.size
}
}
}
}, function(err, httpResponse, body) {
console.log("uploaded");
json.response = body;
});
});
busboy.on("field", function(name, val) {
// Use whatever pattern you want. I used (fileFieldName + ".meta").
// Another good one might be ("meta:" + fileFieldName).
if (/\.meta$/.test(name)) {
// I send an object with { name, size, type, lastModified },
// which are just the public props pulled off a File object.
// Note: Should probably add error handling if val is somehow not parsable.
fileMetas[name] = JSON.parse(val);
console.log(`file metadata: name: ${name}, value: ${value}`);
return;
}
// Otherwise, process field as normal.
console.log(`name: ${name}, value: ${value}`);
});
busboy.on("finish", function() {
res.send(json);
});
req.pipe(busboy);
});
module.exports = router;
On the client, you need to then send the metadata on the so-named field before the file itself. This can be done by ordering an <input type="hidden"> control before the file and updating its value onchange. The order of values sent is guaranteed to follow the order of inputs in appearance. If you're building the request body yourself using FormData, you can do this by appending the appropriate metadata before appending the File.
Example with <form>
<script>
function extractFileMeta(file) {
return JSON.stringify({
size: file.size,
name: file.name,
type: file.type,
lastUpdated: file.lastUpdated
});
}
function onFileUploadChange(event) {
// change this to use arrays if using the multiple attribute on the file input.
var file = event.target.files[0];
var fileMetaInput = document.querySelector('input[name=fileUpload.meta]');
if (fileMetaInput) {
fileMetaInput.value = extractFileMeta(file);
}
}
</script>
<form action="/upload-to-cloud">
<input type="hidden" name="fileUpload.meta">
<input type="file" name="fileUpload" onchange="onFileUploadChange(event)">
</form>
Example with FormData:
function onSubmit(event) {
event.preventDefault();
var form = document.getElementById('my-upload-form');
var formData = new FormData();
var fileUpload = form.elements['fileUpload'];
var fileUploadMeta = JSON.stringify({
size: fileUpload.size,
name: fileUpload.name,
type: fileUpload.type,
lastUpdated: fileUpload.lastUpdated
});
// Append fileUploadMeta BEFORE fileUpload.
formData.append('fileUpload.meta', fileUploadMeta);
formData.append('fileUpload', fileUpload);
// Do whatever you do to POST here.
}
I downloaded my XML sitemap from the sitemap xml generator website. I placed my sitemap.xml on my public directory but when I tried to submit the sitemap.xml into google console i received the following error: General HTTP error: 404 not found
HTTP Error: 404So i codedapp.get('/sitemap.xml', function( req, res, next ) {
res.header('Content-Type', 'text/xml');
res.render( 'sitemap' );
)};And when i navigate to the 'website/sitemap.xml' I am getting the following error: This page contains the following errors:
error on line 1 at column 42: Specification mandate value for attribute itemscope
Thanks for your help
Generate your sitemap.xml file using a tool like https://www.xml-sitemaps.com/
upload the sitemap.xml in your project
then add this to your .js file:
router.get('/sitemap.xml', function(req, res) {
res.sendFile('YOUR_PATH/sitemap.xml');
});
make sure you change YOUR_PATH for the actual path where your sitemap.xml file is.
Sitemaps do not have to be XML documents. A simple text file with URLs is all you need so something like below works fine. In the following example, fetchMyUrls() would be a function/method that asynchronously gets and returns the available URLs as an array of strings (URL strings).
async function index (req, res){
return fetchMyUrls().then((urls) => {
var str = '';
for (var url of urls) {
str = str + url + '\n';
}
res.type('text/plain');
return res.send(str);
});
}
For those looking for a way to create the XML dynamically on your code and don't want to use another library nor have a file stored in the public folder, you can use this:
app.get('/sitemap.xml', async function(req, res, next){
let xml_content = [
'<?xml version="1.0" encoding="UTF-8"?>',
'<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">',
' <url>',
' <loc>http://www.example.com/</loc>',
' <lastmod>2005-01-01</lastmod>',
' </url>',
'</urlset>'
]
res.set('Content-Type', 'text/xml')
res.send(xml_content.join('\n'))
})
In my NodeJS express project and without installing any library I was able to add this to my routes with my preferred view engine (handlebar).
export const routes: RouteMapper[] = [
{
"/sitemap.xml": [
{
method: "get",
handler: (req, res) =>
res.sendFile("/src/views/sitemap.xml", { root: "." }),
},
],
},
];
Cheers!
The best way is to create a script that would automatically generate a sitemap. In a lot of cases, the URLs should be dynamic based on data from the database.
Great package for creating the sitemap in Express is sitemap package:
STEP 1
Create a middleware that will generate the sitemap dynamically and then cache it for each next call to the server. We can extract logic in separate file called sitemap_generator.js for example, and we can define and export generate_sitemap middleware for it:
const { SitemapStream, streamToPromise } = require('sitemap');
const { Readable } = require('stream');
let sitemap;
const generate_sitemap = async (req, res, next) => {
res.header('Content-Type', 'application/xml');
if (sitemap) return res.status(200).send(sitemap); // If we have a cached entry send it
let changefreq = 'weekly';
try {
let links = [
{ url: '', changefreq, priority: 1 },
{ url: 'aboutus', changefreq, priority: 0.9 },
{ url: 'blog', changefreq },
{ url: 'login', changefreq },
{ url: 'register', changefreq },
];
// Additionally, you can do database query and add more dynamic URLs to the "links" array.
const stream = new SitemapStream({ hostname: 'https://example.com', lastmodDateOnly: true })
return streamToPromise(Readable.from(links).pipe(stream)).then((data) => {
sitemap = data; // Cache the generated sitemap
stream.end();
return res.status(200).send(data.toString())
});
} catch (error) {
return res.status(500).end();
}
}
module.exports = { generate_sitemap };
STEP 2
Import generate_sitemap middleware from sitemap_generator.js in your server configuration file and mound it to the /sitemap.xml endpoint:
const { generate_sitemap } = require('./sitemap_generator');
...
app.get('/sitemap.xml', generate_sitemap);
That's it. Your sitemap should be available on /sitemap.xml endpoint now so navigate in the browser to that endpoint and check if it is there.