Node/Express: No download on stream.pipe() - node.js

Is there any way to force download when i pipe stream to response?
If I look into Chrome tools, I see that response is OK, with fine headers:
HTTP/1.1 200 OK
Server: Cowboy
Connection: keep-alive
X-Powered-By: Express
Content-Type: application/pdf
Date: Mon, 10 Oct 2016 20:22:51 GMT
Transfer-Encoding: chunked
Via: 1.1 vegur
Request Headers
view source
I even see the pdf file code in detailed response, but no file download is initiated. Maybe I miss something?
My code on route looks like:
router.post('/reports/create', access.Regular, function (req, res, next) {
...
pdf.create(html).toBuffer(function(err, buffer){
res.writeHead(200, {
'Content-Type': 'application/pdf',
'Content-Disposition': 'attachment; filename=some_file.pdf',
'Content-Length': buffer.length
});
res.end(buffer)
});
});

You need to add a Content-Disposition Header to signal to the browser that there is content attached that needs to be downloaded.
app.get('/:filename', (req, res) => {
// Do whatever work to retrieve file and contents
// Set Content-Disposition Header on the response
// filename is the name of the file the browser needs to download when
// receiving the response to this particular request
res.setHeader('Content-Disposition', `attachment; filename=${req.params.filename}`);
// stream the fileContent back to the requester
return res.end(fileContent)
});

it works for me like this:
var html = '<h1>Hello World</h1>';
let options = {
format: 'Letter',
border: '1cm'
};
pdf.create(html, options).toBuffer(function (err, Buffer) {
if (err) {
res.json({
data: 'error PDF'
})
} else {
res.set({
'Content-Disposition': 'attachment; filename=test.pdf',
'Content-Type': 'application/pdf; charset=utf-8'
});
res.write(Buffer);
res.end();
}
});

Related

Node.js REST webservice with wrong Content-Type

I'm developing a REST webservice in Node.js, with fastify framework, designed to respond to a specific client.
This is how the client calls my webservice:
POST /myws/scan HTTP/1.1
Host: myhost.io
User-Agent: AGENT
Content-Length: 45
Accept: */*
Content-Type: application/x-www-form-urlencoded
X-Forwarded-For: xxx.xxx.xxx.xxx
X-Forwarded-Proto: http
Accept-Encoding: gzip
{"FIELD1":"testvalue","FIELD2":True,"FIELD3":90}
As you can see the Content-Type is "application/x-www-form-urlencoded" but the request payload is in "application/json" format.
I can't change the client, so I have to adapt my webservice to manage this kind of calls.
I tried with #fastify/formbody package but I receive an error because I think it expects the request body in "application/x-www-form-urlencoded" format.
I tried also with this code:
app.addContentTypeParser('application/x-www-form-urlencoded', function (req, body, done) {
try {
var json = JSON.parse(body)
done(null, json)
} catch (err) {
err.statusCode = 400
done(err, undefined)
}
})
but I have a JSON deserializing error.
How I can manage these kind of calls and which is the best way to do that?
body is a stream.
You need to add the parseAs option:
fastify.addContentTypeParser(
'application/x-www-form-urlencoded',
{ parseAs: 'string' },
function (req, body, done) {
try {
const json = JSON.parse(body)
done(null, json)
} catch (err) {
err.statusCode = 400
done(err, undefined)
}
}
)
I would check the first char before running the JSON.parse and I would use https://www.npmjs.com/package/secure-json-parse (as Fastify does under the hood)

NodeJS Request for unencoded PDF buffer returns some html code with PDF content that corrupts PDF file

I make a NodeJS request with encoding:null in the options, but the buffer that I receive has some <html><script> tags before the %PDF-1.6 that causes the PDF file I create to be corrupt.
If I edit the PDF file afterwards in Notepad++ and remove all the <html> tags before the %PDF-1.6, the PDF opens just fine.
My Question: Is there a request header I can change so that I don't receive the <html> and only receive the PDF content? I tried adding 'Content-Type': 'application/pdf' but it didn't work.
Here's what my request and file creation looks like:
var headers = {
'Accept': 'application/pdf', // i also tried this instead of */* but it didnt work
'Accept-Language': 'en-US,en;q=0.9',
'Connection': 'keep-alive',
'Content-Type': 'multipart/form-data; boundary=----WebKitFormBoundaryk1mzLxNfnADQgWyG',
'Cookie': myCookie,
'Origin': 'https://privateUrl',
'Referer': 'https://privateUrl',
};
var dataString = '' //some data i need to post to get access to pdf
var options = {
url: 'https://privateUrl',
method: 'POST',
encoding: null, //i need to receive buffer instead of binary to search for text using https://www.npmjs.com/package/pdf-parse
headers: headers,
body: dataString
};
request(options, function callback(error, response, body) {
//create the pdf:
var binary = Buffer.from(body, 'base64')
fsPath.writeFile('filename.pdf', binary, 'binary', (err) => {
if (err) throw err;
}); // this file is corrupt because the `binary` has a bunch of `<html>` prior to the `%PDF-1.6`
});

Using Node/Express to stream file to user for download

I want to use a Node/Express server to stream a file to the client as an attachment. I would like to make an async request from the client to a /download endpoint and then serve an object received via API proxy to the client as a downloadable file (similar to the way res.attachment(filename); res.send(body); behaves).
For example:
fetch(new Request('/download'))
.then(() => console.log('download complete'))
app.get('/download', (req, res, next) => {
// Request to external API
request(config, (error, response, body) => {
const jsonToSend = JSON.parse(body);
res.download(jsonToSend, 'filename.json');
})
});
This will not work because res.download() only accepts a path to a file. I want to send the response from an object in memory. How is this possible with existing Node/Express APIs?
Setting the appropriate headers does not trigger a download, either:
res.setHeader('Content-disposition', 'attachment; filename=filename.json');
res.setHeader('Content-type', 'application/json');
res.send({some: 'json'});
This worked for me.
I use the content type octet-stream to force the download.
Tested on chrome the json was downloaded as 'data.json'
You can't make the download with ajax according to: Handle file download from ajax post
You can use a href / window.location / location.assign. This browser is going to detect the mime type application/octet-stream and won't change the actual page only trigger the download so you can wrap it a ajax success call.
//client
const endpoint = '/download';
fetch(endpoint, {
method: 'POST',
credentials: 'include',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
}
})
.then(res => res.json())
.then(res => {
//look like the json is good to download
location.assign(endpoint);
})
.catch(e => {
//json is invalid and other e
});
//server
const http = require('http');
http.createServer(function (req, res) {
const json = JSON.stringify({
test: 'test'
});
const buf = Buffer.from(json);
res.writeHead(200, {
'Content-Type': 'application/octet-stream',
'Content-disposition': 'attachment; filename=data.json'
});
res.write(buf);
res.end();
}).listen(8888);
You can set the header to force the download, then use res.send
see those links
Force file download with php using header()
http://expressjs.com/en/api.html#res.set

express.js not streaming chunked 'text/event-stream' resposne

I'm trying to send a SSE text/event-stream response from an express.js end point. My route handler looks like:
function openSSE(req, res) {
res.writeHead(200, {
'Content-Type': 'text/event-stream; charset=UTF-8',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
'Transfer-Encoding': 'chunked'
});
// support the polyfill
if (req.headers['x-requested-with'] == 'XMLHttpRequest') {
res.xhr = null;
}
res.write(':' + Array(2049).join('\t') + '\n'); //2kb padding for IE
res.write('id: '+ lastID +'\n');
res.write('retry: 2000\n');
res.write('data: cool connection\n\n');
console.log("connection added");
connections.push(res);
}
Later I then call:
function sendSSE(res, message){
res.write(message);
if (res.hasOwnProperty('xhr')) {
clearTimeout(res.xhr);
res.xhr = setTimeout(function () {
res.end();
removeConnection(res);
}, 250);
}
}
My browser makes the and holds the request:
None of the response gets pushed to the browser. None of my events are fired. If I kill the express.js server. The response is suddenly drained and every event hits the browser at once.
If I update my code to add res.end() after the res.write(message) line It flushes the stream correctly however it then fallsback to event polling and dosen't stream the response.
I've tried adding padding to the head of the response like
res.write(':' + Array(2049).join('\t') + '\n');
as I've seen from other SO post that can trigger a browser to drain the response.
I suspect this is an issue with express.js because I had been previously using this code with nodes native http server and it was working correctly. So I'm wondering if there is some way to bypass express's wrapping of the response object.
This is the code I have working in my project.
Server side:
router.get('/listen', function (req, res) {
res.header('transfer-encoding', 'chunked');
res.set('Content-Type', 'text/json');
var callback = function (data) {
console.log('data');
res.write(JSON.stringify(data));
};
//Event listener which calls calback.
dbdriver.listener.on(name, callback);
res.socket.on('end', function () {
//Removes the listener on socket end
dbdriver.listener.removeListener(name, callback);
});
});
Client side:
xhr = new XMLHttpRequest();
xhr.open("GET", '/listen', true);
xhr.onprogress = function () {
//responseText contains ALL the data received
console.log("PROGRESS:", xhr.responseText)
};
xhr.send();
I was struggling with this one too, so after some browsing and reading I solved this issue by setting an extra header to the response object:
res.writeHead(200, {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Content-Encoding": "none"
});
Long story short, when the EventSource is negotiating with the server, it is sending an Accept-Encoding: gzip, deflate, br header which is making express to respond with an Content-Encoding: gzip header. So there are two solutions for this issue, the first is to add a Content-Encoding: none header to the response and the second is to (gzip) compress your response.

status code 400 with no error object when uploading to box

I'm trying to use the request.jslibrary to upload a file to box.com using request.post.
I consistently get a return code of 400 and a null body in the response. Not sure how to get at the actual error that box is seeing. The err argument to the callback is null, so there is a response from box.com, but with a statuscode of 400 and a null body.
FYI, the upload succeeds using curl, so the auth token etc. is fine.
I pointed the function below to http://echo.200please.com', and it seems the HTTP POST request I'm sending out is fine.
How do I get to see what error is being seen by box?
request = require"request");
UploadFile = function(filename, callback) {
var formData = {
attributes: JSON.stringify( {
name: filename,
parent: { id: '' + 2764913765 }
}),
file: fs.createReadStream('./temp.bin')
}
var options = {
url: 'https://upload.box.com/api/2.0/files.content',
headers: { Authorization: 'Bearer ' + tokens.access_token},
formData: formData
}
request.post(options,
function(err, response, body) {
if (err) {
console.log('Error Uploading the file');
} else {
console.log('returned:' + body + JSON.stringify(response.headers))
}
});
If I change the URL to point to echo.200please.com, the response I get from echo.200please.com is below, which seems to be the correct format for a file upload request.
> POST / HTTP/1.0
Host: echo.200please.com
Connection: close
Content-Length: 1951
Authorization: Bearer bVPDzG8PgIVRNoqb5LOzD61h6NXhJ6h0
content-type: multipart/form-data; boundary=--------------------------799592280904953105406767
----------------------------799592280904953105406767
Content-Disposition: form-data; name="attributes"
{"name":"testchunkname.1","parent":{"id":"2764913765"}}
----------------------------799592280904953105406767
Content-Disposition: form-data; name="file"; filename="temp.bin"
Content-Type: application/octet-stream
<... file data ...>
OK ... I found the bug :-)
It's a typo in the url
in my program it got set to api/2.0/files.content whereas the correct path in the url should be api/2.0/files/content

Resources