I have and API link what automatically starts downloading file if I follow it in address bar. Let's call it my-third-party-downloading-link-com.
But when in Express framework I set res.redirect(my-third-party-downloading-link-com). I get status code 301 and can see file content in Preview tab in developer tools. But I can't make Browser downloading this file.
My appropriate request handler is following:
downloadFeed(req, res) {
const { jobId, platform, fileName } = req.query;
const host = platform === 'production' ? configs.prodHost :
configs.stageHost;
const downloadLink = `${host}/api/v1/feedfile/${jobId}`;
// I also tried with these headers
// res.setHeader('Content-disposition', 'attachment;
// filename=${fileName}.gz');
// res.setHeader('Content-Type', 'application/x-gzip');
res.redirect(downloadLink)
}
P.S Now, to solve this problem, I build my-third-party-downloading-link-com on back-end, send it with res.end and then:
window.open(**my-third-party-downloading-link-com**, '_blank').
But I don't like this solution. How can I tell browser to start downloading content form this third-party API ?
According to the documentation, you should use res.download() to force a browser to prompt the user for download.
http://expressjs.com/es/api.html#res.download
Related
I have an API built on node.js with Fastify.
The server generates a PDF which I'm then trying to send to a client via an API request. However the request I see in Chrome under network shows the type as XHR, despite me having set the content type like so:
const fs = require('fs')
const stream = fs.createReadStream('../test.pdf', 'binary')
reply.header('Content-Type', 'application/pdf')
reply.send(stream).type('application/pdf').code(200)
On the client side, I'm just making a simple POST request and assuming if its a PDF, the browser will just download it.
axios.post('http://127.0.0.1:8080/contract', requestBody)
.then((response: any) => {
setIsSubmitting(false)
})
.catch((error: any) => {
//TODO
});
(Also, the other thought I had was is this actually the correct way to send i.e. with fs.createReadStream - it's just a static file).
Any ideas what I'm doing wrong in the above.
I had the same problem. Found out this is a problem with fastify, you can upgrade to the latest (Today is 3.27.2).
The issue: https://github.com/fastify/fastify/issues/3283
Bug fix: https://github.com/fastify/fastify/pull/3285
I have the code below in my express server
app.get('/download', async (req, res)=> {
res.sendFile( `/dl/myfile.mp4`, {headers: {'Content-Type': 'mp4'},root: __dirname})
});
The sent file does not cause a download prompt in any of my tested browsers ( Chrome, Opera ) instead it shows me a player and starts playing the file while I need to it cause a download prompt.
How can I avoid the player and trigger a download prompt using Express?
You shoule use res.download(filePathToServer)
Transfers the file at path as an “attachment”. Typically, browsers will prompt the user for download if you use it.
You can read more about it here res.download - express
So i'm learning web scraping with node 8, followed this
npm install --save request-promise cheerio puppeteer
The code is simple
const rp = require('request-promise');
const url = 'https://www.examples.com'; //good
rp(url).then( (html) => {
console.log(html);
}).catch( (e) => {
console.log(e);
});
Now if url is examples.com, i can see the plain html output, great.
Q1: If yahoo.com, it outputs binary data, e.g.
�i��,a��g�Z.~�Ż�ڔ+�<ٵ�A�y�+�c�n1O>Vr�K�#,bc���8�����|����U>��p4U>mś0��Z�M�Xg"6�lS�2B�+�Y�Ɣ���? ��*
why is this ?
Q2: Then with nasdaq.com,
const url = 'https://www.nasdaq.com/earnings/report/msft';
the above code just won't finish, seems hangs there.
Why is this please ?
I'm not sure about Q2, but I can answer Q1.
It seems like Yahoo is detecting you as a bot and preventing you from scraping the page! The most common method sites use to detect bots is via the User-Agent header. When you make a request using request-promise (which uses the request library internally), it does not set this header at all. This means websites can infer your request came from a program (instead of a web browser) because there is not User-Agent header. They will then treat you like a bot and send you back gibberish or never serve you content.
You can work around this by manually setting a User-Agent header to mimic a browser. Note this seems to work for Yahoo, but might not work for all websites. Other websites might use more advanced techniques to detect bots.
const rp = require('request-promise');
const url = 'https://www.yahoo.com'; //good
const options = {
url,
headers: {
'User-Agent': 'Mozilla/5.0 (Android 4.4; Mobile; rv:41.0) Gecko/41.0 Firefox/41.0'
}
};
rp(options).then( (html) => {
console.log(html);
}).catch( (e) => {
console.log(e);
});
Q2 might be related to this, but the above code does not solve it. Nasdaq might be running more sophisticated bot detection, such as checking for various other headers.
I have both a React APP and a Express API server on the same server/domain. Nginx is serving the React APP and proxying the express server to /api.
Nginx configuration
https://gist.github.com/dvriv/f4cff6e07fe6f0f241a9f57febd922bb
(Right now I am using the IP directly instead of a domain)
From the React APP, when the user does something I want him to download a file. I used a express route on my API server that serve the file. This works fine when the user put the URL.
This is my express route:
donwloadFile.route('/')
.get((req, res) => {
const file = '/tmp/PASOP180901.txt';
res.download(file);
});
This is my react redirect:
if (this.state.downloadFile === true) {
this.setState({ downloadFile: false });
setTimeout(() => {
window.location.href = '/api/downloadFile';
}, 100);
}
The address changes but the download don't start. If I press F5 then the download starts just fine. If I use a external URL to host my file, the download start just fine too.
Thanks
First things first. Don't use setTimeout, but rather use the callback function of setState to execute code after the state is set ensuring it has been modified. Calling the callback function will guarantee the state is changed before that code in the callback is executed.
from the official docs:
setState() enqueues changes to the component state and tells React
that this component and its children need to be re-rendered with the
updated state. This is the primary method you use to update the user
interface in response to event handlers and server responses.
setState() does not always immediately update the component. It may
batch or defer the update until later. This makes reading this.state
right after calling setState() a potential pitfall. Instead, use
componentDidUpdate or a setState callback (setState(updater,
callback)), either of which are guaranteed to fire after the update
has been applied.
setState(stateChange[, callback])
The second parameter to setState() is an optional callback function
that will be executed once setState is completed and the component is
re-rendered. Generally we recommend using componentDidUpdate() for
such logic instead.
So, instead of:
if (this.state.downloadFile === true) {
this.setState({ downloadFile: false });
setTimeout(() => {
// execute code, or redirect, or whatever
}, 100);
}
you should do:
if (this.state.downloadFile === true) {
this.setState({ downloadFile: false }, () => {
// execute code, or redirect, or whatever
});
}
Now, for your specific problem
Set headers in your server side
You can set the Content-Disposition header to tell the browser to download the attachment:
from here:
In a regular HTTP response, the Content-Disposition response header is
a header indicating if the content is expected to be displayed inline
in the browser, that is, as a Web page or as part of a Web page, or as
an attachment, that is downloaded and saved locally.
Set it like this:
('Content-Disposition: attachment; filename="/tmp/PASOP180901.txt"');
Force download from the client
There are multiple ways to force the download from the client, but this is the only one I've tried.
For this to work, you have to have the content of text somehow in the client (your express route can return it for example) and create a filename for the file that will be downloaded.
let element = document.createElement('a');
element.setAttribute('href', 'data:text/plain;charset=utf-8,' + encodeURIComponent(text));
element.setAttribute('download', filename);
document.body.appendChild(element);
element.click();
document.body.removeChild(element);
Basically you are creating an empty link component, setting the data attribute to it with the text's content, attaching the link to the body, clicking the link and then removing the link from the body.
Open the link in a new tab
Opening the link in a new tab will trigger the download as well:
window.open('/api/downloadFile');
Redirect programatically
Have a look at this question in SO
You can do this:
this.props.history.push("/api/downloadFile")?
If cannot access this.props.history you can import { withRouter } from 'react-router-dom'; and export default withRouter(yourComponent); to access it.
I have a react frontend and a nodejs/express backend. The backend can serve files (downloads) via a protected route:
GET /api/file/:id
When the frontend wants to start a file download, it sends a request to this endpoint using this code (and the Authorization header is set with a valid token):
axios.get(`${apiURL}/file/${id}`)
.then(file => {
...
})
The backend responds with this code:
router.get('/file/:id', requireAuth, = (req, res, next) => {
...
res.set('Content-Type', 'application/pdf');
res.download(file.path, file.filename);
};);
This works fine: the download starts and the binary data is present in the file object of the .then() case of the axios call.
My question:
Instead of downloading the complete file I would like to pass the file to the browser BEFORE the file download starts, so that the browser handles the download like a usual browser-triggered-download, where the browser prompts wether it should download or display the file. How can this be done...?
The solution:
Thanks to #FakeRainBrigand tip I added token based authorization via request parameters to the route (using passport strategy):
const jwtOptions = {
jwtFromRequest: ExtractJwt.fromExtractors([
ExtractJwt.fromHeader('authorization'),
ExtractJwt.fromUrlQueryParameter('token'),
]),
secretOrKey: secretKey,
};
Now I can pass the download directly to the browser to handle it:
window.location.href = `${apiURL}/file/${_id}?token=${token}`;
The Authorization header is going to cause issues here. You should use cookies, at least for this feature. Otherwise, you'll have to include a token in the URL.
The two solutions are window.open in response to an event, or navigating to the url (e.g. location.href = '...').
At least in some browsers, they won't actually navigate when there's a file download, despite you changing the location.