Nodejs - React download file from s3 bucket using pre-signed url - node.js

I am trying to make an onClick button to download a file from S3 bucket using pre-signet url. The problem comes when I received my url. I want an automatic redirect or kind of. In other words, how can I lunch the download file after getting back my signed url?
this is my document list
The onClick event is on the Download button.
redux action
Redux action call my nodejs route
api route nodejs
Ask for pre-signed url then send it to my redux reducer.
Now in my front-end page, I got my link but I want an automatic redirect to start the file download.
Part of Component
Hope my first post isn't too messy.

I resolved my problem with a redux action. With one click I call my action, who return my pre-signed URL, then automatically click the link. This trigger download event with the original file name when I upload it to S3.
export const downDoc = (docId) => async dispatch => {
const res = await axios({ url: 'myApiCall', method: 'GET', responseType: 'blob' })
.then((response) => {
console.log(response)
const url = window.URL.createObjectURL(new Blob([response.data]));
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', `${docId.originalName}`);
document.body.appendChild(link);
link.click();
});

The other answer does direct DOM manipulation, creates a blob, which looks as though it buffers the whole file in memory before sending it to the user and also creates a new link each time you download. A react-y of doing is:
const downloadFileRef = useRef<HTMLAnchorElement | null>(null);
const [downloadFileUrl, setDownloadFileUrl] = useState<string>();
const [downloadFileName, setDownloadFileName] = useState<string>();
const onLinkClick = (filename: string) => {
axios.get("/presigned-url")
.then((response: { url: string }) => {
setDownloadFileUrl(response.url);
setDownloadFileName(filename);
downloadFileRef.current?.click();
})
.catch((err) => {
console.log(err);
});
};
return (
<>
<a onClick={() => onLinkClick("document.pdf")} aria-label="Download link">
Download
</a>
<a
href={downloadFileUrl}
download={downloadFileName}
className="hidden"
ref={downloadFileRef}
/>
</>)
See here for more info https://levelup.gitconnected.com/react-custom-hook-typescript-to-download-a-file-through-api-b766046db18a

The way I did it was different and has the advantage of being able to see the progress of the download as the file is being downloaded. If you're downloading a large file then it makes a difference UX wise as you see feedback immediately.
What I did was:
When creating the S3 presigned URL I set the content-disposition to `attachment
I used an anchor element to download the actual item <a url='https://presigned-url' download>Download me</a>

Others have mentioned simulating a click within the DOM or React, but another option is to use window.open(). You can set the target attribute to _blank to open a tab, but you do need window.open() inside the click event to prevent popup blockers from stopping the functionality. There's some good discussion on the subject here. I found this to be a better solution than simulating a click event.
Here's an example (though there may be more needed depending on how you fetch the signed_url).
function downloadDocument() {
const signedurlPromise = fetch("/signed_url")
signedurlPromise.then((response) => {
window.open(response.signed_url, "_blank");
})
}

Related

How to serve file as Readable Stream?

I'm trying to create a Request object that serves a static file.
The issue is: I need to do return new Response(<readable stream>). However, Node's fs.createReadStream returns a ReadStream (Node API) instead of a ReadableStream (which I believe is a web API).
What is the correct way to generate a ReadableStream in Node from a file?
The context:
I am using remix.run. In remix.run, I need to return a plain Response object. A plain Response object can either accept a buffer or a ReadableStream.
This is a Request object: https://developer.mozilla.org/en-US/docs/Web/API/Request
I'm not sure I have all the elements to understand the reason for return new Response(<readable stream>) so I'm going to assume that you just need to send a file in a Response.
Remix has what we call "Resource Routes", which are used for something other than rendering a component (this type of route does not export any), as a more general-purpose endpoint. You'll be able to export a loader that can manage different request methods and returns a new Response with a file and a header.
Take this example from the doc linked above which wants to render a component allowing to download a pdf:
// app/routes/reports/$id.js
export async function loader({ params }) {
return json(await getReport(params.id));
}
export default function Report() {
const report = useLoaderData();
return (
<div>
<h1>{report.name}</h1>
<Link to="pdf" reloadDocument>
View as PDF
</Link>
{/* ... */}
</div>
);
}
You will notice <Link to="pdf" reloadDocument> where "pdf" matches the route below:
// app/routes/reports/$id/pdf.ts
export async function loader({ params }) {
const report = await getReport(params.id);
const pdf = await generateReportPDF(report);
return new Response(pdf, {
status: 200,
headers: {
"Content-Type": "app/pdf",
},
});
}
Hoping that it will be useful to you if I guessed correctly what you needed!

Why is the S3 Url not able to be added to the object and submitted to MongoDB?

I am working on this nextjs jobboard website app and I have a job object which has several key value pairs that gets filled up over several sibling components and is to be submitted by the current component. The current component's job is to upload the job file attachment to s3, retrieve the url, and store it in the job object and upload it to mongodb. The file was successfully added to s3 and able to grab the url. However after retrieving the file url from s3, it was not able to add the string url to the job object which has a specific string field for the url. When i click submit job from the UI, the joblisting was able to be submitted to mogodb with all the other fields filled with values from other sibling components but failed to add the string URL. I have tried to find a way to solve it but was unable to even though the logic seems so clear.
I need help. Thank you in advance
const submit = async (e) =>{
setsubmitjob(true);
let {url} = await uploadToS3(file);
setFileUrl(url);
setjob({
...jobarray, joblistingattachment: url
})
createjoblist();
}
const createjoblist = async () =>{
try {
const res = await axios.post("http://localhost:3000/api/job", jobarray);
setuploadstatus(true);
sleep(3500).then(() => {
router.push("/");
})
} catch (e) {
console.log("Connection to server failed. Please try again in a minute")
}
}

KendoReact upload - passing file Id to React after saving file

I'm a bit stuck with the kendo react upload control.
I need to customise the rendering of the kendo react upload control.
After the user upload the file, I want to save the file in database. Then I need to pass the database Id back to the client because if the user then wants to remove the file I also need to clear the database.
To give you an idea this is what I would like to achieve.
https://stackblitz.com/edit/react-ghna5h
(When you start stackblitz open file app/main.jsx)
Is it possible?
Thanks for your help
I solved this particular issue using a combination of the onStatusChange and onBeforeRemove events. The server's 'save' endpoint returns an id (responseUID) that can be accessed through the onChange event's response object. I set this id on the 'file' object. Then when you're removing the file I pass the responseUID along in the additionalData field which gets put into the POST body.
const onStatusChange = (e) => {
if (e.response) {
const fileId = e.response.response.responseUID;
// This does not deal with multiple or batch uploads.
e.affectedFiles[0].responseUID = fileId;
}
};
const onBeforeRemove = (e) => {
e.additionalData.responseUID = e.files[0].responseUID;
};
return (
<Upload
batch={false}
defaultFiles={[]}
withCredentials={false}
saveUrl="https://localhost/upload/save"
removeUrl="https://localhost/upload/remove"
onStatusChange={onStatusChange}
onBeforeRemove={onBeforeRemove}
/>
);

Sending an Excel file from backend to frontend and download it at the frontend

I had created an Excel file at the backend (Express JS) using Exceljs npm module. I have it stored in a temp directory. Now I would want to send the file from the back-end to the front-end and download it there when the user clicks a button. I am struck on two things
1. How to send the file from the backend to the frontend through an HTTP POST request
2. How to then download the file in the front-end
Edited content:
I need the front end to be a button that appends the file to it and then download it. This is how my code looks, I am not getting the file properly from the backend to the front-end
front end file:
function(parm1,parm2,parm3){
let url =${path}?parmA=${parm1}&parmB=${parm2}&parmC=${parm3};
let serviceDetails = {};
serviceDetails["method"] = "GET";
serviceDetails["mode"] = "cors";
serviceDetails["headers"] = {
"Content-Type": "application/json"
};
fetch(url, serviceDetails)
.then(res => {
if (res.status != 200) {
return false;
}
var file = new Blob([res], { type : 'application/octet-stream' });
a = document.createElement('a'), file;
a.href = window.URL.createObjectURL(file);
a.target = "_blank"; 
a.download = "excel.xlsx";
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
}).catch(error => {
return false;
});
}`
router.js
var abc = ... // this is a object for the controller.js file
router.get('/path', function(req, res) {
abc.exportintoExcel(req, res);
});
controller.js
let xyz = ... //this is a object for the service.js file
exports.exportintoExcel = function(req, res) {
xyz.exportintoExcel(reqParam,res);
}
service.js
exportintoExcel(req,response){
//I have a excel file in my server root directory
const filepath = path.join(__dirname,'../../nav.txt');
response.sendFile(filepath);
})
}
This is a complete re-write of an earlier answer, so sorry if anyone needed that one, but this version is superior. I'm using a project created with express-generator and working in three files:
routes/index.js
views/index.ejs
public/javascripts/main.js
index.ejs
Start with an anchor tag that has the download attribute, with whatever filename you wish, and an empty href attribute. We will fill in the href in the main.js file with an ObjectURL that represents the Excel file later:
<body>
<a id="downloadExcelLink" download="excelFile.xlsx" href="#">Download Excel File</a>
<script type="text/javascript" src="/javascripts/main.js"></script>
</body>
public/javascripts/main.js
Select the anchor element, and then make a fetch() request to the route /downloadExcel. Convert the response to a Blob, then create an ObjectURL from this Blob. You can then set the href attribute of the anchor tag to this ObjectURL:
const downloadExcelLink = document.getElementById('downloadExcelLink');
(async () => {
const downloadExcelResponse = await fetch('/downloadExcel');
const downloadExcelBlob = await downloadExcelResponse.blob();
const downloadExcelObjectURL = URL.createObjectURL(downloadExcelBlob);
downloadExcelLink.href = downloadExcelObjectURL;
})();
routes/index.js
In the index router, you simply need to call the res.sendFile() function and pass it the path to the Excel file on your server.
router.get('/downloadExcel', (req, res, next) => {
const excelFilePath = path.join(__dirname, '../tmp/excel.xlsx');
res.sendFile(excelFilePath, (err) => {
if (err) console.log(err);
});
});
That's it! You can find a git repo here of the project. Clone into it and try it out for yourself if you can't get this code to work in your project as it is.
How It Works
When the page loads, 4 requests are fired off to our server, as we can see in the console output:
GET / 200 2.293 ms - 302
GET /stylesheets/style.css 200 1.123 ms - 111
GET /javascripts/main.js 200 1.024 ms - 345
GET /downloadExcel 200 2.395 ms - 4679
The first three requests are for index.ejs (/), the CSS stylesheet, and our main.js file. The fourth request is sent by our call to fetch('/downloadExcel') in the main.js file:
const downloadExcelResponse = await fetch('/downloadExcel');
I have a route-handler setup in routes/index.js at this route that uses res.sendFile() to send a file from our filesystem as the response:
router.get('/downloadExcel', (req, res, next) => {
const excelFilePath = path.join(__dirname, '../tmp/excel.xlsx');
res.sendFile(excelFilePath, (err) => {
if (err) console.log(err);
});
});
excelFilePath needs to be the path to the file on YOUR system. On my system, here is the layout of the router file and the Excel file:
/
/routes/index.js
/tmp/excel.xlsx
The response sent from our Express server is stored in downloadExcelResponse as the return value from the call to fetch() in the main.js file:
const downloadExcelResponse = await fetch('/downloadExcel');
downloadExcelResponse is a Response object, and for our purposes we want to turn it into a Blob object using the Response.blob() method:
const downloadExcelBlob = await downloadExcelResponse.blob();
Now that we have the Blob, we can call URL.convertObjectURL() to turn this Blob into something we can use as the href for our download link:
const downloadExcelObjectURL = URL.createObjectURL(downloadExcelBlob);
At this point, we have a URL that represents our Excel file in the browser, and we can point the href to this URL by adding it to the DOM element we selected earlier's href property:
When the page loads, we selected the anchor element with this line:
<a id="downloadExcelLink" download="excelFile.xlsx" href="#">Download Excel File</a>
So we add the URL to the href here, in the function that makes the fetch request:
downloadExcelLink.href = downloadExcelObjectURL;
You can check out the element in the browser and see that the href property has been changed by the time the page has loaded:
Notice, on my computer, the anchor tag is now:
<a id="downloadExcelLink" download="excelFile.xlsx" href="blob:http://localhost:3000/aa48374e-ebef-461a-96f5-d94dd6d2c383">Download Excel File</a>
Since the download attribute is present on the link, when the link is clicked, the browser will download whatever the href points to, which in our case is the URL to the Blob that represents the Excel document.
I pulled my information from these sources:
JavaScript.info - Blob as URL
Javascript.info - Fetch
Here's a gif of how the download process looks on my machine:
OK, now that I see your code, I can try and help out a little. I have refactored your example a little bit to make it easier for me to understand, but feel free to adjust to your needs.
index.html
I don't know what the page looks like that you're working with, but it looks like in your example you are creating an anchor element with JavaScript during the fetch() call. I'm just creating one with HTML in the actual page, is there a reason you can't do this?
<body>
<a id="downloadLink" download="excel.xlsx" href="#">Download Excel File</a>
<script type="text/javascript" src="/javascripts/test.js"></script>
</body
With that in hand, here is my version of your front end JS file:
test.js
const downloadLink = document.getElementById('downloadLink');
sendFetch('a', 'b', 'c');
function sendFetch(param1, param2, param3) {
const path = 'http://localhost:3000/excelTest';
const url = `${path}?parmA=${param1}&parmB=${param2}&parmC=${param3}`;
const serviceDetails = {};
serviceDetails.method = "GET";
serviceDetails.mode = "cors";
serviceDetails.headers = {
"Content-Type": "application/json"
};
fetch(url, serviceDetails).then((res) => {
if (res.status != 200) {
return false;
}
res.blob().then((excelBlob) => {
const excelBlobURL = URL.createObjectURL(excelBlob);
downloadLink.href = excelBlobURL;
});
}).catch((error) => {
return false;
});
}
I had to fill in some details because I can't tell what is going on from your code. Here are the things I changed:
Selected the DOM element instead of creating it:
Your version:
a = document.createElement('a'), file;
My version:
index.html
<a id="downloadLink" download="excel.xlsx" href="#">Download Excel File</a>
test.js
const downloadLink = document.getElementById('downloadLink');
This saves us the trouble of creating the element. Unless you need to do that for some reason, I wouldn't. I'm also not sure what that file is doing in your original.
Name the function and change parm -> param for arguments list
Your version:
function(parm1,parm2,parm3){
My version:
function sendFetch(param1, param2, param3) {
I wasn't sure how you were actually calling your function, so I named it. Also, parm isn't clear. Param isn't great either, should describe what it is, but I don't know from your code.
Create a path variable and enclose url assignment in backticks
Your version:
let url =${path}?parmA=${parm1}&parmB=${parm2}&parmC=${parm3};
My version:
const path = 'http://localhost:3000/excelTest';
const url = `${path}?parmA=${param1}&parmB=${param2}&parmC=${param3}`;
In your version, that url assignment should throw an error. It looks like you want to use string interpolation, but you need backticks for that, which I added. Also, I had to define a path variable, because I didn't see one in your code.
Cleaned up some formatting
I used 'dot' notation for the serviceDetails, but that was just personal preference. I also changed the spacing of the fetch() call, but no need to reprint that here. Shouldn't effect anything.
Create a blob from the fetch response
Your version:
var file = new Blob([res], { type : 'application/octet-stream' });
My version:
res.blob().then((excelBlob) => {
I'm not sure why you are calling the Blob constructor and what that [res] is supposed to be. The Response object returned from fetch() has a blob() method that returns a promise that resolves to a Blob with whatever MIME-type the data was in. In an Excel documents case, this is application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.
Create an ObjectURL from the Blob and add this URL to the href of the anchor tag.
Your version:
a = document.createElement('a'), file;
a.href = window.URL.createObjectURL(file);
a.target = "_blank";
a.download = "excel.xlsx";
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
My version:
const excelBlobURL = URL.createObjectURL(excelBlob);
downloadLink.href = excelBlobURL;
You have to do a bunch of DOM manipulation, which I'm not sure why you need. If you do have to dynamically create this element, then I'm not sure why you are 'clicking' it, then removing it, if the user is supposed to be able to click it. Maybe clarify for me why you are doing this, or if you really need to do it. Either way, in my version I create the ObjectURL and then assign it, but you could just as easily not store it in a variable.
Call the function that sends the fetch request.
As my function signature is:
function sendFetch(param1, param2, param3)
I needed to call it somewhere in order to fire off the request, so I did so like this:
sendFetch('a', 'b', 'c');
Right when the page loads, as you can see from the server logs:
GET / 304 0.448 ms - -
GET /javascripts/test.js 304 1.281 ms - -
GET /excelTest?parmA=a&parmB=b&parmC=c 304 0.783 ms - -
The first two requests are for the index.html page and the test.js file, then the fetch request is fired with the param's I passed in. I'm not sure how you are doing this in your app, because that is not included in your code.
Everything I just covered is Front-End. I'm assuming your server-side code is actually sending an excel file with your call to response.sendFile() in service.js. If you are sure that the file is getting sent, then the code I've given you should work, when adjusted to your app.
So, in conclusion, what this code does is:
Load an HTML page with an anchor tag with no href attribute set.
Send off a fetch() request to the server.
Turn the fetch response into a Blob, then create an ObjectURL from this Blob.
Assign that ObjectURL to the anchor tag's href attribute.
When the user clicks the 'Download Excel File' link, the Excel sheet should be downloaded. If you didn't want them to see the link until after the fetch request, you could definitely do create the anchor tag in JS instead, let me know if you want to see how to do that.
As before, here is a gif showing how it looks on my machine (this is with your version and my modifications):

Downloading files from expressJS (sendFile) server to VueJS, is returning corrupted files?

I have an API written in expressjs, that sends a file when provided with a fileID.
The backend is working fine, as it sends the correct file (uncorrupted) when the route url is typed directly int the browser.
ex. http://localhost:8080/download?id=1234 (downloads all files just fine ie.txt, xlsx, jpg, zip)
My express route actually uses res.download, which is really jsut a wrapper for sendFile.
When I try calling these route urls from a Vue http get request, it only returns txt files uncorrupted. All other files download, but they can be opened, due to corruption.
Can anyone please point me in the right direction as to why its not working in Vue?
For clarity, "item" is passed as an argument to this function.
this.$http.get("http://localhost:8000/files/download", {params:{id:item._id}}, {responseType: 'arraybuffer'})
.then(response => {
var blob = new Blob([response.data], {type:response.headers.get('content-type')});
var link = document.createElement('a');
link.href = window.URL.createObjectURL(blob);
link.download = item.filename;
link.click();
})
For reference, this is my express route
Shout out to #Helpinghand for helping me troubleshoot this.
"I found the solution within the link you posted. The problem is that i was explicitly sending params in its own object before assigning "content-type". The example you posted concatenates the query params to the url. After making this switch, its working perfectly".
this.$http.get(`http://localhost:8000/files/download?id=${item._id}`, {responseType: 'arraybuffer'})
.then(response => {
console.log(response.headers.get('content-type'));
console.log(response);
var blob = new Blob([response.body], {type:response.headers.get('content-type')});
var link = document.createElement('a');
link.href = window.URL.createObjectURL(blob);
link.download = item.filename_version;
link.click();
})

Resources