node app download file from gcloud storage to client - node.js

Seems like this should have a canned answer of some sort, but I can't find anything.
The scenario is thus:
I have a web page that lists files to download
The files are located in gcloud storage
I have a node server (express) app to reach into gcloud and fetch the file.
The web page js looks like this:
<script>
function downloadFile(client, fn) {
var x=new XMLHttpRequest();
x.open("GET", "https://<domain>/fetch/" + client + '/' + fn, true);
x.responseType = 'blob';
x.onload=function(e){download(x.response, fn, "<content-type>" ); }
x.send();
}
</script>
It's doing a straight-up XMLHttpRequest() to the server with the requred fetch args.
The relevant part of the node js looks like this:
app.get('/fetch/:client/:filename', async function(req,rsp) {
try {
const storage = new Storage();
const fileLoc = req.params.client + '/' + req.params.filename
const options = {
destination: './' + req.params.filename,
};
// Downloads the file
let file = await storage
.bucket('<bucket-name>')
.file(fileLoc)
let rstream = await file.createReadStream()
.on("error", (err) => {
console.log(" --> rstream error: " + err)
})
.on("response", (strRsp) => {
console.log(' --> response received, sending headers.')
rsp.setHeader('Content-Length', strRsp.headers['content-length'])
rsp.setHeader('Content-Type', strRsp.headers['content-type'])
rsp.header('Content-Disposition', 'attachment; filename=' + req.params.filename)
})
.on("end", () => {
console.log(' --> end received.')
rsp.status(200).end()
return true
})
.pipe(rsp)
} catch(err) {
console.log('download error: ' + err)
rsp.status(400).send('failed')
}
});
The thing is that it works, but it downloads the entire file (silently) before it puts up the save-as dialog.
Is there a way to put up the save dialog before downloading? (Perhaps I need to do the 'content-disposition' header on the client side...?)
Barring that, is there a way to reflect progress back to the web page js so that I can put up a dialog and show progress and maybe a cancel button until it is done and shows the save-as dialog?

Ok, I went with the latter answer. Doing the following:
On download request, pop up a loading dialog with information about the download.
Register a progress callback with the XMLHttpRequest instance
Display the progress information in the dialog
Dialog has a 'Cancel' button which calls 'abort()' on the XMLHttpRequest instance
On completion (or abort) update the dialog and then close it via a timeout.

Related

Download CSV file from browser after making axios call from React to Nodejs Api

I have MERN application and I want to download a CSV file on button click. I implemented everything but when I click the button there is no download in browser.
Axios call
const handleDownloadCsv = async () => {
try {
await axios.get('http://localhost:2000/users/admin-panel/csv');
} catch (error) {
console.log(error);
}
};
NodeJs controller
export const admin_panel_csv = async (req,res) => {
try {
let __dirname = res.locals.__dirname;
const myReadStream = fs.createReadStream(__dirname + '/documents/admin_csv.csv');
//myReadStream.pipe(res);
res.download(__dirname + '/documents/admin_csv.csv')
} catch (error) {
console.log('Error csv: ',error.message);
res.status(400).json({msg:error.message});
}
}
I've tried both createReadStream(with pipe) and res.download(path to file) but non of them seams to work. I am not getting any errors when making this api call through axios. Is there some way to accomplish this without using React libraries.
There is no download prompt in the browser since you are initiating the download via axios and not through the browser (e.g., through a <form> POST or an <a> click).
Change the back-end code back to res.download and on the front-end, initiate the download through an <a> click:
const handleDownloadCsv = () => {
const tempLink = document.createElement('a')
tempLink.href = 'http://localhost:2000/users/admin-panel/csv'
tempLink.click()
}
I think that you should use js-file-download in React and just :
const FileDownload = require('js-file-download');
Axios.get(API_URL + "download")
.then((response) => {
FileDownload(response.data, 'file.txt');
});

NodeJS Express Routing - Only first request gets rendered

In the server script I try to deliver different html files. When app.post('/login'...) comes in, res.sendFile() is working and the html gets rendered. On the second call, whenn app.get('/go') comes in, the file gets served, but not displayed. I cannot explain why the second HTML file is not displayed. What am I doing wrong?
the second request comes from a fetch request in a javascript
socket.on('gameStarted', (data) => {
console.log("Game started");
fetch('/go', {method: 'GET'});
})
served but not displayed
app.post('/login', async (req, res, next) => {
var roomNR = req.body.player.gameCode;
var playerName = req.body.player.nickname;
var codeValid = await checkCode(activeRoomsCollection, gameCodes, roomNR);
var playerExists = await playerCollection.findOne({ playerName: playerName })
if (codeValid) {
if ((playerExists === null) || !playerExists) {
playerCollection.insertOne({ room: roomNR, playerName: playerName, state: false });
console.log(`Added player '${playerName}' with roomnumber '${roomNR}'`);
res.sendFile(path.join(__dirname + '/../../public/lobby.html'), function (err) {
if (err) {
console.log(err);
res.status(err.status).end();
}
else {
console.log('Sent Lobby');
}
});
} else {
// updateDomElement(player, elementId, data)
//res.send('Benutzername existiert bereits');
}
} else {
res.send('Code ungültig');
}
});
app.get('/go', (req, res, next ) => {
res.sendFile(path.join(__dirname + '/../../public/raetsel1.html'), function (err) {
if (err) {
console.log(err);
res.status(err.status).end();
}
else {
console.log('Sent Raetsel1');
}
});
});
fetch() never displays anything on its own. It's a way for your Javsascript to issue http requests to remote servers and those servers then return content back to your Javascript. The result from those http requests ONLY goes to your Javascript. Nothing in the view of the page is affected at all by a fetch() call.
If you want the result of a fetch() call to display something in your page, you would need to write Javascript to do that (to insert content into the current page).
If, instead, you just want the browser to go to a new page, then change from this:
fetch('/go', {method: 'GET'});
to this:
window.location = "/go";
This will cause the browser to go to the URL, retrieve the content and display it. This will shut-down the current page and load and display a new page and the URL in the URL-bar in the browser will show the updated location.
Note that if you have socket.io code in both pages, it will disconnect the current socket.io connection and then run the Javascript in the new page - causing it to create a new socket.io connection (if you have code in the new page to do that) as that is what happens to socket.io connections when you load and display a new web page in the browser.

empty response body in HTTP response in github actions

I'm trying to create a github action which requires sending an http request to https://www.instagram.com/<username>/?__a=1.
When I'm running it locally, it runs perfectly fine and gives me the number of followers.
But when I use it in github actions, it isn't able to parse the JSON string as the response is null
Here is a link to the github action file https://github.com/ashawe/actions-check/blob/e80ca115544979cdb3180207b99c7724e4446849/index.js
Here is the code to get the followers ( starts at line #94 )
promiseArray.push(new Promise((resolve, reject) => {
const url = 'https://www.instagram.com/' + INSTAGRAM_USERNAME + '/?__a=1';
core.info("url is");
core.info(url);
http.get(url, (response) => {
let chunks_of_data = [];
response.on('data', (fragments) => {
chunks_of_data.push(fragments);
});
response.on('end', () => {
let response_body = Buffer.concat(chunks_of_data);
core.info(response_body.toString());
let responseJSON = JSON.parse(response_body.toString());
resolve((responseJSON.graphql.user.edge_followed_by.count).toString());
});
response.on('error', (error) => {
reject(error);
});
});
}));
and then I'm processing it like:
Promise.allSettled(promiseArray).then((results) => {
results.forEach((result, index) => {
if (result.status === 'fulfilled') {
// Succeeded
// core.info(runnerNameArray[index] + ' runner succeeded. Post count: ' + result.value.length);
// postsArray.push(result.value);
instagram_followers = result.value;
} else {
jobFailFlag = true;
// Rejected
//core.error(runnerNameArray[index] + ' runner failed, please verify the configuration. Error:');
core.error(result.reason);
}
});
}).finally(() => {
try {
const followers = instagram_followers;
const readmeData = fs.readFileSync(README_FILE_PATH, 'utf8');
// core.info(readmeData);
const shieldURL = "https://img.shields.io/badge/ %40 " + INSTAGRAM_USERNAME + "-" + followers + "-%23E4405F?style=for-the-badge&logo=instagram";
const instagramBadge = "<img align='left' alt='instagram-followers' src='" + shieldURL + "' />";
const newReadme = buildReadme(readmeData, instagramBadge);
// core.info(newReadme);
// if there's change in readme file update it
if (newReadme !== readmeData) {
core.info('Writing to ' + README_FILE_PATH);
fs.writeFileSync(README_FILE_PATH, newReadme);
if (!process.env.TEST_MODE) {
// noinspection JSIgnoredPromiseFromCall
commitReadme();
}
} else {
core.info('No change detected, skipping');
process.exit(0);
}
} catch (e) {
core.error(e);
process.exit(1);
}
});
But when I run the action, it gives this error:
which means that the response_body isn't complete JSON response but a request to https://www.instagram.com/USERNAME/?__a=1 does send a json response.
UPDATE
Basically every time you hit that endpoint it returns the login html page, which causes the json parse to fail. It appears that you may need to use the api which requires you to authenticate before getting info from users. Or figure out other scraping methodologies.
I was able to recreate this failure in my local pc by jumping into a vpn and private browser. When I hit the endpoint it took me to the login screen. And when i hit the endpoint through curl in terminal it returned nothing. But when i got off the vpn, all worked fine. I think the reason it worked in your local is because there's some caching happening in the browser and you're probs not in a vpn. I am thinking there's some network blacklisting happening when on vpn. I don't know the github hosted network so I would recommend opening a ticket with them if you want to learn more about that.
Here are the instagram api docs for quick reference
https://developers.facebook.com/docs/instagram-basic-display-api/getting-started
Previews Response: Leaving here for other users future reference.
You are not passing username so it's trying to query the endpoint with empty username
Instead of running just node index.js in your action, you need to call your action and provide it with the parameters that it needs
- name: Your github action
uses: ./ # Uses an action in the root directory
with:
username: '_meroware'
Then your code will pick it put properly
const INSTAGRAM_USERNAME = core.getInput('username');
const url = 'https://www.instagram.com/' + INSTAGRAM_USERNAME + '/?__a=1';
Resources:
https://docs.github.com/en/actions/creating-actions/creating-a-javascript-action

How do I create a file in express and node on my server and then download it to my client. I am using NextJS for my frontend and backend

How do I create a file in express and node on my server and then download it to my client. I am using NextJS for my frontend and backend. I am confused on how I would download the file on the front end after the file is created on the root of the server folder. Since I am using React for my frontend whenever I try to visit that filepath it tries to take me to a page instead of the file
Here is what I have in my express route in node
var xls = json2xls(json, {
fields
});
// If there isn't a folder called /temp in the
// root folder it creates one
if (!fs.existsSync('./temp')) {
fs.mkdirSync('./temp');
}
const fileName = `temp/${req.user.first_name}${req.body._id + Date.now()}.xlsx`
// fs.writeFileSync(fileName, xls, 'binary');
fs.writeFile(fileName, xls, 'binary', function (err, result) {
if (err) {
return console.log(err);
}
console.log(result, 'this is result')
});
Here is what I have on my frontend
axios.post('api/download',payload)
.then(res => {
const link = document.createElement('a');
link.href = res.data.url;
link.download
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
})
.catch(err => {
throw err
})
Can you make request with GET on api, and.
Make request with GET.
Make temp directory to be static resources directory:
app.use(express.static('temp')); // app is your express instance.
// Maybe you have to correct temp's path
Response the post request with file url data
fs.writeFile(fileName, xls, 'binary', function (err, result) {
if (err) {
return console.log(err);
res.status(500).json({err});
}
console.log(result, 'this is result');
res.json({url: 'http://localhost:8080/temp/' + fileName}); // res is response object of you router handler.
// Maybe you have correct the server address
});
On other way, you can send the xls binary direct to client, in the client you create a BLOB object from the response, then create download link for the blob object.

Sending Zip file from server to client browser with Express and Archiver

I am a beginner with Node and I am trying to figure out how to create a zip file at the server then send it to the client and then download the zip file to the user's browser. I am using the Express framework and I am using Archiver to actually do the zipping. My server code is the following which was taken from Dynamically create and stream zip to client
router.get('/image-dl', function (req,res){
res.writeHead(200, {
'Content-Type': 'application/zip',
'Content-disposition': 'attachment; filename=myFile.zip'
});
var zip = archiver('zip');
// Send the file to the page output.
zip.pipe(res);
// Create zip with some files. Two dynamic, one static. Put #2 in a sub folder.
zip.append('Some text to go in file 1.', { name: '1.txt' })
.append('Some text to go in file 2. I go in a folder!', { name: 'somefolder/2.txt' })
.finalize();
});
So its zipping two text files and returning the result. On the client side I am using the following function in a service to actually call that endpoint
downloadZip(){
const headers = new Headers({'Content-Type': 'application/json'});
const token = localStorage.getItem('token')
? '?token=' + localStorage.getItem('token')
: '';
return this.http.get(this.endPoint + '/job/image-dl' + token, {headers: headers})
.map((response: Response) => {
const result = response;
return result;
})
.catch((error: Response) => {
this.errorService.handleError(error.json());
return Observable.throw(error.json());
});
}
and then I have another function which calls downloadZip() and actually downloads the zip file to the user's local browser.
testfunc(){
this.jobService.downloadZip().subscribe(
(blah:any)=>{
var blob = new Blob([blah], {type: "application/zip"});
FileSaver.saveAs(blob, "helloworld.zip");
}
);
}
When testfunc() is called, a zip file is downloaded to the user's browser however when I try to unzip it it creates a zip.cpgz file which then turns back into a zip file when clicked in an infinite loop which indicates that some kind of corruption happened. Can anyone see where I went wrong here?

Resources