After i upload a file to my google drive, i am saving the information about it in the database and when someone wants to download that particular file, the only information needed to be provided is the id of the saved file in the database.
The sample code below is working very well but the only problem is that when there is an internet connection problem the file downloading process is terminated and when the user try to assume the download, the file download will start afresh.
Note: When requesting a file from google drive, i can also provide ranges but i don't know how to know when the client is requesting for a partial file so that i can include them in the request. My english is bad but i hope my question is understood
app.get("/download", async (req, res) => {
try {
const fileId = req.query.file;
if (!fileId) return res.status(400).json({ msg: "file is needed" });
const file = await File.findById(fileId);
if (!file) return res.status(404).json({ msg: "not found" });
const title = file.title
.replace(/[-&\/\\#, +()$~%.'":*?<>{}]/g, " ")
.trim();
const ext = file.file_type == "audio" ? ".mp3" : ".mp4";
const resp = await drive.files.get(
{
fileId: file.file_id,
alt: "media",
},
{ responseType: "stream" }
);
res.set({
"Content-Length": file.file_size,
"Content-Disposition": `attachment; filename=${title}${ext}`,
});
resp.data.pipe(res);
} catch (error) {
console.log(error.message);
res.status(500).send("something went wrong");
}
})
Related
So i built an e-learning platform with node.js and vue.js, and i am using GCP buckets to store my videos privately, everything works perfectly asides the fact that my videos can not fast forward or rewind, if you try moving the video to a specific position (maybe towards the end of the video) it returns to the same spot where you were initially, at first i taught it was a vue problem, but i tried playing this videos from my GCP bucket dashboard directly but it does the same thing. it only works fine when i use the firefox browser.
i am using the Uniform: No object-level ACLs enabled access control and the Not public permission settings. I am new the GCP, i have no idea what could be the problem
here is the node.js function i am using
const upload = async (req, res) => {
try {
if (!req.file) {
res.status(400).send('No file uploaded.');
return;
}
const gcsFileName = `${Date.now()}-${req.file.originalname}`;
var reader = fs.createReadStream('uploads/'+req.file.originalname);
reader.pipe(
bucket.file(gcsFileName).createWriteStream({ resumable: false, gzip: true })
.on('finish', () => {
// The public URL can be used to directly access the file via HTTP.
const publicUrl = format(
`https://storage.googleapis.com/bucketname/` + gcsFileName
);
// console.log('https://storage.googleapis.com/faslearn_files/' + gcsFileName)
fs.unlink('uploads/' + req.file.originalname, (err) => {
if (err) {
console.log("failed to delete local image:" + err);
} else {
console.log('successfully deleted local image');
}
});
res.status(200).send(publicUrl);
})
.on('error', err => {
console.log(err);
return
})
//.end(req.file.buffer)
)
// Read and display the file data on console
reader.on('data', function (chunk) {
console.log('seen chunk');
});
} catch (err) {
console.log(" some where");
res.status(500).send({
message: `Could not upload the file: ${req.file.originalname}. ${err}`,
});
}
};
the issue was comming from the way i encoded the video, i was supposed to use the blob but i used the pipe
Below I have selected 3 files. On submit the the files get uploaded successfully to the destination. But also I want to read all the 3 files attributes like filename, path etc. on server side means
in nodejs. How to do that?
I am trying to fnd it in req.body and req.query by printing it in console but i am getting all as one object : [object Object]
Below is my node side code:
router.post('/api/upload/:cid',function(req,res){
// console.log("REQ",req); //file is there in the body
console.log("yes it came under api");
console.log("req.body = "+ req.body);
console.log("req.query = "+ req.query);
console.log("req.data = "+ req.data);
upload2(req,res,function(err) {
if(err) {
return res.end("Error uploading file.");
}
else
{
console.log("File is uploaded");
res.redirect('/taskswriter');
}
});
})
Here is the AJAX code:
$.ajax({
type: "POST",
url: $(this).attr('action'),
data: formData,
processData: false,
contentType: false,
success: function(r){
console.log("result",r)
},
error: function (e) {
console.log("some error", e);
}
});
if you are using multer on server side, to get files data (filename, path etc.) you need to get it from:
// If you are uploading multiples files
console.log(request.files);
// If you are uploading one file
console.log(request.file);
So, probably you will be able to get it with:
router.post('/api/upload/:cid',function(req,res){
console.log("req.files", req.files);
I want to be able to send the file downloaded from google cloud directly to the client and not have to first save on my server then create a download to client from the saved version on my server, cause this make the process slow, as the file is downloaded two times, first from google cloud to my own server then from my own server to client.
router.get("/:filename", async(req, res) => {
try {
// Grab filename from request parameter
const fetchURL =req.params.filename;
const file = await File.findOne({fetchURL});
const srcFileName = file.originalname;
// Call GCS with bucketName and check the file method with srcFileName and check again with download method which takes download path as argument
storage
.bucket(bucketName)
.file(srcFileName)
.download({
destination: path.join(process.cwd(), "downloads", srcFileName)
})
.then(() =>
res.download(path.join(process.cwd(), "downloads", srcFileName), err =>
err ? console.log(err) : null
)
)
.catch(err =>res.status(400).json({
message: err.message
}));
} catch (err) {
res.status(res.statusCode).json({
message: `There was an error downloading your file. ${err.message}`
});
}
});
This works for me in NodeJS+Express server:
const {Storage} = require('#google-cloud/storage');
const storage = new Storage({projectId, keyFilename});
router.get('/:id', async function (req, res) {
let fileName = 'test.jpg'; //For example
let contetType = 'image/jpg;' //For example
res.writeHead(200, {
'Content-Disposition': `attachment;filename=${filename}`,
'Content-Type': `${contetType}`
});
await storage
.bucket('my-bucket')
.file(`Images/${req.params.id}/${filename}`)
.createReadStream() //stream is created
.pipe(res);
});}
Here is my scenario:
I have an application built in Node with Express;
I have an external API that returns a Base64 PDF file;
I have to get this Base64 and open the file for the user;
I am unable to save the PDF on server.
I've tried many ways and I can't open the file to the user.
Ways I've tried:
const buff = Buffer.from(myBase64, 'base64');
const file = fs.writeFileSync('boleto.pdf', buff, { encoding: 'base64' });
try {
res.setHeader('Content-Length', file.size);
res.setHeader('Content-Type', 'application/pdf');
res.setHeader('Content-Disposition', 'attachment; filename=boleto.pdf');
} catch (e) {
return res.status(404).send({ error: e, message: 'File does not exist.', statusCode: 404 });
}
const buff = Buffer.from(myBase64, 'base64');
const file = fs.writeFileSync('boleto.pdf', buff, { encoding: 'base64' });
try {
res.contentType('application/pdf');
return res.status(200).sendFile('boleto');
} catch (e) {
return res.status(404).send({ error: e, message: 'File does not exist.', statusCode: 404 });
}
const buff = Buffer.from(myBase64, 'base64');
const file = fs.readFileSync(buff, { encoding: 'base64' });
try {
res.contentType('application/pdf');
return res.status(200).sendFile(file);
} catch (e) {
return res.status(404).send({ error: e, message: 'File does not exist.', statusCode: 404 });
}
Can someone help me?
The proper way to do this here would be to call the service and direct the base64 string response to a decoding stream, then pipe that to the response output. That would save you having to wait for files to download or for the string -> byte translation to finish.
However if you are only dealing with small files (<1MB) and you dont have to handle traffic from thousands of concurrent requests its probably fine to just download the base64 string and use Buffer.from(base64str, 'base64') to decode it before passing it onwards.
This "minimal implementation" approach goes something like this:
const axios = require('axios'); // or any other similar library (request, got, http...)
const express = require('express');
const router = express.Router();
router.get('/invoice/:id.pdf', async (req, res) => {
// Here is the call to my external API to get a base64 string.
const id = req.params.id;
const response = await axios.get('https://myfileapi.domain.com/file/?id=' + id);
const base64str = response.data;
// Here is how I get user to download it nicely as PDF bytes instead of base64 string.
res.type('application/pdf');
res.header('Content-Disposition', `attachment; filename="${id}.pdf"`);
res.send(Buffer.from(base64str, 'base64'));
});
module.exports = router;
Note that there is no authentication here to block other users accessing this file, if you want authentication you'll have to handle that separately.
I'm trying to get the buffer of some drive pdf files so I can parse it and use the data.
I've managed to get the file names and id using async/await and a "drive.files.list" wrapped with promise. Now I need to use the file ids to get the buffer and then read it.
The function I need should return a promise that I can wait (using await) to be fulfilled to get a buffer. (My parser works fine when I get pdf buffer from website responses)
function getBuffer(drive, file) {
return new Promise((resolve, reject) => {
/////Google Auth
var jwToken = new google.auth.JWT(
key.client_email,
null,
key.private_key, ["https://www.googleapis.com/auth/drive"],
null
);
jwToken.authorize((authErr) => {
if (authErr) {
return reject([false, "Auth Error: " + authErr]);
}
});
drive.files.get({
auth: jwToken,
fileId: file.id,
alt: 'media',
supportsAllDrives: true
}, function (err, res) {
if (err) {
return reject('The API returned an error: ' + err);
};
console.log(res);
const buffer = res;
resolve(buffer);
});
});
}
And I use it this way:
var buffer = await getBuffer(drive,files[i]);
The output I get in "console.log(res)" is something like this:
...
��M�7�|�ı�[��Ξ�A����EBS]��P��r�����j�3�|�I.��i�+ϢKU���U�:[�═�,^t덲�v��=}'*8���ѻ��#ғ�s��No��-��q8E9�/f� �(�`�j'3
"╚�-��� ������[jp&��╚k��M��vy� In�:a�զ�OlN��u����6�n���q�/Y�i4�?&%��q�,��p╚.ZV&n�Ɨ��2G������X����Y
D],�ggb�&�N���G����NS�Lח\U�^R|_f<��f*�|��]�{�3�-P�~�CS��t��>g�Y��#�#7Wjۋ╗=�5�����#ջ���5]>}&v�╝═�wg��eV�^>�#�{��Ѿ��ޤ��>O�� z�?{8Ij�0╗B�.�Cjm�4������║��m�,╗�������O���fS��ӂcE��g�3(�G��}d^O������7����|�
H�N��;
{��x�bȠ��i]=���~��=��ٟ<��C��
wi��'a�-��p═M�6o��ϴ��ve��+��'
...
And when I try to use the parser (pdf2json) I get this error:
"An error occurred while parsing the PDF: stream must have data"
Thanks in advance :D
You want to download a file from Google Drive.
You want to convert the downloaded data to the buffer.
You have already been able to download files from Google Drive using googleapis with Node.js.
If my understanding is correct, how about this modification? In this modification, the file is downloaded as the stream type and the data is converted to the buffer.
Modified script:
From:
drive.files.get({
auth: jwToken,
fileId: file.id,
alt: 'media',
supportsAllDrives: true
}, function (err, res) {
if (err) {
return reject('The API returned an error: ' + err);
};
console.log(res);
const buffer = res;
resolve(buffer);
});
To:
drive.files.get(
{
auth: jwToken,
fileId: file.id,
alt: "media",
supportsAllDrives: true
},
{ responseType: "stream" },
function(err, { data }) {
if (err) {
return reject("The API returned an error: " + err);
}
let buf = [];
data.on("data", function(e) {
buf.push(e);
});
data.on("end", function() {
const buffer = Buffer.concat(buf);
console.log(buffer);
// fs.writeFile("filename", buffer, err => console.log(err)); // For testing
resolve(buffer);
});
}
);
Note:
As a test case, I could confirm that when buffer is saved to a file using fs.writeFile("filename", buffer, err => console.log(err));, the downloaded file can be created.
Reference:
google-api-nodejs-client
If I misunderstood your question and this was not the direction you want, I apologize.