I'm trying to create a system that would read a file from another site and serve this as a file download to the user. The file size could vary from Mbs to Gbs.
I have have already created a proof of concept using vanilla nodejs.
const app = express();
app.get('/', (req, res) => {
res.setHeader('Content-disposition', 'attachment; filename=FILENAME');
res.setHeader('Content-type', 'application/octet-stream');
http.get('URL_TO_FILE', data => {
data.pipe(res);
});
});
I would like to do this using loopback 4. Is this possible? Thanks!
You can do this easily with nodejs request and using the loopback 4 controllers.
You need to import this packages:
var request = require('request');
var fs = require('fs');
And put this code in the endpoint method:
#get('/getImage')
async retrieveImage(): Promise<any> {
let file = fs.createWriteStream(__dirname + `/file.jpg`);
console.log('the file is here: ' + file.path);
return await new Promise((resolve, reject) => {
request({
url: 'URL_TO_FILE',
headers: {
'Content-disposition': 'attachment; filename=FILENAME',
'Content-type': 'application/octet-stream',
},
gzip: true,
})
.pipe(file)
.on('finish', () => {
console.log(`The file is finished downloading.`);
resolve();
})
.on('error', (error: any) => {
reject(error);
})
}).catch(error => {
console.log(`something happened: ${error}`)
});
}
From this you need to go to this url:
http://localhost:3000/getImage
And the file would be in your controllers folder with the "file.jpg" name.
Related
I'm trying to upload a file to nanosets API. I uploaded the following node js function to firebase and trying to excess it with following URL for example with a file in body (trying to test this with postman)
Node js function looks like:
exports.uploadFile = functions.https.onRequest((req, res) => {
cors(req, res, () => {
if (req.method !== "POST") {
return res.status(500).json({
message: "Not allowed"
});
}
const busboy = new Busboy({headers: req.headers});
let uploadData = null;
busboy.on("file", (fieldname, file, filename, encoding, mimetype) => {
const filepath = path.join(os.tmpdir(), filename);
uploadData = {modelId: '4bc54977-60cf-4415-a417-c39f1c18b83f', file: fs.createReadStream(filename), type: mimetype};
const options = {
url: 'https://app.nanonets.com/api/v2/OCR/Model/XXXXXXX-60cf-4415-a417-c39f1c18b83f/LabelFile/',
formData: uploadData,
headers: {
'Authorization': 'Basic ' + Buffer.from('tiOJNxuDbdl40lXXXXXXXXXXFTYbY' + ':').toString('base64')
}
};
request.post(options, function (err, httpResponse, body) {
if (err) {
console.log(err);
}
console.log(body)
});
});
busboy.on("finish", () => {
res.status(200).json({
message: "It worked!"
});
});
busboy.end(req.rawBody);
});
});
Why I check the logs with firebase functions:log I get the following results:
2020-06-06T09:35:06.168774140Z D uploadFile: Function execution started
2020-06-06T09:35:06.344Z I uploadFile: invoice_4.pdf
2020-06-06T09:35:06.432Z I uploadFile: FileStream {
2020-06-06T09:35:06.439Z E uploadFile: TypeError: source.pause is not a function
Anyone an idea? How to pass the file to nanosets?
In order to make outgoing requests from Cloud Functions, your project must be on the Blaze payment plan.
I am a beginner in Reactjs and Nodejs. I am trying to download a file(.pdf,.jpg) from mongodb atlas in react using node js. I have used createReadStream method from Gridfs to fetch the data at server, need to send this data as a file to react so that user can download it when required. But can't get the file at client side.
FileServer.js
app.use('/download', (req, res) => {
// Check file exist on MongoDB
var filename = req.query.filename;
console.log(req.body.filename);
gfs.exist({ filename: req.body.filename ,"root": "uploads"}, (err, file) => {
console.log(file);
if (err || !file) {
console.log("error")
res.status(404).send('File Not Found');
return
}
res.setHeader('Content-Type', 'application/octet-stream');
res.setHeader('Content-Disposition', 'attachment; filename="'+req.body.filename+'"');
var str='';
var readstream = gfs.createReadStream({ filename: req.body.filename });
readstream.pipe(res);
});
});
FileClient.js
onSubmitDownload(e){
e.preventDefault();
axios.post("http://localhost:3000/file_server/download", {filename:'MyFile.pdf'})
.then(res => res.data).then(res=>this.setState({
msg:res
}));
}
Unable to take response in file format
The trick was in getting the response in the format we want, here we need to receive the data in file format. So the code at client side using Filesaver is
onSubmitDownload(e){
e.preventDefault();
axios({
method: "GET",
url: "http://localhost:8000/download",
responseType: "blob"
}).then(response => {
this.setState({ fileDownloading: true }, () => {
FileSaver.saveAs(response.data, data1);
console.log(response);
});
})
.then(() => {
this.setState({ fileDownloading: false });
console.log("Completed");
});
}
you can also check https://github.com/ANS-Developers113/File-Upload-Download-Application
I have a service, what receive a post request with a file and json data too. I use the body-parser package either in the app.js. I want to send the file to a "filer" service, and process the answer from that, but I don't want to pipe the request, because I need to process the json content too and make some actions after the filer answered.
const Busboy = require('busboy');
const request = require('request');
const sendFile = (req, file, callback) => {
return request.post({
uri: 'http://localhost:5000/stream',
headers: req.headers,
formData: { value: file }
}, (err, resp, body) => {
if (err) return callback(err);
return callback();
});
};
app.post('/route', (req, res, next) {
const busboy = new Busboy({ headers: req.headers });
busboy.on('file', (fieldName, file) => {
file.on('error', err => reject(err));
return sendFile(req, file, (err, link) => {
file.resume();
if (err) return reject(err);
});
});
busboy.on('field', (fieldName, val) => {
// process the json here...
});
busboy.on('finish', () => {
console.log('busboy.on(finish)');
return next();
});
req.pipe(busboy);
}
The filer service the following:
app.post('/stream', (req, res, next) => {
const busboy = new Busboy({ headers: req.headers });
// here we are ok
busboy.on('file', function (fieldName, file, name) {
// but this part never run
res.send(200, { fileId: fileDoc._id });
});
return req.pipe(busboy);
});
Unfortunatelly the filer service never answer, and I don't know, where is a missing part. Tried to put the file.resume() to some places inside the busboy.on('file'), but doesn't helped.
Its probably because the file stream from busboy is never processed properly by request formData.
Another way is:
Temporarily write the stream to a local file in main service.(using fs.createWriteStream)
Create a stream from that file and pipe it to filer service.(using fs.createReadStream)
Do whatever processing you need to do in main service.
Wait for response from filer service and call next()
This way you can even use the file in main service if you need to or send the file to another service.
Working with express to create a file transfer tool, and I've almost gotten everything completed. Just need to figure out how to get the data from the request written to file.
My issue appears to be stemming from not knowing where the file contents are placed in the request object.
My code to process sending the request
let file = watcher.getOneFile(config.thisLocation);
console.dir(file);
let contents = fs.readFileSync(file.fullPath, 'utf-8');
console.log(contents);
let form = {
attachments: [
contents
]
}
rq.post({
url: `http://${homeAddress}:${port}/files/?loc=${config.thisLocation}&file=${file.fileName}`,
headers: {'content-type':'application/x-www-form-urlencoded'},
formData: form
}, (err, res, body) => {
// body = JSON.parse(body);
console.log(body);
});
and when I get the request on the server, I'm not sure where the file contents actually are.
Code for handling the request
app.post('/files', (req, res) => {
console.log(req.query.loc);
// console.dir(req);
let incoming = watcher.getOutputPath(req.query.loc, config.locations);
console.log(incoming);
console.dir(req.body);
// console.log(req.body);
// let body = JSON.parse(req.body);
console.log(req.query);
let filename = path.join(incoming, req.query.file);
console.log(filename);
fs.writeFile(filename, req.body, (err) => {
if(err){
console.error(err);
}
console.log(`Successfully wrote file: ${path.join(incoming, req.query.file)}`);
});
res.sendStatus(200);
});
Where on the Request Object is the file contents?
Unfortunately you can't access the file content in any straightforward way. I recommend you to use busboy or similar package to parse form-data requests.
Here is how can you read file content using busboy and write it to the file system:
const Busboy = require('busboy');
app.post('/files', (req, res) => {
const busboy = new Busboy({ headers: req.headers });
busboy.on('file', (fieldname, file, filename, encoding, mime) => {
const newFilename = `${Date.now()}_${filename}`,
newFile = fs.createWriteStream(newFilename);
file.pipe(newFile);
file.on('end', () => {
console.log(`Finished reading ${filename}`);
});
});
busboy.on('finish', () => {
console.log('Finished parsing form');
res.sendStatus(200);
});
req.pipe(busboy);
});
I am using node.js and express. I want to create a file on the server and then download when the end point is hit.
Here is the code I currently have.
router.get('/download', (req, res) => {
const fileController = new FileController();
fileController.generateJSONFile()
.then((file) => {
fs.writeFile('fooFile.json', file, 'utf8');
}).then((success) => {
res.download('fooFile.json');
})
.catch((error) => {
res.status(500).send();
});
});
I would also like it to immediately delete the file off the server after the download is completed.
I do not need to use res.download() if there is a better way to accomplish this goal.
Would this help? This should tell the browser to download the file as fooFile.json. This would not require saving of temporary file.
router.get('/download', (req, res) => {
const fileController = new FileController();
fileController.generateJSONFile()
.then((file) => {
res.setHeader('Content-Type', 'application/octet-stream; charset=utf-8');
res.setHeader('Content-Disposition', 'attachment; filename="fooFile.json"');
res.send(file);
})
.catch((error) => {
res.status(500).send();
});
});
if you just want to send json.
res.setHeader('Content-Type', 'application/json; charset=utf-8');
res.send(file);