Currently writing code to make a skipper gridfs adapter. When you call upload, files are passed in the callback and I would like to add an extra field to the files that contain the metadata of a gridfs store and the ID of a gridfs store.
I'm looking through the Upstream code in skipper and see something called stream.extra. I'm guessing it's for passing extra data, how would I go about using this?
Thanks for working on this! You can add extra metadata to your stream by putting it on the __newFile object in the receiver's _write method. For example, in the bundled s3Receiver, you can see this on line 61:
__newFile.extra.fsName = fsName;
which is adding the newly generated filename as metadata on the uploaded file object. In your controller's upload callback, you can retrieve the extra data from the returned file objects:
req.file('myFile').upload(function (err, files) {
var newFileName = files[0].extra.fsName;
});
Related
I am using Gridfs to store large files in mognodb.
Now I am using PDFTron for pdf editing and want to watermark pdf.
The problem is i am not able to read file from Gridfs stream in pdftron nodejs sdk.
also i want to send it back to the client without storing it locally or anywhere else.
I am doing something like this...
const bucket = new mongodb.GridFSBucket(db);
const stream = bucket.openDownloadStream(ObjectId(file_id))
const pdfdoc = await PDFNet.PDFDoc.createFromFilter(stream);
the error i am getting is ...
TypeError: 1st input argument in function 'createFromFilter' is of type 'object'. Expected type 'Filter'. Function Signature: createFromFilter(PDFNet.Filter)
The PDFDoc.createFromFilter API is expecting a PDFNet Filter, not whatever GridFS is returning.
https://www.pdftron.com/api/pdfnet-node/PDFNet.PDFDoc.html#.createFromFilter__anchor
You can see this sample on creating a PDFDoc object from a Filter
https://www.pdftron.com/documentation/samples/node/js/PDFDocMemoryTest
Though the easiest is to write your GridFD stream to a buffer, and then pass that buffer to PDFDoc.createFromBuffer. https://www.pdftron.com/api/pdfnet-node/PDFNet.PDFDoc.html#.createFromBuffer__anchor
I set up my discord BOT using node.js. For my advantage, I would need to store some data on a external file, but I don't seem to be able to access it from my index.js file (the main Bot file).
I've tried having one static array in the external js/json files, but I can only retrieve undefined/empty values. Additionally, when I tried with a .txt file, once retrieved the content, I found it unable to call functions such as string.split().
Did I miss something in the package content perhaps?
Assuming the data you are storing is in UTF-8 encoding:
var fs = require('fs');
fs.readFile('path/to/file', 'utf8', function(err, contents) {
// code using file data
});
Assuming no errors contents will be a string of the data that is inside that file.
https://code-maven.com/reading-a-file-with-nodejs
I am using Node's formidable module for form processing. Works perfect. Now I need to have access to the non-file field names/values posted before the file upload starts. The field names/values are available only AFTER the file uploads are done. Is there any way to get the field names before the file uploads start?
formProcess = new formidable.IncomingForm();
...
formProcess.parse(req, function(error, myFields, myFiles) {
//I get access to the field values here...
//But only after the files are uploaded.
//I need this info before the file uploads start.
}
..
formProcess.on('progress', function(alreadyReceived, expectedToRcv) {
//Fileupload progress info available here...
//I need field names here while processing the upload progress.
//Application specific requirement...
}
Does this have to do with how http post method works or is it specific to the implementation of Node's Formidable module?
There are file and field events that are emitted. Just make sure your non-file fields come before your file fields in your form since fields are sent/received in order.
I have a file upload system in sails.js app. I want to process the uploads before saving them in the server. My form on the client side allows multiple file uploads. Now on the server side how do I know how many files were sent?
For example I can find the total bytes to be expected from the upload using the following:
req._fileparser.form.bytesExpected
However, I couldn't find something similar that helps me find the total number of files sent to the server.
Also the above code req._fileparser.form.bytesExpected, is there a better way to get total combined file size of the files sent through the upload form by the client?
In the github repository for Skipper there is a file: index.js
Line 92 from the above file, which appears to deal with multipart file uploads, contains the following:
var hasUpstreams = req._fileparser && req._fileparser.upstreams.length;
You should check the length of upstreams in your code, and see if that contains the number of files you sent.
Another option: send a parameter in your request from the client with the number of files uploaded.
See the skipper ReadMe section about Text Parameters.
Skipper allows you to access the other non-file metadata parameters (e.g "photoCaption" or
"commentId") in the conventional way. That includes url/JSON-encoded HTTP body parameters
(req.body), querystring parameters (req.query), or "route" parameters (req.params); in other words,
all the standard stuff sent in standard AJAX uploads or HTML form submissions. And helper methods
like req.param() and req.allParams() work too.
I've just found a previous question/answer on stackoverflow.
You might try using var upload = req.file('file')._files[0].stream to access and validate, as shown in the above answer.
hi i am a newbie to gridfs and am able to insert a file and view the file in gridfs using the query below
mongofiles -d myfiles put hi.txt
db.fs.files.findOne({'filename':'hi.txt'});
I need to view the contents of the file(hi.txt) i tried getResources but it didnt seem to work.Am stuck here,any help will be much helpful
As far as I know, in nodeJS (though you didn't specifically mentioned it in your question, just tagged it), the GridStore object is to be used for manipulating the files:
(quoted form the GridStore doc)
Opening a GridStore (a single file in GridFS) is a bit similar to opening a database. At first you need to create a GridStore object and then open it.
var gs = new mongodb.GridStore(db, filename, mode[, options])
Reading from GridStore can be done with read
gs.read([size], callback)
where
size is the length of the data to be read
callback is a callback function with two parameters - error object (if an error occured) and data (binary string)
Streaming from GridStore:
You can stream data as it comes from the database using stream
gs.stream([autoclose=false])
where
autoclose If true, current GridStore will be closed when EOF and ‘close’ event will be fired
The function returns read stream based on this GridStore file. It supports the events ‘read’, ‘error’, ‘close’ and ‘end’.
Also, at the quoted doc site, there are a lot of useful examples for storing file, etc...
Recommended reading:
A primer for GridFS using the Mongo DB driver