How to convert string into binary buffer? - node.js

I have two pieces of code:
var mmmagic = require('mmmagic');
var request = require('request');
var magic = new mmmagic.Magic(mmmagic.MAGIC_MIME_TYPE);
data = fs.readFileSync('/Users/myaccount/Desktop/test.png');
magic.detect(data,function(err,mime){
console.log(mime); // prints 'image/png'
}
and
var mmmagic = require('mmmagic');
var request = require('request');
var magic = new mmmagic.Magic(mmmagic.MAGIC_MIME_TYPE);
request.get('https://www.google.pl/images/srpr/logo11w.png',function(err,res,data){
data = new Buffer(data); // tried also new Buffer(data,'binary');
magic.detect(data,function(err,mime){
console.log(mime); // prints 'application/octet-stream'
}
})
So first one checks mime type of file from local disk and its 'image/png'.
The second one downloads image from url (its google logo in png format) from url and check its mime type and it is 'application/octet-stream'.
How I can convert response from request (which is a string) to a binary buffer so mime detection would return 'image/png'??

You have to pass in the option encoding: null
var mmmagic = require('mmmagic')
, request = require('request')
, magic = new mmmagic.Magic(mmmagic.MAGIC_MIME_TYPE)
, image = 'https://www.google.pl/images/srpr/logo11w.png';
request({
uri: image,
encoding: null
}, function(err, res, data) {
console.log(typeof data);
console.log(data.constructor);
magic.detect(data, function(err,mime) {
console.log(mime); // prints 'image/png'
});
});
I noticed that data was a string when using request.get(<urlString>). For debugging purposes I used typeof <something> & <something>.constructor to determine what <something> really was.
The docs are a little misleading stating that
encoding - Encoding to be used on setEncoding of response data. If null, the body is returned as a Buffer.
Making one think that the default would be a Buffer!

Related

Base64 Encoding Node.js

I need to get a file from a URL (any file type) and base64 encode it, and then send it to an API that only accepts base64 encoded files.
Here is the documentation description just for clarification:
The file must be base64-encoded. You should not use MIME style
encoding with newline characters at a maximum line length of 76.
Instead, you are required to omit these newlines. (If you don’t omit
the newlines, the stored file won’t be usable.)
Below is the current code I am using. I am able to upload the file to the API in question, but the file has been corrupted for some reason. The current URL I am using for this is (just a dummy image for testing):
dummy image
I am getting the file to upload to the API in question - but when opened it is corrupted. I am assuming it has something to do with the base64 restrictions noted above for the API. My code for the base64 encoding.
UPDATED - async accounted for and still the same issue:
const request = require('request');
function sendBase64EncodedFile( url ) {
request.get( url , function (error, response, body) {
if (!error && response.statusCode == 200) {
var data = new Buffer(body).toString('base64');
/*
Create xml to send to API
*/
var xml = '<qdbapi><usertoken>hidden for privacy issues</usertoken><apptoken>hidden for privacy issues</apptoken><field fid="8">Some testing body text</field><field fid="9" filename="someFileName.png">' + data + '</field></qdbapi>';
var qb_url = "hidden for privacy issues";
request({
url: qb_url,
method: "POST",
'headers': {
'Content-Type': 'application/xml',
'QB-ACTION': 'API_AddRecord'
},
body: xml
}, function (error2, response2, body2){
});
} else {
//console.log( body );
}
});
}

Amazon MWS SubmitFeed Content-MD5 HTTP header did not match the Content-MD5 calculated by Amazon

I know this question is not new but all the solution I get for this are in PHP or my issue is different from them.
I am using MWS feed API to submit flat file for Price and Quantity Updates and always get the following error:
the Content-MD5 HTTP header you passed for your feed did not match the
Content-MD5 we calculated for your feed
I would like to ask 3 questions here:-
ContentMD5Value parameter is optional as given in doc, but if i not passed that than it will say that you must enter ContentMD5Value.
As in doc the ContentFeed which we are given to Amazon. Amazon create contentMD5 for that file and then compares that contentMD5 value with the contentMD5 value we send to Amazon.
If both match then OK, otherwise it will throw an error. But if suppose I will not send the file then also the same errors come that MD5 does not match. How is that possible? Which file are they calculating the MD5 for? Because I haven't send the file in ContentFeed.
If I send the contentMD5 in a header as well as parameter and sending the ContentFeed in body, I still get the error.
Note:- I am sending the contentMD5 in a header as well as in a parameters in form using request module and also calculating the signature with that and then pass the contentFeed in body.
I am using JavaScript (Meteor), I calculate the md5 using the crpyto module.
First, I think that my md5 is wrong but then I tried with an online website that will give me the md5 for a file the md5.
for my file is:
MD5 value: d90e9cfde58aeba7ea7385b6d77a1f1e
Base64Encodevalue: ZDkwZTljZmRlNThhZWJhN2VhNzM4NWI2ZDc3YTFmMWU=
The flat file I downloaded from for Price and Quantity Updates:-
https://sellercentral.amazon.in/gp/help/13461?ie=UTF8&Version=1&entries=0&
I calculated the signature also by giving ContentMD5Value while calculating the signature.
FeedType:'_POST_FLAT_FILE_PRICEANDQUANTITYONLY_UPDATE_DATA_'
As, I read documentation for that I passed the MD5-header in headers and also send as parameter.
Amazon doc says:
Previously, Amazon MWS accepted the MD5 hash as a Content-MD5 header
instead of a parameter. Passing it as a parameter ensures that the MD5
value is part of the method signature, which prevents anyone on the
network from tampering with the feed content.
Amazon MWS will still accept a Content-MD5 header whether or not a
ContentMD5Value parameter is included. If both a header and parameter
are used, and they do not match, you will receive an
InvalidParameterValue error.
I am using the request module for http requests.
I am passing all the required keys, seller id, etc. in form of request module and passing the FeedContent in body.
I tried sending the file as follows:
Method for submitFeed is:-
submitFeed : function(){
console.log("submitFeedAPI running..");
app = mwsReport({auth: {sellerId:'A4TUFSCXD64V3', accessKeyId:'AKIAJBU3FTBCJUIZWF', secretKey:'Eug7ZbaLljtrnGKGFT/DTH23HJ' }, marketplace: 'IN'});
app.submitFeedsAPI({FeedType:'_POST_FLAT_FILE_PRICEANDQUANTITYONLY_UPDATE_DATA_'},Meteor.bindEnvironment(function(err,response){
if(err){
console.log("error in submit feed...")
console.log(err)
}
else{
console.log("suuccess submit feed....")
console.log(response);
}
}))
Method that call Amazon submitFeedAPI is:-
var submitFeedsAPI = function(options, callback){
console.log("submitFeedsAPI running...");
var fileReadStream = fs.createReadStream('/home/parveen/Downloads/test/testting.txt');
var contentMD5Value = crypto.createHash('md5').update(file).digest('base64');
var reqForm = {query: {"Action": "SubmitFeed", "MarketplaceId": mpList[mpCur].id, "FeedType":options.FeedType,"PurgeAndReplace":false,"ContentMD5Value":contentMD5Value}};
mwsReqProcessor(reqForm, 'submitFeedsAPI', "submitFeedsAPIResponse", "submitFeedsAPIResult", "mwsprod-0000",false,file, callback);
}
also try
var fileReadStream = fs.createReadStream('/home/parveen/Downloads/test/testting.txt');
var base64Contents = fileReadStream.toString('base64');
var contentMD5Value = crypto.createHash('md5').update(base64Contents).digest('base64');
mwsReqProcessor function is as below:-
mwsReqProcessor = function mwsReqProcessor(reqForm, name, responseKey, resultKey, errorCode,reportFlag,file, callback) {
reqOpt = {
url: mwsReqUrl,
method: 'POST',
timeout: 40000,
body:{FeedContent: fs.readFileSync('/home/parveen/feedContentFile/Flat.File.PriceInventory.in.txt')},
json:true,
form: null,
headers: {
// 'Transfer-Encoding': 'chunked',
//'Content-Type': 'text/xml',
// 'Content-MD5':'ZDkwZTljZmRlNThhZWJhN2VhNzM4NWI2ZDc3YTFmMWU=',
// 'Content-Type': 'text/xml; charset=iso-8859-1'
'Content-Type':'text/tab-separated-values;charset=UTF-8'
},
}
reqOpt.form = mwsReqQryGen(reqForm);
var r = request(reqOpt, function (err, res, body){
console.log(err)
console.log(res)
})
// var form = r.form();
//form.append('FeedContent',fs.createReadStream('/home/parveen/feedContent//File/Flat.File.PriceInventory.in.txt'))
}
Method for mwsReqQryGen generation:-
mwsReqQryGen = function mwsReqQryGen(options) {
var method = (options && options.method) ? ('' + options.method) : 'POST',
host = (options && options.host) ? ('' + options.host) : mwsReqHost,
path = (options && options.path) ? ('' + options.path) : mwsReqPath,
query = (options && options.query) ? options.query : null,
returnData = {
"AWSAccessKeyId": authInfo.accessKeyId,
"SellerId": authInfo.sellerId,
"SignatureMethod": "HmacSHA256",
"SignatureVersion": "2",
"Timestamp": new Date().toISOString(),
"Version":"2009-01-01",
},
key;
if(query && typeof query === "object")
for(key in query)
if(query.hasOwnProperty(key)) returnData[key] = ('' + query[key]);
if(authInfo.secretKey && method && host && path) {
// Sort query parameters
var keys = [],
qry = {};
for(key in returnData)
if(returnData.hasOwnProperty(key)) keys.push(key);
keys = keys.sort();
for(key in keys)
if(keys.hasOwnProperty(key)) qry[keys[key]] = returnData[keys[key]];
var sign = [method, host, path, qs.stringify(qry)].join("\n");
console.log("..................................................")
returnData.Signature = mwsReqSignGen(sign);
}
//console.log(returnData); // for debug
return returnData;
};
I also tried with following:-
reqOpt = {
url: mwsReqUrl,
method: 'POST',
timeout: 40000,
json:true,
form: null,
body: {FeedContent: fs.createReadStream('/home/parveen/feedContentFile/Flat.File.PriceInventory.in.txt')},
headers: {
// 'Transfer-Encoding': 'chunked',
//'Content-Type': 'text/xml',
// 'Content-MD5':'ZDkwZTljZmRlNThhZWJhN2VhNzM4NWI2ZDc3YTFmMWU=',
// 'Content-Type': 'text/xml; charset=iso-8859-1'
},
}
I also tried without JSON and directly send the file read stream in the
body, i.e:
reqOpt = {
url: mwsReqUrl,
method: 'POST',
timeout: 40000,
form: null,
body: fs.createReadStream('/home/parveen/feedContentFile/Flat.File.PriceInventory.in.txt'),
headers: {
// 'Transfer-Encoding': 'chunked',
//'Content-Type': 'text/xml',
// 'Content-MD5':'ZDkwZTljZmRlNThhZWJhN2VhNzM4NWI2ZDc3YTFmMWU=',
// 'Content-Type': 'text/xml; charset=iso-8859-1'
},
}
But same error comes every time:
the Content-MD5 HTTP header you passed for your feed did not match the
Content-MD5 we calculated for your feed
I want to know where I am doing wrong or what is the right way to submit feed API and sending the file using request module.
I also tried with the code given on MWS to generate the MD5 but same
error occurred each time.
My .txt file as follows:
sku price quantity
TP-T2-00-M 2
Any help is much appreciated
finally i got the solution as Ravi said above. Actually there are few points i want to clear here for you all who are facing the same issue:-
Amazon marketplace API doc is not giving proper information and example. Even i guess the documentation is not updated . As in doc they said that ContentMD5Value parameter value is optional on this page
You can check there they clearly mention that the field is not required but if you not pass than they gives the error that you must pass content MD5 value.
So that is wrong. ContentMD5 is required attribute.
They said in the same doc that you need to send file data weather its a xml or flat-file in the field key name i.e. FeedContent.
But that is also not needed you can send the file with any name no
need to give FeedContent key for the file you just need to send the
file in stream.
They will give the same error of contentMD5 not match weather you send file or not because if they not found file than the contentMD5 you send will not match to that. SO if you are getting the ContentMD5 not match error than check the following:-
Check that you are generating the right MD5 code for your file you can check whether you are generating the right code or not by there java code they given on doc . You can get that from this link
Don't trust on online websites for generating the MD5 hash and base64 encoding.
If your MD5 is matched with the MD5 generated from Java code they given than one thing is clear that your MD5 is right so no need to change on that.
Once your MD5 is correct and after that also if you get the same error that is:-
Amazon MWS SubmitFeed Content-MD5 HTTP header did not match the
Content-MD5 calculated by Amazon
ContentMD5 not matched .Than you need to check only and only you file uploading mechanism.
Because now the file you are sending to Amazon is not either correct or you are not sending it in the right way.
Check for file upload
For checking whether or not you are sending the right file you need to check with following:-
You need to send the required parameters like sellerId, marketplaceId, AWSAccessKey etc. as query params.
You need to send the file in the form-data as multipart , if you are using the request module of node.js than you can see the above code given by Ravi.
you need to set the header as only:-
'Content-Type': 'application/x-www-form-urlencoded'
No need to send the header as chunked or tab separated etc because i don't need them any more they are even confuse me because somewhere someone write use this header on other place someone write use this header.
So finally as i am abel to submit this API i didn't need any of the header rather than application/x-www-form-urlencoded.
Example:-
reqOpt = {
url: mwsReqUrl,
method: 'POST',
formData: {
my_file: fs.createReadStream('file.txt')
},
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
qs: { }// all the parameters that you are using while creating signature.
Code for creating the contentMD5 is:-
var fileData= fs.readFileSync('/home/parveen/Downloads/test/feed.txt','utf8');
var contentMD5Value = crypto.createHash('md5').update(fileData).digest('base64');
As i am facing the issue that is because i am using form and form-data simultaneously via request module so i convert my form data with qs(query string) and file in form-data as multipart.
So in this way you can successfully submit the API for submit feed.
Amazon requires the md5 hash of the file in base64 encoding.
Your code:
var fileReadStream = fs.createReadStream('/path/to/file.txt');
var file = fileReadStream.toString('base64'); //'[object Object]'
var contentMD5Value = crypto.createHash('md5').update(file).digest('base64');
wrongly assumes that a readStream's toString() will produce the file contents, when, in fact, this method is inherited from Object and produces the string '[object Object]'.
Base64-encoding that string always produces the 'FEGnkJwIfbvnzlmIG534uQ==' that you mentioned.
If you want to properly read and encode the hash, you can do the following:
var fileContents = fs.readFileSync('/path/to/file.txt'); // produces a byte Buffer
var contentMD5Value = crypto.createHash('md5').update(fileContents).digest('base64'); // properly encoded
which provides results equivalent to the following PHP snippet:
$contentMD5Value = base64_encode(md5_file('/path/to/file.txt', true));
Hey sorry for late reply but why don't you try to send the file in multipart in the form-data request and other queryStrings in 'qs' property of request module.
You can submit the request as follows:-
reqOpt = {
url: mwsReqUrl,
method: 'POST',
formData: {
my_file: fs.createReadStream('file.txt')
},
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
qs: {
AWSAccessKeyId: '<your AWSAccessKeyId>',
SellerId: '<your SellerId>',
SignatureMethod: '<your SignatureMethod>',
SignatureVersion: '<your SignatureVersion>',
Timestamp: '<your Timestamp>',
Version: '<your Version>',
Action: 'SubmitFeed',
MarketplaceId: '<your MarketplaceId>',
FeedType: '_POST_FLAT_FILE_PRICEANDQUANTITYONLY_UPDATE_DATA_',
PurgeAndReplace: 'false',
ContentMD5Value: '<your file.txt ContentMD5Value>',
Signature: '<your Signature>'
}
}
request(reqOpt, function(err, res){
})
Probably, I'm too late, but here are key points for C#:
1) Multipart form-data didn't work at all. Finished with the following (simplified):
HttpContent content = new StringContent(xmlStr, Encoding.UTF8, "application/xml");
HttpClient client = new HttpClient();
client.PostAsync(query, content)
2) About query:
UriBuilder builder = new UriBuilder("https://mws.amazonservices.com/");
NameValueCollection query = HttpUtility.ParseQueryString(builder.Query);
query["AwsAccessKeyId"] = your_key_str;
query["FeedType"] = "_POST_ORDER_FULFILLMENT_DATA_";
... other required params
query["ContentMD5Value"] = Md5Base64(xmlStr);
builder.Query = query.ToString();
query = builder.ToString();
3) About Md5base64
public static string Md5Base64(string xmlStr)
{
byte[] plainTextBytes = Encoding.UTF8.GetBytes(xmlStr);
MD5CryptoServiceProvider provider = new MD5CryptoServiceProvider();
byte[] hash = provider.ComputeHash(plainTextBytes);
return Convert.ToBase64String(hash);
}

hitting a multipart url in nodejs

I have a client code using form-data module to hit a url that returns a content-type of image/jpeg. Below is my code
var FormData = require('form-data');
var fs = require('fs');
var form = new FormData();
//form.append('POLICE', "hello");
//form.append('PAYSLIP', fs.createReadStream("./Desert.jpg"));
console.log(form);
//https://fbcdn-profile-a.akamaihd.net/hprofile-ak-xfp1/v/t1.0- 1/c8.0.50.50/p50x50/10934065_1389946604648669_2362155902065290483_n.jpg?oh=13640f19512fc3686063a4703494c6c1&oe=55ADC7C8&__gda__=1436921313_bf58cbf91270adcd7b29241838f7d01a
form.submit({
protocol: 'https:',
host: 'fbcdn-profile-a.akamaihd.net',
path: '/hprofile-ak-xfp1/v/t1.0-1/c8.0.50.50/p50x50/10934065_1389946604648669_2362155902065290483_n.jpg?oh=13640f19512fc3686063a3494c6c1&oe=55ADCC8&__gda__=1436921313_bf58cbf91270adcd7b2924183',
method: 'get'
}, function (err, res) {
var data = "";
res.on("data", function (chunks) {
data += chunks;
});
res.on("end", function () {
console.log(data);
console.log("Response Headers - " + JSON.stringify(res.headers));
});
});
I'm getting some chunk data and the response headers i received was
{"last-modified":"Thu, 12 Feb 2015 09:49:26 GMT","content-type":"image/jpeg","timing-allow-origin":"*","access-control-allow-origin":"*","content-length":"1443","cache-control":"no-transform, max-age=1209600","expires":"Thu, 30 Apr 2015 07:05:31 GMT","date":"Thu, 16 Apr 2015 07:05:31 GMT","connection":"keep-alive"}
I am now stuck as how to process the response that i received to a proper image.I tried base64 decoding but it seemed to be a wrong approach any help will be much appreciated.
I expect that data, once the file has been completely downloaded, contains a Buffer.
If that is the case, you should write the buffer as is, without any decoding, to a file:
fs.writeFile('path/to/file.jpg', data, function onFinished (err) {
// Handle possible error
})
See fs.writeFile() documentation - you will see that it accepts either a string or a buffer as data input.
Extra awesomeness by using streams
Since the res object is a readable stream, you can simply pipe the data directly to a file, without keeping it in memory. This has the added benefit that if you download really large file, Node.js will not have to keep the whole file in memory (as it does now), but will write it to the filesystem continuously as it arrives.
form.submit({
// ...
}, function (err, res) {
// res is a readable stream, so let's pipe it to the filesystem
var file = fs.createWriteStream('path/to/file.jpg')
res.on('end', function writeDone (err) {
// File is saved, unless err happened
})
.pipe(file) // Send the incoming file to the filesystem
})
The chunk you got is the raw image. Do whatever it is you want with the image, save it to disk, let the user download it, whatever.
So if I understand your question clearly, you want to download a file from an HTTP endpoint and save it to your computer, right? If so, you should look into using the request module instead of using form-data.
Here's a contrived example for downloading things using request:
var fs = require('fs');
var request = require('request')
request('http://www.example.com/picture.jpg')
.pipe(fs.createWriteStream('picture.jpg'))
Where 'picture.jpg' is the location to save to disk. You can open it up using a normal file browser.

What is the correct way to encode image data in nodejs buffers?

I'm trying to fetch an image, apply a transform and save it in a database like mongodb. Her's my code
var stor = function(inStream, sizeType) {
console.log('entering stor function');
var hashCode = '';
var img = new Buffer(1024 * 1024 * 5 * 1.1, 'binary'); //5mb size + 10% space
var hash = crypto.createHash('sha1');
inStream.on('data', function (chunk){
Buffer.concat([img, chunk]);
hash.update(chunk);
});
inStream.on('end', function() {
hashCode = hash.digest('hex');
var retUrl = "http://playground.com/" + hashCode;
//post this url using requests, set encoding : binary
});
};
server.post('/resize', function(req, res) {
req.accepts('application/json');
console.log('received a resize request for image =', req.body.host + req.body.path);
var request = requests.get({
url: req.body.url,
headers: {'accept-encoding': 'gzip'}
});
//error handling
request.on('response', function (response) {
console.log('succesfully fetched image...');
response.setEncoding('binary');
//save original in stor
stor(response, 'original');
res.writeHead(201);
res.send();
});
});
module.exports = server;
When i do this, where i receive some image from the internet and then save it in my database for future use, the image saved data in the database is not the original image i stored. It is corrupt. I have narrowed the problem down to the encoding of the data I buffer, in the function store( variable 'img'). I did this by directly piping the data from response to the post to database call. I can't do that for my purpose because i need to compute the hash of the image.
I want to know if my assumptions are correct.
Images from the internet can be read as 'binary'.
You can load that data onto a buffer as 'binary'.
PUT the image onto a store with encoding set to 'binary'.
I think one or all of these assumptions are wrong, as i get back only corrupted data back from the database.
The issue was that I was using exec. Exec outputs a buffer. Using spawn solved this issue. Spawn outputs a STREAM, which handles binary correctly. Ofcourse, I also set the encoding to binary as well.

CouchDB/Cradle How do I add images?

So Basically I want to let the user have the option of uploading an image when they register. I have no idea where to start forever. I know that CouchDB supports attachments, but how exactly does that work with Cradle.
I found the following code in Cradle's documentation
saveAttachment: function (/* id, [rev], attachmentName, contentType, dataOrStream */) {
So I know it can save attachments. How would I pass in the image then? I'm assuming that in the html, i have to use
form(action='/upload', enctype='multipart/form-data', method='post')
input(type='file', name='upload')
input(type='submit', value='Upload')
But where do I go from there? Wouldn't this step save the image on the server somewhere. Then do I somehow need to get the address of the image and pass that to cradle to save it as an attachment in the CouchDB database.
Thanks in advance if you can help me out!
You need to take the incoming stream from the form and then send a stream to CouchDB via Cradle.
Sending the stream to Cradle is probably the easy bit. This example shows how to do it with a local file:
db.saveAttachment(
doc.id,
doc.rev,
attachmentId,
mimetype,
fs.createReadStream(path),
function( err, data ){
console.log(data);
}
);
The trickier bit in my opinion is managing incoming files. They arrive as a multipart stream rather than being saved to a file. My preference would be to outsource that code to formidable, either directly, or indirectly via connect-form if you're using Connect or Express.
My current connect-form code can be summarised to this:
req.form.complete(function(err, fields, files){
if ( err ) // handle err
else
{
db.saveAttachment(
doc.id,
doc.rev,
attachmentId,
mimetype,
fs.createReadStream(files.name),
function( err, data ){
console.log(data);
}
);
}
});
This isn't optimal for speed as it creates an actual file on disk rather than streaming data from one place to the other, but it is convenient and may satisfy a lot of use cases.
Another package you should be aware of if you're dealing with image upload is node-imagemagick, which as you might expect from the name is the node.js wrapper for ImageMagick.
I wrote up some documentation on attachments that hopefully will be merged into the cradle readme soon. For now though here is the relevant section
Attachments
Cradle supports writing, reading, and removing attachments. The read and write operations can be either buffered or streaming
Writing
You can buffer the entire attachment body and send it all at once as a single request. The callback function will fire after the attachment upload is complete or an error occurs
Syntax
db.saveAttachment(idData, attachmentData, callbackFunction)
Example
Say you want to save a text document as an attachment with the name 'fooAttachment.txt' and the content 'Foo document text'
var doc = <some existing document>
var id = doc._id
var rev = doc._rev
var idAndRevData = {
id: id,
rev: rev
}
var attachmentData = {
name: 'fooAttachment.txt',
'Content-Type': 'text/plain',
body: 'Foo document text'
}
db.saveAttachment(idAndRevData, attachmentData, function (err, reply) {
if (err) {
console.dir(err)
return
}
console.dir(reply)
})
Streaming
You can use a read stream to upload the attachment body rather than buffering the entire body first. The callback function will fire after the streaming upload completes or an error occurs
Syntax
var doc = savedDoc // <some saved couchdb document which has an attachment>
var id = doc._id
var rev = doc._rev
var idAndRevData = {
id: id,
rev: rev
}
var attachmentData = {
name: attachmentName // something like 'foo.txt'
'Content-Type': attachmentMimeType // something like 'text/plain', 'application/pdf', etc.
body: rawAttachmentBody // something like 'foo document body text'
}
var readStream = fs.createReadStream('/path/to/file/')
var writeStream = db.saveAttachment(idData, attachmentData, callbackFunction)
readStream.pipe(writeStream)
When the streaming upload is complete the callback function will fire
Example
Attach a pdf file with the name 'bar.pdf' located at path './data/bar.pdf' to an existing document
var path = require('path')
var fs = require('fs')
// this document should already be saved in the couchdb database
var doc = {
_id: 'fooDocumentID',
_rev: 'fooDocumentRev'
}
var idData = {
id: doc._id,
rev: doc._rev
}
var filename = 'bar.pdf' // this is the filename that will be used in couchdb. It can be different from your source filename if desired
var filePath = path.join(__dirname, 'data', 'bar.pdf')
var readStream = fs.createReadStream
// note that there is no body field here since we are streaming the upload
var attachmentData = {
name: 'fooAttachment.txt',
'Content-Type': 'text/plain'
}
db.saveAttachment(idData, attachmentData, function (err, reply) {
if (err) {
console.dir(err)
return
}
console.dir(reply)
}, readStream)
Reading
Buffered
You can buffer the entire attachment and receive it all at once. The callback function will fire after the download is complete or an error occurs. The second parameter in the callback will be the binary data of the attachment
Syntax
db.getAttachment(documentID, attachmentName, callbackFunction)
Example
Say you want to read back an attachment that was saved with the name 'foo.txt'
var doc = <some saved document that has an attachment with name *foo.txt*>
var id = doc._id
var attachmentName = 'foo.txt'
db.getAttachment(id, attachmentName, function (err, reply) {
if (err) {
console.dir(err)
return
}
console.dir(reply)
})
Streaming
You can stream the attachment as well. If the attachment is large it can be useful to stream it to limit memory consumption. The callback function will fire once the download stream is complete. Note that there is only a single error parameter passed to the callback function. The error is null is no errors occured or an error object if there was an error downloading the attachment. There is no second parameter containing the attachment data like in the buffered read example
Syntax
var readStream = db.getAttachment(documentID, attachmentName, callbackFunction)
Example
Say you want to read back an attachment that was saved with the name 'foo.txt'. However the attachment foo.txt is very large so you want to stream it to disk rather than buffer the entire file into memory
var doc = <some saved document that has an attachment with name *foo.txt*>
var id = doc._id
var attachmentName = 'foo.txt'
var downloadPath = path.join(__dirname, 'foo_download.txt')
var writeStream = fs.createWriteStream(downloadPath)
var readStream = db.getAttachment('piped-attachment', 'foo.txt', function (err) { // note no second reply paramter
if (err) {
console.dir(err)
return
}
console.dir('download completed and written to file on disk at path', downloadPath)
})
readStream.pipe(writeStream)
Removing
You can remove uploaded attachments with a _id and an attachment name
Syntax
db.removeAttachment(documentID, attachmentName, callbackFunction)
Example
Say you want to remove an attachment that was saved with the name 'foo.txt'
var doc = <some saved document that has an attachment with name *foo.txt*>
var id = doc._id
var attachmentName = 'foo.txt'
db.removeAttachment(id, attachmentName, function (err, reply) {
if (err) {
console.dir(err)
return
}
console.dir(reply)
})
FYI, for future readers, the calling parameters have changed since then, so this appears to be no longer valid. Check the source as the documentation doesn't describe how to use it.

Resources