How to insert an image into CouchDB - node.js

I'm trying to figure out how to insert an image into CouchDB using the node-CouchDB library found here: https://www.npmjs.com/package/node-couchdb
Here's what I've done:
fs.readFile('download.jpeg', (err, data) => {
binary_data = new Buffer(data, 'binary');
couch.insertAttachment("node_db", doc_number, "download.jpeg", binary_data, rev_number).then(({data, headers, status}) => {
}, err => {
console.log("ERROR"+ err.code);
});
});
The result is that CouchDB stores this in the document format like such:
{
"_id": "2741d6f37d61d6bbdf63df3be5000504",
"_rev": "22-bfdbe6db35c7d9873a2cc8a38afb2833",
"_attachments": {
"attachment": {
"content_type": "application/json",
"revpos": 22,
"digest": "md5-on0A+d7045WPI6FyS1ut4g==",
"length": 22482,
"stub": true
}
}
}
//This is what the data looks like in CouchDB using the View Attachment Function through the interface:
{"type":"Buffer","data":[255,216,255,224,0,16,74,70,73,70,0,1,1,0,0,1,0,1,0,0,255,219,0,132,0,9,6,7,18,18,18,21,18,19,19,22,21,21,23,23,23,24,21,21,21,23,23,21,21,24,21,21,21,23,22,22,21,21,22,24,29,40,32,24,26,37,29,21,21,33,49,33,37,41,43,46,46,46,23,31,51,56,51,45,55,40,45,46,43,1,10,10,10,14,13,14,26,16,16,26,45,37,29,37,45,45,45,45,45,45,45,241,...]
I then tried changing the Content-Type attribute to "image/jpeg" in the header of the request resulting in:
{
"_id": "2741d6f37d61d6bbdf63df3be5000504",
"_rev": "23-cf8c2076b43082fdfe605cad68ef2355",
"_attachments": {
"attachment": {
"content_type": "image/jpeg",
"revpos": 23,
"digest": "md5-SaekQP37DCCeGX2M8UVeGQ==",
"length": 22482,
"stub": true
}
}
}
However, this still results in an image that isn't viewable from the CouchDB interface (clicking View Attachments). The image, in this case, is only size 6,904 bytes but it's being stored with a length of ~22k (inflating the size in CouchDB) so I'm assuming I'm not passing the correct representation (encoding) of the image to CouchDB.

You can encode your image data as a base64 string and save it, although i would not recommend it at all. What I would do is to upload the file to a object storage like AWS S3 or it's open source alternative MinIO, and then save in the DB just a reference to the file (e.g. an Image URL).
P.S.: I'm sorry about the lack of links and references in my answer, I'm writing it on my phone. I can edit it and include references as soon as I'm home.

Related

Elasticsearch shows data as buffer type in Kibana

I am trying to index one json to elastic search.
It seems to be working fine as it is not giving any error.
I have indexed document as below.
await client.index({
id: fieldId.toString(),
index: 'project_documents_textfielddata',
body: {
FieldId: fieldId,
DocumentId: documentId,
Value: fieldData.fieldHTMLText,
},
routing: projectId.toString(),
});
But in elasticsearch kibana it is showing as buffer type as below (I have truncated buffer as it was very long).
{
"_index": "documenttextfile.files",
"_id": "6252ab411deaba21fd877c26",
"_version": 1,
"_score": 1,
"_routing": "62505a765ff176cd491f1d1e",
"_source": {
"id": "6252ab411deaba21fd877c26",
"Content": {
"type": "Buffer",
"data": [
10,
// Some extra large binary content removed for convenient
48,
56,
50,
],
"id": [
"6252ab411deaba21fd877c26"
],
"Content.type.keyword": [
"Buffer"
]
}
}
So how can I see my data as is (i.e. in json format) in Kibana. I've seen many tutorials on Kibana, they are able to see data in plain text instead of buffer.
Or am I doing anything wrong while indexing? I am basically trying to see the data the way we can see in mongodb compass.
Your fieldData.fieldHTMLTextfield is probably of type Buffer and you simply need to call fieldData.fieldHTMLText.toString()on it in order to transform the buffer to a string.
PS: the problem has nothing to do with Kibana which shows you exactly what you're sending to Elasticsearch, i.e. a Buffer. So the problem is more related to your understand of Node.js data structures (i.e. Buffer vs string) ;-)

Fauxton CouchDB doesn't show the data from blockfile

I am using fabric 2.0 and CouchDB as a state database.
The data in the blockfile are correct (according the input I entered via the WebApp)
When I go to fauxton (http://localhost:6984/_utils/#database) I can't see the values (e.g. value1 : 4, value2: ID, ...)
I only receive this via fauxton:
Normally there should be also "_value1" :"4" ,...
{
"_id": "\u0000key~timestamp\u0000co\u00002021-04-30T08:47:23.961Z\u0000",
"_rev": "1-210ffdc0",
"~version": "CgMBBgA=",
"_attachments": {
"valueBytes": {
"content_type": "application/octet-stream",
"revpos": 1,
"digest": "md5-n7UoN6woal8ve/9S9DrNTA==",
"length": 48,
"stub": true
}
}
}
Does anyone has an idea, why the data in the blockfile are correct, but doesn't show up in the right way in fauxton?
I checked the docker-compose-couch.yaml and it's correct...
If the value cannot be parsed as JSON, it will be stored in CouchDB as a binary attachment. Fauxton UI only shows the digest of the binary value, not the actual binary attachment.

How to fetch subset of attachments

I have a CouchDB with documents, which look like this:
{
"_id": "000040cc-e3b4-47cc-b051-a5508efb8996",
"_rev": "1-882d7f88cc2e1e767b55d0c82fb638d2",
"state": "uploaded",
"state_since": "2020-02-17T11:20:55.1450252Z"
// more metadata ...
"_attachments": {
"large.jpg": {
"content_type": "image/jpeg",
"revpos": 1,
"digest": "md5-NK7ejYjrErhMAs7tZ4+R8w==",
"length": 87846,
"stub": true
},
"medium.jpg": {
...
},
"small.jpg": {
...
}
}
}
Let's assume, I want to query a set of images like this:
{
"selector": {
"state": "uploaded"
},
"sort": ["state_since"],
"limit": 100
}
If I want to display the thumbnails of those 100 images, I'd have to iterate through the result list and download the corresponding attachments. This would be 101 requests in total.
I could also do it in one request by specifying, that I want to fetch the documents with attachments. But this would return all (potentially very large) attachments.
I know that I can set the fields property in my query to only return the fields I need. But can I apply this to attachments, too? And if yes: how?
No, there's no way to do what you're requesting. The only ways to fetch a subset of attachments are by fetching them one at a time, or by using the atts_since attribute when fetching a single document, which is intended for use in replication.
Perhaps consider re-designing your documents. Perhaps you can store your thumbnails on a separate document, that only contains thumbnails.

Highcharts Export Service Gantt Chart not working

Attempting to pilot a server-side render of a gantt chart using highcharts-export-server, but unable to get the series data to render at all.
As per the docs, I've tried running the export via commandline and as a node module, but I always get an empty graph with the title, subtitle and series name displaying, but none of the actual data.
Failed Gantt Render
All the basic line graph example configurations work and I'm able to render the chart using the Highcharts.ganttChart method via the browser, so I believe it's specific to the Gantt chart configuration for the export server, or related to the millisecond date conversion (as you can see in the screenshot, the X Axis is not rendering as dates, but rather plain numbers). As per suggestions on other threads, I ran build.js in the node_modules/highcharts-export-server, globally for the commandline attempt and project-locally for the node module attempt, making sure to enable the both the gantt and moment libraries, but that didn't help either.
Including my options json below. I haven't been able to find a gantt config example specifically for the export server, so this is my best attempt to interpolate what they'd be:
{
"title": {
"text": "Gantt PoC"
},
"subtitle": {
"text": "Timeline"
},
"series": [
{
"name": "Gantt Demo",
"type": "gantt",
"data": [
{
"name": "Demo Task 1",
"id": "demo_task_1",
"start": 1564113600000,
"end": 1564718400000
},
{
"name": "Demo Task 2",
"id": "demo_task_2",
"start": 1564113600000,
"end": 1564718400000
}
]
}
],
"xAxis": {
"min": 1563681600000,
"max": 1571803200000
}
}
Please let me know if there's anything obviously wrong with my config, if I missed any crucial steps to prep the environment, or any ideas you may have for me to troubleshoot. Thanks!
i also tried to generate image using ganttchart, just i tried constructor which you suggested working perfectly, Thanks.
highcharts-export-server --infile gantt.json --outfile gantt.png --type png --constr ganttChart
actually i tried with HTTP Server as well, working perfectly. Thanks #pawel_d
const fs = require("fs");
const chartExporter = require('highcharts-export-server');
chartExporter.initPool();
const chartDetails = {
type: "png",
constr : 'ganttChart',
options: {
series: [{
data: [{
start: 1,
end: 10
}]
}]
}
};
chartExporter.export(chartDetails, (err, res) => {
// Get the image data (base64)
let imageb64 = res.data;
// Filename of the output
let outputFile = "bar.png";
// Save the image to file
fs.writeFileSync(outputFile, imageb64, "base64", function(err) {
if (err) console.log(err);
});
console.log("Saved image!");
chartExporter.killPool();
});

Post request with encrypted json data with several keys and one image file() multipart , android

Have to pass this json to server, and encrypted param will be created after creating the json and it will return the same json with three parameters, but I am not able to do it with image or file data.
After encrypting the data these params(param1, param2, param3) automatically get created
I tried using #PartMap using Map, #Part #Multipart
{
"fullname": "hello",
"dob": "20-12-1992",
"locale":{
"language: "en",
"version": 2,
},
"media": {
"media":{
"name:"",
"size":""
}
},
"param1":"",
"param2":"",
"param3":""
}

Resources