what is the difference between readFileSync and readAsDataURL? - node.js

NOTE: this is NOT a production application. It is a small app that I am making to explore the capabilities of selecting, opening, saving and reading an image from a DB.
I do not know what method I should call to get the correct Buffer data to store image in mongodb. ( I am aware that mongoDB has a 16mb limit on documents )
In my app, I want the user to be able to select an image from a file on their pc, give the user a preview of the image, then when the user hits submit (not shown here) save the image into mongodb
I know when the user clicks the "choose file" button I can get the file like this:
const file = event.target.files[0]
if (!file) {
// user has hit the cancel button
return;
}
//console.log(file);
I can get information about the file:
console.log(file.type);
console.log(file.size);
and here is how the image is read:
let fr = new FileReader();
fr.readAsDataURL(file);
fr.onload = (e) =>{
this.productImageAdd(e.target.result);
}
This is how the image is stored in the (react) component state
productImageAdd = (file) => {
let temp = this.state["filesArray"] || [];
temp.push(file);
this.setState({"filesArray": temp});
}
Here is the state of my component
this.state = {
filesArray: [],
};
In another component, I get a listing of all the images and then map them to an item for display:
return filesArray.map((file, index) => (
<FileDropZoneItem key={index} id={index} file={file} />
));
In FileDropZoneItem, this is how images are "previewed":
return (
<li className="flex-item ">
<img src={file} width="200" height="200"/>
</li>
);
I saw this stackoverflow question and I am moving towards that to store the image in MongoDB:
Store an image in MongoDB using Node.js/Express and Mongoose
In that post, their schema is like this:
var schema = new Schema({
img: { data: Buffer, contentType: String }
});
And they store the image like this:
var A = mongoose.model('A', schema);
var a = new A;
a.img.data = fs.readFileSync(imgPath);
a.img.contentType = 'image/png';
but I don't really know the difference between fs.readFileSync and fr.readAsDataURL.
Will storing to mongodb work using fr.readAsDataURL or do I need fs.readFileSync ?
I also saw: Difference between readAsBinaryString() and readAsDataURL() in HTML5 FileReader API

From the MongoDB manual GridFS
GridFS uses two collections to store files. One collection stores the file chunks, and the other stores file metadata. The section GridFS Collections describes each collection in detail.
For reference you can read Storing images in MongoDB
readAsDataURL permits to store the images inside the JSON document encoded as base64 characters. Given the size of the Products pictures 200px x 200px seems the best option for your hello world app.
readFileSync can have blocking issues in the server like Nodejs for image handling behavior and reads the binary file to store it as Binary Data.
IMO the best solution is to not overload mongoDB and Nodejs with the image handling and store only an URL to the image in the mongoDB document leaving the image serving process to a proxy like nginx.
The Gridfs option depends on the driver used. Following is Gridfs use in Nodejs that uses its own reader:
fs.createReadStream('./meistersinger.mp3').
pipe(bucket.openUploadStream('meistersinger.mp3')).

Related

Angular8 get image from url but using db

Is there a way to write urls in html, but get the image from db instead of a folder?
my goal is :
HTML (img.path is some url)
<img src={{img.path}}>
ts (as example):
getImage(String : url) {
getFileFromDb(url);
return image;
}
I'm trying to make a angular project using lots of images. For front-end, I know there two ways to get a image, using html (login.html):
<img src='assets/img/loginBanner.jpg'>
or using db (loginBanner.ts):
reader.readAsDataURL(blob);
reader.onload = () => {
img.readerResult = reader.result;
img.blob = blob;
};
Unforturnatly, these two both have disadvantages
There's tons of images we'll use, it's not probable to put all of it in "assets/img" folder.
But if using db, we have to send all the images everytime, that seems like a waste on the boardband.

KendoReact upload - passing file Id to React after saving file

I'm a bit stuck with the kendo react upload control.
I need to customise the rendering of the kendo react upload control.
After the user upload the file, I want to save the file in database. Then I need to pass the database Id back to the client because if the user then wants to remove the file I also need to clear the database.
To give you an idea this is what I would like to achieve.
https://stackblitz.com/edit/react-ghna5h
(When you start stackblitz open file app/main.jsx)
Is it possible?
Thanks for your help
I solved this particular issue using a combination of the onStatusChange and onBeforeRemove events. The server's 'save' endpoint returns an id (responseUID) that can be accessed through the onChange event's response object. I set this id on the 'file' object. Then when you're removing the file I pass the responseUID along in the additionalData field which gets put into the POST body.
const onStatusChange = (e) => {
if (e.response) {
const fileId = e.response.response.responseUID;
// This does not deal with multiple or batch uploads.
e.affectedFiles[0].responseUID = fileId;
}
};
const onBeforeRemove = (e) => {
e.additionalData.responseUID = e.files[0].responseUID;
};
return (
<Upload
batch={false}
defaultFiles={[]}
withCredentials={false}
saveUrl="https://localhost/upload/save"
removeUrl="https://localhost/upload/remove"
onStatusChange={onStatusChange}
onBeforeRemove={onBeforeRemove}
/>
);

Getting octet-stream instead of image in node js

In my front-end I use angular6 and I have this form where you can choose an image either by dropping a file in a div or clicking the div to open a file picker.
The form is
<form [formGroup]="imageUpload" (ngSubmit)="imageUploadSubmitted($event.target)" >
<div id="imageDrop" (click)='imageInput.click()' (drop)="drop($event)" (dragover)="allowDrop($event)" #imageDrop></div>
<input type="file" (change)='imageChange($event)' #imageInput id="imageInput" name = 'imageInput' accept=".png, .jpg, .jpeg, .gif" formControlName="imageInput" required >
<button type="submit" >Submit</button>
</form>
This is the typescript
selectedFile:File=null;
allowDrop(e) {
e.preventDefault();
}
drop(e) {
e.preventDefault();
this.imageUpload.controls.imageInput.reset();
this.selectedFile = e.dataTransfer.files[0];
let input = this.imageUpload.controls.imageInput as any;
input.value = e.dataTransfer.files[0];
}
imageChange(e){
this.selectedFile = e.target.files[0];
}
So, when dropping an image, get it from the event and put it in the file input. When the form is submitted, send the form data to a service for posting. The console.log shows a File object (__proto__ : File) whether you picked an image from the file picker, or dropped one in the div.
imageUploadSubmitted(form){
console.log('imageInput value - ', this.imageUpload.controls.imageInput.value);
this.mapcmsService.uploadImage(form).subscribe((data) =>{
if(data.success){ alert('all good'); }
else{ alert('error'); }
})
}
The service grabs the form controls and builds a FormData object to send in node.
uploadImage(data){
let formData = new FormData(data);
let headers = new Headers();
headers.append('Authorization',this.userToken);
return this.http.post('http://localhost:3000/user/upload/image', formData ,{headers:headers} ).pipe(map(res => res.json()));
}
In node I use formidable to get the file and save it. This is for testing.
var form = new formidable.IncomingForm();
form.parse(req);
form.on('file', function (name, file){
console.log('file' , file);
});
The problem is that if I have chose an image via the file picker, I get a file of type image/jpeg , a name a path and a size.
If I chose an image by drag and drop, I get a file of type application/octet-stream. This has no name and no size.
I would like to get image/jpeg in both cases. I am confused, is this a node or angular issue? How can I fix this ?
Thanks
angular 6 , node 8.11.1, formidable 1.2.1
The issue is that the assignment in the drop event does not actually set the value of the input because file inputs are not supported by Angular reactive forms. I am talking about this lines:
let input = this.imageUpload.controls.imageInput as any;
input.value = e.dataTransfer.files[0];
So when you drop in your case you are not actually sending the file to the server at all. That is why the data you get is wrong. Here are also links to two other stack overflow questions about reactive forms and file upload where there is more information regarding this issue.
How to include a file upload control in an Angular2 reactive form?
Using reactive form validation with input type=“file” for an Angular app
There are two possible solutions to workaround this issue. The first is that you get the ElementRef of the file input by using the ViewChild query. And then assign the files to the native html element directly. The good thing with this approach is that you will see the dropped file name also in the file input. The downside is that this might not work in all browsers. The official documentation on MDN says that it works in modern browsers but for me it did work in Chrome and not in Edge.
Here is a sample of the code:
#ViewChild('imageInput') private imageInput: ElementRef;
public drop(e: DragEvent) {
e.preventDefault();
this.imageUpload.controls.imageInput.reset();
this.selectedFile = e.dataTransfer.files[0];
this.imageInput.nativeElement.files = e.dataTransfer.files;
}
The other approach is that you build the FormData object yourself by adding the selected file yourself in code before sending it to the server. This should work anywhere without issues. Here is a sample code:
let formData = new FormData();
formData.append('imageInput', this.selectedFile);
I have created also a StackBlitz example where you can see all the code.

CSV, Download or manipulate right away for mongoDB in node.js

I am working on a project and in which i need to manipulate a CSV file hosted on a distant URL and then send the formatted content to a distant mongodb server. I have tried many approaches without much success so far, so i was wondering what you would suggest ? Is it easier to download the distant file locally and then reupload it or to just load the contents in memory and insert it right away?
Thanks
I assume you want to save every entry from the CSV to a mongo
An easy way with scramjet:
const {StringStream} = require('scramjet');
const request = require('request');
const mongo = require('mongo-db');
const db = new mongodb(url, cfg.collection, true);
StringStream.from(request.get('https://example.com/csvs/first.csv'))
// get your CSV as a stream and pass it to scramjet
.CSVParse() // parse the stream as CSV
.setOptions({maxParallel: 32}) // set maximum parallel operations
.map(x => { // map every object
const y = await manipulate(x);
return JSON.parse(y);
})
.consume((y) => db.insert(y)); // insert into database
Disclamer: I'm the author of scramjet.

NodeJS/React: Taking a picture, then sending it to mongoDB for storage and later display

I've been working on a small twitter-like website to teach myself React. It's going fairly well, and i want to allow users to take photos and attach it to their posts. I found a library called React-Camera that seems to do what i want it do to - it brings up the camera and manages to save something.
I say something because i am very confused about what to actually -do- with what i save. This is the client-side code for the image capturing, which i basically just copied from the documentation:
takePicture() {
try {
this.camera.capture()
.then(blob => {
this.setState({
show_camera: "none",
image: URL.createObjectURL(blob)
})
console.log(this.state);
this.img.src = URL.createObjectURL(blob);
this.img.onload = () => { URL.revokeObjectURL(this.src); }
var details = {
'img': this.img.src,
};
var formBody = [];
for (var property in details) {
var encodedKey = encodeURIComponent(property);
var encodedValue = encodeURIComponent(details[property]);
formBody.push(encodedKey + "=" + encodedValue);
}
formBody = formBody.join("&");
fetch('/newimage', {
method: 'post',
headers: {'Content-type': 'application/x-www-form-urlencoded;charset=UTF-8'},
body: formBody
});
console.log("Reqd post")
But what am i actually saving here? For testing i tried adding an image to the site and setting src={this.state.img} but that doesn't work. I can store this blob (which looks like, for example, blob:http://localhost:4000/dacf7a61-f8a7-484f-adf3-d28d369ae8db)
or the image itself into my DB, but again the problem is im not sure what the correct way to go about this is.
Basically, what i want to do is this:
1. Grab a picture using React-Camera
2. Send this in a post to /newimage
3. The image will then - in some form - be stored in the database
4. Later, a client may request an image that will be part of a post (ie. a tweet can have an image). This will then display the image on the website.
Any help would be greatly appreciated as i feel i am just getting more confused the more libraries i look at!
From your question i came to know that you are storing image in DB itself.
If my understanding is correct then you are attempting a bad approcah.
For this
you need to store images in project directory using your node application.
need to store path of images in DB.
using these path you can fetch the images and can display on webpage.
for uploading image using nodejs you can use Multer package.

Resources