Good day everyone, i was looking for a similar that i can refer to, but sadly i wasn't found yet until now. Hope someone would give me some guidances on it...
I'm using filepond to send the api request, and use the prisma 2 client to store it
Thank you!
It's usually a good practice to store Images in a Blob Storage and images should not directly be stored in the database, so you can use something like AWS S3 or Cloudinary to store the image and store the Image's S3 Bucket Path in your PostgreSQL Database.
Here's a working example of how you could achieve it.
Related
I need to store mongodb database back up data directly to gcp bucket without storing back up file in local. I have reffered a lot but couldn't find my required solution. Is there any way to solve this ?
Can anyone help me to implement direct upload of client files to minio in nodejs? I just saw that there are two methods for this work, presignedPostPolicy and presignedPutObject, but somewhere about how these two methods worked was not explained at the same time.
I want that when a link is given to the user that he can upload there, it has limitations such as the maximum size that can be uploaded.
I would like to know what is the best way to handle image uploading and saving the reference to the database. What I'm mostly interested is what order do you do the process in?
Should you upload the images first in the front-end (say Cloudinary), and then call the API with result links to the images and save it to the database?
Or should you upload the images to the server first, and upload them from the back-end and save the reference afterwards?
OR, should you do the image uploading after you save the record in the database and then update it once the images were uploaded?
It really depends on the resources, timeline, and number of images you need to upload daily.
So basically if you have very few images to upload then you can upload that image to your server then upload it to any cloud storage(s3, Cloudinary,..) you are using. As this will be very easy to implement(you can find code snippet over the internet) and you can securely maintain your secret keys/credential to your cloud platform on the server side.
But, according to me best way of doing this will be something like this. I am taking user registration as an example
Make server call to get a temporary credential to upload files on the cloud(Generally, all the providers give this functionality i.e. STS/Signed URL in AWS).
The user will fill up the form and select the image on the client side. When the user clicks the submit button make one call to save the user in the database and start upload with credentials. If possible keep a predictable path for upload. Like for user upload /users/:userId or something like that. this highly depends on your use case.
Now when upload finishes make a server call for acknowledgment and store some flag in the database.
Now advantages of this approach are:
You are completely offloading your server from handling file operations which are pretty heavy and I/O blocking and you are distributing that load to all clients.
If you want to post process the files after upload you can easily integrate this with serverless platforms and do that on there and again offload that.
You can easily provide retry mechanism to your users in case of file upload fails but they won't need to refill the data, just upload the image/file again
You don't need to expose the URL directly to the client for file upload as you are using temporary Creds.
If the significance of the images in your app is high then ideally, you should not complete the transaction until the image is saved. The approach should be to create an object in your code which you will eventually insert into mongodb, start upload of image to cloud and then add the link to this object. Finally then insert this object into mongodb in one go. Do not make repeated calls. Anything before that, raise an error and catch the exception
You can have many answers,
if you are working with big files greater than 16mb please go with gridfs and multer,
( changing the images to a different format and save them to mongoDB)
If your files are actually less than 16 mb, please try using this Converter that changes the image of format jpeg / png to a format of saving to mongodb, and you can see this as an easy alternative for gridfs ,
please check this github repo for more details..
I will need to store photo's on a datastore, store the location in Mongo so I can retrieve it later to display. What is a good way to go about doing this? If I just google I end up getting information on accessing the files in node itself.....
At this point I am not sure what datastore I will use, for now just on another server I have, Ubuntu Server.
I made something like what you need. I had to store photos on s3 and store the path on mongo. If you use amanzon too, they provide a sdk, that contains all possible functions for interactions with the cloud. So you'll need a way to work with asynchronous tasks when store the data in the cloud, I used async module to manage the functions. I hope I gave you some direction where go to. I can't help you code, once I don't know your business rules, good lucky
I'm trying to save the images from a third-party API to my own S3 bucket using Node.js and MongoDB. The API provides a URL to the image on the third-party servers. I've never done this before but I'm assuming I have to download the image to my own server and then upload it to S3?
Should I save the image to mongodb with GridFS and then delete it once it is on S3? If so, what's the best way to do that?
I've read and re-read this previous question:
Problem with MongoDB GridFS Saving Files with Node.JS
But I can't find good documentation on how I should determine buffer/chunk size and other attributes for a JPEG image file.
Once I've saved the file on my server/database, then it seems like I should use:
https://github.com/appsattic/node-awssum
To upload it to S3. Is that a good idea?
I apologize if this is an obvious answer, I'm pretty green when it comes to databases and server scripting.
The easiest thing to do would be to save the image onto disk and then stream the upload from there using AwsSum's S3 PutObject operation. Alternatively, if you have the file contents in a Buffer you can just use that.
Have a look at the following two examples and they should help you figure out what to do:
https://github.com/appsattic/node-awssum/blob/master/examples/amazon/s3/put-bucket.js
https://github.com/appsattic/node-awssum/blob/master/examples/amazon/s3/put-object-streaming.js
Let me know if you need any more help. :)
Cheers,
Andy
Disclaimer: I'm the author of AwsSum.