I'm currently considering developing a Meteor node.js app, but am struggling with how best to handle uploading of user images. In particular, I want to create a photography website that will allow the photographer to upload images in an 'admin' section, and these images will then be displayed on the website. I need to create a thumbnail of these images, and save the respective URLs to the database. I'm struggling with how to best accomplish this in meteor.
Is my best bet to use something like s3 combined with an AWS process for generating thumbnails?
Or should I save and host the images directly in the Meteor/node session?
Or should I scrap Meteor and use something like Express.js for this project?
Why don't you just use something like Filepicker.io to handle uploading and hosting images and simply store the image unique url (given to you by filepicker in the callback)?
Thumbnails can also be dynamically generated by Filepicker (using simple url modifications).
Cloudinary is a nicer alternative to filepicker when it comes to images, but integration process will be messier.
I would store the images on the filesystem, not in a database. If you have a unique id, you can use that as part of the url, for example an id of the item the image belongs to. Might look like this:
./uploads/img-<id>-<size>.jpg
You can write to disk and resize if necessary with node-imagemagick and your cdn should just poll these images from time to time. Not exactly sure how that part would work in terms of including the url to the image in the html.
Related
Users should upload photos.
They should be hosted somewhere else.
Only the photo hosting, no photo transformation needed.
What should I use, and how to use it.
Cloudinary.
It works, but I need a very simple solution
I have searched far and wide, but everyone seems to be focused on how to upload images to a NodeJs REST API. I'm already using Multer for that. Now I need to figure out how to download the private images through my REST API. When I say "private" I obviously mean that the images are specific to each user who is logged in - they are NOT publicly accessible nor are they accessible to anyone who is logged in except for the user who uploaded them. I tried ExpressJs's sendFile(), but it sends a binary file that is seemingly impossible for the client to transform and insert into an img element because it's not a blob. But what I'm REALLY asking is: Is there a mechanism/library already available to handle this, like Multer - except for DOWNLOADING PRIVATE images and letting me insert them into img elements?
Thanks for your suggestions. I'm using Vue and Vuetify on the front end, by the way.
Final solution is to use image-to-base64 (https://www.npmjs.com/package/image-to-base64) on server and download the base64 string and plug it into the img src as src="data:image/(ext);base64,...". Ref: https://en.wikipedia.org/wiki/Data_URI_scheme As you know it makes the download 33% bigger, but that's the cost of privacy! Have fun!!
I have a website created using Node Express, this website serves functionality where user can upload an image and it will be stored locally in server folder and the path will be saved in database.
The problem is the images size is taking too much space on the server disk, so i need to use cdn as a storage for those images and to show the image to the user. The problem is i don't know what is the proper end-to-end flow to store this image to cdn.
The end-to-end flow means the customer upload the picture , the server save it, and can be used again when the user need to see it.
My thought is, when the user uploaded the image, then the server save it first locally, the image path, there will be cron running to store the image to CDN, at the end the image stored in the server will be deleted after success store the image to CDN.
Is that the correct way? or there are any other way to do this?
you can do something like this
store images on a cheap storage for long term like s3. this serve as source of truth.
configure cdn to use the s3 url or your server as source => you don't neeed to upload to cdn.
bonus: create an image resizer service to sit in front of the source and configure cdn to use the image resizer service as source. this way, it will reduce the load to your resizer service
In addition to the Tran answer, you can perform some optimizations.
For example, converting to WebP image format which can reduce the size.
Also, you can look at Image CDNs available which are optimized for images. Following article can be helpful for you
https://imagekit.io/blog/what-is-image-cdn-guide/
Am not saying that i dont know how because i tried it before using multer and it works perfectly fine but the thing is these images get saved to a foldet called uploads with a namd and an id that i generatr using js
What i want to know : is this the propper way of doing it and if so wherr does that folder go to after hosting your website and does it contain alll the images from my website, i mean it will be a really large folder so where does it go to when hosting ??
PS: And another thing whenever i go to youtube or facebook etc i see a domain like i.ytimg.com containing these images so how is that acheived
I don't know if my reply would be helpful, but the way I use it is I save the image in a mongoDB database (Buffer) and just load it from the database.
I would like to know what is the best way to handle image uploading and saving the reference to the database. What I'm mostly interested is what order do you do the process in?
Should you upload the images first in the front-end (say Cloudinary), and then call the API with result links to the images and save it to the database?
Or should you upload the images to the server first, and upload them from the back-end and save the reference afterwards?
OR, should you do the image uploading after you save the record in the database and then update it once the images were uploaded?
It really depends on the resources, timeline, and number of images you need to upload daily.
So basically if you have very few images to upload then you can upload that image to your server then upload it to any cloud storage(s3, Cloudinary,..) you are using. As this will be very easy to implement(you can find code snippet over the internet) and you can securely maintain your secret keys/credential to your cloud platform on the server side.
But, according to me best way of doing this will be something like this. I am taking user registration as an example
Make server call to get a temporary credential to upload files on the cloud(Generally, all the providers give this functionality i.e. STS/Signed URL in AWS).
The user will fill up the form and select the image on the client side. When the user clicks the submit button make one call to save the user in the database and start upload with credentials. If possible keep a predictable path for upload. Like for user upload /users/:userId or something like that. this highly depends on your use case.
Now when upload finishes make a server call for acknowledgment and store some flag in the database.
Now advantages of this approach are:
You are completely offloading your server from handling file operations which are pretty heavy and I/O blocking and you are distributing that load to all clients.
If you want to post process the files after upload you can easily integrate this with serverless platforms and do that on there and again offload that.
You can easily provide retry mechanism to your users in case of file upload fails but they won't need to refill the data, just upload the image/file again
You don't need to expose the URL directly to the client for file upload as you are using temporary Creds.
If the significance of the images in your app is high then ideally, you should not complete the transaction until the image is saved. The approach should be to create an object in your code which you will eventually insert into mongodb, start upload of image to cloud and then add the link to this object. Finally then insert this object into mongodb in one go. Do not make repeated calls. Anything before that, raise an error and catch the exception
You can have many answers,
if you are working with big files greater than 16mb please go with gridfs and multer,
( changing the images to a different format and save them to mongoDB)
If your files are actually less than 16 mb, please try using this Converter that changes the image of format jpeg / png to a format of saving to mongodb, and you can see this as an easy alternative for gridfs ,
please check this github repo for more details..