wondering what the modern paradigm is for securing upload images/files.
In other words, I have a website which requires authentication. Once a user logs in they can upload files/images which they can later access.
I'm using multer for the file upload, so a couple questions:
1) Should I be storing files in a database (MongoDB) or in folders? Pros and cons? Security considerations?
2) Assuming the answer to #1 is "folders", how do I practically control access to (i.e. authorization) to those files. In other words, if a user upload file image.jpg, and I save it to http://API_URL/images/image.jpg, I want that jpg to be accessible only after I receive a validation token from that user, but that isn't possible since the image will be accessed via , i.e. not a get or post where I can attach such token. Is that something I should be doing via the express router? Maybe make the path of the saved file /images/:id and then attach some long user id which cannot be guessed as part of the path to get to the image?
Thanks
Related
I'm building an app which allows users to upload their images to IPFS and the image will be loaded using an <img> tag, the file type checking is done only in the frontend.
However, I'm also aware of the File Upload Vulnerability in normal centralized servers. So here's my question, would hackers be able to explore this?
The following is a JavaScript file I tried to bypass frontend checking and upload to IPFS, however, when I try to access its URL it returns the file in text instead of executing it. As a sophisticated hacker, would he/she be able to upload a malicious file somehow and cause damage on my site or my users?
https://cloudflare-ipfs.com/ipfs/bafybeigynjetni7b2z52qqv75u5c6k3fgrowqdp6a4qtcbfd4rq7nnj3pu/
All data on ipfs in the public node is Public. If you want to secure files you need to encrypt them. https://docs.ipfs.tech/concepts/privacy-and-encryption/
I have a MEVN Stack app. This requires to upload media files to the database. From the client-side, I am posting a multipart type form which will send the file to the server through Axios. I don't know how to store that in the MongoDB database and retrieve the same.
What you are looking for, is creating an API in node.js to handle uploaded file. (You can use express.js to make your life easier if you are not already using it)
In Vue.js create a component to upload a file and store the files in the file system on the server might be in a folder that is accessible through the web server (like in a public folder).
Finally, save the URL to access the file in MongoDB so, you can reference it later. You can google for tutorials on this, an example is here.
Context
I am building Stateless REST APIs for a browser-based platform that needs to store some user-generated files. These files could potentially be in the GBs.
I am using AWS S3 for storage. I have used AWS SDK in the past for this to route the file uploads through the NodeJS server (Basically - Upload to Server, Server uploads to S3).
I am trying to figure out how to improve it using the Pre-signed urls. I understand the dynamics and the flow on how to get the presigned urls and how to upload the file to S3 directly.
I cannot use SQS or Lambda to trigger object created event.
The architecture needs to be AWS independent.
Question
The simplest of flows I need to achieve is pretty common -
User --> Opens Profile
Clicks Upload Photo
Client Sends Request to /getSignedUrl
Server Returns the signedURL for the file name/type
The client executes the PUT/POST request to upload the file to the signedUrl
Upload Successful
After this - my understanding is -
Client tells the server - File Uploaded Successfully
Server associates the S3 Url for the Photo to the User.
...and that's my problem. How do I associate the successfully uploaded file back to the user on the server in a secure way?
Not sure what I've been missing. It seems like a trivial use case but I haven't been able to find anything regarding it.
1/ I think for the avatar, you should set it as public-read.
When create signed upload url in the
GET: /signed-upload-url
You need to set the image as public-read. After that you are free to interact with image through the direct url. Since this is an avatar, so you can compress it, reduce the size of image by the AWS Lambda function.
2/ If you don't want to have the public-read, you need to associate with server to get signed-download-url to interact with image
GET: /signed-download-url
I can't find a response to my question.
I'm building a React app using NodeJS and CRA and i need to implement an uploading avatar system. But i'm not sure where to save the uploaded image. My Node server serve a static folder 'public', so does i need to save images in /public/avatar? But each time i will make update on the app and re-build the client-side folder, this will overwrite the public folder and remove all the previous uploaded avatar ? I'm right ? So what are you suggesting me ?
Thanks,
There are multiple locations that you can store your user uploaded images, though storing them in your public is probably not the best location.
In the case where you were using a database like MongoDB, you could store the image inside Mongo using gridfs and serve the data using a route when you retrieve the user information. Similarly, you can also store in the database a path to the file, and return the path, or the file data, from the route as well.
Be careful with user uploads, however, as arbitrarily allowing uploaded data can lead to unanticipated results if you're not careful.
You could also use Gravatar (https://gravatar.com/).
User can choose an avatar assigned to their mail address hash, or use an automatically generated one by default.
Though with this solution you cannot let users change their avatar directly on your website.
It is widely used on well known websites like StackOverflow.
I would like to know what is the best way to handle image uploading and saving the reference to the database. What I'm mostly interested is what order do you do the process in?
Should you upload the images first in the front-end (say Cloudinary), and then call the API with result links to the images and save it to the database?
Or should you upload the images to the server first, and upload them from the back-end and save the reference afterwards?
OR, should you do the image uploading after you save the record in the database and then update it once the images were uploaded?
It really depends on the resources, timeline, and number of images you need to upload daily.
So basically if you have very few images to upload then you can upload that image to your server then upload it to any cloud storage(s3, Cloudinary,..) you are using. As this will be very easy to implement(you can find code snippet over the internet) and you can securely maintain your secret keys/credential to your cloud platform on the server side.
But, according to me best way of doing this will be something like this. I am taking user registration as an example
Make server call to get a temporary credential to upload files on the cloud(Generally, all the providers give this functionality i.e. STS/Signed URL in AWS).
The user will fill up the form and select the image on the client side. When the user clicks the submit button make one call to save the user in the database and start upload with credentials. If possible keep a predictable path for upload. Like for user upload /users/:userId or something like that. this highly depends on your use case.
Now when upload finishes make a server call for acknowledgment and store some flag in the database.
Now advantages of this approach are:
You are completely offloading your server from handling file operations which are pretty heavy and I/O blocking and you are distributing that load to all clients.
If you want to post process the files after upload you can easily integrate this with serverless platforms and do that on there and again offload that.
You can easily provide retry mechanism to your users in case of file upload fails but they won't need to refill the data, just upload the image/file again
You don't need to expose the URL directly to the client for file upload as you are using temporary Creds.
If the significance of the images in your app is high then ideally, you should not complete the transaction until the image is saved. The approach should be to create an object in your code which you will eventually insert into mongodb, start upload of image to cloud and then add the link to this object. Finally then insert this object into mongodb in one go. Do not make repeated calls. Anything before that, raise an error and catch the exception
You can have many answers,
if you are working with big files greater than 16mb please go with gridfs and multer,
( changing the images to a different format and save them to mongoDB)
If your files are actually less than 16 mb, please try using this Converter that changes the image of format jpeg / png to a format of saving to mongodb, and you can see this as an easy alternative for gridfs ,
please check this github repo for more details..