Images not showing up in folder using Multer and nodejs - node.js

I have a nodejs backend with Multer to upload images to a local folder named /uploads and after that, I store the image name in my MySQL database. When I run this on localhost the file gets uploaded to the folder and everything seems to work, but after I push the code to Heroku, the image doesn't get uploaded to the folder anymore. I can still access the file when I fetch the URL of where the file should be, but I can't see it. and when I push new code to Heroku then since the image is not in the folder. I'm guessing it gets overridden.
Anyone can explain to me why or how? thanks!
app.post('/api/register', upload.single('avatar'), function (req, res, next) {
pool.getConnection(async (err, connection) => {
if (err) throw err
console.log(req.file);
const username = req.body.username;
var password = req.body.password;
const email = req.body.email;
const picture = req.file.filename;
const salt = await bcrypt.genSalt();
password = await bcrypt.hash(password, salt);
connection.query(`INSERT INTO users (username, password, email, picture) VALUES (?,?,?,?)`,
[username, password, email, picture], (err, rows) => {
connection.release() // return the connection to pool
if (!err) {
console.log("insert succes");
res.send(`user with the record ID has been added.`)
} else {
console.log(err)
}
})
})
})
const storage = multer.diskStorage({
destination(req, file, callback) {
callback(null, './uploads');
},
filename(req, file, callback) {
callback(null, `${file.fieldname}_${Date.now()}_${file.originalname}`);
},
});
const upload = multer({ storage })
const app = express()
app.use('/uploads', express.static(process.cwd() + '/uploads'));
app.use(express.json());
app.use(cors());

As mentioned in the comment via the link, because Heroku is more of a 'container' all storage is ephemeral (meaning it on the running container and is not persistent) So each time a container spins up its just loading your base code into the running image nothing more nothing less. When the image restarts or rebuilds it's a fresh instance and any items added while it was running are basically lost..
So, unless your application is 'stateless' then will need to seperate things like
Sessions
Persistent storage
Fortunately, this is not too difficult to setup. For sessions you would want to use something like Redis to store session information and for images / assets you can use S3 or other object store. Multer has a good s3 connector ( works with any s3 compatible object store), then in your DB you just store the file name and to return it, use the s3 url ( media.example.com/images/[image]) in your application..
Now, to your question.. You should still be able to upload files to the running container (unless permissions prevent you ) there are use-cases where it does make sense, for example its just a temp file that will be read right away in the same thread as the request etc.
From your code it looks like you are trying to upload to the 'uploads' folder off the root context of your app, so I would first ensure you have an 'uploads' folder and it has permissions to read / write, also the correct owner as well. Node should give you an error message in the console if there is an error so you will need to SSH to the running heroku instance to see what is going on from that point.

Related

Upload Images to Web Server directory using Multer [Nodejs]

I am upload images using multer and Nodejs. The upload works perfectly. The problem is I want to upload those images directly to a folder inside wwwroot folder of the IIS Webserver.
var Storage = multer.diskStorage({
destination: function (req, file, callback) {
// callback(null, "../grit-server/uploads/");
callback(null, "https://example.com/grit/images/photogallery/");
},
filename: function (req, file, callback) {
const sqc = sqlConn;
var fileNames = (file.originalname.split('_'))
let query = 'INSERT INTO dbo.MS_PhotoGallery (MinorId,Photo_Name) VALUES (#MinorId,#PhotoName)'
let params = [
sqc.paramize('MinorId', fileNames[1], sql.Int()),
sqc.paramize('PhotoName',fileNames[2], sql.VarChar(255))
]
sqc.SqlExecuteStatementCallback(
query,params,(result)=>{
res.send(result)
}
)
callback(null, file.originalname);
}
});
The above code is of the Multer storage where I am setting the URL where to upload the images. But it is throwing error
'Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client'
What modifications shall I do to upload the images in the URL mentioned under Storage to save any photos in the location.
I have solved this problem hardcoding the path like '../../../../wwwroot/abc/def/'. This is not the efficient way but it works for my specific problem as the images folder is not going to change in future.

Heroku | Cannot GET

I deployed an API to heroku. One of its functions is file uploading. I made a static folder called "uploads" and upload files using multer package.
app.use('/api/uploads', uploadRouter);
app.use('/uploads', express.static(path.join(__dirname, '/uploads')));
Above code snippet comes from server.js. Below is the router and multer setup.
const storage = multer.diskStorage({
destination(req, file, cb) {
cb(null, 'uploads/');
},
filename(req, file, cb) {
cb(null, `${Date.now()}.jpg`);
},
});
const upload = multer({ storage });
uploadRouter.post('/', upload.single('image'), (req, res) => {
res.send(`https://app.herokuapp.com/${req.file.path}`);
});
uploadRouter.post('/imgs', upload.array('images'), (req, res) => {
try{
let filesArray = [];
req.files.forEach(file => {
filesArray.push(`https://app.herokuapp.com/${file.path}`)
})
res.status(201).send(filesArray);
}catch(error) {
res.status(400).send(error.message);
}
});
Something tells me that my approach is far from an ideal one. However, it used to work just perfect until I introduced multiple file upload. Now, images after being uploaded stay there for a little bit (I can access them via links like https://app.herokuapp.com/uploads/1623782012131.jpg) and disappear over few minutes. Then, these URLs throw the following:
Cannot GET /uploads/1623782012131.jpg
The thing is that my previous images that I uploaded before introducing multiple file upload are still accessible. I thought it had to do with heroku file size limitations but it was not the case since I've only used about 50mb out of 500mb. What might be the reason?
Heroku's filesystem is ephemeral/short lived. This means that any images you save on the disk will disappear after you restart or deploy your app. You can't rely on Heroku's filesystem to store images that need to persist for a longer time. Read more here: https://help.heroku.com/K1PPS2WM/why-are-my-file-uploads-missing-deleted
For a reliable image storage, have a look at cloud storage solutions such as Cloudinary or AWS S3.

How to accept upload in multer only after approval from server?

I'm writing an application that accepts uploads from a client to server using electron, node and multer.
What I wanna do is only allow the upload after the server has accepted it. Maybe some kind of prompt. How would I go about it in JS?
Even though I mentioned multer, I imagine it's not gonna play a big part in this. Rn, my upload function goes through like this.
var upload = multer({ storage: storage, })
app.post('/uploadfile', upload.single('single'), (req, res, next) => {
const file = req.file
if (!file) {
const error = new Error('Please upload a file')
error.httpStatusCode = 400
return next(error)
}
res.send(file)
})
When the request comes, instead of running "upload.single", I imagine I have to use something else that prompts for validation and then run the actual upload function. But I'm very clueless. Any help is appreciated.

Failed to insert image into MongoDB server using NodeJS

I am trying to get the image when user submits the form and inserting it into mongoDB server.For image I am using Multer plugin but its showing me the error.Here is my code of NodeJS
const multer = require('multer');
mongoose.connect('mongodb://localhost:27017/mytable',{useNewUrlParser:true} )
.then(()=>
console.log("Mongodb connected"))
.catch(err => console.error("could not connected",err));
const Schema =new mongoose.Schema({
name:String,
email:String,
lastname:String,
pass:String,
phonenumber:String,
zipcode:String,
birthdate:String,
img: {contentType:String,data:Buffer }
});
Schema.plugin(mongoosePaginate)
var user = mongoose.model('mytable', Schema);
//Multer for include image into directory
app.use(multer({ dest: '/public/'}).single('files'));
app.post('/save',(req,res)=>{
console.log("image is" +req.body.img);
var model = new user();
model.name = req.body.name,
model.email=req.body.email,
model.lastname=req.body.lastname,
model.pass=req.body.pass,
model.phonenumber=req.body.phonenumber,
model.zipcode=req.body.zipcode,
model.birthdate=req.body.birthdate,
/* model.img.data = req.body.img, */
model.img.data = fs.readFileSync(req.files.userPhoto.path);
newPic.image.contentType = 'image/png';
model.save(function(err,doc){
});
res.json({result:'sucess'});
res.end();
});
I just uploaded the required code. I am getting the error of Cannot read property 'userPhoto' of undefined .I don't know what should I write in fs.readFilesync.Please help me to insert image into a server .
You ask Multer to handle a .single() file that is expected to be referred to by the input name "files". According to the doc:
The single file will be stored in req.file
But you try to access req.files instead. (And it seems you're expecting this file to be referred to as "userPhoto", maybe?).
See also what information Multer exposes to retrieve the uploaded file's path.
Finally, you might want to restrict your middleware usage to the routes that need it.
EDIT: a few comments
// This tells the WHOLE app that:
// when a request comes in, execute this
app.use(
// Multer will do:
// when I'm given an incoming request (here it's every request)
// then I'm looking for *one* file being uploaded
// this file should be named "files" (i.e. client has <input type="file" name="files">)
// if I find it, then I store it in /public
multer({ dest: '/public/'}).single('files')
);
// This tells the app that:
// if a request comes in for endpoint /save with method POST, this is the code to execute
app.post('/save', (req, res) => {
// in here, Multer will have been executed already
});
So:
does your form really names its file to be uploaded "files"? I'd guess your form names the file "userPhoto"... just a guess!
if such a file exists in the request, Multer documentation says that your route handler can access it in req.file (not req.files)
if not, Multer will just let the request pass (that's what middlewares do), so you won't have a req.file
if req.file is mounted on the request by Multer, it exposes several data fields, such as req.file.path
I also hinted that you may not want to enable Multer for the whole app, but just for the routes that require it. Instead of a "global" app.use, you can define several times a route (or you could explicitly use the router, I don't see much of a difference), like:
app.post('/save', multer(...));
app.post('/save', (req, res) => {...});
// which can also be written as
app.post('/save', multer(...), (req, res) => {...});
This way, all other routes do not consider file uploading, I'm sure I don't need to highlight how better this is.
the problem is not with mongoose! as it says in your error message, req.files is undefined. it's a problem with multer documentation! when you're using single your file will be available in req.file
so this would fix your problem:
app.post('/save',(req,res)=>{
console.log("image is" +req.body.img);
var model = new user();
model.name = req.body.name,
model.email=req.body.email,
model.lastname=req.body.lastname,
model.pass=req.body.pass,
model.phonenumber=req.body.phonenumber,
model.zipcode=req.body.zipcode,
model.birthdate=req.body.birthdate,
/* model.img.data = req.body.img, */
model.img.data = fs.readFileSync(req.file.path); // here's the fix
newPic.image.contentType = 'image/png';
model.save(function(err,doc){
});
res.json({result:'sucess'});
res.end();
});
date: { type: Date, default: Date.now } // stick that into the Schema

NodeJS Server Uploading Large Files with Multer in AWS

I am able to upload very large files (12-15GB) when running my NodeJS server locally using multer. However, I'm having trouble getting large files to upload when deploying the node server on AWS.
When the node server is deployed, I can upload small files, but when uploading large files it hangs and never returns until I kill the node server. This seems like an environment issue inherent to being on AWS because I never get this problem locally.
In AWS I have my node server deployed on a t2.micro Windows Server 2016 Datacenter.
Is there something I'm missing fundamentally about AWS setup when transferring large files over the network? Some file limit somewhere? Something with Windows Firewall or a setting in my AWS profile?
Since I have been able to FTP any size file to my AWS server, I didn't think there was a file limit size imposed. So I'm inclined to believe it's probably some other issue.
Let me know if any further information is needed to help out with this if it's not obvious immediately.
Here is the multer part of my nodejs code in case that helps.
import { Router } from 'express';
import multer from 'multer';
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, 'C:/inetpub/wwwroot/Archive/' + req.body.destination)
},
filename: function (req, file, cb) {
cb(null, file.originalname)
}
});
let upload = multer({ storage });
export default ({ config, db }) => {
let api = Router();
var cpUpload = upload.fields([
{ name: 'destination' },
{ name: 'files' }])
api.post('/upload', cpUpload, function (req, res, next) {
res.json(req.files);
});
return api;
}

Resources