HTTP 413 Request Entity Too Large in Node JS Project in GAE - node.js

I have my backend app deployed on GAE. Inside it, I am having an API which will upload a file to a GCS bucket.
Recently I tried uploading a file of more than 50mb size and got 413 Request entity too large
Did some research and found out that the issue is with ngnix. The API will give 413 for any file > 32Mb.
Found one solution where it was mentioned to include a ngnix.conf file and add client_max_body_size 80M in it.
I did so but still getting the same error.
This is my ngnix-app.conf file
server{
location / {
client_max_body_size 80m;
client_body_buffer_size 512k;
}
}
Anything obvious that I am missing out here?

You can change your request buffer size in your Node.Js application using
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));
Also you can increase your request in nginx configuration
server{
location / {
client_max_body_size xxm;
client_body_buffer_size xxm;
}
}

Just Modify NGINX Configuration File
sudo nano /etc/nginx/nginx.conf
Search for this variable: client_max_body_size. If you find it, then just increase the value to 100M. If you can't find this then just add
this line inside HTTP.
client_max_body_size 100M;
To apply changes just restart ngnix
sudo service nginx restart
See screenshot for better understand

Related

Azure App service (Linux Nginx) file upload error 413 (Payload Too Large)

I am getting 413 (Payload Too Large) when I try to upload a video with size 10MB. How can I change the app service upload size.
Server:Azure App service (Linux Nginx)
API: PHP 8
I tried adding client_max_body_size 100M;didn't work
you will need to modify the client_max_body_size directive in the nginx configuration file. The location of the configuration file is:
/etc/nginx/nginx.conf
add or edit the following setting:
client_max_body_size 100M;

client_max_body_size is in Nginx conf but seems to be getting ignored

I'm running into status code 413 Request Entity Too Large. I'm running an Amazon Linux 2 AMI instance on AWS's Elastic Beanstalk, which is running an express server with a post route that uploads files to an S3 Bucket and then both adds some data to a table and produces a kafka message. Everything is working properly with files below 1MB size.
I understand nginx's default max-size value is 1MB and that I must change it.
I tried every answer in this thread Increasing client_max_body_size in Nginx conf on AWS Elastic Beanstalk but despite getting the client_max_body_size 10M; inside the nginx.conf file, and restarting nginx everytime I changed a configuration, using nginx -t to see if anything was wrong with the syntax, resulting in everything being ok, and finally proving via this command that the client_max_body_size 10M; line was in fact there, when it accused of there being a duplicate of it inside the file, all of these configs seemed to be completely ignored by my micro-service whenever I try to post a file greater than 1MB.
i added client_max_body_size 10M; manually to show that, when testing, nginx tells me it's duplicate, proving it was already included in the nginx.conf file
I also tried to put my conf files inside a .platform/conf.d/ structure, which did make the client_max_body_size 10M; go inside the nginx.conf file, but still it made no difference for my request.
I've also tried reload and restarting the nginx service, both to no avail.
I don't have much ideas on where to proceed from here. Any tips?
The link you are giving is for Amazon Linux 1 (AL1). These days all EB platform are based on AL2, and nginx is set differently. Namely, you should create .platform/nginx/conf.d/myconfig.conf file in the root of your application, with the content of:
client_max_body_size 10M;

How can I track upload progress from app behind Nginx reverse proxy?

I have a node.js server behind an Nginx reverse proxy. The node.js app has an endpoint to receive a file upload using busboy. As the file is uploaded I would like to track progress. However, Nginx I believe buffers it, so my app receives the file all at once. How can I make it so that my node app receives the packets as soon as possible? I have tried setting the following in my nginx.conf file:
http {
....
proxy_busy_buffers_size 0;
}
and
http {
....
proxy_buffering off;
}
In the documentation it covers this. set proxy_request_buffering off; In my case, I set it has follows
location / {
...
proxy_request_buffering off;
...
}

Configuring nginx client_max_body_size on Elastic Beanstalk Node

I have a Node 10 app running on Elastic Beanstalk, and it throws 413 errors when the request payload is larger than ~1MB.
<html>
<head>
<title>413 Request Entity Too Large</title>
</head>
<body>
<center>
<h1>413 Request Entity Too Large</h1>
</center>
<hr>
<center>nginx/1.16.1</center>
</body>
</html>
The request is not hitting my app at all; it's being rejected by nginx.
I have tried configuring AWS to increase the size of the allowed request body based on this answer, to no avail.
I've tried adding a file at .ebextensions/01_files.config with the contents:
files:
"/etc/nginx/conf.d/proxy.conf" :
mode: "000755"
owner: root
group: root
content: |
client_max_body_size 20M;
That didn't work, so I tried adding the file directly to .ebextensions/nginx/conf.d/proxy.conf with only:
client_max_body_size 20M;
And this also didn't work. Then I SSH'ed into the instance and added the file directly. Upon re-deploy, the entire conf.d directory was deleted and re-written, without this file.
How can I get AWS Elastic Beanstalk with Node.js 10 running on 64bit Amazon Linux 2/5.1.0 to accept nginx configuration?
The nginx setting you are trying to use (/etc/nginx/conf.d/proxy.conf) is for Amazon Linux 1.
Since you are using Amazon Linux 2 you should be using different files for setting nginx. For AL2, the nginx settings should be in .platform/nginx/conf.d/, not in .ebextentions as shown in the docs.
Therefore, you could have the following .platform/nginx/conf.d/myconfig.conf with content:
client_max_body_size 20M;
The above is an example only of the config file. I can't verify if the setting will actually work, but you are definitely using wrong folders to set nginx options.
My recommendation would be to try to make it work manually through ssh as you are attempting now. You may find that you need to overwrite entire nginx setting if nothing works by providing your own .platform/nginx/nginx.conf file.
adding the client_max_body_size 20M; in the folder
.platform/nginx/conf.d/proxy.conf fixed it for me. Please make sure to check whether you are using Amazon Linux 2

CORS error upload file ~4mb

I'm building an app where angular front-end is on s3 as static website and Sails (0.10.3) API inside dokku with Node 0.11.13 and SSL on EC2. If file is larger than about 4mb I got error "No 'Access-Control-Allow-Origin' header is present on the requested resource." OPTIONS request is hitting my API and I can catch it in customMiddleware but the POST with data is not reaching that far. On front-end side I'm using angularjs-file-upload.
If I turn off SSL then it works without any problems but I would prefer to keep it on.
One more thing you can try, if it's Nginx that's causing issues. Look into your error log file. Generally its /var/log/nginx/error.log
In that if you see this line
*133774 client intended to send too large body: 3141911 bytes
It means the issue is of the size and you might wanna fix it.
The way you do it is in your nginx.conf in the HTTP context paste this anywhere.
client_max_body_size 50M;
This will allow you to increase your body size to 50M.
Hence fixing the issue
I went up the chain app itself -> dokku -> SSL and the problem was even higher, in nginx.
nginx.conf required one line more:
proxy_read_timeout 1200s;
I had a similar solution and I solved it with the help of this. It's because of the client_max_body_size configuration variable that Nginx has.
1.Go to nginx configuration setting file:
sudo nano /etc/nginx/nginx.conf
2.Change in nginx.conf file:
http {
clientmaxbody_size 100m;
}
3.Restart Your Nginx:
sudo service nginx restart
or
service nginx reload
for my MEAN Stack (node & Expreess js) application it worked.
Hoping for your case too !
Another EXPRESS code if 1,2 & 3 point won't work for you
const express = require('express');
const cors = require('cors');
const app = express();
app.use(cors());
Below link for reference full detail explanation:
https://www.digitalocean.com/community/questions/how-to-edit-nginx-setting-for-big-file-upload

Resources