Getting proxy error when using "gulp serve" when API returns 204 with no content - node.js

I am using Gulp to develop an Angular application generated by Yeoman's gulp-angular generator. I have configured it to proxy requests to /api to another port, the port my API is listening on. That port is actually forwarded via an SSH tunnel to an external server.
Here is the config generated by Yeoman that I have edited for my own API:
gulp/server.js
'use strict';
var gulp = require('gulp');
var browserSync = require('browser-sync');
var httpProxy = require('http-proxy');
/* This configuration allow you to configure browser sync to proxy your backend */
var proxyTarget = 'http://localhost:3434/api'; // The location of your backend
var proxyApiPrefix = 'api'; // The element in the URL which differentiate between API request and static file request
var proxy = httpProxy.createProxyServer({
target: proxyTarget
});
function proxyMiddleware(req, res, next) {
if (req.url.indexOf(proxyApiPrefix) !== -1) {
proxy.web(req, res);
} else {
next();
}
// ...rest of config truncated
stdout
[BS] Watching files...
/Users/jason/dev/web/node_modules/http-proxy/lib/http-proxy/index.js:114
throw err;
^
Error: Parse Error
at Socket.socketOnData (http.js:1583:20)
at TCP.onread (net.js:527:27)
I get the above error when my application attempts to hit a particular API url which sends back a response of 204, no content.
url structure: POST /api/resource/delete
(API doesn't support actual DELETE http method so we POST to this endpoint)
Response: 204 No Content
The API is also in development and is being served via the built in PHP web server. What the server is telling us is that the client (aka Node in this case because it is the proxy) is hanging up before PHP can send the response.
I thought perhaps it was just choking on the fact that there was no content. So, we created a second endpoint that also returned 204 No Content and it seemed to work fine. But, to be fair, this issue appears to be intermittent - it works sometimes and sometimes it does not. It's very confusing.
As far as we can tell, it only happens on this delete URL, however. I am pretty new to Node and am having a very hard time figuring out what the issue is or where to look. Does anyone have any clues or has anyone seen this before?

It turns out that the developer of the API was sending me content along with his 204 when he shouldn't have been - some debug code left in. The HTTP parser that node-proxy uses was then reading that content from the buffer at the beginning of the subsequent request and then throwing an error because it wasn't seeing a properly formed HTTP request - since the first thing in the buffer was a PHP var_dump.
As it happens, my front end app did the delete call and then refreshes another object via the GET request. They happen so fast that it seemed like the DELETE call killed the gulp server, when it was actually the GET command afterwards.
The http-proxy module for node explicitly does not do error handling, leaving the onus on the end user. If you don't handle an error, it bubbles up into an uncaught exception and will cause the application to close, as I was seeing.
So, the fix was simply:
gulp/server.js
var proxy = httpProxy.createProxyServer({
target: proxyTarget
}).on('error', function(e) {
console.log(JSON.stringify(e, null, ' '))
});
The console will now log all proxy errors, but the process won't die and subsequent requests will continue to be served as expected.
For the error in question, the console output is:
{
"bytesParsed": 191,
"code": "HPE_INVALID_CONSTANT"
}
Additionally, we've fixed the API to honor its 204 and actually, you know, not send content.

Related

Failed to load resource: the server responded with a status of 431 (Request Header Fields Too Large)

when i am trying to use my NodeJS api from the react app(building a MERN stack app) i get the error mentioned in question
"Failed to load resource: the server responded with a status of 431 (Request Header Fields Too Large)"
the api is working fine from postman
const onSubmit=async(e)=>{
e.preventDefault()
if(password!==password2){
console.log('passwords dont match')
}else{
const newUser={
name:name,
email:email,
password:password
}
try {
const config={
headers:{
'Content-Type':'application/json'
}
}
const body=JSON.stringify(newUser)
//axios has been set up as proxy
//http://localhost:3000
//we dont need to add the above to url
const res =await axios.post('/api/users',body,config)
console.log(res.data)
} catch (error) {
console.error(error.response.data)
}
}
}
The HTTP 431 Request Header Fields Too Large response status code indicates that the
server refuses to process the request because the request’s HTTP headers are too long.
The request may be resubmitted after reducing the size of the request headers.
431 can be used when the total size of request headers is too large, or when a single
header field is too large. To help those running into this error, indicate which of the > two is the problem in the response body — ideally, also include which headers are too
large. This lets users attempt to fix the problem, such as by clearing their cookies.
Servers will often produce this status if:
The Referer URL is too long
There are too many Cookies sent in the request
In my case I was sending too many cookies because localhost:4200 was used as the domain of 3 different projects ... The solution (delete useless cookies)
Hope this helps...
I was getting this error when I used the same local port in the proxy destination as the port of the React app by accident. This created an internal forwarding loop, resulting in "Request Header Fields Too Large".
I was getting that error when forgetting to run my server first, before running React app.
I used a knex.js and express.js based simple back end and forgot to initialise it before starting React. Now it all works fine.
Clean your browser cache or Open with an incognito window.
Go to chrome dev tool > Application > Cookies,
Clear the cookies, and you are ready to rock!
You must be following Traversy Media !
remove proxy statement from package.json and write node url in axios call
After I used chrispytoe's suggestion to debug, I knew the problem is related to wrong URL, but not axios or React. To be more specific, something related to the URL on the server side.
Here's what I would suggest to help debug. On the server side in that
route, do console.log(req.headers). Then make the request from
postman, then make it from your react app and see what the differences
are. – chrispytoe
How I fixed this problem is to make sure the proxy server port in my client's packcage.json the same as my server.js' port number. Add the line below to the end of your's client's package.json (before the last "}").
"proxy": "http://localhost:5500"
Next, set your server.js' port the same as your proxy's port:
const PORT = process.env.PORT || 5500;
In addition, it doesn't matter what port you configure your client to use. It has nothing to do with the server and its proxy server.
Hope my 2 cents help. Please feel free to correct me. Thanks.
This error arises when the API server port you inputted in your front-end proxy(e.g "proxy":"http://localhost:5000/") isn't the same as the port on the server.
when you get back to the front end and start building React hooks, you'll have a to do a bit of jumping around on a PC
install cross-env lib:
npm i --save-dev cross-env
alter your start section in package.json
"start": "cross-env PORT=8000 react-scripts start",
allow you to declare the start port of the project to avoid conflict with other libs, in this case the create-react-app
431 HTTP response status code is sent from the server when client's HTTP Header is greater than the server's accepting HTTP Header limit. Maximum size of HTTP Header for well-known web-server is provided here.
This makes complete sense that the API is working fine from Postman and not browser; As there won't be any residue cookies present in Postman.

"Cross Origin Request Blocked" No solutions seem to work

Background
I'm building a MERN full stack application as a personal project. I am running the frontend client on localhost:3000 and the server on localhost:5000.
Problem
All of my API routes work as expected except for a GET request, router.get('/get-friends', ...) which queries the mongoDB to return a list of collection documents. Calling that get request on Postman returns the expected output. I decided to write a simple GET request that returns a method and it works just fine in my browser
When making the request the get-friends request in my browser, I get the following log:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost:5000/api/users/get-friends/. (Reason: CORS request did not succeed)
What I've Already Tried
Enabling cors in my Express server
Enabling cors preflight
Adding a proxy to the server from the client's package.json
Switching from Axios to vanilla JS's fetch() method
Turning off cors in my browser
I suspect the issue occurs when I make the request to the database from Express. I am really not sure how to solve this issue.
Here is the route in question:
router.get('/get-friends', (req, res) =>{
var species_ = req.body.species;
var gender_ = req.body.gender;
var neutered_ = req.body.neutered;
// query db
Friend.find({species: species_},{gender:gender_},{neutered:neutered_}).then((friends_) =>{
if(!friends_){
return res.status(404).send('query error, nothing returned');
}
return res.send(friends_);
}).catch((e) =>{
res.status(400).send(4);
})
});
Here is the project repo and the relevant files are:
https://github.com/edgarvi/foster-friends/server.js (Express server)
https://github.com/EdgarVi/foster-friends/blob/master/routes/api/users.js (Routes for the express server)
https://github.com/EdgarVi/foster-friends/blob/master/client/src/components/layout/SearchFriends.js (React component which calls the server)
I would gladly appreciate any help!
I have highlighted possible problems.
Reason: CORS request did not succeed
The HTTP request which makes use of CORS failed because the HTTP
connection failed at either the network or protocol level. The error
is not directly related to CORS, but is a fundamental network error of
some kind.
> In many cases, it is caused by a browser plugin (e.g. an ad blocker or
privacy protector) blocking the request.
Other possible causes include:
Trying to access an https resource that has an invalid certificate
will cause this error.
Trying to access an http resource from a page with an https origin
will also cause this error.
As of Firefox 68, https pages are not permitted to access
http://localhost, although this may be changed by Bug 1488740.
> The server did not respond to the actual request (even if it responded
to the Preflight request). One scenario might be an HTTP service being
developed that panicked without returning any data.
https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS/Errors/CORSDidNotSucceed
Thank you all for the help and the suggestions. After struggling through this for multiple days, I finally encountered a solution.
In my react client, I made the API call:
axios.get('http://localhost:5000/api/users/get-friends',
{
params: {
species: this.state.species,
gender: this.state.gender,
neutured: neutered_
}}
);
and then I changed the Mongoose query to look like:
router.get('/get-friends', (req, res) =>{
var species_ = req.query.species;
var gender_ = req.query.gender;
var neutered_ = req.query.neutered;
// query db
Friend.find({species: species_},{gender:gender_},{neutered:neutered_}).then((_friends) => {
return res.send(_friends);
})
});
I'm not exactly sure why these changes made my code finally work but once again, thank you all for the help and suggestions!

Why the callback function gets called twice for each request?

Guys I am new to node js and this behavior is strange to me!
In the code snippet below,
'use strict';
var http = require('http');
var numberOfRequests = 0;
http.createServer(function (request, responce) {
console.log('Request number ' + numberOfRequests + ' received!');
responce.writeHead(200);
responce.write("Here is responce to your request..");
responce.end();
numberOfRequests++;
}
).listen(8080);
console.log('listening ...');
for each
localhost:8080
call at Chrome, the app writes twice onto console? e.i
for a single 8080 call, it prints out:
Request number 0 received!
Request number 1 received!
I am using Visual studio to run this node js app.
Usually, when you see two requests for each page request, one is for the desired page and one is for the favicon for the website. This is what most browsers do unless there is meta tag in the page that tells the browser not to request a favicon resource. If you do this in your handler:
console.log(request.url)
That will likely show you what's going on. In general, you don't want to have a web server where you never look at what resource is being requested. If you based your logic on a specific resource being requested such as /, then you would easily be able to ignore other types of requests such as the favicon.
Run curl <hostname:port> to make a single request. This proves it's the browser making multiple.

Socket.io + Express CORS Error on localhost (not allowed by Access-Control-Allow-Origin)

I have a working node.js Express server to which I would to add socket.io support (allow javascript clients to connect via socket.io). I can connect to the express server via a Javascript $.get(), but the socket.io.connect() command fails due to a CORS error.
My testing machine is OSX with Apache to serve the client, thus port 80 is taken, so I have node.js/express running on port 8888. I added socket.io per the documentation:
var exp = express();
var server = require('http').createServer(api.server);
exp.listen(8888);
var io = require('socket.io').listen(server);
io.sockets.on('connection', function(socket) {
console.log('connection');
});
I properly see "info: socket.io started" in my node.js logs.
Then, on the client, I attempt to connect to the server...
this.socket = io.connect('http://localhost:8888');
this.socket.on('connect',function() {
socket.emit('install','test');
});
However, I'm getting a CORS error in the console in Chrome:
XMLHttpRequest cannot load http://localhost:8888/socket.io/1/?t=1358715637192. Origin http://localhost is not allowed by Access-Control-Allow-Origin.
HOWEVER, THIS works fine!
$.get('http://localhost:8888',function(e,d){
console.log(e,d);
});
So I double checked my headers, for both localhost:8888 and localhost -- both are properly returning the headers which (should) allow for the cross-domain requests...
Access-Control-Allow-Origin: *
Any ideas?
CORS is a very tricking thing to get working (or at least it was for me). I recommend this resource here: http://enable-cors.org/
Following what they do very carefully helped me. I also found that different browsers gave different visibility over the CORS request/responses which helped.
I found that Chrome was easier to get working than firefox, but firefoxes tools such as firebug, were quite nice to work with.
My gut feel from your information is that you might need your request to have an X-Request-With in your request attributes.
I also found using fidler to send the http requests allowed me to narrow my problems down to the server side initially and get that working. You will find browser enforce CORS, but something like fidler doesn't and thus provides another way of inspecting what is happening.
I definately recommend trying to break the problem in half so that you can see if it is server side or client side that is not behaving how you expect.
My problem was related to returning the same CORS response for the OPTIONS header as the POST or GET. That was wrong. Chrome allowed it. Firefox didnt. Any options request that is sent out will be sent out once, then in the future it will be cached and not resent (Which caused alot of confusion for me initially). For the options request you just need a standard response saying its ok to proceed, then in the post or get response i believe you want your cors responses there only.

Express request is called twice

To learn node.js I'm creating a small app that get some rss feeds stored in mongoDB, process them and create a single feed (ordered by date) from these ones.
It parses a list of ~50 rss feeds, with ~1000 blog items, so it's quite long to parse the whole, so I put the following req.connection.setTimeout(60*1000); to get a long enough time out to fetch and parse all the feeds.
Everything runs quite fine, but the request is called twice. (I checked with wireshark, I don't think it's about favicon here).
I really don't get it.
You can test yourself here : http://mighty-springs-9162.herokuapp.com/feed/mde/20 (it should create a rss feed with the last 20 articles about "mde").
The code is here: https://github.com/xseignard/rss-unify
And if we focus on the interesting bits :
I have a route defined like this : app.get('/feed/:name/:size?', topics.getFeed);
And the topics.getFeed is like this :
function getFeed(req, res) {
// 1 minute timeout to get enough time for the request to be processed
req.connection.setTimeout(60*1000);
var name = req.params.name;
var callback = function(err, topic) {
// if the topic has been found
if (topic) {
// aggregate the corresponding feeds
rssAggregator.aggregate(topic, function(err, rssFeed) {
if (err) {
res.status(500).send({error: 'Error while creating feed'});
}
else {
res.send(rssFeed);
}
},
req);
}
else {
res.status(404).send({error: 'Topic not found'});
}};
// look for the topic in the db
findTopicByName(name, callback);
}
So nothing fancy, but still, this getFeed function is called twice.
What's wrong there? Any idea?
This annoyed me for a long time. It's most likely the Firebug extension which is sending a duplicate of each GET request in the background. Try turning off Firebug to make sure that's not the issue.
I faced the same issue while using Google Cloud Functions Framework (which uses express to handle requests) on my local machine. Each fetch request (in browser console and within web page) made resulted in two requests to the server. The issue was related to CORS (because I was using different ports), Chrome made a OPTIONS method call before the actual call. Since OPTIONS method was not necessary in my code, I used an if-statement to return an empty response.
if(req.method == "OPTIONS"){
res.set('Access-Control-Allow-Origin', '*');
res.set('Access-Control-Allow-Headers', 'Content-Type');
res.status(204).send('');
}
Spent nearly 3hrs banging my head. Thanks to user105279's answer for hinting this.
If you have favicon on your site, remove it and try again. If your problem resolved, refactor your favicon url
I'm doing more or less the same thing now, and noticed the same thing.
I'm testing my server by entering the api address in chrome like this:
http://127.0.0.1:1337/links/1
my Node.js server is then responding with a json object depending on the id.
I set up a console log in the get method and noticed that when I change the id in the address bar of chrome it sends a request (before hitting enter to actually send the request) and the server accepts another request after I actually hit enter. This happens with and without having the chrome dev console open.
IE 11 doesn't seem to work in the same way but I don't have Firefox installed right now.
Hope that helps someone even if this was a kind of old thread :)
/J
I am to fix with listen.setTimeout and axios.defaults.timeout = 36000000
Node js
var timeout = require('connect-timeout'); //express v4
//in cors putting options response code for 200 and pre flight to false
app.use(cors({ preflightContinue: false, optionsSuccessStatus: 200 }));
//to put this middleaware in final of middleawares
app.use(timeout(36000000)); //10min
app.use((req, res, next) => {
if (!req.timedout) next();
});
var listen = app.listen(3333, () => console.log('running'));
listen.setTimeout(36000000); //10min
React
import axios from 'axios';
axios.defaults.timeout = 36000000;//10min
After of 2 days trying
you might have to increase the timeout even more. I haven't seen the express source but it just sounds on timeout, it retries.
Ensure you give res.send(); The axios call expects a value from the server and hence sends back a call request after 120 seconds.
I had the same issue doing this with Express 4. I believe it has to do with how it resolves request params. The solution is to ensure your params are resolved by for example checking them in an if block:
app.get('/:conversation', (req, res) => {
let url = req.params.conversation;
//Only handle request when params have resolved
if (url) {
res.redirect(301, 'http://'+ url + '.com')
}
})
In my case, my Axios POST requests were received twice by Express, the first one without body, the second one with the correct payload. The same request sent from Postman only received once correctly. It turned out that Express was run on a different port so my requests were cross origin. This caused Chrome to sent a preflight OPTION method request to the same url (the POST url) and my app.all routing in Express processed that one too.
app.all('/api/:cmd', require('./api.js'));
Separating POST from OPTIONS solved the issue:
app.post('/api/:cmd', require('./api.js'));
app.options('/', (req, res) => res.send());
I met the same problem. Then I tried to add return, it didn't work. But it works when I add return res.redirect('/path');
I had the same problem. Then I opened the Chrome dev tools and found out that the favicon.ico was requested from my Express.js application. I needed to fix the way how I registered the middleware.
Screenshot of Chrome dev tools
I also had double requests. In my case it was the forwarding from http to https protocol. You can check if that's the case by looking comparing
req.headers['x-forwarded-proto']
It will either be 'http' or 'https'.
I could fix my issue simply by adjusting the order in which my middlewares trigger.

Resources