express cant handle too many requests? - node.js

Im currently working on my new MERN stack project.
Im trying to get information about time spent on every location and everything seemed ok until i' ve discovered a bug. After changing location and sending post request to server about 10 times whole localhost server is frozing. Any calls to the server stop working. This is very frustrating and i cant figure it out. Do you guys have any ideas?
UseEffect calling action:
useEffect(() => {
dispatch(savePageStats())
}, [location])
Redux action call:
export const savePageStats = () => async (dispatch) => {
try{
const arr = []
await api.savePageSession(arr)
}catch(err){
console.log(err)
}
Axios api:
export const savePageSession = (arr) => API.post('/stats/savepagesession', arr)
Express router:
const app = express()
app.use(bodyParser.json({ limit: '30mb', extended: true}))
app.use(bodyParser.urlencoded({ limit: '30mb', extended: true}))
app.use(cors())
app.use('/users', usersRoutes)
app.use('/messages', messagesRoutes)
app.use('/stats', statsRouter)
dotenv.config()
mongoose.connect(process.env.CONNECTION_URL)
.then(() => app.listen(process.env.PORT, () => console.log(`server running on port ${process.env.PORT}`) ))
.catch((err) => console.log(err))
Express controler
export const savePageSession = async (req, res) => {
try{
console.log('im here')
}catch(err){
res.status(500).json({message: 'something went wrong'})
}
}

The savePageSession route handler doesn't send any response back to the client. That means the browser will still be waiting for a response and will eventually stop sending requests to your server until the previous requests finish or eventually timeout.
Add at least something simple like res.send("ok"); to the route. All http request handlers on your server MUST send some kind of response back to the client or it will mess up the client.

Related

Cannot run .JSON on response from the Fetch API

I am working with React, Express, PostgreSQL, Node, and the Fetch API. When I try to run a "GET" request (within a try block) to get data from my database, the request fails (and enters the catch block) with the following error:
Unexpected token < in JSON at position 0
Here is the failing code that I have on the front end:
const getRequests = async () => {
try {
const responseInfo = await fetch("/api/advice-requests", {
headers: { "Accept": "application/json" },
});
if (responseInfo.status === 200) {
console.log("200 running"); // This is logged to the console.
const data = await responseInfo.json();
console.log("data :", data); // This is NOT logged to the console. It fails.
setAdviceRequests(data.requests);
setResponses(data.responses);
return;
}
} catch (error_o) {
// The UI is updated with the text of the below error
setError(
"Something went wrong on our end. We are looking into how we can improve..."
);
return;
}
};
Here is some of my server code (there is more, but it is not relevant), including some changes I made that worked to solve this problem for other people.
const adviceRequests = require("./controllers/adviceRequests");
const express = require("express");
const app = express();
const cors = require("cors");
app.use(express.json());
app.use(cors());
app.options("*", cors());
if (process.env.NODE_ENV === "production") {
app.use(express.static(path.join(__dirname, "../build")));
app.get("/*", (req, res) => {
res.sendFile(path.join(__dirname, "../build", "index.html"));
});
}
app.get("/api/advice-requests", adviceRequests.getAll);
app.listen(PORT, () => {
console.log(`SERVER RUNNING ON ${PORT}.`);
});
Lastly, here is the adviceRequests.getAll function:
getAll: async (req, res) => {
const db = req.app.get("db");
try {
let requests = await db.requests.getAll(req.session.user.id);
let responses = await db.responses.getAll([req.session.user.id]);
return res.status(200).send([requests, responses]);
} catch (error) {
return res.status(500).send({
message:
"Something went wrong on our end. We are looking into how we can improve.",
error,
});
}
},
A bit more information:
The code works just fine when I run it locally
Even on the live server, I can successfully run several POST requests for authentication and adding requests. I just cannot get them.
I have done quite a bit of research into this and am posting my own as a last resort. Nothing that has worked for other people has worked for me so far.
Everytime I have had this "Unexpected token < in JSON at position 0" it was because i was trying to parse an html plain response as if it was a json. Note that every html file starts with a <.
I suggest you change this console.log("200 running"); with a console.log(responseInfo);, that way you'll notice if the response is a json or not.
From what I see, the problem might be the order in which the app.get are defined. Note that express serves first come first served, so since you have already defined an app.get("/*"), everything will be served by that route. Also note that you are sending back an index.html, which matches the issue shown in the frontend.

How do I make server-side fetch calls?

I have a React web application which currently does fetch calls client-side to update a dashboard with live information (let's say current weather, as an example), meaning that with an increase in users it will cause unnecessary traffic calls and could potentially crash this weather website.
What I am trying to understand is how can I make those fetch calls be server-side? I have looked into creating a Node.js Express server, but I am unsure if it has the functionality to make fetch calls to a remote host.
Here is my code with request-weather which does not really work, unfortunately.
const { response } = require('express');
const express = require('express');
const app = express();
var fetch = require('node-fetch');
const port = process.env.PORT || 5000;
app.use(express.json());
// This displays message that the server running and listening to specified port
app.listen(port, () => console.log(`Listening on port ${port}`));
// create a GET route
app.get('/request-info', (req, res) => {
res.send({ information: 'information call successful' });
});
app.get('/request-weather', (req, res) => {
fetch('http://thisotherwebsite.com/weather-query-that-returns-json',
{method: 'GET',
headers: {' Accept': 'application/json'}})
.then(res => {
return res;
})
});
Couple things:
Your /request-weather handler makes the request to thisotherwebsite but doesn't do anything with the response.
Your .then(res => { return res; }) doesn't actually do anything. You're just taking what fetch already returns and returning it.
If you want to send the response back to the browser you might do something like this:
fetch(...) // make the request
.then(result => result.json()) // extract the data
.then(data => {
res.json(data); // send it to the browser
})
If you want to do additional processing you could await the fetch call and then do whatever else you need to do with it:
app.get('/request-weather', async (req, res) => { // make handler async
// get data from the other site
const data = await fetch(...)
.then(response => response.json());
// package it up with some other stuff
responseData = {
fromOtherSite: data,
myExpressStuff: {
foo: 1,
bar: 2,
}
}
// return it to the browser
res.json(responseData);
Reference:
fetch: response.json() - Extracting data from a fetch response
express response.json() - Sending json to the response (usually to the browser)

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client - Node.Js, Express, Postgres

I'm having trouble with the error message in the title when trying to retrieve all users in my express .get('/users') method. I am using Node.js, Express, and node-postgres. I have my
getUsers(); function defined in my queries.js file, and I call the function in my app.get() function in my index.js file.
queries.js
const client = require('./object models/db_client_pool')
const Pool = require('pg').Pool
const pool = new Pool(client.client)
async function getUsers(request, response) {
await pool.connect()
pool.query('select * from discord_users', (error, results) => {
if (error) {
throw error
}
response.sendStatus(200).json(results.rows)
pool.release();
})
}
module.exports = {
getUsers
}
index.js
const express = require('express');
require('dotenv').config();
//const bodyParser = require('body-parser'); deprecated
const app = express();
const port = 3000;
const db = require('./queries');
app.use(express.json())
app.use(express.urlencoded({
extended: true
}))
app.get('/', (request, response) => {
response.json({ info: 'Node.js, Express, and Postgres API' })
})
app.get('/users', (req, res) => {
db.getUsers(req, res)
})
app.listen(port, () => {
console.log(`App is listening on port ${port}`);
});
As I said, I keep getting the "cannot set headers after they are sent to the client" error and I'm at a loss of what to do. Thanks in advance for your help!
Change from this:
response.sendStatus(200).json(results.rows)
to this:
response.status(200).json(results.rows);
or even just to this:
response.json(result.rows); // 200 is the default status already
The last one is fine because 200 is already the default status so you don't need to set that yourself.
The problem is that response.sendStatus(200) sends a complete response with an empty body and then you try to call response.json(result.rows) which tries to send ANOTHER response to the same request. Trying to send that second response to the same request is what triggers the error message you are getting.
response.status(200) just sets the status to 200 as a property on the waiting response object and waits for some other method to actually send the response itself which you can then do with .json(...).
So my guess is, you're running express 4.x and that doesn't support response.sendStatus(200) anymore. You have to use response.status(200) instead.
Now, another issue I see in your code is, I don't recognize pool.release() method from pg library. You can release a client back to a pool but you can't release a pool of clients. Maybe you meant pool.end()?

Handle simultaneous requests in express on local machine from arduino

I'm creating a local webserver (on my windows PC) for my arduino to send POST requests to. The data from these requests would then be written to Cloud Firestore via the local webserver since the Arduino cannot support HTTPS which is required for Firebase. I have the server set up using express:
const express = require('express');
const bodyParser = require('body-parser')
const app = express();
const HOST_PORT = 6500;
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
app.use(bodyParser.raw());
app.get('/',(req,res,next)=>{
return res.status(200).send()
})
app.post('/',(req,res,next)=>{
return res.status(200).send()
})
app.post('/machine',(req,res,next)=>{
console.log('Got body:', req.body);
//Send to firebase here use async/await or just send the request?
return res.status(200).send()
})
app.listen(HOST_PORT, () => console.log(`Started server at http://localhost:${HOST_PORT}!`));
If three arduinos make a request to /machine would three instances of that functions be created? I'm wondering because I would like to know how to handle simultaneous requests on the webserver. Do I just make the call to firebase and not await the result?
This is the nice thing about Node.js asynchronous runtime: while your /machine route awaits for a first request, it can handle other requests, no problem!
app.post('/machine', async (req,res,next)=>{
console.log('Got body:', req.body);
await firestore.doc("collection/docId").set({foo: "bar"})
return res.status(200).send()
})
More info can be found here

Real-time listener doesn't work in PWA with Pusher

I'm trying to build my first PWA. I managed to do it, however, pushers functionality for some reason doesn't work.
I was debugging this for a few hours now and I can't make it work. I already tried rebuilding the app several times.
So, I'm subscribing to this
this.prices = this.pusher.subscribe('coin-prices');
then I have this function
sendPricePusher(data) {
console.log('Sending data from React')
console.log(data)
axios.post('/prices/new', {
prices: data
})
.then(response => {
console.log(response)
})
.catch(error => {
console.log(error)
})
}
I'm calling the function above every 10 seconds from my
componentDidMount()
setInterval(() => {
axios.get('https://min-api.cryptocompare.com/data/pricemulti?fsyms=BTC,ETH,LTC&tsyms=USD')
.then(response => {
this.sendPricePusher(response.data)
})
.catch(error => {
console.log(error)
})
}, 10000)
NodeJs handles it perfectly. I see 200 in dev console.
app.post('/prices/new', (req, res) => {
// Trigger the 'prices' event to the 'coin-prices' channel
pusher.trigger( 'coin-prices', 'prices', {
prices: req.body.prices
});
res.sendStatus(200);
})
For some reason this magic piece of code doesn't work.
this.prices.bind('prices', price => {
this.setState({ btcprice: price.prices.BTC.USD });
this.setState({ ethprice: price.prices.ETH.USD });
this.setState({ ltcprice: price.prices.LTC.USD });
}, this);
It should recreate the state and the values will be updated.
So, I came to the conclusion that something is wrong with my server code. I want to host the app on heroku. I tried to write different variations of servers but none of them seem to work. However, I'm not 100% sure that my server is the problem. Can you please have a look at my server code? Here's my server.js file and a link to the project on github in case the problem is not so obvious. Pusher seems like a cool tech. I want to keep using it in my future projects just need to understand how.
// server.js
const express = require('express')
const path = require('path')
const bodyParser = require('body-parser')
const app = express()
const Pusher = require('pusher')
const HTTP_PORT = process.env.PORT || 5000;
//initialize Pusher with your appId, key, secret and cluster
const pusher = new Pusher({
appId: '593364',
key: '8d30ce41f530c3ebe6b0',
secret: '8598161f533c653455be',
cluster: 'eu',
encrypted: true
})
// Body parser middleware
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: false }))
app.use(express.static("build"));
// CORS middleware
app.use((req, res, next) => {
// Website you wish to allow to connect
res.setHeader('Access-Control-Allow-Origin', '*')
// Request methods you wish to allow
res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS, PUT, PATCH, DELETE')
// Request headers you wish to allow
res.setHeader('Access-Control-Allow-Headers', 'X-Requested-With,content-type')
// Set to true if you need the website to include cookies in the requests sent
// to the API (e.g. in case you use sessions)
res.setHeader('Access-Control-Allow-Credentials', true)
// Pass to next layer of middleware
next()
})
// API route in which the price information will be sent to from the clientside
app.post('/prices/new', (req, res) => {
// Trigger the 'prices' event to the 'coin-prices' channel
pusher.trigger( 'coin-prices', 'prices', {
prices: req.body.prices
});
res.sendStatus(200);
})
app.use((req, res) => {
res.sendFile(path.join(__dirname + "/build/index.html"));
});
app.listen(HTTP_PORT, err => {
if (err) {
console.error(err)
} else {
console.log('Server runs on ' + HTTP_PORT)
}
})
A good way to distinguish if the problem is in your server or Pusher's functionality is to test the Pusher part of the code separately with dummy data to ensure that at least the Publish/Subscribe functionality is working fine, i.e you have set it up as expected.
So you could just try the following in separate files:
//publisher
pusher.trigger( 'coin-prices', 'prices', {
prices: dummyprices
});
//subscriber
var prices = pusher.subscribe('coin-prices');
prices.bind('prices', ({ price }) => {
console.log(price);
})
Assuming you have initialized the Pusher SDKs correctly, this should work, if so, then the Pusher side of things is just fine and you can concentrate on finding out what in your server is causing the app to not work.
btw, in your existing code, you might wanna change:
this.prices.bind('prices', price => {})
to
this.prices.bind('prices', ({ price }) => {})
Hope this helps.

Resources