I have two different versions of an endpoint for testing.
The first one is this:
app.get('/order', async (req, res) => {
console.log("OK")
getPrintFile()
})
And the second one is this:
app.get('/order', async (req, res) => {
console.log("OK")
await getPrintFile()
})
the getPrintFile is an async function that returns a promise when every operation is done. Withing the function I upload an image to a
server, I download a new image, and re upload that new image to another server.
I noticed that in the first example, without the await, if I send a lot of requests to the "order" endpoint,
I get the "OK" instantly for each request, which is what I want because that "OK" will get replaced by a res.status("200"). I need
to send a status 200 immediatly after getting the endpoint hit for various reasons. Then I don't care how long it takes for the server to do all the processing of the images/uploading, as long as the res.send(200) is executed instantly when there is a new incoming request.
However, when I use the await, even if new requests are coming in, it takes a lot to display the next "OK" if a previous request
is still processing. Usually it displays the OK only when the code within the getPrintFile function is done executing (that is, images are uploaded and everything is done)
It's like the event loop is blocked but I don't understand why.
What is happening here?
Edit:
So it is clearer, I tested it. If I send 5 requests to the "order" endpoint, the "OK" is displayed in the console immediately for all of them, and then the images are processed and uploaded at their own speed for each request. In the second example, if I send 5 requests, the first OK is displayed, and then the remaining 4 are displayed one at a time when the previous request is done executing, or if not exactly in that order, they get logged with tremendous delay, and not consecutively
I'll try to answer based on my understanding of your problem. The first thing missing is the res.sendStatus(200) in your examples to make them actually work. But then, indeed it happens as you describe it: the /order endpoint actually is "blocking" you from making another request as the await is blocking you from reaching the final statement (res.send). Here a full example
const express = require("express");
const app = express();
async function takeTime() {
return new Promise((resolve, reject) => {
setTimeout(() => {
console.log("resolved");
resolve(1);
}, 2000);
});
}
app.get("/order", (req, res) => {
console.log("ok"); // Happens immediately
await takeTime();
return res.sendStatus(200);
});
app.get("/", async (req, res) => {
console.log("ok"); // Happens immediately
takeTime();
return res.sendStatus(200);
});
app.listen(3000, () => {
console.log("server running on port 3000");
});
When testing the API (i.e. from Postman), you won't be able to run immediately another request on the /order endpoint, because you will be locked in waiting for the answer of the request just sent.
In the / endpoint, on the other hand, you will receive immediately the HTTP 200 response (as there is no await) and the code for the takeTime function will keep running asynchronously until it's done.
Here you can find more information, I hope it's useful.
Edit 1
here I add the .html page I'm using to test the await loop requests
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Document</title>
</head>
<body></body>
<script>
for (let i = 0; i < 10; i++) {
fetch("http://localhost:3000/order");
}
</script>
</html>
The fetch is not working for me in the browser, but having a web page making those request works
Related
our app need to do around 37000 pages generation with an array generated by getStaticPath
the getStaticProps need to do a request on our api for build each page, but we have currently some timeout request.
Then, i will try to add a little timer in each request (i think about 0.1 second) but i didn't find anywhere a nextjs parameter to do this
Anyone allready do this?
Ok so for your case I think you should use getServerSideProps, you told that you need Dynamic SEO for each page and that is exactly what getServerSideProps id for, the idea is to render specific meta that depend on the data you get from your request, here is an example below:
export const getServerSideProps = async (context) => {
try {
let { data } = await axios.get("YOUR_URL")
return {
props: {
data,
},
}
} catch (e) {
console.log(e)
}
}
And in your component
<Head>
<link rel="canonical" href={canonicalUrl} />
<title>{title}</title>
<meta name="title" content={data.productTitle} />
<meta name="description" content={data.description} />
...etc
</Head>
return res.end() not closing script execution.
Initially it renders the form and when i submit it executes the code block below closing brace of if statement.
I think execution should have ended on the line return res.end().
const http = require("http");
const server = http.createServer((req, res) => {
const url = req.url;
if (url === "/") {
res.write("<html>");
res.write("<head><title>FORM</title></head>");
res.write(
"<body><form action='/message' method='POST'><input type='text' name='message' /><button>Send</button></from></body>"
);
res.write("</html>");
return res.end();
}
res.setHeader("Content-Type", "text/html");
res.write("<html>");
res.write("<head><title>APP</title></head>");
res.write("<body><h1>Demo APP</h1></body>");
res.write("</html>");
});
server.listen(4000);
Please add console.log(req.url) to the start of your request handler. You will find that:
return res.end();
does indeed stop the execution of that particular request by returning from the request handler callback.
But, you are most likely then getting another incoming http request, probably for favicon.ico. In general you should never have code in a request handler that pays zero attention to what resource is being requested unless it's sending back an error like a 404 because there are lots of requests that can potentially be made to your http server for somewhat standard resources sucn as robots.txt, favicon.ico, sitemap.xml and so on and you shouldn't be answering for those requests unless you are sending the appropriate data for those requests.
You might want to put the remainder of the code in ELSE block to ensure it is run only when URL!=="/"
So first of all here is my code
router
.get('/find/:_id',(req,res)=>{
const id = req.params._id;
console.log(id);
if(id !== 'jumbotron.css'){
Doc.findById(id)
.then((doc)=>{
res.render('table/content',{
title: doc.subject,
docs: doc
})
})
.catch((err)=>{
console.log(err)
})
}
});
This is the output
enterthe server running on 3000
Connected successfully to mongodb
5b00373bbea0a253106e0930
jumbotron.css
so whenever I make a GET req using that Id, two items are printed,
first one is id and the second one is 'jumbotron.css',
I have no idea where I am getting jumbotron.css as req.params.id.
This would not let me get into that url.
On your table/content HTML, you probably have:
<link rel="stylesheet" type="text/css" href="jumbotron.css">
Instead of href="/jumbotron.css" or href="/path/to/css/jumbotron.css"
And since it's relative to the current URL it is performing a request to:
/find/jumbotron.css
That's why you're getting a second request with id === 'jumbotron.css'
Im trying to make an API that works through the power of the framework nodejs and the node module/package express
I made an API that once you call localhost:3000/?username=nameHere
It makes some http requests using the request module, waits for the callback and then sends the result out as html to the person requesting localhost:3000/?username=nameHere
However if I refresh it quickly, the ending result would be a stack of 2 results instead of one
Example: localhost:3000/?username=nameHere would return 1 array
if i quickly call/refresh it 2 times, the route acts the same for everyone and the ending result would stack up 2 arrays
How do I overcome this? I'm seriously new to routing.
Here's my code, app is simply express() and i Did define some variables at the top, GetAllItems basically is a function that does a bunch of requests made by the the request module, wraps up all the data and sends it back as a callback
app.use('/', function(req, res,next) {
console.log("request");
if (req.query.username!==undefined) {
GetID(req.query.username,function(ID){
if(ID!==undefined || ID!=="undefined"){
console.log("Cool");
GetAllItems(ID,function(cb){
res.send(JSON.stringify(cb));
next();
});
}else{
console.log("Not found");
res.send("Not Found");
}
})
}else{
res.send("No username");
}});
I am sorry that I can't come up with a better title.
I always have this problem (when coding in node.js also python) but I think my solution is kind dirty.
I am here to seek a better solution for this problem.
Here is the scenario:
Your server is doing a very very heavy task upon a special http request (like generating browser screenshot for an URL/generating game server banner with statistics). Whoever did a HTTP request to your server will get the same response. The response will be cached for a long time.
For example, in the browser screenshot generating HTTP request, your server is expected to spawn a phantomjs, capture the screenshot, save it and cache it for a long time, then respond with the PNG captured. The HTTP request after this should hit the cache.
The pseudo code to scenario:
server.get(":urlname.png", function(req, res, next) {
var cached = cache.get(req.params_urlname);
if (cached) {
res.send(cached);
return;
}
// This will take very long time
generateScreenshot(req.params_urlname, function(pngData) {
cache.set(req.params_urlname, pngData, LONG_TIME);
res.send(cached);
});
});
Here is the problem:
Imagine that you have a screenshot generating URL
(http://yourserver.com/generate-screenshot/google.png). The screenshot
is not generated nor cached yet.
Your posted the URL in a very popular forum, and there are 1000 HTTP requests to the that URL at the same time! It means that your server will have to spawn 1000 phantomjs and all of them together will generate screenshot of google.com at the same time, which is crazy!
In other words, the heavy function should be executed only once for generating cache.
My current code solution to the problem:
var pendingResponse = {};
server.get(":urlname.png", function(req, res, next) {
var cached = cache.get(req.params_urlname);
if (cached) {
res.send(cached);
return;
}
// The screenshot is currently generating for other request. Let's mark this response as pending.
if (req.params_urlname in pendingResponse) {
pendingResponse[req.params_urlname].push(res);
return;
}
// The screenshot needs to be generated now. Let's mark the future response as pending.
req.params_urlname[req.params_urlname] = [];
// This will take very long time
generateScreenshot(req.params_urlname, function(pngData) {
cache.set(req.params_urlname, pngData, LONG_TIME);
res.send(cached);
// Let's respond all the pending responses with the PNG data as well.
for (var i in pendingResponse[req.params_urlname]) {
var pRes = pendingResponse[req.params_urlname][i];
pRes.send(cached);
}
// No longer mark the future responses as pending.
delete pendingResponse[req.params_urlname];
});
});
This solution works. However, I consider this solution dirty, because it not reusable at all. Also, I think it may cause resource leak. Is there any better solution / library?
Here's a proof-of-concept server doing this result caching using a memoizee package (not only removes the necessity to cache computations in progress, but also allows to remove the "cache" altogether):
var express = require('express');
var memoize = require('memoizee');
function longComputation(urlName, cb) {
console.log('called for ' + urlName);
setTimeout(function () {
console.log('done for ' + urlName);
cb();
}, 5000);
}
var memoizedLongComputation = memoize(longComputation, {async: true, maxAge: 20000});
var app = express();
app.get('/hang/:urlname', function (req, res, next) {
memoizedLongComputation(req.params.urlname, function () {
res.send('hang over');
});
});
app.listen(3000);
Here we make the result be cached for 20 seconds.
When I start the server and then run in the shell
for i in `seq 1 10`; do curl http://localhost:3000/hang/url1; done
(or just open several browser tabs and quickly navigate them all to http://localhost:3000/hang/url1), I see one "called for url1" and in 5 s one "done for url1" message in the console, meaning only one "real" longComputation call was made. If I repeat it shortly after (less than in 20 s), there are no additional messages, and results are returned instantaneously, because they are cached. If I repeat the command later (in more than 20 s), there's again one call only.