I have a really simple file upload example
var http = require('http');
http.createServer(function(req, res){
console.log(req.method);
if (req.method == 'GET'){
res.writeHead(200);
res.write('<html><head></head><body><form method="POST" enctype="multipart/form-data"><input type="file" id="f" name="f"><input type="submit"></body></html>')
res.end();
}else if(req.method == 'POST'){
req.pause();
res.writeHead(200);
res.end();
}else{
res.writeHead(404).end();
}
}).listen('8081');
What I want to do is pause the upload. While it works fine with large files, the small ones (<= 100kB), that are probably sent along with the request in a single part are not paused (that's fine and understandable), but instead the next request to the server is paused (i.e. when I try to load the page, it never reaches the console.log(req.method) part, but when I refresh again it's back to normal), which is definitely not fine.
It seems like an error that would pop out once in a while, so I was suprised when I didn't find any complaints about it. What are your thoughts/possible explanation/workaround/fix suggestions? For now I check whether the file's size is below a certain threshold, but it doesn't look very safe or ellegant.
I am of the opinion (could be wrong) that req.pause causes the underlying socket to be paused. A way to verify this claim would be to add the header Connection: Close in the res.writeHead(200); which is just after the line req.pause();.
Now, if everyting start working fine (next request is not paused), you can say that the underlying socket was paused and was being reused for the next request.
Related
I'm running a small server that needs to receive webforms. The server checks the request and sends back "success" or "fail" which is then displayed on the form (client screen).
Now, checking the form may take a few seconds, so the user may be tempted to send the form again.
What is the corret way to ignore the second request?
So far I have come out with this solutions: If the form is duplicate of the previous one
Don't check and send some server error back (like 429, or 102, or some other one)
Close directly the connection req.destroy();res.destroy();
Ignore the request and exit from the requestListener function.
With solution 1 and 2 the form (on client's browser) displays a message error (even if the first request they sent was correct, so as the duplicates). So it's not a good one.
Solution 3 gives the desired outcome... but I'm not sure if it is the right way around it... basically not changing req and res instead of destroying them. Could this cause issues, or slow down the server? (like... do they stack up?). Of course the first request, once it has been checked, will be sent back with the outcome code. My concern is with the duplicate requests, which I don't destroy nor answer...
Some details on the setup: Nodejs application using the very default code by the http module.
const http = require("http");
const requestListener = function (req, res) {
var requestBody = '';
req.on('data', (data)=>{
requestBody += data;
});
req.on('end', ()=>{
if (isduplicate(requestBody))
return;
else
evalRequest(requestBody, res);
})
}
return res.end() not closing script execution.
Initially it renders the form and when i submit it executes the code block below closing brace of if statement.
I think execution should have ended on the line return res.end().
const http = require("http");
const server = http.createServer((req, res) => {
const url = req.url;
if (url === "/") {
res.write("<html>");
res.write("<head><title>FORM</title></head>");
res.write(
"<body><form action='/message' method='POST'><input type='text' name='message' /><button>Send</button></from></body>"
);
res.write("</html>");
return res.end();
}
res.setHeader("Content-Type", "text/html");
res.write("<html>");
res.write("<head><title>APP</title></head>");
res.write("<body><h1>Demo APP</h1></body>");
res.write("</html>");
});
server.listen(4000);
Please add console.log(req.url) to the start of your request handler. You will find that:
return res.end();
does indeed stop the execution of that particular request by returning from the request handler callback.
But, you are most likely then getting another incoming http request, probably for favicon.ico. In general you should never have code in a request handler that pays zero attention to what resource is being requested unless it's sending back an error like a 404 because there are lots of requests that can potentially be made to your http server for somewhat standard resources sucn as robots.txt, favicon.ico, sitemap.xml and so on and you shouldn't be answering for those requests unless you are sending the appropriate data for those requests.
You might want to put the remainder of the code in ELSE block to ensure it is run only when URL!=="/"
I have a page with file uploading/downloading functionality.
When I try to download a file AND cancel the save file prompt, which happens after the res.writeHead part, it leaves the headers and it waits for the res.write and res.end parts.
The problem is that these are escaped if the prompt is cancelled, making every other response fail with the error "Can't set headers after they are sent".
Is there anyway to end the response catching the cancelled prompt event is some way, or any other way to avoid this?
The part that sets headers and streams data for the file download (located in a function that is called in the /download/:filename route) is :
res.writeHead(200, {'Content-Type': 'application/octet-stream'});
var readstream = gfs.createReadStream({
filename: files[0].filename
});
readstream.on('data', function(data){
res.write(data);
});
readstream.on('end', function(){
res.end();
});
If this sequence is not completed, every other response fails.
example:
res.status(403).send('You have no access to this file');
in another controller, called in the same page.
(I guess if I redirected to another page headers would actually get cleared?)
*If I select download location and press ok, no error occurs
*I am not having a double response in a loop, to avoid this common mistake answer :)
in express you can check for res.headersSent, this way you would be able to avoid the exception
if(res.headersSent){
return;
} else {
//set you headers
}
I am sorry that I can't come up with a better title.
I always have this problem (when coding in node.js also python) but I think my solution is kind dirty.
I am here to seek a better solution for this problem.
Here is the scenario:
Your server is doing a very very heavy task upon a special http request (like generating browser screenshot for an URL/generating game server banner with statistics). Whoever did a HTTP request to your server will get the same response. The response will be cached for a long time.
For example, in the browser screenshot generating HTTP request, your server is expected to spawn a phantomjs, capture the screenshot, save it and cache it for a long time, then respond with the PNG captured. The HTTP request after this should hit the cache.
The pseudo code to scenario:
server.get(":urlname.png", function(req, res, next) {
var cached = cache.get(req.params_urlname);
if (cached) {
res.send(cached);
return;
}
// This will take very long time
generateScreenshot(req.params_urlname, function(pngData) {
cache.set(req.params_urlname, pngData, LONG_TIME);
res.send(cached);
});
});
Here is the problem:
Imagine that you have a screenshot generating URL
(http://yourserver.com/generate-screenshot/google.png). The screenshot
is not generated nor cached yet.
Your posted the URL in a very popular forum, and there are 1000 HTTP requests to the that URL at the same time! It means that your server will have to spawn 1000 phantomjs and all of them together will generate screenshot of google.com at the same time, which is crazy!
In other words, the heavy function should be executed only once for generating cache.
My current code solution to the problem:
var pendingResponse = {};
server.get(":urlname.png", function(req, res, next) {
var cached = cache.get(req.params_urlname);
if (cached) {
res.send(cached);
return;
}
// The screenshot is currently generating for other request. Let's mark this response as pending.
if (req.params_urlname in pendingResponse) {
pendingResponse[req.params_urlname].push(res);
return;
}
// The screenshot needs to be generated now. Let's mark the future response as pending.
req.params_urlname[req.params_urlname] = [];
// This will take very long time
generateScreenshot(req.params_urlname, function(pngData) {
cache.set(req.params_urlname, pngData, LONG_TIME);
res.send(cached);
// Let's respond all the pending responses with the PNG data as well.
for (var i in pendingResponse[req.params_urlname]) {
var pRes = pendingResponse[req.params_urlname][i];
pRes.send(cached);
}
// No longer mark the future responses as pending.
delete pendingResponse[req.params_urlname];
});
});
This solution works. However, I consider this solution dirty, because it not reusable at all. Also, I think it may cause resource leak. Is there any better solution / library?
Here's a proof-of-concept server doing this result caching using a memoizee package (not only removes the necessity to cache computations in progress, but also allows to remove the "cache" altogether):
var express = require('express');
var memoize = require('memoizee');
function longComputation(urlName, cb) {
console.log('called for ' + urlName);
setTimeout(function () {
console.log('done for ' + urlName);
cb();
}, 5000);
}
var memoizedLongComputation = memoize(longComputation, {async: true, maxAge: 20000});
var app = express();
app.get('/hang/:urlname', function (req, res, next) {
memoizedLongComputation(req.params.urlname, function () {
res.send('hang over');
});
});
app.listen(3000);
Here we make the result be cached for 20 seconds.
When I start the server and then run in the shell
for i in `seq 1 10`; do curl http://localhost:3000/hang/url1; done
(or just open several browser tabs and quickly navigate them all to http://localhost:3000/hang/url1), I see one "called for url1" and in 5 s one "done for url1" message in the console, meaning only one "real" longComputation call was made. If I repeat it shortly after (less than in 20 s), there are no additional messages, and results are returned instantaneously, because they are cached. If I repeat the command later (in more than 20 s), there's again one call only.
I'm really new to node.js so please bear with me if I'm making a obvious mistake.
To understand node.js, i'm trying to create a webserver that basically:
1) update the page with appending "hello world" everytime the root url (localhost:8000/) is hit.
2) user can go to another url (localhost:8000/getChatData) and it will display all the data built up from the url (localhost:8000/) being triggered
Problem I'm experiencing:
1) I'm having issue with displaying that data on the rendered page. I have a timer that should call get_data() ever second and update the screen with the data variable that stores the appended output. Specifically this line below response.simpleText(200, data); isn't working correctly.
The file
// Load the node-router library by creationix
var server = require('C:\\Personal\\ChatPrototype\\node\\node-router').getServer();
var data = null;
// Configure our HTTP server to respond with Hello World the root request
server.get("/", function (request, response) {
if(data != null)
{
data = data + "hello world\n";
}
else
{
data = "hellow world\n";
}
response.writeHead(200, {'Content-Type': 'text/plain'});
console.log(data);
response.simpleText(200, data);
response.end();
});
// Configure our HTTP server to respond with Hello World the root request
server.get("/getChatData", function (request, response) {
setInterval( function() { get_data(response); }, 1000 );
});
function get_data(response)
{
if(data != null)
{
response.writeHead(200, {'Content-Type': 'text/plain'});
response.simpleText(200, data);
console.log("data:" + data);
response.end();
}
else
{
console.log("no data");
}
}
// Listen on port 8080 on localhost
server.listen(8000, "localhost");
If there is a better way to do this, please let me know. The goal is to basically have a way for a server to call a url to update a variable and have another html page to report/display the updated data dynamically every second.
Thanks,
D
The client server model works by a client sending a request to the server and the server in return sends a response. The server can not send a response to the client that the client hasn't asked for. The client initiates the request. Therefore you cannot have the server changing the response object on an interval.
The client will not get these changes to the requests. How something like this is usually handled as through AJAX the initial response from the server sends Javascript code to the client that initiates requests to the server on an interval.
setTimeout accepts function without parameter which is obvious as it will be executed later in time. All values you need in that function should be available at the point of time. In you case, the response object that you are trying to pass, is a local instance which has scope only inside the server.get's callback (where you set the setTimeout).
There are several ways you can resolve this issue. you can keep a copy of the response instance in the outer scope where get_data belongs or you can move the get_data entirely inside and remove setTimeout. The first solution is not recommended as if getChatData is called several times in 1sec the last copy will be prevailing.
But my suggestion would be to keep the data in database and show it once getChatData is called.