In my index.js, I have an exports function that is supposed to send data back to the client via ajax on pressing a submit button. However, when the user presses submit, the data seems to get sent over before it the data gets modified. When pressing submit one more time, it sends the data that was previously modified as if clicking the submit button only sends the 'previously' set data. This is my code:
var tabledata = getRecordFromDatabase(key);
if(tabledata.length === 0)
tabledata = 'There is no matched record in the database';
res.contentType('text/html');
res.send({'matched':tabledata});
So to illustrate the error: I click submit after filling out a form and receive back the message "There is no matched record in the database". I hit submit a second time without changing anything in the form I just filled. This time record data is actually sent to me. Why could this be?
If whatever you're doing in getRecordFromDatabase is asynchronous and non-blocking, then node.js is behaving as it should. Node.js is non-blocking - it doesn't stop and wait for processes to complete (unless those processes are intentionally written to block, which is usually avoided in node.js). This is beneficial, because it keeps the server free to accept new requests and process many requests at once.
If your database call is asynchronous, you're not waiting for it to return before you res.send(). That's why your first submit returns back empty. Most likely, by the time you hit submit a second time, your DB call has finally returned, and that's why you get a result.
It's hard to give you a code-based answer to your problem, because you abstracted away what is happening in your DB call method. But typically, an asynchronous call would go something like:
getRecordFromDatabase(key, function(err, data){
if(data.length === 0)
data = 'There is no matched record in the database';
res.contentType('text/html');
res.send({'matched':data});
});
This way, you are passing a function to execute as a callback to your asynchronous method - when the async call completes, it executes the callback, which then executes the res.send() with the appropriate data.
Related
I am new to node.js and working through the API. In the stream module docs I came across this example of the "unpipe event" (actually a fusion of two examples in the docs).
const fs = require("fs);
const writable = fs.createWriteStream("write.txt");
const readable = fs.createReadStream("read.txt");
readable.pipe(writable);
setTimeout(function(){
console.log("Stop writing to file.txt");
readable.unpipe(writable);
console.log("Manually close the file stream");
writable.end();
}, 0);
writable.on("unpipe", function(src){
console.log("Something has stopped piping into the writer");
});
I can't understand the following console.log order:
"Stop writing to file.txt"
"Something has stopped piping into the writer"
"Manually close the file stream"
Given the setTimeout callback is running - which is the first phase of the event loop as I understand - how on earth does the callback for the "unpipe" event start to run before the setTimeout callback has finished.
Originally I had the setTimeout firing after a time above zero seconds, however I was finding that the unpipe call back was always called first. I reasoned that my computer was reading the file always first before the setTimeout was ready. (Although I can't see any mention in the docs about the completion of the write to the file eliciting the "unpipe" event, but this makes sense I suppose). However I can't for the life of me reason how the above program flow is occurring. Thanks in advance for any help.
As specified by the node.js documentation:
The EventEmitter calls all listeners synchronously in the order in which they were registered.
That is, when .emit is called, it synchronously runs through all listeners for the emitted event and calls them.
Note that if necessary you can wrap your callback code in process.nextTick to ensure that it will always run asynchronously, but in your case it's likely that's unnecessary.
Also the source of the call to .emit (the emission of the event) will often be asynchronous.
I have this problem only when I try refresh the page and I can not solve it, I tried everything but still happens the same. It began to happen when I add socket.io at the project. The project run in several servers which are connected one each other throught sockets.
TEST CASES: When I render the page, at the first time everything goes well but, if I refresh the same page, I get this error:
ERROR: "Error: Can't set headers after they are sent. at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:344:11)"
ATTENTION: when get in IF() and send "return res.end('The Activation Code is INVALID!');" it DOESN'T HAPPEND! I refresh it and refresh it and everything goes well. My problem is in the RENDER.
MY CODE BELOW:
activationUser = function(req,res,next){
var data = {
activationCode : req.params.activationCode,
now : new Date().valueOf(),
ip : req.connection.remoteAddress,
fId : frontalId
}
socketCore.emit('activationUser', data);
socketCore.on(frontalId + 'activationUserResp', function(data){
if(data.msg == "CHECKED!"){
next();
}else{
return res.end(data.msg);
}
});
}
router.get('/activationUser/:activationCode',activationUser,function(req,res){
var data = {
activationCode : req.params.activationCode,
fId : frontalId
}
socketCore.emit('step2', data);
socketCore.on(frontalId + 'step2Resp', function(data){
if(data.msg == 'err'){
return res.end('The Activation Code is INVALID!');
}else{
return res.render('registro2', {title: 'title | '+ data.name + ' ' + data.lastname, user:data});
}
});
});
Thank you!
The particular error you are getting happens when you try to send anything on the res object after the complete response has already been sent. This often occurs because of errors in asynchronous logic. In your particular case, it apepars to be because you are assigning a new event handler with socketCore.on() every single time the router is hit. Those event handlers will accumulate and after the first time the route is hit, they will execute multiple times triggering the sending of multiple responses on the same response object, thus trigger that error.
The main ways to fix your particular problem are:
Use .once() instead of .on() so the event handler automatically removes itself after being triggered.
Manually remove the .on() event handler after you get the response.
Move the event handler outside of the route so it's only ever installed once.
In your particular case, since socketCore is a shared object available to all requests, it appears that you also have a race condition. If multiple users trigger the '/activationUser/:activationCode' route in the same general time frame, then you will register two event handlers with socketCore.on() (one for each route that is hit) and you will do two socketCore.emit('step2', data);. But, you have no way of associating which response belongs with which request and the two responses could easily get mixed up - going to the wrong request.
This highlights how socket.io connections are not request/response. They are message/answer, but unless you manually code a correspondence between a specific message request and a specific answer, there is no way to correlate which goes with which. So, without assigning some particular responseID that lets you know which response belongs to which message, you can't use a socket.io connection like this in a multi-user environment. It will just cause race conditions. It's actually simpler to use an HTTP request/response for this type of data fetching because each response goes only with the request that made it in the HTTP architecture.
You can change your architecture for making the socketCore request, but you will have to manually assign an ID to each request and make sure the server is sending back that ID with the response that belongs to that request. Then, you can write a few lines of code on the receiving side of things that will make sure the right response gets fed to the code with the matching request.
I am trying to stream some audio to my server and then stream it to a service specified by the user, the user will be providing me with someHostName, which can sometimes not support that type of request.
My problem is that when it happens the clientRequest.on('end',..) is never fired, I think it's because it's being piped to someHostReq which gets messed up when someHostName is "wrong".
My question is:
Is there anyway that I can still have clientRequest.on('end',..) fired even when the stream clientRequest pipes to has something wrong with it?
If not: how do I detect that something wrong happened with someHostReq "immediately"? someHostReq.on('error') doesn't fire up except after some time.
code:
someHostName = 'somexample.com'
function checkIfPaused(request){//every 1 second check .isPaused
console.log(request.isPaused()+'>>>>');
setTimeout(function(){checkIfPaused(request)},1000);
}
router.post('/', function (clientRequest, clientResponse) {
clientRequest.on('data', function (chunk) {
console.log('pushing data');
});
clientRequest.on('end', function () {//when done streaming audio
console.log('im at the end');
}); //end clientRequest.on('end',)
options = {
hostname: someHostName, method: 'POST', headers: {'Transfer-Encoding': 'chunked'}
};
var someHostReq = http.request(options, function(res){
var data = ''
someHostReq.on('data',function(chunk){data+=chunk;});
someHostReq.on('end',function(){
console.log('someHostReq.end is called');
});
});
clientRequest.pipe(someHostReq);
checkIfPaused(clientRequest);
});
output:
in the case of a correct hostname:
pushing data
.
.
pushing data
false>>>
pushing data
.
.
pushing data
pushing data
false>>>
pushing data
.
.
pushing data
console.log('im at the end');
true>>>
//continues to be true, that's fine
in the case of a wrong host name:
pushing data
.
.
pushing data
false>>>>
pushing data
.
.
pushing data
pushing data
false>>>>
pushing data
.
.
pushing data
true>>>>
true>>>>
true>>>>
//it stays true and clientRequest.on('end') is never called
//even tho the client is still streaming data, no more "pushing data" appears
if you think my question is a duplicate:
it's not the same as this: node.js http.request event flow - where did my END event go? , the OP was just making a GET instead of a POST
it's not the same as this: My http.createserver in node.js doesn't work? , the stream was in paused mode because the none of the following happened:
You can switch to flowing mode by doing any of the following:
Adding a 'data' event handler to listen for data.
Calling the resume() method to explicitly open the flow.
Calling the pipe() method to send the data to a Writable.
source: https://nodejs.org/api/stream.html#stream_class_stream_readable
it's not the same as this: Node.js response from http request not calling 'end' event without including 'data' event , he just forgot to add the .on('data',..)
The behaviour in case of a wrong host name seems some problem with buffers, if the destination stream buffer is full (because someHost is not getting the sended chunks of data) the pipe will not continue to read the origin stream because pipe automatically manage the flow. As pipe is not reading the origin stream you never reach 'end' event.
Is there anyway that I can still have clientRequest.on('end',..) fired
even when the stream clientRequest pipes to has something wrong with
it?
The 'end' event will not fire unless the data is completely consumed. To get 'end' fired with a paused stream you need to call resume() (unpiping first from wrong hostname or you will fall in buffer stuck again) to set the steam into flowMode again or read() to the end.
But how to detect when I should do any of the above?
someHostReq.on('error') is the natural place but if it takes too long to fire up:
First try to set a low timeout request (less than someHostReq.on('error') takes to trigger, as seems too much time for you) request.setTimeout(timeout[, callback]) and check if it doesn't fail when correct hostname. If works, just use the callback or timeout event to detect when the server timeOut and use one of the techniques above to reach to the end.
If timeOut solution fails or doesn't fits your requirements you have to play with flags in clientRequest.on('data'), clientRequest.on('end') and/or clienteRequest.isPaused to guess when you are stuck by the buffer. When you think you are stuck just apply one of the techniques above to reach to the end of the stream. Luckily it takes less time to detect buffer stuck than wait for someHostReq.on('error') (maybe two request.isPaused() = true without reach 'data' event is enought to determine if you are stuck).
How do I detect that something wrong happened with someHostReq
"immediately"? someHostReq.on('error') doesn't fire up except after
some time.
Errors triggers when triggers. You can not "immediately" detect it. ¿Why not just send a prove beacon request to check support before piping streams? Some kind of:
"Cheking service specified by the user..." If OK -> Pipe user request stream to service OR FAIL -> Notify user about wrong service.
I just started developing nodejs. I'm confused to use async model. I believe there is a way to turn most of SYNC use cases into ASYNC way. Example, by SYNC, we load some data and wait until it returns then show them to user; by ASYNC, we load data and return, just tell the user data will be presented later. I can understand why ASYNC is used in this scenario.
But here I have a use case. I'm building an web app, allowing user to place a order (buying something). Before saving the order data into db, I want to put some user data together with order data (I'm using document NoSql db by the way). So I think by SYNC, after I get order data, I make a SYNC call to database and wait for its returned user data. After I get returned data, integrate them together and ingest into db.
I think there might be an issue if I make ASYNC call to db to query user data because user data may be returned after I save data to db. And that's not what I want.
So in this case, how can I do this thing ASYNCHRONOUSLY?
Couple of things here. First, if your application already has the user data (the user is already logged in), then this information should be stored in session so you don't have to access the DB. If you are allowing the user to register at the time of purchase, you would simply want to pass a callback function that handles saving the order into your call that saves the user data. Without knowing specifically what your code looks like, something like this is what you would be looking for.
function saveOrder(userData, orderData, callback) {
// save the user data to the DB
db.save(userData, function(rec) {
// if you need to add the user ID or something to the order...
orderData.userId = rec.id; // this would be dependent on your DB of choice
// save the order data to the DB
db.save(orderData, callback);
});
}
Sync code goes something like this. step by step - one after other. There can be ifs and loops (for) etc. all of us get it.
fetchUserDataFromDB();
integrateOrderDataAndUserData();
updateOrderData();
Think of async programming with nodejs as event driven. Like UI programming - code (function) is executed when an event occurs. E.g. On click event - framework calls back registered clickHandler.
nodejs async programming can also be thought on these lines. When db query (async) execution completes, your callback is called. When order data is updated, your callback is called. The above code goes something like this:
function nodejsOrderHandler(req,res)
{
var orderData;
db.queryAsync(..., onqueryasync);
function onqueryasync(userdata)
{
// integrate user data with order data
db.update(updateParams, onorderudpate);
}
function onorderupdate(e, r)
{
// handler error
write response.
}
}
javascript closure provides the way to keep state in variables across functions.
There is certainly much more to async programming and there are helper modules that help with basic constructs like chain, parallel, join etc as you write more involved async code. but this probably gives you a quick idea.
In my app (node / express / redis), I use some code to update several items in DB at the same time:
app.put('myaction', function(req, res){
// delete stuff
db.del("key1");
db.srem("set1", "test");
// Add stuff
db.sadd("set2", "test2");
db.sadd("set3", "test3");
db.hmset("hash1", "k11", "v11", "k21", "v21");
db.hmset("hash2", "k12", "v12", "k22", "v22");
// ...
// Send response back
res.writeHead(200, {'content-type': 'application/json'});
res.write(JSON.stringify({ "status" : "ok" }));
res.end();
});
Can I be sure ALL those actions will be performed before the method returns ? My concern is the asynchronous processing. As I do not use callback function in the db actions, will this be alright ?
While all of the commands are sent and responses parsed asynchronously, it's useful to note that the callbacks are all invoked in order. So you can use the callback of the last Redis command to send the response to the client, and then you'll know that all of the Redis commands have been executed before responding.
Use the MULTI/EXEC command to create a queue of your commands and execute them in a row. Then use a callback to send a coherent response back (success/failure).
Note that you must use Redis' AOF to avoid that - in case of crash - the db state is not coherent with your logic because only a part of the commands in the queue were executed: i.e. MULTI/EXEC is not transactional upon execution. This is a useful reference.
I haven't worked with redis, but if this works(if you it doesn't call undefined function) and it should be asynchronous, then you can use it. But if there is an error in updating, then you can't handle it, this way.
No, you can't be sure if all those actions complete successfully, because your redis server might crash.. To speed things up, you can group all your update commands into one with pipelining (does your redis driver support that?), then get the success or failure of the whole operation via a callback and proceed..