I have an API endpoint, which executes a function, which in turn listens to some events via a web socket and then throws the response of it back via a callback to the api endpoint, so the endpoint can respond with this object to a http request.
Sometimes I get only one event, sometimes I get two events from the listener. When only the first event appears, I have no problems. Because the second event has priority, I am using setTimeout to wait for it, in case it still appears.
The problem ist, when the second event appears, my API endpoint responds with its object, but afterwards it crashes. I suppose it has something to do with the second callback and I don't know how to handle it. This is how the code is structured:
app.get('/api/myendpoint', function(req, res) {
myVar_1 = req.query.myvar1;
myVar_2 = req.query.myvar2;
myFunction(myVar_1, myVar_2, function(data, error) {
if (!error) res.json(data);
else res.json({"error": String(error)});
});
});
function myFunction(myVar_1, myVar_2, callback){
listenOnSocket.someEvent(myVar_1,
function(error, event) {
if (error) callback(null, error);
else setTimeout(function() {callback(event, null);}, 6000);
}
);
listenOnSocket.someEvent(myVar_2,
function(error, event) {
if (!error) callback(event, null);
else callback(null, error);
}
);
};
This is what my errors look like:
Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent
to the client
Why its Happening?
1st event occurs at which you set a timeout for 6000 ms to call the callback function.
2nd event occurs before 6000 and the callback executes due to event 2. Response is sent here.
timeout completes and the pending function passed to setTimeout executes the callback which would send the response again and its not possible to send headers to a response after its sent.
Solution
A basic solution would be to check a flag variable in the function passed to set timeout that whether the 2nd event has occured or not.
function myFunction(myVar_1, myVar_2, callback){
let executed = false;
listenOnSocket.someEvent(myVar_1,
function(error, event) {
if(!executed){
executed = true
if (error) callback(null, error);
else setTimeout(function() {
callback(event, null);
}, 6000);
}
}
);
listenOnSocket.someEvent(myVar_2,
function(error, event) {
if(!executed){
executed = true;
if (!error) callback(event, null);
else callback(null, error);
}
}
);
};
Related
Basically, it is a web proxy. In a request listener, I create another http request, read its response and pass it to the response. But I need to wait for the other request to end. Please see the function structure below.
I have searched for existing answers, but the existing answers were using await or Promise etc, which I think, do not fit my structure. I think I need something like the ManualResetEvent of C#. Right after sending the request (POS 1), I need to mark the thread to block so that before finishing the response (POS 3) it can be blocked. When the request's response ends (POS 2), I need to mark the thread to continue. How do I do this in TypeScript/NodeJS?
function onRequest(req: http.IncomingMessage, res: http.ServerResponse)
{
....
if(arguments are valid)
{
... prepare options for request
try
{
const remoteReq = https.request(options, (remoteRes) =>
{
remoteRes.on('data', (d) =>
{
... pass it to the response.
});
remoteRes.on('end', (d) =>
{
//POS 2: resetevent.Set() allow thread to proceed
});
});
remoteReq.end();
//POS 1:resetevent.Reset() block thread
}
}
catch
{
}
}
else
{
}
//POS 3: resetevent.WaitOne() wait for the remote response to end.
res.end("");
}
You don't "wait" in nodejs. You register a listener for an event and you finish the request when that listener is called. You move the res.end() and and res.write() into the listener that tells you you're done or you have data. Nodejs is a non-blocking, event driven, asynchronous I/O model. You have to program it that way.
You don't show enough of your real code for us to write something that would actually work, but the general scheme would be like this where you listen for the data, end and error events on the http request you sent and you handle the original request in those event handlers. There is no crying in baseball. There is no "waiting" in nodejs:
function onRequest(req: http.IncomingMessage, res: http.ServerResponse) {
....
if(arguments are valid) {
...prepare options
try {
const remoteReq = https.request(options, (remoteRes) => {
remoteRes.on('data', (d) => {
...pass it to the response.
res.write(...)
});
remoteRes.on('end', (d) => {
res.end(...);
});
});
remoteReq.on('error', err => {
console.log(err);
if (res.headersSent) {
// not much to do other than just hangup
res.end();
} else {
res.statusCode = 500;
res.end();
}
});
remoteReq.end();
} catch (e) {
// highly unlikely anything gets here because this is all
// asynchronous. Instead, you need to listen for the 'error' event.
}
}
else {
// need to send some sort of response here, probably a 400 status
// if arguments are invalid
}
}
I'm having an issue sending the same json object to two separate endpoints. I need to send one copy to a database, and the other copy back to the client. I'm getting can't set headers after they are sent. Which I have gathered is an error saying that res.json() is called once, and can not be called a second time because the headers have been "baked in". I'm pretty newb to development, any suggestions or explanation would be very helpful.
Note: the whole code executes successfully, then crashes the server with the error message above.
paypal.payment.execute(paymentId, execute_payment_json, function (error, payment) {
if (error) {
console.log(error.response);
throw error;
} else {
console.log("Get Payment Response");
console.log(JSON.stringify(payment));
const userData = {paymentID : payment.id};
UserData.addUserData(userData, function(err, userData) {
if (err) {
throw err;
}
res.json(userData);
});
res.json(userData)
}
});
})
You are right when you write that you can't call res.json() a second time. You can only send one response per request. res.json() sends a response, so you can only call it once (you have probably seen this question already).
You don't have to send a response to the database. Only the client that sent the request should receive a response. Calling res.json() will not send anything to the database. In order to store the userData object in the database, you have to call the function that does that. In your case, I assume you are doing that with UserData.addUserData(). This is where the storing happens, not by sending a response to it.
The function you send in as an argument to UserData.addUserData() is most likely a callback that is called AFTER storing the userData object is finished. Basically, UserData.addUserData() will do all the stuff it's supposed to do, and then continue with the function you have written after that. In there you can call res.json(). You can remove the last call to res.json(). It's not needed as you will call it in the callback function after storing in the database is finished.
paypal.payment.execute(paymentId, execute_payment_json, function (error, payment) {
if (error) {
console.log(error.response);
throw error;
} else {
console.log("Get Payment Response");
console.log(JSON.stringify(payment));
const userData = {paymentID : payment.id};
UserData.addUserData(userData, function(err, userData) {
if (err) {
throw err;
}
res.json(userData);
});
}
});
})
I have a DialogFlow V2 node.js webhook.
I have an intent that is called with a webhook action:
const { WebhookClient } = require('dialogflow-fulfillment');
const app = new WebhookClient({request: req, response: res});
function exampleIntent(app) {
app.add("Speak This Out on Google Home!"); // this speaks out fine. no error.
}
Now, if I have an async request which finishes successfully, and I do app.add in the success block like this:
function exampleIntent(app) {
myClient.someAsyncCall(function(result, err) {
app.add("This will not be spoken out"); // no dice :(
}
// app.add("but here it works... so it expects it immediately");
}
... then Dialog Flow does not wait for the speech to be returned. I get the error in the Response object:
"message": "Failed to parse Dialogflow response into AppResponse, exception thrown with message: Empty speech response",
How can I make DialogFlow V2 wait for the Webhook's Async operations to complete instead expecting a speech response immediately?
NOTE: This problem only started happening in V2. In V1, app.ask worked fine at the tail-end of async calls.
exampleIntent is being called by the main mapper of the application like this:
let actionMap = new Map();
actionMap.set("my V2 intent name", exampleIntent);
app.handleRequest(actionMap);
And my async request inside myClient.someAsyncCall is using Promises:
exports.someAsyncCall = function someAsyncCall(callback) {
var apigClient = getAWSClient(); // uses aws-api-gateway-client
apigClient.invokeApi(params, pathTemplate, method, additionalParams, body)
.then(function(result){
var result = result.data;
var message = result['message'];
console.log('SUCCESS: ' + message);
callback(message, null); // this succeeds and calls back fine.
}).catch( function(error){
console.log('ERROR: ' + error);
callback(error, null);
});
};
The reason it worked in V1 is that ask() would actually send the request.
With V2, you can call add() multiple times to send everything to the user in the same reply. So it needs to know when it should send the message. It does this as part of dealing with the response from your handler.
If your handler is synchronous, it sends the reply immediately.
If your handler is asynchronous, however, it assumes that you are returning a Promise and waits till that Promise resolves before sending the reply. So to deal with your async call, you need to return a Promise.
Since your call is using Promises already, then you're in very good shape! The important part is that you also return a Promise and work with it. So something like this might be your async call (which returns a Promise):
exports.someAsyncCall = function someAsyncCall() {
var apigClient = getAWSClient(); // uses aws-api-gateway-client
return apigClient.invokeApi(params, pathTemplate, method, additionalParams, body)
.then(function(result){
var result = result.data;
var message = result['message'];
console.log('SUCCESS: ' + message);
return Promise.resolve( message );
}).catch( function(error){
console.log('ERROR: ' + error);
return Promise.reject( error );
});
};
and then your Intent handler would be something like
function exampleIntent(app) {
return myClient.someAsyncCall()
.then( function( message ){
app.add("You should hear this message": message);
return Promise.resolve();
})
.catch( function( err ){
app.add("Uh oh, something happened.");
return Promise.resolve(); // Don't reject again, or it might not send the reply
})
}
I am trying to pass messages between content script and the extension
Here is what I have in content-script
chrome.runtime.sendMessage({type: "getUrls"}, function(response) {
console.log(response)
});
And in the background script I have
chrome.runtime.onMessage.addListener(
function(request, sender, sendResponse) {
if (request.type == "getUrls"){
getUrls(request, sender, sendResponse)
}
});
function getUrls(request, sender, sendResponse){
var resp = sendResponse;
$.ajax({
url: "http://localhost:3000/urls",
method: 'GET',
success: function(d){
resp({urls: d})
}
});
}
Now if I send the response before the ajax call in the getUrls function, the response is sent successfully, but in the success method of the ajax call when I send the response it doesn't send it, when I go into debugging I can see that the port is null inside the code for sendResponse function.
From the documentation for chrome.runtime.onMessage.addListener:
This function becomes invalid when the event listener returns, unless you return true from the event listener to indicate you wish to send a response asynchronously (this will keep the message channel open to the other end until sendResponse is called).
So you just need to add return true; after the call to getUrls to indicate that you'll call the response function asynchronously.
The accepted answer is correct, I just wanted to add sample code that simplifies this.
The problem is that the API (in my view) is not well designed because it forces us developers to know if a particular message will be handled async or not. If you handle many different messages this becomes an impossible task because you never know if deep down some function a passed-in sendResponse will be called async or not.
Consider this:
chrome.extension.onMessage.addListener(function (request, sender, sendResponseParam) {
if (request.method == "method1") {
handleMethod1(sendResponse);
}
How can I know if deep down handleMethod1 the call will be async or not? How can someone that modifies handleMethod1 knows that it will break a caller by introducing something async?
My solution is this:
chrome.extension.onMessage.addListener(function (request, sender, sendResponseParam) {
var responseStatus = { bCalled: false };
function sendResponse(obj) { //dummy wrapper to deal with exceptions and detect async
try {
sendResponseParam(obj);
} catch (e) {
//error handling
}
responseStatus.bCalled= true;
}
if (request.method == "method1") {
handleMethod1(sendResponse);
}
else if (request.method == "method2") {
handleMethod2(sendResponse);
}
...
if (!responseStatus.bCalled) { //if its set, the call wasn't async, else it is.
return true;
}
});
This automatically handles the return value, regardless of how you choose to handle the message. Note that this assumes that you never forget to call the response function. Also note that chromium could have automated this for us, I don't see why they didn't.
You can use my library https://github.com/lawlietmester/webextension to make this work in both Chrome and FF with Firefox way without callbacks.
Your code will look like:
Browser.runtime.onMessage.addListener( request => new Promise( resolve => {
if( !request || typeof request !== 'object' || request.type !== "getUrls" ) return;
$.ajax({
'url': "http://localhost:3000/urls",
'method': 'GET'
}).then( urls => { resolve({ urls }); });
}) );
I'm combining the async and request modules to make api requests asynchronously and with rate limiting.
Here is my code
var requestApi = function(data){
request(data.url, function (error, response, body) {
console.log(body);
});
};
async.forEachLimit(data, 5, requestApi, function(err){
// do some error handling.
});
Data contains all the urls I make request to. Am limiting the number of concurrent request to 5 using forEachLimit method. This code makes the first 5 request then stops.
In the async docs it says "The iterator is passed a callback which must be called once it has completed". But I don't understand this, what should I be doing to signal that the request has completed?
First, you shall add callback to your iterator function:
var requestApi = function(data, next){
request(data.url, function (error, response, body) {
console.log(body);
next(error);
});
};
next(); or next(null); tells Async that all processing is done. next(error); indicates an error (if error not null).
After processing all requests Async calls its callback function with err == null:
async.forEachLimit(data, 5, requestApi, function(err){
// err contains the first error or null
if (err) throw err;
console.log('All requests processed!');
});
Async calls its callback immediately after receiving the first error or after all requests completed succesfully.