I'm experimenting with the translation service on the Microsoft bot framework. I've written a method to which I pass a callback function which receives my translated text.
I've got an existing bot that calls an HTTP endpoint to create my output in English. I want to translate the output to the different language before returning it to the user. My unaltered code looks like this:
await request.post(ENDPOINT,
{
headers: HEADERS,
json: BODY
},
async function (error, response, body) {
if (response.statusCode == 202) {
var msg = body.mainResponse.text;
context.sendActivity(msg);
}
});
This runs just fine. Data passed in the HTTP response body gets parsed sent back to the user.
Now I want to plug in my translation service. I've got a single function that I call to do this called Translator.translate(text, callback). I've added this call to my existing function to get:
await request.post(ENDPOINT,
{
headers: HEADERS,
json: BODY
},
async function (error, response, body) {
if (response.statusCode == 202) {
var msg = body.mainResponse.text;
await Translator.translate(msg, function (output) {
context.sendActivity(output);
});
}
}
);
My translation process runs and I get the translation in the output variable, but nothing gets sent back to the user. Looking at the terminal, I see the error "Cannot perform 'get' on a proxy that has been revoked" relating to the context.sendActivity line in my callback.
Can anyone suggest how I keep the context object active?
Thanks in advance.
Many thanks for the assistance everyone - I never completely got to the bottom of this, but I finally fixed it with a complete re-write of the code. I think the problem was caused by a large number of nested synchronous and asynchronous calls. My ultimate solution was to completely get rid of all the nesting - first calling the translation service (and waiting for it), then doing the original call.
I think there are a number of other asynchronous threads inside the methods of both pieces of functionality. I don't have a great understanding of how this works in node, but I'm guessing that the response was getting popped off the stack at the wrong point, which is why I wasn't seeing it. The "cannot perform get" error was a bit of a red herring, it turns out. I get the same error from some of Microsoft's working demo code. I'm sure there's a separate issue there that ought to be fixed, but it wasn't actually caused by this issue. The code was running, but the output was getting lost.
Related
I'm using Cloud Functions for Firebase to:
Receive parameters from api.ai
Make a call to a third-party API and
Respond back to api.ai.
My call to the third-party API uses the request Node.js module and is wrapped within a function (getInfoFromApi()) in index.js.
The problem I'm having is that the execution of the secondary function call is consistently taking between 15-20 seconds. Note: The cloud function itself completes its execution consistently in the 400 ms range.
By logging simple comments to the console I can see when the function starts, when the secondary function is being called and when it receives a response from the third party, so I think I can see what's happening.
Roughly, the timings look like this:
0: cloud function initialises
400 ms: cloud function completes
16 s: getInfoFromApi() function is called (!)
17 s: third-party API returns results
My questions:
Is there an obvious reason for the delay in calling the secondary function? This doesn't seem to be caused by the cold start issue since the cloud function springs to life quickly and the delay is consistent even after repeated calls.
Is the use of the 'request' node module causing the issue? Is there a better module for creating/managing http requests from cloud functions?
You can see a simplified Gist of the index.js here: https://gist.github.com/anonymous/7e00420cf2623b33b80d88880be04f65
Here is a grab of the Firebase console showing example timings. Note: the output is slightly different from the above code as I simplified the above code to help understanding.
The getInfoFrom3rdParty() call is an asynchronous event. However, you haven't returned a promise from your function, so Functions is not waiting for the async event to complete.
It looks to me like, since you are returning undefined, the Function also assumes that it failed and retries. At some point in the retry process, the async event is probably completing before the function exits (i.e. unintentional success due to race conditions). I've seen similar outcomes in other cases where users don't return a promise or value in their functions.
I can't tell from the gist what you're trying to do--it doesn't appear to actually do anything with the third party results and probably isn't a realistic mcve of your use case. But something like this is probably what you want:
exports.getInfo = functions.https.onRequest((request, response) => {
// ....
// NOTE THE RETURN; MOST IMPORTANT PART OF THIS SAMPLE
return getInfoFromThirdParty(...).then(() => {
response.writeHead(200, {"Content-Type": "application/json"});
response.end(JSON.stringify(payload));
}).catch(e => /* write error to response */);
});
function getInfoFrom3rdParty(food) {
reqObj.body = '{"query": "'+food+'"}';
return new Promise((resolve, reject) => {
mainRequest(reqObj, function (error, response, body) {
// ....
if( error ) reject(error);
else resolve(...);
// ....
});
});
}
From my experience with cloud functions, once the "execution finish" flag completes, it will then cause a delay (from 3s up to 15s). This blocks the remaining execution and increases your total response time.
To overcome this, you could try placing a Promise to your 3rd party call, once it completes, then do the "response.send()" which ends the function.
I dont know how to ask my question correctly, but for example I have some structure like this
get_data:function(){
this.unblock();
request("example.com", Meteor.bindEnvironment(function(error, response, body) {
if (!error && response.statusCode == 200) {
$ = Cheerio.load(body);// get HTML of example.com
$(".someclass").each(function() {
if (!somedata_doesnt_exist_in_Mongo) {
request(nexturl, Meteor.bindEnvironment(function(error, response, body)
//... logic
}));
}
});
}
}))
}
Main idea is that I get data from many sites like agregator and have a lot of methods like this. And it'a a lot of time. So I have 2 questions
1 - for Meteor guys. When I use this.unblock() this ensures that my method will work without taking time with customers, like work in other thread ?
2 - How can I optimaze code stucture like above ?
Sorry if it's not in StackOverflow format but
I am waiting for any help !
this.unblock is relevant only to each client individually. It
allows subsequent method calls from client A to run without
having the previous method calls from that client A to finish.
It is like working in a new thread asynchronously in the sense that
the previous method calls are not blocking for client A for this
function using this.unblock. If you have client B, his/her
method invocation wouldn't be blocking A's regardless of whether
you use this.unblock.
I recommend using this.unblock whenever you are sure subsequent method calls will not rely on the result of the function you use this.unblock in. Sending out emails is the most common example. Subsequent method calls will not need the emails to finish sending before doing its job. For your example, I think it should be good to use this.unblock, but of course it depends on what you plan to do with the results following the execution of code after this.unblock.
I'm developing a website in NodeJS with Mongo. Part of the website has url localhost/api/ and returns some JSON, it works fine for fetching from clientside. Now I want to work with these data from server (to prerender it). Basically, I have a function which should return the result from the API part. It looks like this:
request('http://localhost:8000/api/', function (error, response, body) {
if (!error && response.statusCode == 200) {
return body // return the JSON array from API which works OK
}
})
Unfortunately, it doesn't work and only returns "500: TypeError: Cannot call method 'map' of undefined at App". The code simply doesn't have the value at the moment it renders, so I would ideally like to make the function somehow synchronous as I'm used to from other languages.
If I return the JSON array I need directly from the function (without asking for it from request module), it works. Therefore, I know that the problem comes from my wrong usage of asynschronous programming. What would you recommend as a solution? (I could also ask the mongo directly, not via request, but that's not the problem now - I tried and it was the same).
Just in case someone was interested.. I have solved this problem quickly with Promises. The function which waits for the result of request expects a Promise, which is made in the request call.
3 years ago I could do multiple res.send in express.js.
even write a setTimeout to show up a live output.
response.send('<script class="jsbin" src="http://code.jquery.com/jquery-1.7.1.min.js"></script>');
response.send('<html><body><input id="text_box" /><button>submit</button></body></html>');
var initJs = function() {
$('.button').click(function() {
$.post('/input', { input: $('#text_box').val() }, function() { alert('has send');});
});
}
response.send('<script>' + initJs + '</script>');
Now it will throw:
Error: Can't set headers after they are sent
I know nodejs and express have updated. Why can't do that now? Any other idea?
Found the solution but res.write is not in api reference http://expressjs.com/4x/api.html
Maybe you need: response.write
response.write("foo");
response.write("bar");
//...
response.end()
res.send implicitly calls res.write followed by res.end. If you call res.send multiple times, it will work the first time. However, since the first res.send call ends the response, you cannot add anything to the response.
response.send sends an entire HTTP response to the client, including headers and content, which is why you are unable to call it multiple times. In fact, it even ends the response, so there is no need to call response.end explicitly when using response.send.
It appears to me that you are attempting to use send like a buffer: writing to it with the intention to flush later. This is not how the method works, however; you need to build up your response in code and then make a single send call.
Unfortunately, I cannot speak to why or when this change was made, but I know that it has been like this at least since Express 3.
res.write immediately sends bytes to the client
I just wanted to make this point about res.write clearer.
It does not build up the reply and wait for res.end(). It just sends right away.
This means that the first time you call it, it will send the HTTP reply headers including the status in order to have a meaningful response. So if you want to set a status or custom header, you have to do it before that first call, much like with send().
Note that write() is not what you usually want to do in a simple web application. The browser getting the reply little by little increases the complexity of things, so you will only want to do it it if it is really needed.
Use res.locals to build the reply across middleware
This was my original use case, and res.locals fits well. I can just store data in an Array there, and then on the very last middleware join them up and do a final send to send everything at once, something like:
async (err, req, res, next) => {
res.locals.msg = ['Custom handler']
next(err)
},
async (err, req, res, next) => {
res.locals.msg.push('Custom handler 2')
res.status(500).send(res.locals.msg.join('\n'))
}
I've been using Google Translate API for a while now, without any problems.
I recently pushed my app to my new server and even if it has been working perfectly on my local server, the same source code always gives me the "Required parameter: q" as error message.
I'm using NodeJS + ExpressJS + Request to send this request. Here's my test case:
var request = require('request');
request.post({
url: "https://www.googleapis.com/language/translate/v2",
headers: {"X-HTTP-Method-Override": "GET"},
form: {
key: /* My Google API server key */,
target: "en",
q: ["Mon premier essai", "Mon second essai"]
}
}, function(error, response, data) {
if (!error && response.statusCode == 200) {
console.log("everything works fine");
} else {
console.log("something went wrong")
}
});
Running on my local machine gives me "everything works fine", and running it on my server gives me "something went wrong". Digging more into it, I get the error message mentioned above.
As you can see, I'm trying to translate in one request two sentences. It's just a test case, but I really need to use this through POST request instead of doing two GET request.
I have no idea what this is happening, and I double checked my Google settings and I can't find something wrong there.
Also, I'm having no problem using Google Places APi with this same api key on my server.
I'm stuck. Anyone has any idea what's wrong here?
Well I finally found what was wrong: the new version of RequestJS doesn't work as the old one and my server was running 2.16 when my local machine was running 2.14.
The difference is the way the array is sent. I debugged and the old version was sending
key=my_api_key&target=en&q=Mon%20premier%20essai&q=Mon%20second%20essai
When the new version is sending
key=my_api_key&target=en&q[0]=Mon%20premier%20essai&q[1]=Mon%20second%20essai
So I just added 2.14.x instead of 2.x in my package.json file for now, hopefully it will get fixed soon — or maybe it's not a bug? I don't know.
This answer is a little late but to help people out there with this problem. The problem comes from the way the querystring module converts array parameters:
https://github.com/visionmedia/node-querystring
Its function qs.stringify converts fieldnames (q in the given example) that have an array value to the format:
q[0]=..q[1]=...
This is not a bug but an intended functionality. To overcome this problem without reverting to an old version of the request module you need to manually create your post by using the body option instead of the form option. Also you will need to manually add the content-type header with this method:
var request = require('request');
request.request({
url: "https://www.googleapis.com/language/translate/v2",
headers: {
"X-HTTP-Method-Override": "GET",
'content-type':'application/x-www-form-urlencoded; charset=utf-8'
},
body:'key=xxxx&target=en&q=q=Mon%20premier%20essai&q=Mon%20second%20essai'
}, function(error, response, data) {
if (!error && response.statusCode == 200) {
console.log("everything works fine");
} else {
console.log("something went wrong")
}
});
Obviously this is not as clean but you can easily create a utility function that creates the body string from the object the way you want it to.
things that pop into my head:
jquery file version on server and local PC are not the same
file encoding issues (UTF8 on PC ascii on server?)
have you tried testing it with chrome with Developer Tools open, then check "Network Tab" and verify exactly what is being sent to Google.
For me at least, when it works on one machine and not the other, it is usually due to the first 2 options.
Good Luck!