So I've got a working node.js code that processes data from a website's API. I'd like to speed it up a bit and I figured the best way to do that would be to send a request and while waiting for a response some code would execute and not just wait for a response like it is now. Right now my code is essentially this:
function httpGet(url){
var response = requestSync(
'GET',
url
);
return response.body;
}
var returnCode;
var getUrl = "url"
returnCode = httpGet(getUrl);
var object = JSON.parse(returnCode);
//Some code executes
As you can see with this way some time is lost because you're waiting for the response. I'd be looking for something in this sense (pseudocode):
Send a request
Some code that's not related to the request is executed right after the request is sent
After the part above is done the request result is parsed
In conclusion I'm looking for a way to send a request and not waste time waiting for a response. If you have any other ideas on how to speed up the code please let me know :)
You are looking for asynchronous code. When you use a function like requestSync it means that it "blocks" until it's done. It's synchronous. When you use something asynchronous, you will usually do so with a callback (a function to call when the desired action is completed) or a promise (an abstraction over callbacks). There are lots of questions about using those on SO. This post: How do I return the response from an asynchronous call? has a bunch of info related to your question.
Related
I was asked a question in an interview. below is the question.
const JsonFromHTTPCall = function(){
// make get request to some api url and return json object.
}
// code below is not editable
let result = JsonFromHTTPCall();
console.log("result ", result);
I am not finding a way to make console.log statement wait until I get the result from http call.
Please give me a way to solve it.
Thanks in advance.
Nodejs does not offer synchronous networking in any way. All built-in networking is asynchronous. Therefore, you cannot directly return a value from a function retrieved via networking. Instead, you need to communicate back the result either via a callback function, an event you trigger or a returned promise.
For a summary of this issue see this highly active question/answer:
How do I return the response from an asynchronous call?
There is a gross hack that involves using a synchronous child process and having it do the networking for you, but it's unlikely that is what they were asking for in your interview.
So, the main answer to the question is that "nodejs does not offer synchronous networking" and further "you cannot change an asynchronous result into a synchronous result". Therefore the proper way to code this is to use nodejs asynchronous coding techniques.
The cleanest way I know of to make http get calls is using a library such as request-promise() or my newer favorite got() and use the promise interface plus async/await to make a nice clean code path:
const got = require('got');
async function getSomeJSON(url) {
let data = await got(url).json();
console.log(data);
return data;
}
getSomeJSON(myURL).then(data => {
console.log("got my data");
}).catch(err => {
console.log(err);
});
I'm experimenting with the translation service on the Microsoft bot framework. I've written a method to which I pass a callback function which receives my translated text.
I've got an existing bot that calls an HTTP endpoint to create my output in English. I want to translate the output to the different language before returning it to the user. My unaltered code looks like this:
await request.post(ENDPOINT,
{
headers: HEADERS,
json: BODY
},
async function (error, response, body) {
if (response.statusCode == 202) {
var msg = body.mainResponse.text;
context.sendActivity(msg);
}
});
This runs just fine. Data passed in the HTTP response body gets parsed sent back to the user.
Now I want to plug in my translation service. I've got a single function that I call to do this called Translator.translate(text, callback). I've added this call to my existing function to get:
await request.post(ENDPOINT,
{
headers: HEADERS,
json: BODY
},
async function (error, response, body) {
if (response.statusCode == 202) {
var msg = body.mainResponse.text;
await Translator.translate(msg, function (output) {
context.sendActivity(output);
});
}
}
);
My translation process runs and I get the translation in the output variable, but nothing gets sent back to the user. Looking at the terminal, I see the error "Cannot perform 'get' on a proxy that has been revoked" relating to the context.sendActivity line in my callback.
Can anyone suggest how I keep the context object active?
Thanks in advance.
Many thanks for the assistance everyone - I never completely got to the bottom of this, but I finally fixed it with a complete re-write of the code. I think the problem was caused by a large number of nested synchronous and asynchronous calls. My ultimate solution was to completely get rid of all the nesting - first calling the translation service (and waiting for it), then doing the original call.
I think there are a number of other asynchronous threads inside the methods of both pieces of functionality. I don't have a great understanding of how this works in node, but I'm guessing that the response was getting popped off the stack at the wrong point, which is why I wasn't seeing it. The "cannot perform get" error was a bit of a red herring, it turns out. I get the same error from some of Microsoft's working demo code. I'm sure there's a separate issue there that ought to be fixed, but it wasn't actually caused by this issue. The code was running, but the output was getting lost.
I have a node process running that do some unirest.get and unirest.post from time to time. I also have a type of web terminal from which I can see the overall progress of said unirest requests.
The problem is that I need to be able to cancel a specific request, but I can't find out how.
The structure is something like this:
var requests = [];
requests.push(unirest.post('someurl').end(somecallback));
requests.push(unirest.post('someurl').end(somecallback));
requests.push(unirest.post('someurl').end(somecallback));
And I want to do something like:
requests[1].cancel(); // but of course this method doesn't exist
since in this case I can't let the callback fires, since the goal is to cancel it to request the same url again, and without canceling it the callback would fire twice.
Anyone knows how to cancel/interrupt/destroy it?
Result of unirest.post('someurl').end(some callback) is Request object from request module.
So, you can use abort method:
requests[1].abort();
I'm new in Node JS and i wonder if under mentioned snippets of code has multisession problem.
Consider I have Node JS server (express) and I listen on some POST request:
app.post('/sync/:method', onPostRequest);
var onPostRequest = function(req,res){
// parse request and fetch email list
var emails = [....]; // pseudocode
doJob(emails);
res.status(200).end('OK');
}
function doJob(_emails){
try {
emailsFromFile = fs.readFileSync(FILE_PATH, "utf8") || {};
if(_.isString(oldEmails)){
emailsFromFile = JSON.parse(emailsFromFile);
}
_emails.forEach(function(_email){
if( !emailsFromFile[_email] ){
emailsFromFile[_email] = 0;
}
else{
emailsFromFile[_email] += 1;
}
});
// write object back
fs.writeFileSync(FILE_PATH, JSON.stringify(emailsFromFile));
} catch (e) {
console.error(e);
};
}
So doJob method receives _emails list and I update (counter +1) these emails from object emailsFromFile loaded from file.
Consider I got 2 requests at the same time and it triggers doJob twice. I afraid that when one request loaded emailsFromFile from file, the second request might change file content.
Can anybody spread the light on this issue?
Because the code in the doJob() function is all synchronous, there is no risk of multiple requests causing a concurrency problem.
If you were using async IO in that function, then there would be possible concurrency issues.
To explain, Javascript in node.js is single threaded. So, there is only one thread of Javascript execution running at a time and that thread of execution runs until it returns back to the event loop. So, any sequence of entirely synchronous code like you have in doJob() will run to completion without interruption.
If, on the other hand, you use any asynchronous operations such as fs.readFile() instead of fs.readFileSync(), then that thread of execution will return back to the event loop at the point you call fs.readFileSync() and another request can be run while it is reading the file. If that were the case, then you could end up with two requests conflicting over the same file. In that case, you would have to implement some form of concurrency protection (some sort of flag or queue). This is the type of thing that databases offer lots of features for.
I have a node.js app running on a Raspberry Pi that uses lots of async file I/O and I can have conflicts with that code from multiple requests. I solved it by setting a flag anytime I'm writing to a specific file and any other requests that want to write to that file first check that flag and if it is set, those requests going into my own queue are then served when the prior request finishes its write operation. There are many other ways to solve that too. If this happens in a lot of places, then it's probably worth just getting a database that offers features for this type of write contention.
I'm new to node.js and have this very simple code. I just want to say a Hi User every second to the users who have connected to the server.
Here's the code I have:
var http = require('http');
function newfunc(request, response) {
response.writeHead(200, {
"Content-Type": "text/plain",
"connection" : "keep-alive"
});
setInterval(function() {
response.write('Hi User\n');
response.end('');
}, 1000);
}
http.createServer(newfunc).listen(7070);
I see the Hi User message only once, and seems as if setInterval is writing it only once.
What am I doing wrong in this?
EDIT: I stand corrected in the comments... Just remove the response.end() call and you should see something like what you were expecting.
ORIGINAL RESPONSE: What you are trying to do cannot be done in this fashion... The response to an HTTP request is only ever sent once: when response.end() is called (the first time). Commenter points out that this is not really correct: what would be correct to say is that no further data can be sent in the response after end() is called.
If you want to show an HTML page whose contents change every second based on server-side actions, you will need to use something like WebSockets (e.g. the Node-compatible http://socket.io/ library) and some client-side Javascript, which is somewhat more complicated than the code you have above. In general, for non-trivial UI's that do more than just append to the response or do bi-directional communication, this type of approach is preferrable