I have created a rest client to make calls outside of my application. I want to send the data I receive back from the rest calls to the client's web browser.
Something like the following, but what is the best way to structure the code to allow access to the response to write back to the web browser with as loose coupling as possible? I don't want to define the rest client within a request handler.
var servReq = http.request(options, function(restResponse){
var status = restResponse.statusCode
var headers = restResponse.headers
restResponse.setEncoding("utf8");
d='';
restResponse.on('data', function(chunk){
d += chunk;
})
restResponse.on('end', function(restResponse){
// res would be a response to write back to the client's web browser
// with the data received from the rest client.
res.writeHead(200, {"content-type":"text/plain"})
res.write(d)
res.end();
})
}
Using request, you can pipe the API response straight to your application's response. This way it'll be completely loosely coupled—your server will return precisely what the API returns.
request(options).pipe(res);
https://github.com/mikeal/request#streaming
Related
I'm trying to display a list of comments on my react page.
For this I have setup a NodeJS server which loads the data from Firebase and passes it on to React. I am able to get it to load the comments list and display them, but when I try to add a comment, the server crashes with the following error:
#firebase/database: FIREBASE WARNING: Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
This is because I am using:
firebase.database().ref('my-path').on("value", ...)
However, if I use firebase.database().ref('my-path').once("value", ...) then I lose the ability to update the comments as soon as a new comment is posted. Is there a way to be able to have a listener attached to the database and still be able to update the contents of that database?
Here is my NodeJS code:
app.get("/comments/:id", (req, res) => {
const itemsRef = firebase.database().ref(`comments/${req.params.id}`);
itemsRef.on('value', (snapshot) => {
let comments = snapshot.val();
return res.status(200).json(comments);
})
})
app.post("/comments/:id", (req, res) => {
const itemsRef = firebase.database().ref(`comments/${req.params.id}`);
itemsRef.push(req.body);
})
The error occurs after the post request is called.
You're sending a response back to the client with:
res.status(200).json(comments)
This sets at least two headers (the status, and the response type) and then sends the response. Next time you get an update from the database, this code runs again, and again tries to send the two headers. But in HTTP all headers must be before the main body of the response. So the second time this code runs, it throws an error.
If you want to keep sending more data to the client, you'll need to use more primitive methods of the response object to prevent sending headers, or other illegal data. While possible, it's more complex than you may think, as the client needs to handle this response stream, which most clients won't.
I'd highly recommend looking at Doug's alternative, which is to just use the Firebase Realtime Database from the client directly. That way you can use the client SDK that it has, which handles this (and many more complexities) behind the scenes.
When sending the first request to the calendarView API, the request does not return or timeout. This only happens to some of the requests and seems to happen only on the first request (perhaps because the first request has larger response sizes).
An example request:
GET /me/calendarView/delta?startDateTime=2019-06-27T22:00:00.000Z&endDateTime=2019-09-08T13:17:30.659Z
The current solution I found was reducing the odata.maxpagesize to a very small number (currently 2 is the highest number which works for all calendars I have tested).
The requests are sent using the nodejs client "#microsoft/microsoft-graph-client": "1.7.0".
// Initialize client with credentials
const client = graph.Client.init({
authProvider: done => {
done(null, credentials.access_token);
}
});
const url = "/me/calendarView/delta?startDateTime=2019-06-27T22:00:00.000Z&endDateTime=2019-09-08T13:17:30.659Z
console.log("Request start")
const result = await oAuth2Client
.api(url)
.header("prefer", "odata.maxpagesize=10")
.get();
console.log("Got result", result);
Here the last console.log is never called.
The expected result is that the request returns, at least with an error code. I also expect the API to be able to handle a lot more items than 2 per page.
The current solution with setting a small maxpagesize works temporarily, however, I expect that there is another root cause issue.
Any idea what is wrong, and how this can be resolved?
After a lot of debugging i traced the issue to the node library. When asking for a raw response from the API, I got back the result regardless of page size.
The solution was to manually parse the response myself, after asking for the raw response from the library. This was based on the implementation in the library at https://github.com/microsoftgraph/msgraph-sdk-javascript/blob/dev/src/GraphResponseHandler.ts#L98
I could not find the root cause issue in the library, and ended up just parsing it on my end. I also analysed the raw response from the API, but the content-type header was correctly application/json, and the response status was 200 OK.
const rawResponse = client.api(url).response(ResponseType.RAW).get()
const parsedResponse = await rawResponse.json()
What is the difference between the request in this line of code:
http.createServer(function(request,response){. . .}
and request in
http.request()
Are both requests done to the server?
I am new to node.js and I am sorry if I sound dumb!
How does http.request() work?
In http.request() we fetch data from another site but in order to fetch data from another site we first need to go to our site and then make a request? Explain it with a simple real-life example!
http.request() makes a request to another HTTP server. Suppose for some reason I wanted to go download Stack Overflow's home page...
http.request('https://stackoverflow.com/', (res) => {
// ...
});
http.createServer()... it creates an HTTP server. That is, it binds your application to a socket to listen on. When a new connection is made from somewhere or something else, it handles the underlying HTTP protocol of that request and asks your application to deal with it by way of a callback. From the Node.js documentation:
http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('okay');
});
These two methods have absolutely nothing to do with each other. http.request() is for making a request to an HTTP server. http.createServer() is for creating your own HTTP server. Don't get confused by the callbacks.
Based on the source code of nodejs (extract below), createServer is just a helper method to instantiate a Server.
Extract from line 1674 of http.js.
exports.Server = Server;
exports.createServer = function(requestListener) {
return new Server(requestListener);
};
The http.request() API is for when you want your server code to act as a client and request content from another site and has GET, POST, PUT, DELETE methods.
For example, I want to signal to the client side that a username sent via the POST method in an HTML form already exists in my database.
I know how to recuperate POST data with body-parser and I know how to look it up in a MySQL database.
I know that I could use Ajax to write an error message directly on the form. What does my NodeJS server need to send and how does it send this information?
I've searched through numerous tutorials and I just found solutions where they send a new HTML page. I want to keep my web page the same and use functions like appendChild() to post the error message.
There are a couple of ways you could send data from server-side, so NodeJS, to client-side - which I assume in your case would be some JavaScript file like main.js that handles DOM manimulation.
So, the 1st way you could send data is through a templating engine like Handlebars, for example. There is an easy to use module for express you could get here: hbs.
Now to quickly summarize how an engine like that works, we are basically sending an HTML file like you probably saw in the tutorials, however, a templating engine like Handlebars allows us to send actual data with that file dynamically, so what we would do is render a specific Handlebars template (which in core is just HTML), and pass in a JavaScript object to the render call which would contain all the data you want to pass into that file and then access it in the .hbs file.
So on the server-side, we would write something like this, assuming we have a file called home.hbs and set up Handlebars as the templating engine:
router.get('/home', function(req,res) {
var dataToSendObj = {'title': 'Your Website Title', 'message': 'Hello'};
res.render('home',dataToSendObj);
});
And access in home.hbs like this:
<html>
<header>
{{title}}
</header>
<body>
message from server: {{message}}
</body>
</html>
Now, the issue with this approach is that if you wanted to update the data on the page dynamically, without having to reload the page, using a templating engine would not be ideal. Instead, like you said, you would use AJAX.
So, the 2nd way you could send data from your NodeJS server to the front-end of your website, is using an asynchronous AJAX call.
First, add a route to whatever route handler you are using for AJAX to make a call to. This where you have some logic to perhaps access the database, make some checks and return some useful information back to client.
router.get('/path/for/ajax/call', function(req,res) {
// make some calls to database, fetch some data, information, check state, etc...
var dataToSendToClient = {'message': 'error message from server'};
// convert whatever we want to send (preferably should be an object) to JSON
var JSONdata = JSON.stringify(dataToSendToClient);
res.send(JSONdata);
});
Assuming you have some file such as main.js, create an AJAX request with callbacks to listen to certain event responses like this:
var req = new XMLHttpRequest();
var url = '/path/for/ajax/call';
req.open('GET',url,true); // set this to POST if you would like
req.addEventListener('load',onLoad);
req.addEventListener('error',onError);
req.send();
function onLoad() {
var response = this.responseText;
var parsedResponse = JSON.parse(response);
// access your data newly received data here and update your DOM with appendChild(), findElementById(), etc...
var messageToDisplay = parsedResponse['message'];
// append child (with text value of messageToDisplay for instance) here or do some more stuff
}
function onError() {
// handle error here, print message perhaps
console.log('error receiving async AJAX call');
}
To summarize the above approach using AJAX, this would be the flow of the interaction:
Action is triggered on client-side (like button pressed)
The event handler for that creates a new AJAX request, sets up the callback so it knows what to do when the response comes back from the server, and sends the request
The GET or POST request sent is caught by our route handler on the server
Server side logic is executed to get data from database, state, etc...
The new data is fetched, placed into a JSON object, and sent back by the server
The client AJAX's event listener for either load or error catches the response and executes the callback
In the case of a successful response load, we parse the response, and update the client-side UI
Hope this is helpful!
Say I have the following code that sends back json data (that I get from example.com first) back to a user that made a post request
app.post('/riot', function(request, response) {
var riotAPI = https.get("https://example.com", function(riotResponse){
var body = "";
riotResponse.on('data', function(chunk){
body+= chunk;
});
riotResponse.on('end', function(){
response.type('json');
response.end(body);
});
});
});
What do I do if I want to get more data from a different website and send json data from both website back to user? I am using express.
There are a number of ways you can do this. I would suggest using the request npm module instead of calling https directly. With request you can simply pass in callback which is called when a request finishes, so no need to deal with chunks of data.
If you take this approach then you can just use something like async.parallel() to run both requests in parallel. async.parallel takes one callback that is called when all of its async functions have finished.. and that is where you would send your response.