Ok, this is not what you think it is, I am not asking for help with the async/wait pattern or asynchronous programming I am well versed with those. I am however querying whether something is possible within a Node.JS Express service.
The Scenario
I have a web service which is developed in Node.JS and uses Express.JS to expose some REST endpoints that a client can connect to and send a POST request. For the most part these are Synchronous and will create a SOAP message and send that on to an external service and receive an immediate response which can then be returned to the client, all really simple stuff which is already implemented. So what's your point I hear you say, I am coming to that.
I have a couple of POST interactions that will build a SOAP message to send to an Asynchronous external endpoint where the response will be received asynchronously through an inbound endpoint.
Option 1: What I am looking for in these cases is to be able to build the SOAP message, create a listener (so I can listen for the response to my request), and then send the request to the external service which immediately returns a 200.
Option 2: When I setup the service I want to also setup and listen for incoming requests from the external service whilst also listening for REST requests from the internal service.
The Question
Is either option possible in Node and Express? and, if so how would one achieve this?
NOTE: I know its possible in C# using WCF or a Listener but I would like to avoid this and use Node.JS so any help would be greatly appreciated.
First of all check node-soap if it fits your needs.
Option 1: What I am looking for in these cases is to be able to build the SOAP message, create a listener (so I can listen for the response to my request), and then send the request to the external service which immediately returns a 200.
Here's a very basic non-soap service implementation.
let request = require('request-promise');
let express = require('express');
let app = express();
//Validate the parameters for the request
function validateRequest(req) { ... }
//Transform the request to match the internal API endpoint
function transformRequest(req) { ... }
app.post('/external', function(req, res) {
if(!validateRequest(req))
return res.status(400).json({success: false, error: 'Bad request format');
res.status(200).send();
let callbackUrl = req.query.callback;
let transformedRequest = transformRequest(req);
let internalServiceUrl = 'https://internal.service.com/internal'
request.post(internalServiceUrl, {body: transformedRequest}).then(function (internalResponse){
//Return some of the internal response?
return request.get(callbackUrl, {success: true, processed: true});
}).catch(function (e) {
request.get(callbackUrl, {success: false, error: e});
});
});
Option 2: When I setup the service I want to also setup and listen for incoming requests from the external service whilst also listening for REST requests from the internal service.
There is no "listening" in http. Check socket.io if you need realtime listening. It uses websockets.
Your other option is to poll the internal service (say if you want to check for its availability).
Related
I am using socket.io with nodejs on the server side to provide two-way communication between my web application and the server.
I am using the socket.io API to issue commands and receive responses, but I am not sure if there is a more methodical way of defining a "language" for sending commands to the server and receiving results from it.
For sending commands to the server, I am emitting events from the web application like the following (I am using pseudo code below):
socket.emit('commandRequest', {
msg_id: '...'
username: '...',
command: '...'
});
The server evaluates the command and emits responses like the following:
socket.on('commandRequest', (data) => {
// parse and execute data.command
socket.emit('commandResponse', {
msg_id: data.msg_id,
username: data.username,
response: ...,
error: ...
});
})
Finally, the web application is listening to responses from the server and it updates the app content accordingly:
socket.on('commandResponse', (data) => {
if (data.error) {
...
} else {
// interpret data.response
}
})
So I am using the commandRequest/commandResponse event naming paradigm and the event data structure has corresponding {command: ...} and {response: ...} properties.
Is there a more formal way of defining a request/response "language" that can be used for more complex client/server interactions? Something similar to what REST APIs achieve with HTTP request, but using web sockets?
You could try looking into Primus and its Responder plugin as described in this blog post: Request-Response Oriented Websockets
An excerpt from the post:
It allows to request the opposite peer for a response using a simple API known from other libraries.
Primus Responder adds two things to Primus' core functionality:
A new event called request is emitted as soon as a peer requests a response.
The new method named writeAndWait(data, fn) writes data to a peer and runs given callback function as the peer sends its response.
Primus Responder is available on the client too.
I'm writing a node.js web service which needs to communicate with another server. So its basically server to server communication. I don't have any previous experience of writing web services so I have very limited knowledge. For unit tests I'm using Mocha.
Now, I intend to test the behavior of my service for a particular scenario when this other server doesn't respond to my GET request and the request is actually timed out. For tests I've created a fake client and server around my web service. My web service now takes request from this fake client and then gets information from another fake server that I created which then returns the response in the expected format. To simulate timeout I don't do response.end() from my route handler. The problem is that Mocha judges it to have failed this test case.
Is there a way I could catch this intentional timeout in Mocha and the test is a success?
As mido22 suggested you should use handle the timeout generated by whatever library you use to connect. For instance, with request:
var request = require("request");
it("test", function (done) {
request("http://www.google.com:81", {
timeout: 1000
}, function (error, response, body) {
if (error && error.code === 'ETIMEDOUT') {
done(); // Got a timetout: that's what we wanted.
return;
}
// Got another error or no error at all: that's bad!
done(error || new Error("did not get a timeout"));
});
});
Here is a short snippet of node.js code (express.js and socket.io). Could sending POST requests and emitting socket responces be considered as a bad practice and why?E.g.:
var io = require('socket.io')(http);
app.post('/tickets', jsonParser, function(req, res) {
io.emit('ticket', req.body);
return res.sendStatus(200);
}
I see no problem with that. I actually created a notification system that receives the message and destination as a post and sends notifications to multiple sockets like that.
From your code it looks like that's what your are doing, someone creates a ticket and you send a notification to all listeners.
That seems to be the most practical way and added bonus of being a proper api for use with external server like php or .net. If you're just using it from your own node app than perhaps you could just make it a socket event instead unless you are planning on getting requests from outside your app.
I'd like to add a live functionality to a PHP based forum - new posts would be automatically shown to users as soon as they are created.
What I find a bit confusing is the interaction between the PHP code and NodeJS+socket.io.
How would I go about informing the NodeJS server about new posts and have the server inform the clients that are watching the thread in which the post was posted?
Edit
Tried the following code, and it seems to work, my only question is whether this is considered a good solution, as it looks kind of messy to me.
I use socket.io to listen on port 81 to clients, and the server running om port 82 is only intended to be used by the forum - when a new post is created, a PHP script sends a POST request to localhost on port 82, along with the data.
Is this ok?
var io = require('socket.io').listen(81);
io.sockets.on('connection', function(socket) {
socket.on('init', function(threadid) {
socket.join(threadid);
});
});
var forumserver = require('http').createServer(function(req, res) {
if (res.socket.remoteAddress == '127.0.0.1' && req.method == 'POST') {
req.on('data', function(chunk) {
data = JSON.parse(chunk.toString());
io.sockets.in(data.threadid).emit('new-post', data.content);
});
}
res.end();
}).listen(82);
Your solution of a HTTP server running on a special port is exactly the solution I ended up with when faced with a similar problem. The PHP app simply uses curl to POST to the Node server, which then pushes a message out to socket.io.
However, your HTTP server implementation is broken. The data event is a Stream event; Streams do not emit messages, they emit chunks of data. In other words, the request entity data may be split up and emitted in two chunks.
If the data event emitted a partial chunk of data, JSON.parse would almost assuredly throw an exception, and your Node server would crash.
You either need to manually buffer data, or (my recommendation) use a more robust framework for your HTTP server like Express:
var express = require('express'), forumserver = express();
forumserver.use(express.bodyParser()); // handles buffering and parsing of the
// request entity for you
forumserver.post('/post/:threadid', function(req, res) {
io.sockets.in(req.params.threadid).emit('new-post', req.body.content);
res.send(204); // HTTP 204 No Content (empty response)
});
forumserver.listen(82);
PHP simply needs to post to http​://localhost:82/post/1234 with an entity body containing content. (JSON, URL-encoded, or multipart-encoded entities are acceptable.) Make sure your firewall blocks port 82 on your public interface.
Regarding the PHP code / forum's interaction with Node.JS, you probably need to create an API endpoint of sorts that can listen for changes made to the forum. Depending on your forum software, you would want to hook into the process of creating a new post and perform the API callback to Node.js at this time.
Socket.io out of the box is geared towards visitors of the site being connected on the frontend via Javascript. Upon the Node server receiving notification of a new post update, it would then notify connected clients of this new post and its details, at which point it would probably add new HTML to the DOM of the page the visitor is viewing.
You may want to arrange the Socket.io part of things so that users only subscribe to specific events being emitted by them being in a specific room such as "subforum123" so that they only receive notifications of applicable posts.
I have a very limited knowledge about node and nob-blocking IO so forgive me if my question is too naive.
In order to return needed information in response body, I need to
Make a call to 3rd party API
Wait for response
Add some modifications and return JSON response with the information I got from API.
My question is.. how can I wait for response? Or is it possible to send the information to the client only when I received response from API (as far as I know, connection should be bidirectional in this case which means I won't be able to do so using HTTP).
And yet another question. If one request waits for response from API, does this mean than other users will be forced to wait too (since node is single-threaded) until I increase numbers of threads/processes from 1 to N?
You pass a callback to the function which calls the service. If the service is a database, for example:
db.connect(host, callback);
And somewhere else in the code:
var callback = function(err, dbObject) {
// The connection was made, it's safe to handle the code here
console.log(dbObject.status);
res.json(jsonObject, 200)
};
Or you can use anonymous functions, so:
db.connect(host, function(err, dbObject) {
// The connection was made, it's safe to handle the code here
console.log(dbObject.status);
res.json(jsonObject, 200)
});
Between the call and the callback, node handles other clients / connections freely, "non-blocking".
This type of situation is exactly what node was designed to solve. Once you receive the request from your client, you can make a http request, which should take a callback parameter. This will call your callback function when the request is done, but node can do other work (including serving other clients) while you are waiting for the response. Once the request is done, you can have your code return the response to the client that is still waiting.
The amount of memory and CPU used by the node process will increase as additional clients connect to it, but only one process is needed to handle many simultaneous clients.
Node focuses on doing slow I/O asynchronously, so that the application code can start a task, and then have code start executing again after the I/O has completed.
An typical example might make it clear. We make a call to the FB API. When we get a response, we modify it and then send JSON to the user.
var express = require('express');
var fb = require('facebook-js');
app.get('/user', function(req, res){
fb.apiCall('GET', '/me/', {access_token: access_token}, function(error, response, body){ // access FB API
// when FB responds this part of the code will execute
if (error){
throw new Error('Error getting user information');
}
body.platform = 'Facebook' // modify the Facebook response, available as JSON in body
res.json(body); // send the response to client
});
});