I am trying to write a piece of latency-sensitive Rust code that makes a few HTTP requests very quickly (ideally within a few microseconds of one another -- I know that seems ridiculous when considering the effects of network latency, but I'm co-located with the server I'm trying to work with here). The server I'm trying to work with supports HTTP/2 (I know it at least does ALPN -- not sure if it also does prior-knowledge connections).
I've tried using Reqwest's tokio-based client, but tokio's task spawning time seems to be anywhere from 10-100us (which is more than I'd like).
I've tried doing independent threads with their own HTTP clients that just make their own requests when they get an incoming message on their channels, but the time between sending the message and them receiving it can be anywhere from 1-20us (which is, again, more than I'd like).
I've also tried using libcurl's multi library, but that seems to add milliseconds of latency, which is far from ideal.
I've also tried using hyper's client, but any way that I try to enable HTTP/2 seems to run into issues.
I know that HTTP/2 can multiplex many requests into a single connection. And it seems to me that it's probably possible to do this in a single thread. Would anyone be able to point me in the right direction here?
Alright, here's some of the code I've got so far: https://github.com/agrimball/stack-overflow-examples
Not sure if this question is scoped appropriately for StackOverflow. The basic idea is simple (I want to do HTTP/2 request multiplexing and haven't found a usable lib for it in Rust -- any help?), but the story of how I got to that question and the list of alternatives I've already tried does end up making it a bit long...
In any case, thanks in advance for any help!
ninja edit: And yes, being able to do so from a single thread would be ideal since it would remove a bit of latency as well as some of the code complexity. So that part of the title is indeed relevant as well... Still striving for simplicity / elegance when possible!
Related
My project is a full stack application where a web client subscribes to an unready object. When the subscription is triggered, the backend will run an observation loop to that unready object until it becomes ready. When that happens it sends a message to the frontend through socketIO (suggestions are welcome, I'm not quite sure if it's the best method). My question is how do I construct the observation loop.
My frontend basically subscribes to the backend, and gets a return 200 and will connect to the server per Websocket (socketIO) if it got subscribed correctly, or an error 4XX code if there was something that went wrong. On the backend, when the user subscribes, it should start for that user, a "thread" (I know Nodejs doesn't support threads, it's just for the mental image) that polls an information from an api every 10 or so seconds.
I do that, because the API that I poll from does not support WebHooks, so I need to observe the API response until it's at the state that I want it (this part I already got cleared).
What I'm asking, is there a third party library that actually is meant for those kinds of tasks? Should I use worker threads or simple setTimeouts abstracted by Classes? The response will be sent over SocketIO, that part I already got working as well, it's just the method I'm using im not quite sure how to build.
I'm also open to use another fitting programming language that makes solving this case easier. I'm not in a hurry.
A polling network request (which it sounds like this is) is non-blocking and asynchronous so it doesn't really take much of your nodejs CPU unless you're doing some heavy-weight computation of the result.
So, a single nodejs thread can make a lot of network requests (for your polling and for sending data over socket.io connection) without adding WorkerThreads or clustering. This is something that nodejs is very, very good at.
I'm not aware of any third party library specifically for this as you have to custom code looking at the results of the network request anyway and that's most of the coding. There are a bunch of libraries for making http requests of other servers from nodejs listed here. My favorite in that list is got(), but you can look at the choices and decide what you like.
As for making the repeated requests, I would probably just use either repeated setTimeout() calls or a setInterval() call.
You don't say whether you have to make separate requests for every single client that is subscribed to something or whether you can somehow combine all clients watching the same resource so that you use the same polling interval for all of them. If you can do the latter, that would certainly be more efficient.
If, as you scale, you run into scaling issues, you can then move the polling code to one or more child processes or WorkerThreads and then just communicate back to the main thread via messaging when you have found a new state that needs to be sent to the client. But, I would not anticipate you would need to code that extra step until you reach larger scale. As with most scaling things, you would need to code up the more basic option (which should scale well by itself) and then measure and benchmark and see where any bottlenecks are and modify the architecture based on data, not speculation. Far too often, the architecture is over-designed and over-implemented based on where people think the bottlenecks might be rather than where they actually turn out to be. Not only does this make the development take longer and end up with more complicated implementation than required, but it can target development at the wrong part of the problem. Profile, measure, then decide.
I'm very new to Node.js, so I might just not be getting it, but after searching quite a bit, and trying a few different solutions, I am still not able to find a decent way to mock API responses using Node for acceptance testing.
I've got a javascript app (written in elm actually) that interacts with an API (pretty common, I imagine), and I want to write some acceptance tests... so I setup WebdriverIO with selenium and mocha, write some tests, and of course now I need to mock some API responses so that I can setup some theoretical scenarios to test under.
mock-api-server: Looked pretty nice, but there's no way to adjust the headers getting sent back from the server!
mock-http-server: Also looked pretty nice, lets me adjust headers, but there's no way to reset the mock responses without shutting down the whole server... !? And that has issues because the server won't shut down while the browser window is still open, so that means I have to close and relauch the browser just to clear the mocks!
json-server: Simple and decent way to mock some responses, but it relies entirely on files on disk for the responses. I want something I can configure from within a test run without reading and writing files to disk.
Am I missing something? Is this not how people do acceptance testing in the Node universe? Does everyone just use a fixed set of mock data for their entire test suite? That just sounds insane to me... Particularly since it seems like it wouldn't be that hard to write a good one based on express server that has all the necessary features... does it exist?
Necessary Features:
Server can be configured and launched from javascript
Responses(including headers) can be configured on the fly
Responses can also be reset easily on the fly, without shutting down the server.
I hit this problem too, so I built one: https://github.com/pimterry/mockttp
In terms of the things you're looking for, Mockttp:
Lets you start & reconfigure the server dynamically from JS during the test run, with no static files.
Lets you adjust headers
Lets you reset running servers (though I'd recommend shutting down & restarting anyway - with Mockttp that takes milliseconds, is clear & easily automatable, and gives you some nice guarantees)
On top of that, it:
Is configurable from both Node & browsers with identical code, so you can test universally
Can handle running tests in parallel for quicker testing
Can fake HTTPS servers, self-signing certificates automatically
Can mock as an intercepting proxy
Has a bunch of nice debugging support for when things go wrong (e.g. unmatched requests come back with a readable explanation of the current configuration, and example code that would make the request succeed)
Just to quickly comment on the other posts suggesting testing in-process: I really wouldn't. Partly because a whole bunch of limitations (you're tied to a specific environment, potentially even specific node version, you have to mock for the whole process, so no parallel tests, and you can't mock subprocesses), but mainly because it's not truly representative. For a very very small speed cost, you can test with real HTTP, and know that your requests & responses will definitely work in reality too.
Is this not how people do acceptance testing in the Node universe? Does everyone just use a fixed set of mock data for their entire test suite?
No. You don't have to make actual HTTP requests to test your apps.
All good test frameworks lets you fake HTTP by running the routes and handlers without making network requests. Also you can mock the functions that are making the actual HTTP requests to external APIs, which should be abstracted away in the first place, so no actual HTTP requests need to take place here as well.
And if that's not enough you can always write a trivially simple server using Express, Hapi, Restify, Loopback or some other frameworks, or plain http, or even net module (depending on how much control do you need - for example you should always test invalid responses that don't use HTTP protocol correctly, or broken connections, incomplete connections, slow connections etc. and for that you may need to use lower lever APIs in Node) to provide the mock data yourself.
By the way, you also always need to test responses with invalid JSON because people often wrongly assume that the JSON they get is always valid which it is not. See this answer to see why it is particularly important:
Calling a JSON API with Node.js
Particularly since it seems like it wouldn't be that hard to write a good one based on express server that has all the necessary features... does it exist?
Not everything that "wouldn't be that hard to write" necessarily has to exist. You may need to write it. Hey, you even have a road map ready:
Necessary Features:
Server can be configured and launched from javascript
Responses(including headers) can be configured on the fly
Responses can also be reset easily on the fly, without shutting down the server.
Now all you need is choose a name, create a repo on GitHub, create a project on npm and start coding.
You now, even "it wouldn't be that hard to write" it doesn't mean that it will write itself. Welcome to the open source world where instead of complaining that something doesn't exist people just write it.
You could try nock. https://github.com/node-nock
It supports all of your feature requests.
I have a question regarding the examples out there when using Nodejs, Express and Jade for templates.
All the examples show how to build some sort of a user administrative interface where you can add user profiles, delete them and manage them.
Those are considered beginner's guides to NodeJs. My question is around the fact that if I have have 10 users concurrently accessing the same interface and doing the same operations, surely NodeJs will block the requests for the other users as they are running on the same port.
So let's say I am pulling out a list of users which may be something like 10000. Yes I can do paging, but that is not the point. While I am getting the list from the server another 4 users want to access the application. They have to wait for my process to end. That is my question - how can one avoid that using NodeJS & Express?
I am on this issue for a couple of months! I currently have something in place that does the following:
Run the main processing of stuff on a port
Run a Socket.io process on a different port
Use a sticky session
The idea is that I do a request (like getting a list of items), and immediately respond with some request reference but without the requested items, thus releasing the port.
In the background "asynchronously" I then do the process of getting the items. Upon which when completed, I do an http request from one node to the socket node port node SENDING the items through.
When that is done I then perform a socket.io emit WITH the data and the initial request reference so that the correct user gets the message.
On the client side I have an event listening for the socket which then completes the ajax request by populating the list.
I have SOME success in doing this! It actually works to a degree! I have an issue online which complicates matters due to ip addresses, and socket.io playing funny.
I also have multiple workers using clustering. I use it in the following manner:
I create a master worker
I spawn workers
I take any connection request and pass it to the relevant worker.
I do that for the main node request as well as for the socket requests. Like I said I use 2 ports!
As you can see I have had a lot of work done on this and I am not getting a proper solution!
My question is this - have I gone all around the world 10 times only to have missed something simple? This sounds way to complicated to achieve a non-blocking nodejs only website.
I asked myself - surely all these tutorials would have not missed on something as important as this! But they did!
I have researched, read, and tested a lot of code - this is my very first time I ask anything on stackoverflow!
Thank you for any assistance.
P.S. One example of the same approach is this: I request a report using jasper, I pass parameters, and with the "delayed ajax response" approach as described above I simply release the port, and in the background a very intensive report is being generated (and this can be very intensive process as a lot of calculations are being performed)..! I really don't see a better approach - any help will be super appreciated!
Thank you for taking the time to read!
I'm sorry to say it, but yes, you have been going around the world 10 times only to have been missing something simple.
It's obvious that your previous knowledge/experience with webservers are from a blocking point of view, and if this was the case, your concerns had been valid.
Node.js is a framework focused around using a single thread to execute code, which means if it does any blocking operations, no one else would be able to get anything done.
There are some operations that can do this in node, like reading/writing to disk. However, most node operations will be asynchronous.
I believe you are familiar with the term, so I won't go into details. What asynchronous operations allows node to do, is to keep this single thread idle as much as possible. By idle I mean open for other work. If your code is fully asynchronous, then handling 4 concurrent users (or even 400) shouldn't be a problem, even for a single thread.
Now, in regards to your initial problem of ports: Once a request is received on a given port, node.js execute whatever code you have written for it, until it encounters an asynchronous operation as soon as that happens, it is available to to pick up more requests on the same port.
The second problem you inquire about, is the database operation. In this case, node-js would send the query to the database (which takes no time at all) and the database does that actual execution of the query. In the meantime, node is free to do whatever it wants, until the database is finished, and lets node know there is a result to fetch.
You can recognize async operations by their structure: my_function(..., ..., callback). Function that uses a callback function, is in most cases asynch.
So bottom line: Don't worry about the problems around blocking IO, as you will hardly encounter any in node. Use a single port if you want (By creating multiple child processes, you can even have multiple node instances on the same port).
Hope this explains it good enough. If you have any further questions, let me know :)
The title pretty well says it.
I need the microcontrollers to stay connected to the server to receive updates within a couple of seconds and I'm not quite sure how to do this.
The client in this case is very limited to say the least and it seems like all the solutions I've found for polling or something like socket.io require dropping some significant JavaScript to the client.
If I'm left having to reimplement one of those libraries in C on the micro I could definitely use some pointers on the leanest way to handle it.
I can't just pound the server with constant requests because this is going to increase to a fair number of connected micros.
Just use ordinary long polling: each controller initially makes an HTTP request and waits for a response, which happens when there's an update. Once the controller receives the response, it makes another request. Lather, rinse, repeat. This won't hammer the server because each controller makes only one request per update, and node's architecture is such that you can have lots of requests pending, since you aren't creating a new thread or process for each active connection.
I'm in process of making realtime multiplayer racing game. Now I need help writing game logics in Node.js TCP (net) server. I don't know if it's possible, I don't know if i'm doing that right, but I'm trying my best. I know it's hard to understand my broken english, so i made this "painting" :)
Thank you for your time
To elaborate on driushkin's answer, you should use remote procedure calls (RPC) and an event queue. This works like in the image you've posted, where each packet represents a 'command' or RPC with some arguments (i.e. movement direction). You'll also need an event queue to make sure RPCs are executed in order and on time. This will require a timestamp or framecount for each command to be executed on (at some point in the future, in a simple scheme), and synchronized watches (World War II style).
You might notice one critical weakness in this scheme: RPC messages can be late (arrive after the time they should be applied) due to network latency, malicious users, etc. In a simple scheme, late RPCs are dropped. This is fine since all clients (even the originator!) wait for the server to send an RPC before acting (if the originating client didn't wait for the server message, his game state would be out of sync with the server, and your game would be broken).
Consider the impact of lag on such a scheme. Let's say the lag for Client A to the server was 100ms, and the return trip was also 100ms. This means that client input goes like:
Client A presses key, and sends RPC to server, but doesn't add it locally (0ms)
Server receives and rebroadcasts RPC (100ms)
Client A receives his own event, and now finally adds it to his event queue for processing (200ms)
As you can see, the client reacts to his own event 1/5 of a second after he presses the key. This is with fairly nice 100ms lag. Transoceanic lag can easily be over 200ms each way, and dialup connections (rare, but still existent today) can have lag spikes > 500ms. None of this matters if you're playing on a LAN or something similar, but on the internet this unresponsiveness could be unbearable.
This is where the notion of client side prediction (CSP) comes in. CSP is made out to be big and scary, but implemented correctly and thoughtfully it's actually very simple. The interesting feature of CSP is that clients can process their input immediately (the client predicts what will happen). Of course, the client can (and often will) be wrong. This means that the client will need a way of applying corrections from the Server. Which means you'll need a way for the server to validate, reject, or amend RPC requests from clients, as well as a way to serialize the gamestate (so it can be restored as a base point to resimulate from).
There are lots of good resources about doing this. I like http://www.gabrielgambetta.com/?p=22 in particular, but you should really look for a good multiplayer game programming book.
I also have to suggest socket.io, even after reading your comments regarding Flex and AS3. The ease of use (and simple integration with node) make it one of the best (the best?) option(s) for network gaming over HTTP that I've ever used. I'd make whatever adjustments necessary to be able to use it. I believe that AIR/AS3 has at least one WebSockets library, even if socket.io itself isn't available.
This sounds like something socket.io would be great for. It's a library that gives you real time possibilities on the browser and on your server.
You can model this in commands in events: client sends command move to the server, then server validates this command and if everything is ok, he publishes event is moving.
In your case, there is probably no need for different responses to P1 (ok, you can move) and the rest (P1 is moving), the latter suffices in both cases. The is moving event should contain all necessary info (like current position, velocity, etc).
In this simplest form, the one issuing command would experience some lag until the event from server arrives, and to avoid that you could start moving immediately, and then apply some compensating actions if necessary when event arrives. But this can get complicated.