Say you have microservice A,B, and C which all currently communicate through HTTP. Say service A sends a request to service B which results in a response. The data returned in that response must then be sent to service C for some processing before finally being returned to service A. Service A can now display the results on the web page.
I know that latency is an inherent issue with implementing a microservice architecture, and I was wondering what are some common ways of reducing this latency?
Also, I have been doing some reading on how Apache Thrift and RPC's can help with this. Can anyone elaborate on that as well?
Also, I have been doing some reading on how Apache Thrift and RPC's can help with this. Can anyone elaborate on that as well?
The goal of an RPC framework like Apache Thrift is
to significantly reduce the manual programming overhead
to provide efficient serialization and transport mechanisms
across all kinds of programming languages and platforms
In other words, this allows you to send your data as a very compactly written and compressed packet over the wire, while most of the efforts required to achieve this are provided by the framework.
Apache Thrift provides you with a pluggable transport/protocol stack that can quickly be adapted by plugging in different
transports (Sockets, HTTP, pipes, streams, ...)
protocols (binary, compact, JSON, ...)
layers (framed, multiplex, gzip, ...)
Additionally, depending on the target language, you get some infrastructure for the server-side end, such as TNonBlocking or ThreadPool servers, etc.
So coming back to your initial question, such a framework can help to make communication easier and more efficient. But it cannot magically remove latency from other parts of the OSI stack.
Related
TLDR;
I am wondering what is the best way to reliably share an object or other data between n number of webservers on n number of machines?
I have looked at the likes of redis but it seems that this would not be what I am actually looking for here. I am now thinking that something like IPC over remote / RPC might be more appropriate? Is there a better way to do this given it will be called at minimum 10 times over a 30 second interval which can exponentially grow as the number of users running servers grows too.
Example & current use case:
I run a multiplayer mod for a game which receives a decent level of traffic and we are starting to notice cases where requests get dropped sometimes. The backend webserver is written in NodeJS and uses express in a couple of places too. We are in the process of restructuring the system and we have now come to restructuring the part of the system that handles a heartbeat from each server that members of the public host. This information is then shared out to the players so they can decide which server to join.
Based on my own research I am looking to host the service on several different machines for redundancy. These machines are then linked over vlan / vswitch so that they have a secure method to communicate with eachother. The database system is already setup to replicate this way however I cannot see a performance inclined way to handle the sharing of objects containing information about the servers that have communicated with each webhost.
If it helps the system works something like this:
Users server -> my load balancer -> webhost (backend).
Player -> my load balancer -> webhost (backend) returns info on all currently online servers.
In the example above and what is currently in use is a single instance webserver which handles the requests and processes needed.
Just an idea while the community proposes answers: consider reading about Apache Thrift. It is not such as IPC like, but more an RPC like. If the architecture of your servers, or the different components or the "backend network" is in "star"... with one "master" I shoud consider that possibility.
If the architecture of your backend is not like that... but a group of "independent" entities, it comes to my mind to solve the functionality with some "data bus" such as private MQTT broker and a group of members, subscribed or publishing data for the rest of the network. The most optimal serialization strategy for the object would be in my opinion Google Protobuf.
The integration of Mqtt with nodeJS is very simple, and if the weight of the packets is not too big, and you can admit some latency, I would really recommend you to make some tests using Mqtt with a publish/subscription QoS=2. It would not take great efforts to substitute de underlying communications library that you are using.
Once that is said, it seems that there's another solution: Kafka, that seems very interesting (I don't really know it).
Your choice will depend on the nature of your data, mostly: weight of the packets, frequency per user, and the latency you are willing to admit for the worst of the scenarios.
I would like to develop an application that will allow a user to live streams audio from his computer and broadcast it through the network.
My main concern is the performance of this application. I want the quality of the sound to be preserved as its highest and I would like as few "delay" as possible.
Can I reach such a level of performance with web-based languages such as AJAX, NodeJS, or Ruby (Ruby on Rails?) or should I consider more low-level languages such as C/C++?
Thanks!
The performance of sound of the application is not affected so much by the language you choose. That level of performance is achieved by using the correct protocols to send the audio stream over the network. One common mistake is to use HTTP which user TCP protocol because is has a lot of certificates to be verified and this delay the information from streaming live. The best protocol which is used by all big stream companies like Facebook, Instagram, Skype, Lync is the UDP protocol which is super fast because of the minimum level of certificates to be checked.
The companies mentioned above uses different programming languages in order to build the streaming servers and get almost the same results so this is my opinion that the protocol makes the difference.
You can use Java if you want, I have friends that work for 8x8 one of the best companies of live streaming in the USA and they have used JAVA for building the servers and the clients.
i am using the node.js as client to jms topic.There are two protocols available to make the connection on Topic.
Theses are Stomp and AMQP. I am read brief about them at http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol
and http://en.wikipedia.org/wiki/Streaming_Text_Oriented_Messaging_Protocol. Both seems to be wire level protocol i.e
data that is sent across the network as a stream of octets.I do not find any
concrete reason which one should be preferred. If someone can shed light on it, it would be helpful.
Another point is both the protocol takes pride in stating them as interoprable. Does the interoperable term means that if someone
want to take off specific message broker implementation say apache active MQ and instead want to plugin Websphere MQ , transition
will be smooth(provided both support AMQP/STOMP or any other wire levl protocol)?
You may see a difference in performance (refer to this benchmark based upon many factors including message size and persistence requirements for the queue entries.
As is often the case, there are other factors to consider as well, especially if your message size/count/etc. doesn't mean there's a clear winner in terms of performance and no one protocol meets your functional requirements in a way the other does not.
This article in particular hints there may be more fragmentation in the different STOMP broker implementations. Quoting from that article
STOMP...uses a SEND semantic with a “destination” string. The broker must map onto something that it understands internally such as a topic, queue, or exchange. Consumers then SUBSCRIBE to those destinations. Since those destinations are not mandated in the specification, different brokers may support different flavours of destination. So, it’s not always straightforward to port code between brokers.
At least with AMQP (which touts interoperability as one of its most important advantages) the only issues you should have with switching providers/languages are those inherent in setting up said new providers. For example, I've read ZeroMQ is likely to take more configuration work on your part than RabbitMQ, but that's not really due to any attributes specific to AMQP.
We have an embedded box. The specs of the CPU is medium speed (600MHz) and RAM is between 512MB to 2GB depending on configuration. The system consists of data layer to process incoming data from hardware and needing to be displayed both remotely and on an HDMI output.
Seeing the remote aspect is as important as the local display, we have architected a client-server solution. For the server, it just needs to respond to requests for data. The data needs to come from the internals of another process (IPC interaction) and return it formatted for the client.
For the server we are thinking of using node.js. The data is spec'ed to be formatted into json messages, so using JavaScript and json is simple.
However, are there any better/other server options to consider?
The main requirement is the server can be extended to interact with the data process and be able to process and respond to requests.
We could write it ourselves, but feel that there must be usable tech to leverage already.
Thanks.
I am assuming that you need output as a webpage only.
It depends.
If your team knows java/servlet well, you can do using simple jetty/servlet/jsp combination.
But if you have team good with javascript/node.js, go with it. Although, I am not sure about stability requirements you have, because node.js is quite stable but it haven't reached 1.0 yet.
I'm reading about different approaches for scaling request handling capabilities on a single machine being taken by node.js, ruby, jetty and company.
Being an application developer, i.e. having very little understanding in Kernel/Networking I'm curious to understand the different approaches taken by each implementation (kernel select, polling the socket for connection, event based and company.) ?
Please note that I'm not asking about special handling features (such as jetty continuations (request->wait->request), a pattern which is typical for AJAX clients) but more generally, should you like to implement a server that can respond with "Hello World" to the maximum number of concurrent clients how would you do it? and Why?
Information / References to reading material would be great.
Take a look at The C10K problem page.