Everytime i get or set couchbase was slow;i thank the reason is when i get or set a key i must wait couchbase return me result;and couchbase use tcp connect,everytime must via three-way handshaking;i know redis support unix domain socket connect, and it is faster than via tcp connect when try unix domain socket connect; and is couchbase server support unix domain socket? and how to do it;or have any method can improve the speed;
Standard practice for any production Couchbase cluster is to place the application (i.e. client) on their own servers - this allows you to scale the app layer independently of the database layer.
Therefore you would be not be running Couchbase and the application on the same machine, and hence unix domain sockets (even if they were supported) wouldn't be of any use.
Related
I'm making a nodejs application that will act a server for other sites in different countries as the data being transmitted will be business related data. I would like to know how I can safely/securely send this data.
I am currently using socket.io to act as my main server (Master) on other sites there are (Slave) servers that handle the data from the master server.
I have got this working in a local environment but want to deploy this in the other sites.
I have tried to Google this to see if anyone else has done this but came across socket.io sessions but I don't know if this will fit with (Server->Server) connections.
Any help or experience would be grateful.
For server-server communication where you control both ends of the communication you can use WebSocket over HTTPS, you can use TCP over SSH tunnel or any other encrypted tunnel. You can use a PubSub service, a queue service etc. There are a lot of ways you can do it. Just make sure that the communication is encrypted either natively by the protocols you use or with VPN or tunnels that connect your servers in remote locations.
Socket.io is usually used as a replacement for WebSocket where there is no native support in the browser. It is rarely used for server to server communication. See this answer for more details:
Differences between socket.io and websockets
If you want a higher level framework with focus on real-time data then see ActionHero:
https://www.actionherojs.com/
For other options of sending real-time data between servers you can use some shared resource like a Redis database or some pub/sub service like Faye or Kafka, or a queue service like ZeroMQ or RabbitMQ. This is what is usually done to make things like that work across multiple instances of the server or multiple locations. You could also use a CouchDB changes feed, or a similar feature of RethinkDB to make sure that all of your instances get all the data as soon as it is posted by any one of them. See:
http://docs.couchdb.org/en/2.0.0/api/database/changes.html
https://rethinkdb.com/docs/changefeeds/javascript/
https://redis.io/topics/pubsub
https://faye.jcoglan.com/
https://kafka.apache.org/
Everything that uses HTTP is easy to encrypt with HTTPS. Everything else can be encrypted with a tunnel or VPN.
Good tools that can add encryption for protocols that are not encrypted themselves (like e.g. the Redis protocol) are:
http://www.tarsnap.com/spiped.html
https://www.stunnel.org/index.html
https://openvpn.net/
https://forwardhq.com/help/ssh-tunneling-how-to
See also:
https://en.wikipedia.org/wiki/Tunneling_protocol
Note that some hosting services may give you preconfigured tunnels or internal network interfaces that pass data encrypted between your servers located in different data centers of that provider. Some providers give you tools and tutorials to that easily as well.
I'm working in a project where we need to connect clients to devices behind LAN networks.
Brief description: there are "devices" connected, in a home for example, under a LAN created by a router. These devices create a full webserver, operating under linux, and using nodejs as the backend implementation language. They also have access to Internet, through the public IP of the router. On the other side, there are clients which can choose to which device to connect to.
The goal is to connect the clients with the webServer created by any device.
Up to now, my idea is to try to implement something similar to how TeamViewer works. As I understand, Teamviewer has a central server, which the agents connect to. When an agent connects to the central server, this one gets hold of the TCP connection, keeping it alive. When another client wants to access to the first client, the server bypasses both TCP connections. That way the server acts like a proxy, where it additionally routes the TCP connections. This also allows to connect to clients under LAN or firewalls (because the connections are created always from the clients).
If this is correct, what I would like to implement is a central server, in nodejs as well, which manages a pool of socket connections coming from the different active devices, and when a client wants to connect to one specific device, the server bypasses the incoming TCP connection of the client with the already existing connection of the device.
What I first would like to know is if this is possible in nodejs. My idea is to keep the device connections alive, so clients can inmediately connect to them, creating some sort of pool of device connections.
If implemented in C, I guess I could get hold of the socket descriptor, keeping it alive, and bypassing it to the incoming client request. But in nodejs I can't seem to find any modules that manage TCP connections.
Are there any high level npm packages which do this function? Else, is it possible to use lower level modules (like net) which have those functionalities.
Ideally I would like to implement it with high level modules (express), but if it's not possible, I could always rewrite the server using low level modules.
Thanks in advance
I have spun 3 node instances using pm2. They are all running a websocket server using these ports: (9300, 9301, and 9302).
My main server acts as a nginx load balancer. The nginx upstream block:
upstream websocket {
least_conn;
server 127.0.0.1:9300;
server 127.0.0.1:9301;
server 127.0.0.1:9302;
}
After 10 players have connected, they are distributed in round-robin fashion. I am also utilizing Redis for Pub/Sub for all the node instances.
I am curious if it's possible for a connected player that is on instance 9300, switch to 9302 while not losing their connection?
The reasoning is because my game is instance based. I have "games" if you will, that players can create or join. If I can get the connected players onto the same node instance for their games, I would reduce all the extra Pub/Sub signals and achieve better latency. (Or so I think, but just curious if this is possible)
I am curious if it's possible for a connected player that is on
instance 9300, switch to 9302 while not losing their connection?
No, it is not possible. A TCP socket is a connection between two specific endpoints and it cannot be moved from one endpoint to another after it is established. There are very good security reasons why this is prohibited (so connections can't be hijaacked).
The usual way around this problem is for the server to tell the client to reconnect and give it instructions for how to connect to the particular server you want it connected to (e.g. connect to a specific port or specific hostname or some other means that your load balancer might use).
We have a node.js server that is primarily used with socket.io for browser inter connectivity in a web application.
We want to have a high availability solution which would theoretically consist of two node.js servers, one as a primary server and the other as a backup should the primary fail. The solution would allow that if or when the primary node.js server goes down the backup would take over to provide seamless functionality without interruption.
Is there a solution that allows socket.io to maintain the array of client connections over multiple servers without duplication of clients or of messages sent?
Is there another paradigm we should be considering for HA and node.js?
There is no way to have a webSocket auto fail-over without any interruption to a new server when the one it is currently connected to goes down. The webSockets that were connected to the server that went down will die. That's just how TCP sockets work.
Fortunately with socket.io, the client will quickly realize that the connection has been lost (within seconds) and the clients will try to reconnect fairly quickly. If your backup server is immediately in place (e.g. hot standby) to handle the incoming socket.io connections, then the reconnect will be fairly seamless from the client point of view. It will appear to just be a momentary network interruption from the client's point of view.
On the server, however, you need to not only have a backup, but you have to be able to restore any state that was present for each connection. If the connections are just pipes for delivering notifications and are stateless, then this is fairly easy since your backup server that receives the reconnects will immediately be in business.
If your socket.io connections are stateful on the server-side, then you will need a way to restore/access that state when the backup server takes over. One way of doing this is by keeping the state in a redis server that is separate from your web server (though you will then need a backup/high availability plan for the redis server too).
SocketIO in primary and backup server can be connected to a redis server. This will maintain the sessions in the primary server and can be used by backup server, The clients should once again connect to the new server(when primary fails).
SoketIO- Redis
HA - proxy is used for load balancing between multiple node.js instances. The usage of HA proxy will depend on how you are going to deal with failure of primary server. If you have any method to automatically switch primary server, then HA-proxy will not be much useful, else you can configure HA-Proxy to forward request to backup server if the primary server is unreachable.
Other options similar to HA-Proxy are:
node-http-proxy
Nginx
There will be no human being in the loop, and both endpoints are autonomous Node.js applications operating as independent services.
Endpoint A is responsible for contacting Endpoint B via secure web socket, and maintaining that connection 24/7/365.
Both endpoints will initiate messages independently (without human intervention), and both endpoints will have an API (RESTful or otherwise) to receive and process messages. You might say that each endpoint is both a client of, and a server to, the other endpoint.
I am considering frameworks like Sails.js and LoopBack (implemented on both endpoints), as well as simply passing JSON messages over ws, but remain unclear what the most idiomatic approach would be.
Web Sockets have a lot of overhead for connecting to browsers and what not, since they try to remain compatible with HTTP. If you're just connecting a pair of servers, a simple TCP connection will suffice. You can use the net module for this.
Now, once you have that connection, how do you initiate communication? You could go through the trouble of making your own protocol, but I don't recommend it. I found that a simple RPC was easiest. You can use the rpc-stream package over any duplex stream (including your TCP socket).
For my own application, I actually installed socket.io-client and let my servers use it for RPC. Although if I were to do it again, I would use rpc-stream to skip all the overhead required for setting up a Web Socket connection.