I'm developing a set of gRPC microservices using rust and tonic library. One of those microservices is behind a firewall that does not allow any inbound connections. I'd like to know if it's still possible for this service to act as a server for unary grpc calls. I was thinking of rearranging parts of hyper/tower stack to force the server to connect to client and then proceed with regular gRPC communication but I'm not sure where to start (if at all possible). In my research I also came across yamux library, is this something I could use for this purpose?
Related
I'm working on a data sharing app using node js, the idea is to create an application that can connect peers using node-js without using any kind of central dependency i.e a signalling server or something of its kind. After a lot of research I'm always back to this diagram but it only makes sense if there is no signalling server I'm targeting LAN networks so that I don't have to deal with NAT.
To be specific, I would like some to answer these specific question
Is it possible to connect to webrtc on LAN, i.e the webrtc connection will connect using my client's ip
Is it possible to use websockets without a central server, or something like websockets
Is it possible to connect two clients on LAN using node js without using any hardcoded Ips or asking users to enter ip.
Since you are using node.js, you can very well use raw UDP (dgram) and use UDP broadcasting for device discovery - then you do not need any form of centralization required by websockets/webRTC.
The answer to all your questions is Yes!
Also, there is a lof WebRTC server that you can use them on a simple Linux box, like Janus, Kurento, etc. I've tested them and they have worked with some mischief, lol. So, run it and next read their API to exchange anything you want on their medium.
I'm not sure about Janus but the Kurento has a nodeJS client itself. Read the Local Installation and JavaScript Kurento Client.
Additionally, if you want to make a WebSocket connection, it has enabled by default configuration.
To change the port, enter this command at the final step:
npm start -- --ws_uri=ws://https://185.164.72.144/:8888/kurento --as_uri=https://185.164.72.144:6008/
Suppose I have 2 servers. I want to establish a connection between them using socket.io library, and one of them would reach another by .onion link using SOCKS proxy. all the traffic between servers should also go through tor.
I was able to create a simple net socket using onion links, but cannot find how to pass it to socket.io. Is it possible to make socket.io using existing net socket? or maybe there is another way to achieve the ultimate goal?
I have some working experiences with socket.io, but not with the other technologies you mention (so I can't resolve your question entirely). Until someone else resolves it, I will show you a link with about the two modules socket.io-redis and socket.io-emitter that allow socket.io to communicate with the "outside world":
https://socket.io/docs/rooms-and-namespaces/#sending-messages-from-the-outside-world
I'm making a nodejs application that will act a server for other sites in different countries as the data being transmitted will be business related data. I would like to know how I can safely/securely send this data.
I am currently using socket.io to act as my main server (Master) on other sites there are (Slave) servers that handle the data from the master server.
I have got this working in a local environment but want to deploy this in the other sites.
I have tried to Google this to see if anyone else has done this but came across socket.io sessions but I don't know if this will fit with (Server->Server) connections.
Any help or experience would be grateful.
For server-server communication where you control both ends of the communication you can use WebSocket over HTTPS, you can use TCP over SSH tunnel or any other encrypted tunnel. You can use a PubSub service, a queue service etc. There are a lot of ways you can do it. Just make sure that the communication is encrypted either natively by the protocols you use or with VPN or tunnels that connect your servers in remote locations.
Socket.io is usually used as a replacement for WebSocket where there is no native support in the browser. It is rarely used for server to server communication. See this answer for more details:
Differences between socket.io and websockets
If you want a higher level framework with focus on real-time data then see ActionHero:
https://www.actionherojs.com/
For other options of sending real-time data between servers you can use some shared resource like a Redis database or some pub/sub service like Faye or Kafka, or a queue service like ZeroMQ or RabbitMQ. This is what is usually done to make things like that work across multiple instances of the server or multiple locations. You could also use a CouchDB changes feed, or a similar feature of RethinkDB to make sure that all of your instances get all the data as soon as it is posted by any one of them. See:
http://docs.couchdb.org/en/2.0.0/api/database/changes.html
https://rethinkdb.com/docs/changefeeds/javascript/
https://redis.io/topics/pubsub
https://faye.jcoglan.com/
https://kafka.apache.org/
Everything that uses HTTP is easy to encrypt with HTTPS. Everything else can be encrypted with a tunnel or VPN.
Good tools that can add encryption for protocols that are not encrypted themselves (like e.g. the Redis protocol) are:
http://www.tarsnap.com/spiped.html
https://www.stunnel.org/index.html
https://openvpn.net/
https://forwardhq.com/help/ssh-tunneling-how-to
See also:
https://en.wikipedia.org/wiki/Tunneling_protocol
Note that some hosting services may give you preconfigured tunnels or internal network interfaces that pass data encrypted between your servers located in different data centers of that provider. Some providers give you tools and tutorials to that easily as well.
I am having trouble understanding the difference between net.createserver and http.createserver in node.js.
I have read the documentation for both methods located at these two urls
https://nodejs.org/api/net.html#/net_net,
https://nodejs.org/api/http.html#/http_class_http_server.
I understand that http.createserver creates an http server. However, the documentation says that net.createserver creates a tcp server. I understand that tcp is the transmission protocol that http is on top of and that http servers are set up to read http request headers. I also understand the concept of even emitters in node.js pretty well. However, I don't understand this notion of a tcp server and why one would be made in node.js. The context is I am coding a chat application example in the "node.js in action" book.
http.createServer() sets up a server that handles the HTTP protocol, which is indeed transmitted over tcp. net.createServer() creates a server that simply understands when a TCP connection has happened, and data has been transmitted, and so on, but doesn't know anything about whether a valid HTTP request has been received, etc.
If you are writing a web server, favor http.createServer() over net.createServer() as it will save you a lot of work. If you are writing some other kind of server, do not use http.createServer().
I don't know much of a Node.js, but I know something about networks. HTTP is a protocol that works on 7th (Application) layer of model OSI. TCP is protocol that works on 4th (Transport) layer of model OSI. As you said, yes HTTP works on top of the TCP. The option of creating HTTP server by http.createServer() is there so you don't have to implement it by yourself by using net.createServer(). The protocol TCP might by used by lot of applications, you might create your own, or implement some different protocol than HTTP, for example: FTP, DNS, SMTP, Telnet and much much more.
Straight from the Node Net documentation. NET is the basic bare-bones server you can create. It's particularly useful for setting up a cluster of servers and allows simple connections but on that you'll want communication protocols, namely HTTP, which HTTP is in fact a NET server at it's core.
The net module provides an asynchronous network API for creating stream-based TCP or IPC servers (net.createServer()) and clients (net.createConnection()).
And from the HTTP documentation. HTTP is the common way to transmit large sets of data as requested by the client and then a response is generated. It's the standard way of communicating over the internet and introduces the concept of handshakes and is done through REST protocol, you know the usual request and response way of communicating.
The HTTP interfaces in Node.js are designed to support many features of the protocol which have been traditionally difficult to use. In particular, large, possibly chunk-encoded, messages. The interface is careful to never buffer entire requests or responses — the user is able to stream data.
Websockets are an upgrade over the HTTP headers and offer low latency and less server load and are a much more minimal conversation. If you're talking peer to peer communication, that's the way you'll want to go.
There will be no human being in the loop, and both endpoints are autonomous Node.js applications operating as independent services.
Endpoint A is responsible for contacting Endpoint B via secure web socket, and maintaining that connection 24/7/365.
Both endpoints will initiate messages independently (without human intervention), and both endpoints will have an API (RESTful or otherwise) to receive and process messages. You might say that each endpoint is both a client of, and a server to, the other endpoint.
I am considering frameworks like Sails.js and LoopBack (implemented on both endpoints), as well as simply passing JSON messages over ws, but remain unclear what the most idiomatic approach would be.
Web Sockets have a lot of overhead for connecting to browsers and what not, since they try to remain compatible with HTTP. If you're just connecting a pair of servers, a simple TCP connection will suffice. You can use the net module for this.
Now, once you have that connection, how do you initiate communication? You could go through the trouble of making your own protocol, but I don't recommend it. I found that a simple RPC was easiest. You can use the rpc-stream package over any duplex stream (including your TCP socket).
For my own application, I actually installed socket.io-client and let my servers use it for RPC. Although if I were to do it again, I would use rpc-stream to skip all the overhead required for setting up a Web Socket connection.