I'm making a nodejs application that will act a server for other sites in different countries as the data being transmitted will be business related data. I would like to know how I can safely/securely send this data.
I am currently using socket.io to act as my main server (Master) on other sites there are (Slave) servers that handle the data from the master server.
I have got this working in a local environment but want to deploy this in the other sites.
I have tried to Google this to see if anyone else has done this but came across socket.io sessions but I don't know if this will fit with (Server->Server) connections.
Any help or experience would be grateful.
For server-server communication where you control both ends of the communication you can use WebSocket over HTTPS, you can use TCP over SSH tunnel or any other encrypted tunnel. You can use a PubSub service, a queue service etc. There are a lot of ways you can do it. Just make sure that the communication is encrypted either natively by the protocols you use or with VPN or tunnels that connect your servers in remote locations.
Socket.io is usually used as a replacement for WebSocket where there is no native support in the browser. It is rarely used for server to server communication. See this answer for more details:
Differences between socket.io and websockets
If you want a higher level framework with focus on real-time data then see ActionHero:
https://www.actionherojs.com/
For other options of sending real-time data between servers you can use some shared resource like a Redis database or some pub/sub service like Faye or Kafka, or a queue service like ZeroMQ or RabbitMQ. This is what is usually done to make things like that work across multiple instances of the server or multiple locations. You could also use a CouchDB changes feed, or a similar feature of RethinkDB to make sure that all of your instances get all the data as soon as it is posted by any one of them. See:
http://docs.couchdb.org/en/2.0.0/api/database/changes.html
https://rethinkdb.com/docs/changefeeds/javascript/
https://redis.io/topics/pubsub
https://faye.jcoglan.com/
https://kafka.apache.org/
Everything that uses HTTP is easy to encrypt with HTTPS. Everything else can be encrypted with a tunnel or VPN.
Good tools that can add encryption for protocols that are not encrypted themselves (like e.g. the Redis protocol) are:
http://www.tarsnap.com/spiped.html
https://www.stunnel.org/index.html
https://openvpn.net/
https://forwardhq.com/help/ssh-tunneling-how-to
See also:
https://en.wikipedia.org/wiki/Tunneling_protocol
Note that some hosting services may give you preconfigured tunnels or internal network interfaces that pass data encrypted between your servers located in different data centers of that provider. Some providers give you tools and tutorials to that easily as well.
Related
I'm working on a data sharing app using node js, the idea is to create an application that can connect peers using node-js without using any kind of central dependency i.e a signalling server or something of its kind. After a lot of research I'm always back to this diagram but it only makes sense if there is no signalling server I'm targeting LAN networks so that I don't have to deal with NAT.
To be specific, I would like some to answer these specific question
Is it possible to connect to webrtc on LAN, i.e the webrtc connection will connect using my client's ip
Is it possible to use websockets without a central server, or something like websockets
Is it possible to connect two clients on LAN using node js without using any hardcoded Ips or asking users to enter ip.
Since you are using node.js, you can very well use raw UDP (dgram) and use UDP broadcasting for device discovery - then you do not need any form of centralization required by websockets/webRTC.
The answer to all your questions is Yes!
Also, there is a lof WebRTC server that you can use them on a simple Linux box, like Janus, Kurento, etc. I've tested them and they have worked with some mischief, lol. So, run it and next read their API to exchange anything you want on their medium.
I'm not sure about Janus but the Kurento has a nodeJS client itself. Read the Local Installation and JavaScript Kurento Client.
Additionally, if you want to make a WebSocket connection, it has enabled by default configuration.
To change the port, enter this command at the final step:
npm start -- --ws_uri=ws://https://185.164.72.144/:8888/kurento --as_uri=https://185.164.72.144:6008/
My team and I are working on a digital signage platform.
We have ~ 2000 Raspberry Pi around the world connected to a Nodejs server using Socket IO. The Raspberries are initiating the connection.
We would like to be able to scale horizontally our application on multiple servers but we have a problem that we can’t figure out.
Basically, the application stores the sockets of the connected Raspberry in an array.
We have an external program that calls the API within the server, this results by the server searching which sockets will be "impacted" by the API call and send them the informations.
After lots of search, we assume that we have to stores the sockets (or their ID) elsewhere (Redis ?), to make the application stateless. Then, any server can respond to a API call and look the sockets in a central place.
Unfortunately, we can’t find any detailed example on how to do that.
Can you please help us ?
Thanks
(You can't store sockets from multiple server instances in a shared datastore like redis: they only make sense in the context of the server where they were initiated).
You will need a cluster of node.js servers to handle this. There are various ways to make a cluster. They all involve directing incoming connections from your RPis to a "generic" hostname, for example server.example.com. Behind that server.example.com hostname will be multiple node.js servers.
Each incoming connection from each RPi connects to just one of those multiple servers. (You know this, I believe.) This means one node.js server in your cluster "owns" each individual RPi.
(Telling you how to rig up a cluster of node.js servers is beyond the scope of this answer. Hints: round-robin DNS or a reverse-proxy nginx front end.)
Then, you want to route -- to fan out -- the incoming data from each API call to each server in the cluster, so the server can route it to the RPis it owns.
Here's a good way to handle that:
Set up a redis cache or other shared data store. It can be very small.
When each node.js server starts, have it register itself as active. That is, have it place its own specific address for handling API calls into the shared server. The specific address is probably of the form 12.34.56.78:3000: that is, an IP address and port.
Have each server update that address every so often, once a minute or so, to show it is still alive.
When an API call arrives at server.example.com, it will come to a more-or-less randomly chosen node.js server instance.
Get that server to read the list of server addresses from the redis cache
Get that server to repeat the API call to all servers except itself. Add a parameter like repeated=yes to the repeated API calls.
Then, each server looks at its list of connected sockets and does what your application requires.
On server shutdown, have the server unregister itself -- remove its address from redis -- if possible.
In other words, build a way of fanning out the API calls to all active node.js servers in your cluster.
If this must scale up to a very large number (more than a hundred or so) node.js servers, or to many hundreds of API calls a minute, you probably should investigate using message queuing software.
SECURE YOUR REDIS server from random cybercreeps on the internet.
There will be no human being in the loop, and both endpoints are autonomous Node.js applications operating as independent services.
Endpoint A is responsible for contacting Endpoint B via secure web socket, and maintaining that connection 24/7/365.
Both endpoints will initiate messages independently (without human intervention), and both endpoints will have an API (RESTful or otherwise) to receive and process messages. You might say that each endpoint is both a client of, and a server to, the other endpoint.
I am considering frameworks like Sails.js and LoopBack (implemented on both endpoints), as well as simply passing JSON messages over ws, but remain unclear what the most idiomatic approach would be.
Web Sockets have a lot of overhead for connecting to browsers and what not, since they try to remain compatible with HTTP. If you're just connecting a pair of servers, a simple TCP connection will suffice. You can use the net module for this.
Now, once you have that connection, how do you initiate communication? You could go through the trouble of making your own protocol, but I don't recommend it. I found that a simple RPC was easiest. You can use the rpc-stream package over any duplex stream (including your TCP socket).
For my own application, I actually installed socket.io-client and let my servers use it for RPC. Although if I were to do it again, I would use rpc-stream to skip all the overhead required for setting up a Web Socket connection.
I'm developing chat application using app.js which is webkit+node.js framework.
So i have node.js plus bridged web browser environment on both sides.
I want to make file transfer feature somewhat similar to Skype one.
So, initial idea is to:
1.connect clients to main server.
2.Each client gets ip of oposite ones.
3.Start socket or websocket server on both clients and connect to each other.
4.Sender reads the file and transmits it to the reciver.
Question are:
1.Im not really sure that one client can "see" the other.
2.file is a binary data, but websockets are made for text messages so i need some kind of coding/decoding stuff. I thought about base 64 but it has 30% of "overhead" information. So i need something more effitient (base 128?).
3.If it is not efficient to use websocket should i use TCP sockets instead? What problems can appear if i decide to use them?
Yeah i know about node2node and BinaryJS, i just dont know should i use them or not. And i really what to do something myself.
OK, with your communication looking like this:
(C->N)<->N<->(N->C)
(...) is installed on one client's machine. N's are node servers, C's are web clients.
This is out of your control. Some file sharing apps send test packets from the central server to clients, to check whether ports are open and NAT rules are configured correctly, etc. Your clients will start their own servers on some port, your master server can potentially create a test connection to these servers to see whether they're started correctly and open to the web, BEFORE telling other clients that they can send files.
Websockets are great for status messages from your servers to the web GUIs and general client-to-client communication. For the actual file transfers, I would use TCP sockets, see the next answer. On the other hand base64 encoding is really not a slow process, play with it and benchmark its performance, then decide with some data to back up your decision.
You could use a combination: websockets from your servers to the web GUIs, but TCP communication between the servers themselves. TCP servers (and streams) aren't hard to set up in Node, I see no disadvantages. It might actually be less complicated than installing node2node on those servers, since TCP is already built-in.
Well
It's not a big question, obviously.
But you see, an application that is using a database on the servers, and is installed on multiple clients. Is called Client/Server application.
And an application that is constituted by two parts: Host (or server) part, and the client part.
They are both called client/server apps
How can we distinguish them, and what's the proper name of each type.
P.S. I'm talking about English, you know. I want to say that this application is Client/Server(only database on server) and not a Client/Server(as real client and server).
As others have said, they're both client/server applications. If you want to be more explicit about what kind of client and what kind of server, you can use adjectives or nouns. Please don't go inventing new terms when we've already got plenty of terms that can be combined - computer science & software engineering are already overloaded with unnecessary terms.
web client/web server (note that the web includes more protocols than just HTTP)
http client/http server
ftp client/ftp server
soap client/soap server
client/application server (the app server may include a db, or the appserver may be the db client)
database client/database server
JDBC client/database server
sqlplus/oracle
application server/database server
I don't think I could say it any better than Wikipedia:
Client-server computing or networking
is a distributed application
architecture that partitions tasks or
work loads between service providers
(servers) and service requesters,
called clients. Often clients and
servers operate over a computer
network on separate hardware. A server
machine is a high-performance host
that is running one or more server
programs which share its resources
with clients. A client does not share
any of its resources, but requests a
server's content or service function.
Clients therefore initiate
communication sessions with servers
which await (listen to) incoming
requests.
I am not sure what you are asking in the second part of your question - if both architectures involve clients making requests to servers, then they are both client-server architectures. On what basis would you prefer to distinguish them?
It has to do with which side the request comes from (amongst other things): the Client issues a "request" and the Server responds.
The terminology Client-Server is often associated with a discipline called "System Design Engineering" (there are other attributions for sure but that's one).
Of course, an Agent could be composed of different sub-systems, each having different roles (i.e. S1 has a Client role whilst S2 has a Server role).
One should note that the terminology Client-Server can also be found outside of Computer Sciences. A good example is in the networking field: A Client protocol layer interfaces with a Server protocol layer e.g. IP (Client) uses the services of an Ethernet protocol layer (the Server in this case).