I've been reading on packets a lot today. I was confused for sometime because smtp, http, or ftp, for example, are all called protocols. But that they also somehow utilize transport protocols like TCP. I couldn't locate them on the packet 4 layers. Until I just discovered they're simply part of the application layer.
I want to know what exactly these "protocols" offer. I'm guessing a specific format for the data which applications on the client side know how to handle? If so, does this mean that realistically, I might have to create my own "protocols" if I created an application with a unique functionality?
A protocol, in this case, is just a structured way of communicating between two or multiple parties.
If you write, for example, a PHP-App and offer an API, you created a protocol to interact with your program. It defines how others interact with it and what response they can expect while doing so. Your self-created protocol depends on others, like the HTTP and TCP.
I suggest watching following video of LiveOverflow, explaining exactly this:
https://www.youtube.com/watch?v=d-zn-wv4Di8&ab_channel=LiveOverflow
I want to know what exactly these "protocols" offer.
You can read the definition of each protocol, if you really want to
Related
I'm writing a application in NodeJs where a client sends udp messages to a server with udp. I'm trying to find out how people normally handle different message types in NodeJs but can only find tons of examples of echo servers where the kind of message is not relevant.
The only example I have found so far is https://github.com/vbo/node-webkit-mp-game-template/tree/networking_1/networking
Maybe the best way is to send the udp messages as json?
User Datagram Protocol (UDP) is a network protocol and mechanism for sending short messages from one host to another without any guarantee of delivery. What you put in the message is entirely up to you.
While JSON can be used to encode your message, it suffers from two problems: it is not secure and is self-describing.
The first problem means that bad actors can easily see the content of your message while in flight and the second implies a substantial overhead for any message above and beyond its intended purpose.
Depending on your needs, a better choice might be to define your own binary protocol specific to your purpose using a node Buffer.
Another might be to use a more compact interchange format like thrift.
I want to connect to Unix Domain Socket created by ZeroMQ (IPC model) via command nc. I can connect, but when I sending some messages then, my deamon, which is listening to this socket, is not getting any message...
I'm using nc like:
nc -U /path/to/socket
Very well, here's a longer version.
ZeroMQ implements a message queue transport system over the top of stream connections like sockets, named pipes, etc. To do this it runs a protocol called ZMTP over the top of the stream, which provides all the message demarcation, communication patterns, and so forth. It also has to deal with protocol errors in order to give itself some resiliency.
Comparison to a Web Browser
It's the same idea to a web browser and web server communicating using http over a socket. Http is used to transport html files. If you look at the data flowing over the socket you see the html mixed up with the messages involved in running the http protocol. And because http is a text based protocol, it looks kinda OK to the human eye.
Talking the Same Language
Thus when a program that uses the zmq libraries for communication connects a socket / named pipe / etc, it will be expecting to exchange data over that connection in the way defined by the ZMTP protocol (in the same way a web browser is expecting to talk to a server using http). If the program at the other end is also using zmq, then they're both talking the same protocol and everything is good.
Incompatible Protocols
However, if you connect a program that doesn't of itself use the ZMTP protocol such as a web browser, and that sends a http request, it's unlikely to mean anything. And the zmq library routines will no doubt receive the bytes that make up the http request, attempt to interpret it, fail to understand it, and ultimately reject it as garbage.
Similarly if the program that uses the zmq library wants to send messages, nothing will happen unless the underlying ZMTP protocol driver is content that it is communicating with something else that talks ZMTP. If anything at all emerges from netcap, it won't look anything like the message you were sending (it'll be jumbled up with the bytes that ZMTP uses).
Human Equivalent
The equivalent is an Englishman called Bob picking up the phone and dialling the number for his English friend called Alice living in Paris. However, if a Frenchman called Charlie answers the phone by mistake (wrong number), it'll be very difficult for them to exchange information. Meanwhile Eve, who's tapped the phone line, is laughing her head off at the ineptitude of these two people's failed attempt to communicate. (I make sweeping and partly justifiable generalisations about us Englishmen's poor ability to speak any other language).
Way Forward
There's a ZMQ binding available for almost everything, possibly even bash. Whatever it is you're trying to accomplish it's probably well worth while getting a decent binding of ZMQ for the programming or scripting language your using, and use that to provide a proper ZMQ endpoint.
I am reading about TCP connections for past few days ,I came across NET as a native nodejs library and socket.io ..can anyone suggest which one will be better with pros and cons of both
socket.io is a specific message based protocol built on top of TCP.
If you want to send messages where you define a message name and send a payload for the message and the other side listens for a specific set of message names and you have a socket.io library already implemented for the other end of your connection, then socket.io will work great and will be a lot simpler to use and offer more ready-made capabilities (such as auto-reconnect).
If you intend to implement your own protocol, then you will use TCP in order to implement your own protocol. If the type of data you are sending is not really message-based (such as audio/video streaming or large file uploads as a couple examples), then you will want to either use TCP or use some other protocol that is also built on top of TCP (such as HTTP, FTP, etc...).
As with any feature in a library, define your requirements, understand the options available in your system and find the solution that best matches your requirements. Since you have said absolutely nothing about your requirements, we cannot make a specific recommendation.
Most examples I have seen are are just small demo's and not full stack applications and they use websocket for messaging, but for any chat application there is more data then just messages...suppose like user profile, his contacts etc.
So should I use websockets for all communication between server and client or just use them for sending messages and do other things through http? If I am to use websocket for all communication how do url design of the app...since websockets don't have have any different urls like http.
You might be interested in WAMP, an officially registered WebSocket subprotocol that provides applications with WebSocket based
asynchronous, bidirectional remote procedure calls
real-time publish & subscribe notifications
Disclaimer: I am original author of WAMP and work for Tavendo.
Pretty sure you'll get the usual "it depends" answer, because, well, it depends!
If you are going to build a large application, to be used by a number of different clients in different network arrangements etc then I personally wouldn't recommend using WebSockets for everything. Why?
It's a new standard, so not all clients support it
In some network configurations WebSocket traffic may be filtered out, meaning you end up with no communications - not great
If you end up exposing an external API then HTTP is much better fitted for the job and will likely be easier to code against. There are a lot more tools out there to help you with it and styles that everyone is familiar with, like REST, to follow.
Use WebSockets when you require data being pushed from the server without the client having to poll for it, or when HTTP header overhead becomes a problem. And if you still decide to use it make sure you have a fallback mechanism (e.g. longpolling) so you don't end up with no comms.
I'm afraid I can't help you regarding WebSocket API design... given it's a new standard I don't believe the community has settled on anything just yet so you'll have to come out with your own message-based scheme.
I am working on an application that will send OSC control messages, which is, as I understand a datagram packet, from a web page to an OSC Receiver (server), such as Max/MSP or Node or any other.
I know typically UDP is used because speed is important in the realtime/ audio visual control work done with OSC (which is also the work I will be doing), but I know other methods can be used.
Right now for instance, I am sending OSC from the browser to a node.js server (using socket.io), and then from the node.js server to Max (which is where I ultimately want the data), also using socket.io. I believe this means I am using websockets and the delay/latency has not been bad.
I am curious though, now that WebRTC is out, if I should place the future of my work there. In all of my work with OSC I have always used UDP, and only used the Socket.io/Websockets connection because I didn't know about WebRTC.
Any advice on what I should do. Specifically I am interested in
1. How I could send OSC messages from the browser directly to an OSC server (as opposed to going through a node server first)
2. If I should stay with the Node/Socket.io/Websocket method for sending OSC data or should I look into WebRTC?
Regarding your first question - if there is a solution for a direkt Websocket link between browser and server (for OSC) - you can have a look into this:
ol.wsserver object for Max/MSP by Oli Larkin: https://github.com/olilarkin/wsserver
I published an osc-js utility library which does the same with a UDP/Websocket bridge plugin:
https://github.com/adzialocha/osc-js
This is not really a question about any specific technical challenge, but a discovery challenge. And is pretty much opinion based.
But I will try to provide my thoughts, as others can be useful too.
If OSC server support WebSockets or WebRTC connection, then you might use one of them directly to it.
Based on nature of browsers, you probably need to support many protocols rather than one specific. As many browsers have different support: http://caniuse.com/rtcpeerconnection unless your users can be "forced" to use specific browser.
WebRTC is meant to be high-performance. I'm not sure it is good to compare it with UDP at all, as it is much more than UDP in comparison as well as implementation vary (inside) per browser.
You need to talk to server rather than between clients, while WebRTC mainly is designed for peer-to-peer communications. So you need precisely investigate into benefits on performance from latency as well as CPU point of view on server side.
Benefit of UDP is the fact that some packets can be dropped/skipped/undelivered, as it is not "mandatory" data, like picture frames or piece of sound, that only results on reduced quality, which is "expected" behaviour in streaming world. WebSockets will ensure that data is delivered, and "price" for it is mechanics that at the end slows down the queue of packets, to ensure ordered and guaranteed delivery of data.