I'm developing a system to control a range of IoT devices. Each set of devices is grouped into a "system" that monitors/controls a real-world process. For example system A may be managing process A and have:
3 cameras
1 accelerometer
1 magnetometer
5 thermocouples
The webserver maintains socket connections to each device. Users can connect (via a UI - again with WebSockets) to the webserver and receive updates about systems to which they are subscribed.
When a user wants to begin process A, they should press a 'start' button on the interface. This will start up the cameras, accelerometer, magnetometer, and thermocouples. These will begin sending data to the server. It also triggers the server to set the recording mode to true for each device, which means the server will write output to a database. My question:
Should I send a single 'start' request from javascript code in my UI to the server, and allow the server to start each device individually (how do I then handle an error, for example, if a single sensor isn't working - what about if two sensors don't work?). Or do I send individual requests from the UI to the server for each device, i.e. start camera 1, start camera 2, start accelerometer, start recording camera1, etc. and handle each success/error state individually?
My preference throughout the system so far has been the latter approach - one request, one response; with an HTTP error code. However, programming becomes more complex when there are many devices to control, for example - System B has 12 thermocouples.
Some components of the system are not vital - e.g. if 1 camera fails we can continue, however, if the accelerometer fails the whole system cannot run and so human monitoring is required. If the server started the devices individually from a single 'start' message, should I return an array of errors, or should the server know which components are vital and return a single error if a vital component fails? And in a failure state, should the server then handle stopping each sensor and returning to the original state - and what if that then fails? I foresee this code becoming quite complex with this approach.
I've been going back and forth over the best way to approach this for months, but I can't find much advice online around building complex, production-ready IoT systems for the real world. If anybody has any advice or could point me towards any papers/books/etc. I would really appreciate it.
Thanks in advance,
Tom
I am writing a build script which has some password protected files (keys). I need a way to prompt the user once for the password and then use this key across multiple scripts. These scripts do not live inside the same shell, and may spawn other windows via dbus. I can then send them commands, one of which must have access to the password.
I have this working already, but at a few points the passphrase is either used directly on a command-line (passed via dbus), or is put into a file (the name then passed to the other script). Both of these are less secure than I want*. The command-line ends up in a history which may be stored in a file, as well as appearing in the process list, and the second option stores in a file which can be read by somebody else.
Is there some standard way to create a temporary communications channel between two processes which could communicate the password and not be intercepted by another user on the system (including root)?
*Note: This is primarily an exercise to be fully secure. For my current project the temporary in-file storage of the password is okay.
Setting "root being all-powerful" aside, I would imagine that a Private DBus Connection would do the trick although the documentation I could find seems a little light on what exactly makes a private connection private.
However, the DBus Specification, more specifically, the Message Bus Specification subsection on eavesdropping says in part:
Receiving a unicast message whose DESTINATION indicates a different
recipient is called eavesdropping. On a message bus which acts as a
security boundary (like the standard system bus), the security policy
should usually prevent eavesdropping, since unicast messages are
normally kept private and may contain security-sensitive information.
So you may not even need to use private connections which incur more overhead costs. But on a risk/reward basis with security being paramount, that may be the more secure alternative for you. Hope that helps.
I am using Fedora 18 with the avahi command line tools (version 0.6.31)
I use avahi-resolve-host-name to discover the IP address of units on my subnet, for testing purposes during development. I monitor the request and response with Wireshark. After one successful request and response, no further requests show up on Wireshark, but the tool still returns an IP address. Is it possible the computer/avahi daemon/something else is 'caching' the result?
The Question: I wish to send out the request packet with EVERY CALL of avahi-resolve-host-name. Is this possible?
The Reason: I'm getting 'false positives' so to speak. I try to resolve 'test1.local', and I am getting a resulting IP, but the unit is no longer located at this IP. I want the request sent every time so I can avoid seeing units at incorrect IP addresses.
I see that I'm a bit late to answer your question but I'm going to leave a generic answer in case someone else stumbles upon this.
My answer is based on avahi-0.6.32_rc.
Is it possible the computer/avahi daemon/something else is 'caching' the result?
Yes, avahi-daemon is caching lookup results. While this doesn't seem to be explicitly listed in features, the avahi-daemon(8) manpage tips it:
The daemon [...] provides two IPC APIs for local programs to make use of the mDNS record cache the avahi-daemon maintains.
I wish to send out the request packet with EVERY CALL of avahi-resolve-host-name. Is this possible?
Yes, it is. The relevant option is cache-entries-max (from avahi-daemon.conf(5)):
cache-entries-max= Takes an unsigned integer specifying how many resource records are cached per interface. Bigger values allow mDNS work correctly in large LANs but also increase memory consumption.
To achieve the desired effect, you can simply set:
cache-entries-max=0
This will disable the caching entirely and force avahi-daemon to reissue the MDNS packets on every request, therefore making it possible for you to monitor them.
However, I should note here that this will also render avahi pretty much useless for normal use. While avahi-daemon will be issuing lookup packets, it will be unable to store the results and every call of avahi-resolve-host-name (as well as other command-line tools, nss-mdns, D-Bus API…) will fail.
I just stumbled upon this problem myself and found solution that doesn't require changing the config. It seems that simply killing the daemon (avahi-daemon --kill) flushes the cache. I'm on Ubuntu 18.04 and the daemon is restarted automatically. If on some other distro it isn't running after being killed, it can be restarted with avahi-daemon --daemonize.
Note that root is needed to kill avahi daemon, so this might not be the best option in some cases.
I have been using JmDNSfor a while now. I could use it for the purposes of my application. Every thing works fine for me (I have "announcer" machines and a "listening" one, and this latter machine can see the other devices and discover their information).
It is true that I've managed to work with the JmDNS jar file, but I did it without totally understanding what is going on in this file. Now I want to know about the effect of using JmDNS for the network traffic. I have consulted the documentation but couldn't manage to discover the signification of the constants, like QUERY_WAIT_INTERVAL, PROBE_THROTTLE_COUNT, etc.
I want to know the default frequency with which the announcer machine sends service announcements.
I also noticed DNS_TTL that was described as follows: "The default TTL is set to 1 hour by the standard, so a record is going to stay in the cache of any listening machine for an hour without need to ping the server again".
I understand that it is the Time To Live of the service to stay in the DNS cache, but I couldn't understand what is intended by "purge the server". Does it mean that the listener has to ask the announcer about a service when the DNS_TTL expires? if so, why do need to have the announcer announce its service every 1s (ANNOUNCE_WAIT_INTERVAL = 1000 milliseconds)?
I am so confused.
The way that the Domain Name System works is basically very simple. Fundamentally it's a tree-like system which starts with the root nameservers. These then delegate name space out to the next level. That level in turn delegates out the next level and so on. For example . is the root, which delegates to .com., which can then delegate out example.com.. (Yes, that trailing . is actually part of the domain name, though you almost never have to use it or see it.
When you load a web page there are usually hundreds of elements that load. This is every image, every JS file, every CSS file, etc. To have your computer request that same domain to IP resolution that many times for one page would make load time unbearable and also create massive unnecessary traffic on the nameserver. Therefore DNS caches. The TTL is how long it caches for. If it's set to 24 hours then when you get an answer for that resolution, that's how long you can hold on to it for before you make another request.
The announcing that you're talking about is the nameserver basically announcing that it's responsible for those domains. You want it constantly stating that so other nameservers know where to go to get the correct (authoritative) data.
Throttling is a term used in many fields and applications and means you're limiting your traffic flow so it doesn't get overloaded.
DNS is actually quite simple to understand once you get the basics down.
Here are a few links that could help you get a better grip of it all:
Few paragraphs of basic DNS info
About.com guide
A few definitions
Relatively simple and informative PDF from IETF
Are there any known methods of finding peers without using a dedicated central server?
ie: If I have peers which are disconnecting and reconnecting to the internet but getting a new IP address each time, and I want to connect to them without setting up a dedicated server to register with.
I was thinking about using peers email address to send a manifest of connected peers periodically, with some sort of timecode, negating the need for a dedicated server. This would be a fallback if none of the peers could be connected to after trying all the previously known peer addresses. But existing models of finding peers would be preferable.
There's no way around having to know at least one initial peer to discover more.
Fully P2P protocols, such as Gnutella or Gnutella2, or the simpler Overnet (made famous by Storm Worm), are based on each client having a start-up list of a few peers. These can come off a web-based automated tracker for example. The client will discover the whole network or portions of it by asking other peers for more addresses, for example when delegating a file search.
If you truly can't have any kind of a centralized resource, the best you can do is find the first peer through broadcasted messages and ultimately IP address scanning. The first approach is well-meaning but in at least 98% of cases won't yield any results. The later approach, of course, is abusing the internet, as well as illegal in most countries.
I really would rethink having some kind of a central tracker. It can be something as simple as a PHP script on a webserver (the gnutella network, today, is held up by ten-twenty such scripts, hosted by people who don't even know each other). And this sure is more lightweight than email (which, due to spam filters at the very least, would not work anyway).
In the limited case of peers within an intranet, it is possible to send a broadcast UDP message to a known port asking for peers to report back.
The BitcoinQT client uses a variety of methods to find nodes, some of them might be useful to you.
Satoshi Client Node Discovery
IRC is no longer used, but might be the most easy to implement:
As of version 0.6.x the Bitcoin client no longer uses IRC bootstrapping by default, and as of version 0.8.2 support for IRC bootstrapping has been removed completely. This documentation below is accurate for most prior versions.
In addition to learning and sharing its own address, the node learned about other node addresses via an IRC channel. See irc.cpp.
After learning its own address, a node encoded its own address into a string to be used as a nickname. Then, it randomly joined an IRC channel named between #bitcoin00 and #bitcoin99. Then it issued a WHO command. The thread read the lines as they appeared in the channel and decoded the IP addresses of other nodes in the channel. It did this in a loop, forever, until the node was shutdown.
When the client discovered an address from IRC, it set the timestamp on the address to the current time, but it used a "penalty" of 51 minutes, which means it looked like it was actually seen almost an hour earlier.
Take advantage of any existing forum where data can posted. Think secret IRC channel, embedding data in photos and posting to photo sharing sites 4chan?, any site that would allow your application to login and post data without captia logins etc.
http://chatzilla.hacksrus.com/faq/#password
Another strategy might be to embedded messages in digital currency transactions. Pick a cheap coin that's likely to hang around ... DOGE or MOON coin maybe. Build wallet functionality into your app. such that you can post micro transactions back and forth between addresses that your app controls. There would still be a miners fee, but this is only fractions of pennies. Even if they later prohibit adding metadata to transactions, you could make a transaction equivalent to your IP address in MOON, and use vanity addresses in MOON coin for your app. such that when a new node comes online it knows what to search the blockchain for -- 2daMOON%bootStr#pM3. SEND - 104.003021133 MOON IP = 104.3.21.133 not an expensive proposition.
Old question but I've been thinking about this problem myself so will ad my 2-cents. In short, a central server is not required if a node is aware of at least one valid peer. New nodes must be added to the network by any current member (e.g. invited, or node spawns another node, depending on your application).
Assuming that:
agents keep track of peers; the size of this address book and how entries are managed will depend on the nature of the system; e.g. how long peers remain connected, if peers use stable addresses
agents share peer information with other peers
at least some agents remain available for relatively long periods of time relative to frequency node connects to network to update it's address book (or nodes have stable addresses)
in addition to peer addresses, availability information is also tracked (many options here depending on your system. examples include: whether peer has a stable address, when last seen, some availability metric, content/service type information, address valid-until time if known)
new agents are initialized with at least one valid peer (doesn't have to be a central node, can be any valid node)
trust mechanisms shall be required if malicious peers are a possibility
When a peer comes online, it queries the peers in it's peer table to discover which are active and perhaps removes expired dynamic addresses. Nodes exchange peer information and may become linked themselves. This peer discovery/exchange may continue a certain number of hops or via random walk until peer list if of sufficient size and/or quality.
A few more details:
Nodes connect and share peer information with frequency related to how often node addresses change, so address book doesn't become stale and node becomes disconnected because none of it's former peers are available at their last known addresses
Nodes may need to limit the number of peers they accept, to avoid tendency towards centralization around the most stable nodes.
Nodes should be selective about the peers they keep; i.e. ones in which they are more likely to exchange data (e.g. weight based upon history)
Node links may be asymmetric or symmetric depending on the application
Three ways, off the top of my head, though you're always going to need some central server to start the connection unless you went with option 3.
Central server that maintains known list of peers, with keep-alive.
One or more central servers that maintain some common resource peers can use to discover one another, but once connected no longer need the central server as long as the peer remains connected (something like BitTorrent); can chain peered connections as well.
Port/IP scanning (strongly not recommended).
In your example, you'd still have some kind of central server where the peers would be registered; the protocol is the only difference.
To put it simply no, there is no way to do this without a central sever.
If you want to do this you simply need one or more central servers, whether by dynamic dns or not. The clients need a method to discover where they should connect to, and the only truly sensible way to do this is with your own server, in the simplest scenario it only needs to send an IP address in response.
Virtual severs can be had for around $15/month, which IMO is considerably cheaper than trying to use or abuse someone else's bandwidth.
[Edit].
To put it simply, there is another way, as follows.
Upon reflection I think what I'd do is to designate a set of peers as cluster controllers and use a dynamic DNS service to allow other peers to discover the cluster controllers.
Choose a dynamic DNS provider I'll call it myc.ath.cx (I Use http://www.dyndns.com/).
Each peer has to be capable of becoming a cluster controller. A cluster controller will contain a list of all the other peers connected.
When a peer is started it looks up myc.ath.cx and attempts to connect. If connection cannot be made within a period, say 30 seconds, it takes over the registration of the DNS entry.
Any peer wishing to discover other peers can simply query myc.ath.cx and a list will be provided
All peers are responsible for periodically downloading the list of peers, in case they need to cluster controller.
The cluster controller will periodically query the DNS entry - if has changed from it's IP address then it knows that it is no longer the cluster controller - so it will contact the cluster controller that currently has the DNS entry and provide it's list of known hosts.
The cluster controller will periodically contact hosts on the list to ensure that they are still valid.
Your method of sending email does use a dedicated server, though; the peer's email server, to be precise.
Roughly, I don't think it's possible without using some sort of dedicated storage or server (which the email approach does, albeit obliquely) UNLESS you are able to characterize the connectivity to the internet that your peers are using.
Basically, if you have a set of X number of peers, that connect for Y amount of time, and they are then off the grid for Z amount of time... essentially, you can construct a probability equation about how likely it is that the set of peers that you last contacted is still available; where that probability approaches 1 (for a given set of X, Y, and Z above), you can most likely sustain a peer-to-peer network without using storage.
Possibly more in the spirit; instead of having a "dedicated central server", use simple online free service to specify a peer list. Set up a yahoo group, or something like that; clients can automatically look it up and get a peer address from which to query a set of peers; the client can be coded with the authentication to post to the group, and can post periodically its IP address so that others can request the set of known active peers.
If you want to get really tricky, you can start using basically steganographic methods to hide peer location information. I.e. get a google search for "blah"; find the first site listed in the results that has an unprotected (no CAPTCHA) message board; find the third (or whatever) post that starts with "Indubitably" (or whatever), and find the header of the first message there, and there's the IP address of a peer. If that doesn't work, go down the list of search terms to the next one.
But that's sneaky. :-)
Could you re-use an existing dedicated server for the purpose?
I am thinking in particular of registering each of the peers with a Dynamic DNS, but if you were willing to get a bit uglier, sharing access to a known Hotmail account or Google Doc or the like.
You can either use a central directory or some sort of broadcast protocol for service discovery. Assuming that you could get them indexed by Google, you could conceive of a system whereby each peer runs a web site with some unique, rare words contained on a specific page. You could then use Google search results based on these words to identify potential peers. This would essentially be a (noisy and slow) internet broadcast.
If the page structure was a well known pattern or contained identifiable connection information for that peer, it would be easy to distinguish them in the search results. Using such a public directory leaves you open to compromised nodes in the network that is formed, but this is pretty much true of any P2P network absent some security mechanism.
Getting the web sites crawled and highly ranked by Google (or some other search engine) for your particular arcane set of search terms would be the trick. I can think of a couple of ways, but they aren't ones that I would use. For a legitimate service, I'd rather spend the money or find a free web site that could function as a directory.
What about another P2P system built specifically to track online peers of other P2P systems?
Then we reduce the problem of finding peers for any new P2P system to simply finding peers for the 'main' P2P system, which will give you the addresses of online peers for the system you're interested in using...
This is a typical use of a distributed hash table algorithm. I'd suggest looking at something like pastry. It uses a overlay network (Application layer network) on top of other layers.
Each node has a GUID which is used to route requests across the peer network.
If you're loooking for an already established central server then see the metaserver entry on page here:
http://martindevans.appspot.com/
You can register peers on there and then other peers can find them. Obviously this is a central server, but it requires no maintenance on your part.