I want to build a Raspberry Pi (RPi) based device similar to the Nest Thermostat, except that I want to control light switches. I know I could easily code up a Node.js script on the RPi and then just hook it up to my local router and control it with a PC hooked up to the same router.
But, I want to make it more like a Nest. I want to be able to control 20 (or more) RPis across different subnets through a cloud provider like Heroku.
The Nest thermostat does not require the user to open a port on their router so that the Nest server can communicate with it.
How is Nest able to control customer's thermostats over the web without enabling port forwarding in the customer's router?
Blog post Nest Thermostat API using Node JS and Nest API Update discusses the API
between the thermostat and the nest lab servers. In particular:
The API uses mostly JSON formatted data POSTed to their web servers.
So there is your answer. By periodically POSTing data to the nest web servers using HTTPS, Nest is able to avoid having the user open any ports on their router.
For what it's worth, the unofficial API (in node.js!) from the blog post is available on github: Unofficial Nest API on Node.
I think the thermostat periodically posts a request with the thermostat serial number to nest. It may have timeouts in the http headers set to infinity or more likely, the nest server simply closes the connection after a minute or so if no command for that device has come in. Whenever nest closes the connection the thermostat opens a new one. When a command request comes in with serial number, its a quick lookup to find the thread currently connected to that thermostat. The command is sent in the payload of the current thermostat request, which is executed, the connection is closed, and the thermostat opens a new request to nest .
Take a look at eventSource in the HTML 5 standard. It does everything that is required.
Related
I'm new to working with sockets and have a small system design question:
I have 2 separate node processes for a web app, 1 is a simulator that is constantly running and the 2nd is an api server. Both share the same MongoDB database and we have a React app running for the client, served by the api server.
I'm looking to implement socket.io for real-time notifications and so I've set up a simple connection between the api and client.
My problem is that while the simulator runs, there are some events that I also want to trigger push notifications for so my question is how to hook that into everything?
The file hierarchy is like:
app/
simulator/
api/
client/
I saw this article for communication between node processes and I currently have 3 solutions in mind:
Leave hierarchy as it is and install socket.io package inside simulator as well. I'm not sure if sockets work this way but can both simulator and api connect to the same socket?
Move simulator file into api file to fork as a child process so that the 2 processes can communicate via child/parent messaging. simulator will message api which will then emit updates through the socket to client
Leave hierarchy as is and communicate via node-ipc. Same situation as above with simulator messaging api first before api emits that to client
If 1 is possible, that seems like the best solution in my impression. It seems like extra work to add an additional layer of messaging for 2 and 3.
Leave hierarchy as it is and install socket.io package inside simulator as well. I'm not sure if sockets work this way but can both simulator and api connect to the same socket?
The client would have to create a separate socket.io connection to the simulator process. Then, the client can receive data from the API server over one connection and from the simulator over another connection. You would need two separate, independent socket.io connections from the client, one to the API server and one to the simulator. Simulator and API server cannot share the same socket unless they are in the same process.
Move simulator file into api file to fork as a child process so that the 2 processes can communicate via child/parent messaging. simulator will message api which will then emit updates through the socket to client
This is really part of a broader option that the simulator communicates with the API server and sends it data that the API server can then send to the client over the single socket.io connection that the client made to the API server.
There are lots of different ways for the simulator process to communicate with the API server.
Since it's already an API server, you can just make an API for this (probably non-public). The simulator calls an API to send data to the client. The API server receives that data and sends it to the client.
As you suggest, if the simulator is run from the API server as a child process, then you can use parent/child communication messaging built into node.js. Note, you don't have to move the simulator files into the API file at all. You can just use child_process to launch the simulator as another nodejs app from another project. You just have to know the path to that other project.
You can use any another communication mechanism you want between the simulator process and the API server process. There could be a socket.io connection between them. You could use several forms of IPC, etc...
If 1 is possible, that seems like the best solution in my impression.
Your #1 option is not possible as separate processes can't use the same socket.io connection.
It seems like extra work to add an additional layer of messaging for 2 and 3.
My options #1 and #2 are not much code in each server. You're doing interprocess communication. You should expect to use some code to enable that. But, it's not hard at all.
If the lifetime of the simulator server and the API server are always together (they have no independent uses), then I'd probably do the child process thing where the API server launches the simulator and then use parent/child messaging to communicate between them. You do NOT have to combine sources to do this.
The child_process module can run the simulator process by just knowing what directory it is located in.
Otherwise, I'd probably make a small web server on a non-public port in the API server and have the simulator just send data to that other web server. I often refer to this as a control port. It's a way of "controlling or diagnosing" the API server internals and can only be accessed from within the private network and/or with credentials. The reason I'd use a separate web server (in the same nodejs app as the API server) is to make it easy to secure so it can't be accessed from the outside world like the regular public APIs can. You just put the internal web server on a port that is not exposed to the outside world.
You should check Socket.IO docs about adapters and Emitters. This allows to connect to sockets from different node processes and scalability.
I'm trying to make it so that NodeRed uses a webhook to send an interactive message to a slack app to ask permission to activate something. I can POST to the webhook without any issues from my custom node, but how do I get the reply?
Do I need to use the "http in" node to create an endpoint Slack can use? Is there a way to handle it from the same node that POSTs the message?
Also, I'm running NodeRed locally. Can I use ngrok so that Slack can access the http endpoint (in case I need to use that)?
You can define your own http endpoints that are private to your custom node, this is how nodes that need to do oAuth handle the callback. Also nodes like the serial port node do this to supply information to the config dialog.
Make sure to give the endpoint a unique route so it's unlikely to clash with any other nodes.
You can look at the source for the serial port node here
How you ensure that Node-RED is safely made accessible from the internet should be a separate question.
1) Yes, ngrok works nicely for exposing your local webserver securely to the Internet and Slack. I use it every day for my Slack app development. Keep in mind that if your app can't be reached from the Internet, then also Slack can not reach it.
2) Interactive Messages only work with Slack apps. One reason being that you need to provide the route to your app, so that Slack knows where to send the requests when someone uses your interactive messages (e.g. clicks on a button). But you can use interactive messages with webhooks, as long as both are setup within the same Slack app.
3) Don't believe the "http-in node" approach will work. As said, you need to configure the route to your app for receiving messages from Slack in your Slack appp.
I have a linux machine on which I want to run a local server using ExpressJS. To that machine a couple of instruments are connected via a USB-to-RS485-adapter which can be controlled and updated using NodeJS. What I want to do is display a live status feed of the instruments (mostly pressure gauges) and update some graphs that display the current pressure that the gauges are measuring. I already have a Server that runs ExpressJS, but I don't know how to make the Node code for controlling the hardware influence the status. Do I have to run the node code server side and then send data to the client with socket.io or is there some easier way. I'm completely new to expressJS and webapps, so it would be nice if you could point me in the right direction and explain to me how that works.
Express is a framework to make a web server, while very good for what it does, real time is not what it does.
Instead, socket.io is really good at real time transmissions.
What you should do is serve your "status" page with Express, page that connect to socket.io and subscribe for your "data" event.
Then all you have to do is, when you retrieve the data, send event with socket.io to make the data appear in near real-time on the web page.
I am building an app in which I provide functionality X, Y and chat.
Lets say that X and Y are non-interactive eg. reading articles - which will work fine with REST (on a node.js server) while chat is obviously interactive so it will work best with socket.io!
Questions: 1. Is it possible for me to 'switch on' a socket between the server and the user when the user navigates to the chat part of the application? 2. Can I open up a socket inside a GET request for the url: example.com/chats/usr_id on the node.js server?
3. How can this be accomplished inside a Backbone routing framework?
Yes. Just initialize the connection when the view is rendered (via a controller or script). See socket.io client documentation. You can just connect when the view is rendered and disconnect when the view is terminated. http://socket.io/docs/client-api/
You cannot open sockets with a GET request. Socket.io has it's own build in mechanisms for connecting to a socket server. It will start with Web Socket protocol and fall back to Long Polling. You can however use custom url's for unique things. One again, consult the socket.io documentation: http://socket.io/docs/client-api/
http://www.sitepoint.com/chat-application-using-socket-io/
p.s. I'd suggest reading up on how Web Sockets work, as you don't seem to have a very strong understanding.
I have working PHP application. It allows user create private projects and invite others in it. Now, using node.js and socket.io, I want to make real-time comments, posts etc.
What is the best architecture?
I see two solutions now.
The first is:
User sends AJAX query to PHP backend:
http://example.com/comment_add.php?text=...
comment_add.php adds
comment to database and via AMPQ (or something better?) notifies
node.js server which broadcasts comment to channel's subscribers.
The second is:
User sends AJAX query to node.js server: http://example.com:3000/comment_add
Node.js sends request to PHP backend (But how? And what about authorization?), receives response, and then broadcasts to channel's subscribers.
What is the best way? Is there another methods? How to implement this properly?
When you decided to use node.js + socket.io to make a real-time web-app, you don't need to think about PHP anymore and forget Ajax also... Socket.io will be the communication between client and server.
But yes, you can use Ajax and PHP for building websites fast, and some other functions that don't need real-time
The second way is the best method. You can use http to communicate with PHP from node.js. Authorization can be done in node.js but passing auth credentials everytime to PHP
Finally my working solution is #1.
When user establishing connection to node.js/socket.io he just send 'subscribe' message to node.js with his PHP session id. Node.js checks authorization using POST request to PHP backend and if all is OK allows user to establish connection.
Frontend sends all requests to PHP as it was before node.js.
PHP modifies some object, checks who can access modified object and sends message (via AMQP or redis pub/sub etc.) to node.js:
{
id_object: 125342,
users: [5, 23, 9882]
}
node.js then check who from listed users have active sockets and for each user sends GET request to PHP:
{
userId: 5,
id_object: 125342
}
Special PHP controller receiving this request runs query to get object with rights of given user id and then sends message to node.js with resulting answer. Node.js then via socket sends answer to user's frontend.
I faced this same question a year ago when starting my final year project at University. I realized that my project was much better suited to using Node as a standalone. Node is very good at dealing with I/O, this can be anything from a HTTP requests to a database query. Adding in a PHP based Web Server behind Node is going to add un-needed complexity. If your application needs to perform CPU intensive tasks you can quite easilly spawn 'child' node processed which perform the needed operation, and return the result to your parent node.
However out of the two you methods you have mentioned I would choose #2. Node.js can communicate with your PHP server in a number of ways, you could look at creating a unix socket connection between your PHP server and Node. If that is unavailable you could simply communicate between Node and your PHP back end using HTTP. :)
Take a look here, here is a solution to a question very similar to your own:
http://forum.kohanaframework.org/discussion/comment/57607