I'm trying to figure out how to insert data into a Meteor mongodb database from an external native mobile application that I'm writing (specifically for iOS using Cinder, right now). I'd like events that happen on the mobile device application to be written to my Meteor app's database, so that it can be immediately displayed on a browser elsewhere.
Importantly, I need to stay in my native application on the mobile device - I can't launch into a browser. I'm a bit new to Meteor, so apologies if I'm overlooking something obvious.
Any ideas on how to do this?
Thanks!
Your best bet is to use an iOS DDP client like this one. You can use this client natively in your existing iOS app and subscribe and write back to data in your Meteor ecosystem.
DDP stands for Distributed Data Protocol and is authored by the Meteor group as an external standard for real-time app frameworks to adopt. It's a much preferred method than communicating directly with the database because you can leverage the publish and subscribe methods within the Meteor ecosystem.
The protocol is under rapid development.
I believe there will be a release shortly that will expound on the current state of DDP and the evolution of its official specification. An official spec is slated for the 1.0 release.
Note: Here is a great video overview of DDP in its present form.
Another option is to have your iOS app write directly to the MongoDB instance used by your Meteor app. You can use any MongoDB driver such as NuMongoDB. Meteor polls the MongoDB database every ten seconds so web based users will automatically see updates, albeit with a short lag.
DDP is evolving quickly, as Tim mentioned, so this option might be a little more stable.
In terms of hosting, for scalability I recommend separating your MongoDB instance from the free meteor.com site, by using a Mongo host such as MongoHQ.
And what about other cases ?
I mean browser applications ?
Related
I'm trying to build a website myself but I also want to build a native mobile application that will access the same DB in the future.
What I'm thinking now is using Node.js to build Web Services wrapper for the DB and every DB operation will be executed via web service API. And for the website framework, I'm going to use Rails.
Please let me know whether this is a good architecture or not. I'm not sure whether encapsulate data with Web Services is a good idea. Will there be any performance issue? And if it's feasible, which DB should I use? And can rails communicate with DB via web services?
Thanks a lot!
Update
Why do people down vote this question??
I think you have more technology than needed in your architecture right now.
Personally I would create a REST api on top of the DB (using either node or Rails - both are super easy to do this with and both can use pretty well any db)
Then you can write any number of "apps" for the front end process, whether they are web apps, ios apps, android apps, etc... They will all get their data from your REST api on the backend.
You might even consider writing the front end as a single page app using Angular, Knockout or Backbone, something like that. If you do that with node, your entire stack will essentially be written in javascript. It can get confusing for a newb, but it's super powerful.
I have an app that I would like to create. But I am not sure how to go about it. I am using node.js and would like to use couchdb, but if something like mongodb or riak would be a better choice them im willing to hear ideas. But, i have a site, say
cool.com
and on there is a couchdb instance, as well as a site to manage a store. say a shopping cart. the db houses all the store's items and data. The app itself has an admin backend to manage that data and can change items. What i would like to be able to do, is have the ability to have the user be disconnected from the internet, and still have the admin backend work. I realize for this to work I need to use a client side framework with my models/routes/controllers/whatever. But what I am not sure of, is how to let the site function while offline. couchdb if installed locally can sync the data from local to remote when back online, and if the admin user is on the computer, i could have them install couch. but that could be messy.
Also, what if the admin user is on a tablet or a phone? Would I need to have an actual mobile app and a desktop app to do this? is there some way I can set this up so it is seamless the the end user. I would also like this to be offline for end users too, but the bigger audience is the admin.
Another use case, instore POS system. and the power goes out. But the POS system can be loaded from the web onto a tablet and they can still make card based sales if the wifi is out, because the app is available offline.
Im just not sure how to do this. lets assume i need a client framrwork that can handle the data as well as the backend. something like ember, or angular. theres also all in one stacks like meteor and derby js, but those arent fully offline,but are for the appearance of real time. though meteor does have mini mongo so it might be worth looking into.
I was hoping someone could help me figure out how I would get this setup to work, preferrably with couch, but other nosql's would work too if I can have a way to sync the data.
I'm not sure if it would work for you, but I have been thinking of such an application for quite a long time now and been doing some research on what's possible. The best solution I could come up with is using a server with a couchdb and writing the application clientside based. Then for the data storage use pouchdb and synchronize the pouchdb regularly with your serverside couchdb if the app is online. I know pouch is in an early stage and not production ready but if you are willing to put some work into it I'd say it's doable.
If you want clients that work seemless as they go offline and come online (like a POS with the power out) then I would recommend making the app primarily work off local storage with a background publishing or synchronization to the cloud.
Local storage options could be everything from something light like sqlite, sqlexpress, firebird to no sql options like mongo, couchdb etc...
But for the client or device, consider the ease of configuration and weight of the option. You also need to consider the type of clients - do you have many platforms varying from devices to PCs? You don't want something that has a heavy config and runtime footprint. That's fine on the service side.
On the service side, consider the nature of your data and whether it's fitted better for transactional/relational systems (banking etc...) or eventually consistent/non transactional (no-sql) documents. Don't forget hybrid as an option. Also consider the service platform - for example, node goes well with mongodb (json objects front to back) ...
The device and service storage options can be different (and likely should be) separate by service interfaces (soap, rest/http, sockets etc...).
It's hard to have a one size fits all solution but often something light weight like sqlite on the device or client makes for ease of installation/config while scalability on the service side with something like sqlserver/mysql or couchdb/mongodb makes sense.
Some links to read:
http://www.mongodb.org/display/DOCS/Comparing+Mongo+DB+and+Couch+DB
http://www.sqlite.org/
http://blogs.msdn.com/b/sqlexpress/archive/2011/07/12/introducing-localdb-a-better-sql-express.aspx
You're question is pretty wide open and there's no one size fits all solution. Hopefully I provided some options to think about.
There's an interesting project out there called AppJs (http://appjs.com/), which packages Node.JS and Chrominium as a desktop environment. It's currently very fresh (very little documentation), but it appears to be straight forward enough (you'll be using the same tools as you would for your online application).
As for synchronising the offline and online environments. I doubt you can rely on CouchDB in the way that you envisage. CouchDB mobile support is not as comprehensive as some of the documentation suggests. So in this sense, it would be no different to using SQL/Mongo/Punchcards.
You might have more luck with designing a suitable serialisation scheme based on XML or JSON (or just plain text), and passing files between the online and offline installations.
Edit - Since writing this, Node Webkit - http://nwjs.io/ - is clearly the most obvious replacement for App.js. It has a very simple API, and some great features.
Is there an easy way to manage offline data with a web app, and synchronize with a server when there is a connection? I have been looking at Meteor, CouchDB and the likes, but still not sure what would be the least painfull way.
I could of course implement it myself with sockets or something similar, but if something is already made for the purpose, I don't see a reason to do it again.
I'm planning to work with Node as the server.
Thanks
You're talking about two things; 1) How to store/persist data if/when offline (storage mechanism), and 2) How to synchronize with a server when online (communication mechanism). The answer to 1 is some kind of local storage, and there any several ways of doing that (localstorage, websql, filesystem APIs etc) depending on your platform. The answer to 2 really depend on how urgent your synchronization needs are, but in general you can use HTTP itself with periodic (long-) polling, websockets and similar.
On top of both storage and communication mechanisms there are numerous libraries that make the job simpler, like Meteor (communication) and CouchDB (storage), but also many many more. There are even libraries that take care of the actual synchronization mechanism (with possible conflict resolution as well), but this very much depends on your actual application.
Updated: This framework looks promising, but I haven't tested it myself:
http://blog.nateps.com/announcing-racer-experimental-realtime-model
You might want to look at cloud services as well. These are best if you are developing a new application as they push you more to a serverless model, and of course you have to be happy using a service.
Simperium (simperium) is an interesting cloud service - the only one I can find today that does syncing (unlike Firebase and Spire.io who are similar in other respects), and for iOS it includes offline storage, while for JavaScript clients you'd need to cover the local storage yourself using HTML5 features. Backbone.js seems to have some support for this, and Simperium can integrate with Backbone, using a similar API style.
For non-cloud services, Derbyjs (derbyjs) is an open source project that includes Racer, a data synchronization library (mentioned by the earlier answer) - both are under rapid development and not yet complete, but look interesting if your timescales allow, and don't require a cloud service. There is a comparison of Derbyjs to Meteor that is useful - although it's written by the Derbyjs developers it's not too biased.
I also looked at CouchDB, which has some interesting built-in replication features, but I didn't like its use of indexes that are updated lazily when a query needs them (or by a batch process), and I wasn't happy with exposing the server DB directly to clients to enable replication/sync. Generally I think it's best to decouple the client side local storage from the server side DB, and of course for a web app it would be hard to use CouchDB on the client.
I am going to build a web application to manage notes (think of something similar to Evernote). I have decided to use Backbone.js+JQuery client side.
Server-side, I haven't decided yet: either pure PHP (that I know very well) or Node.js+Socket.io (completely new to me).
I am considering Node.js+Socket.io because I would like to have my web application being real-time (i.e: if a user updates a note, that note gets updated instantly for a collaborator who shares that note, without reloading the page).
I was also considering, as a third alternative, to use Node.js and Socket.io for the UI and PHP for the REST API (I feel more comfortable to build an API with PHP). The PHP and Javascript code will share the MongoDB database.
My question is this: if I develop the REST API for my web application with PHP and a new note for the user gets created through the API (i.e.: the Android app sends an API request to create that note), will Node.js, Socket.it and Backbone.js be able to instantly update the UI of the user and show the new note on their screen? I think that can be called "push notification".
I hope I was clear enough.
Also, is there any alternative outstanding technology to build real time web applications?
Yes Node.js + Socket.IO will do a very good job of this. Node uses an event-loop, this means upon a request it is entered into a queue. Node deals with these requests one-by-one. Traditional web servers deal with a 'Thread-per-request' approach where a thread is created to handle that requests.
The benefit of Node here is that it doesn't need to context switch so often, this means it can deal with these requests very quickly... most likely faster than your PHP server. However Node runs as a single process, on a single CPU core. If your application is CPU intensive it could be that it blocks, meaning the time for each requests will be slower.
However it sounds to me like your application isn't CPU intensive, meaning Node.js will work well.
Decision
If your time is limited, and you don't want to learn a new skill (Node), PHP will be fine. If you have the time I recommend learning Node.js, as it is very strong when it comes to I/O intensive tasks such as a REST API for creating Notes.
Updating the UI
If your intended use is through a mobile device, I recommend using WebSockets but having a fallback such as long-polling. It is possible to update the Client UI using either Node or PHP. However from my experience it is much easier to do so using Socket.IO on Node.js.
Example Updating the client using Node.js / Socket.io
Client-side
socket.on('new-note', function (data) {
placeNewNote(data);
});
Server-side
socket.emit('new-note', data);
Getting Started With Node:
How do I get started with Node.js
Please also note, if you want to build a native Android mobile app which uses WebSockets... you will need to use: Java socket.io client
Using Node.js for both web server and push server is of course the best way. Especially since if you are going to use Node.js anyway then you have to learn it, so learning how to make a web server is only natural (I advice using the most famous Express framework).
Now you can use PHP for web server and Node.js for push server. To make them communicate with each other you would probably want to add Redis to your application. Redis will allow you to push notifications to any client connected to it, like PHP server or Node.js push server (and it scales well). From that point push server will push the data further to the client's browser.
Alternative technology would be for example Twisted server. Of course you'll need to learn Python in order to use it. And I don't know whether it supports WebSockets correctly. I think you should stick with Node.js + socket.io.
For a notification project, would like to push event notifications out. These are things like login, change in profile, etc., and to be displayed to the appropriate client. I would like to discuss some ideas on putting it together, and get some advice on the best approach.
I noticed here that changes made to a CouchDB can be detected with a _changes stream, picked up by Node, and a process kicks off. I would like to implement something like this (I'm using SQL Server, but an entry point at this level may not be the best solution).
Instead of following the CouchDB example (detecting database-based events, I think this just complicates things, since we're interested in client events), I was thinking that when an event occurs, such as a user login, then a message is sent to the Node server with some event details (RESTful request?). This message is then processed and broadcast to all connected clients; the appropriate client displays notification.
Proposed ecosystem:
.Net 4.0
IIS
IISNode
Socket.IO
Node.JS
SQL Server 2008
This will be built on top of an existing project using the .Net framework (IIS, etc.). Many of the clients' browsers do not support web sockets, so using Socket.IO is a good option (fallback support). However, from what I can see, Socket.IO only still only supports long polling through IISNode (which isn't really a problem).
An option would be to expose the Socket.IO/Node endpoint to all clients, so that client-based notifications can be sent through JS to the Node server, which broadcasts the message. (follows the basic chat-server /client/server examples).
Alternately, an IIS endpoint could be used, but could only support long polling (through Socket.IO). This would offer some additional .Net back-end processing, but may be over-complicating the architecture.
Is there SQL Server-based event notification available for Node?
What would be the best approach?
If I didn't get the terminology ecosystem configuration right, please clarify.
Thanks.
I would recommend you check out SignalR first before considering adding iisnode/node.js to the mix of technologies of your pre-existing ASP.NET application.
Regarding websockets, regardless if you use ASP.NET or node.js (socket.io), you can only use HTTP long polling for low latency notifications, as websockets are not supported by HTTP.SYS/IIS until Windows 8. iisnode does not currently support websockets (even on Windows 8), but such support could be added later.
I did some research lately regarding MSSQL access from node.js. There are a few OSS projects out there, some of them use native, platform-specific extensions, some attempt implementing TDS protocol purely in JavaScript. I am not aware of any that would enable you to access the SQL Notifications functionality. However, the MSSQL team itself is investing in a first class MSSQL driver for node.js, so this is something to keep an eye on going forward (https://github.com/tjanczuk/iisnode/issues/139).
If you plan to use SQL Notifications to support low latency notifications, I would strongly recommend starting with performance benchmarks that simulate the desired level of traffic at the SQL server level. SQL Notifications were meant primarily as a mechanism to help maintain in memory cache consistent with the content of the database, so it may or may not meet the requirements of your notification scenario. At the very minimum these measurements would help you start with a better design.
I would highly recommend using Pusher. That is what we use and it makes it easy to implement as it is a hosted solution. So plugging it and making it work is really easy. It doesn't cost much unless you are going to push a crazy amount of messages through it on a massive scale.