I don't think that I am understanding how CouchDB works. My impression is that everything runs on the client side, so wouldn't that mean it is useless for storing user data because anyone can write a simple script to access that information? This doesn't make sense to me, do I have it all wrong?
Aside from map-reduce and update operations, everything in CouchDB does run on the client. In this context, client means client connecting to the database server, which will usually be an application or script running on your web server. That's the case for other database systems, too: to connect to a MySQL database from a PHP script, you need to use a MySQL client library.
One special thing about CouchDB is that instead of using its own transfer protocol (like other systems like MySQL do), it uses HTTP, which is implemented by almost every single available language out there. This makes the development of a CouchDB client extremely easy.
The other special thing about CouchDB is that its security model does allow you to let end users connect directly to the database. In such a situation, you would write a JavaScript application that runs entirely in the users' browsers and queries the database through AJAX. The server would then authenticate the user and grant access only to those databases that the user is allowed to access, in either read-only or read-write mode. While this requires a bid of server-side scaffolding (to register new users and create a brand new database for them).
But you don't have to. My company uses CouchDB as a general-purpose persistent storage that is completely invisible from the internet, and only our web server is allowed to access it.
There's a really good book on CouchDb here: http://guide.couchdb.org/
Related
I wish to use PouchDB - CouchDB for saving user data for my web application, but cannot find a way to control the access per user basis. My DB would simply consists of documents using user id as the key. I know there are some solutions:
One database per user - however it requires to monitor whenever a new user wants to save data in order to create a new DB, and may create a lot of DBs;
Proxy between client and CouchDB - however I don't want PouchDB to sync changes for the whole DB including documents of other users in those _all_docs, _revs_diff request.
Is there any suggestion for user access control for pouchDB for a user base of around 1 million (active users around 10 thousand only)?
The topic of a million or more databases has come up on the mailing list in the past. The conclusion was that it depends on how your operating system deals with that many files. CouchDB is just accessing parts of the .couch file when requested. Performance is related to how quickly it can find, open, access, and close that file.
There are tricks for some file systems like putting / delimiters in the database name--which will cause CouchDB to store them in matching directory structures such as groupA/userA.couch or using email-style database names com/bigbluehat/byoung.couch (or some similar).
If that's not sufficient, Apache CouchDB 2.0 brings in BigCouch code (which IBM Cloudant uses) to provide a fully auto-sharded CouchDB. It's not done yet, but it will provide scalability across multiple nodes using an Amazon Dynamo style sharding system.
Another option is to do your own username-based partitioning between multiple CouchDB servers or use IBM Cloudant (which is built for this level of scale).
All these options provide the same Apache CouchDB replication protocol and will work just fine with PouchDB sitting on the user's computer, phone, or tablet.
The user's device would then have their own database +/- any share databases. The apps on those million user devices would only have the scalability of their own content (aka hard drive space) to be concerned about. The app would replicate directly to the "cloud"-side user database for backup, web use, etc.
Hopefully something in there sounds promising. :)
Before jumping into development, I'd like to get feedback on a change I'm thinking of making, moving from mongo to couch.
Basically I've got a webapp which is used to help organize users activities (todolist, calendar, notes, journal). It currently uses mongodb, but i'm thinking to move it to couch, mainly due to couches replication ability, and clientdb interaction (pouchdb). I have a similar homegrown setup on the browser using localstorage, backed by mongo, but am looking for a more mature solution.
Due to how couchdb differs from mongodb, I'm thinking that each user should have their own couch db, and their documents being each of my app components. Basically I have to move everything up a level with couch due to local db replication, and due to security.
I have 3 questions.
1) I assume that couch does not have document level security/authentication, correct? (Hence me moving each user assets to their own database, good idea?)
2) My plan is have users login to the website, then my backend nodejs code authenticates them, and sends them down some auth/session token. The javascript on the client then uses its local pouchdb data to set itself up, and also sends the replication request directly to the couchdb server (using the auth token it got from my server-side process). They should only have access to their database, since I can do per database auth access (correct?)
What do you think of that setup? It should work?
3) Regarding couchdb service providers, why do they vary so much on their couch version? IE, happycouch, 1.6.1, iris 1.5, cloudant, 1.0.2? And I also hear about couchdb 2.0 coming out soon... I'd like to use cloudant, but 1.0.2 is so many versions back from a 1.6 or 1.5, if I'm not doing anything exotic, does it matter?
Bonus question :p Continuing from the last question, do you know of any services that host node.js and have local instances of couchdb available? I'd like to use my backend server code as a proxy, but not at the expense of another network hop.
Thank you very much for your feedback,
Paul
Due to how couchdb differs from mongodb, I'm thinking that each user should have their own couch db
This is a CouchDB best practice. Good choice.
I assume that couch does not have document level security/authentication, correct? (Hence me moving each user assets to their own database, good idea?)
You are correct: https://github.com/nolanlawson/pouchdb-authentication
My plan is have users login to the website...
Yep. You can just pass the cookie headers straight through from Node.js to CouchDB, and it'll work fine. nano has some docs on how to do that: https://github.com/dscape/nano#using-cookie-authentication
Regarding couchdb service providers, why do they vary so much on their couch version
The Couch community is one big happy fragmented family. :)
I'd like to use cloudant, but 1.0.2 is so many versions back from a 1.6 or 1.5, if I'm not doing anything exotic, does it matter?
1.0.2 refers to when Cloudant forked CouchDB. They've added so many of their own features since then, that they're pretty much feature-equivalent by now.
The biggest difference between the various Couch implementation is in authentication. Everybody (Cloudant, CouchDB, Couchbase) does it differently.
I would like to know, how to get data from MySQL database to my application without using any REST API or PHP code. I was looking over the internet for the solution for this problem. But they say, you can use php code as REST API and then, can communicate with database. For this purpose, i will need a host and domain. I don't want to use that. Is there any other way to communicate with mysql database. Can i use mysql module of node js in titanium application.
There is no way to have direct connection between your mobile client and MySQL database. To retrieve data from MySQL you need to build application which will receive request from your app, retrieve data from MySQL, process and return it as a response.
If you don't want to build mobile and server application at the same time you can try using Appcelerator Cloud service, which plays really nicely with Titanium SDK and allows you to persist users data.
There are two answers to this problem, depending on your situation:
If Your Data Is Specific to One Device...
If you want to store data locally on one device, and that one device is the only one that will ever use it, then you want to use a SQLite database. This is very commonly used in mobile apps, and is very well documented. If you already have a MySQL database with the schema you want to use, then you could really easily convert it to a SQLite db file.
If Your Data Is Centralized...
If you need to store data remotely, in one central place, that the mobile app can access, then you need to use a remote database.
MySQL is one such option. You say that hosting PHP (which is itself run through something like Apache or IIS) is not something you want to do. But if you can host MySQL somewhere, or run it on a machine that your mobile app can access, then you can also easily host PHP and Apache.
If you don't want to spend money on a domain, then use one of the free dynamic DNS providers, which map a domain name (such as foo.hopto.org) to an IP address. If you don't want to pay for a server, then use your home computer, and keep it on whenever the mobile app needs to access it. There's easy, well documented ways around any of the issues you're having.
Alternatively, as #daniula pointed out, use Appcelerator Cloud Services. Then you can interact with simple objects, and they'll be stored for you in a central server. You can control who can access what data, and more. (Full disclosure -- I work for Appcelerator.)
I have an app that I would like to create. But I am not sure how to go about it. I am using node.js and would like to use couchdb, but if something like mongodb or riak would be a better choice them im willing to hear ideas. But, i have a site, say
cool.com
and on there is a couchdb instance, as well as a site to manage a store. say a shopping cart. the db houses all the store's items and data. The app itself has an admin backend to manage that data and can change items. What i would like to be able to do, is have the ability to have the user be disconnected from the internet, and still have the admin backend work. I realize for this to work I need to use a client side framework with my models/routes/controllers/whatever. But what I am not sure of, is how to let the site function while offline. couchdb if installed locally can sync the data from local to remote when back online, and if the admin user is on the computer, i could have them install couch. but that could be messy.
Also, what if the admin user is on a tablet or a phone? Would I need to have an actual mobile app and a desktop app to do this? is there some way I can set this up so it is seamless the the end user. I would also like this to be offline for end users too, but the bigger audience is the admin.
Another use case, instore POS system. and the power goes out. But the POS system can be loaded from the web onto a tablet and they can still make card based sales if the wifi is out, because the app is available offline.
Im just not sure how to do this. lets assume i need a client framrwork that can handle the data as well as the backend. something like ember, or angular. theres also all in one stacks like meteor and derby js, but those arent fully offline,but are for the appearance of real time. though meteor does have mini mongo so it might be worth looking into.
I was hoping someone could help me figure out how I would get this setup to work, preferrably with couch, but other nosql's would work too if I can have a way to sync the data.
I'm not sure if it would work for you, but I have been thinking of such an application for quite a long time now and been doing some research on what's possible. The best solution I could come up with is using a server with a couchdb and writing the application clientside based. Then for the data storage use pouchdb and synchronize the pouchdb regularly with your serverside couchdb if the app is online. I know pouch is in an early stage and not production ready but if you are willing to put some work into it I'd say it's doable.
If you want clients that work seemless as they go offline and come online (like a POS with the power out) then I would recommend making the app primarily work off local storage with a background publishing or synchronization to the cloud.
Local storage options could be everything from something light like sqlite, sqlexpress, firebird to no sql options like mongo, couchdb etc...
But for the client or device, consider the ease of configuration and weight of the option. You also need to consider the type of clients - do you have many platforms varying from devices to PCs? You don't want something that has a heavy config and runtime footprint. That's fine on the service side.
On the service side, consider the nature of your data and whether it's fitted better for transactional/relational systems (banking etc...) or eventually consistent/non transactional (no-sql) documents. Don't forget hybrid as an option. Also consider the service platform - for example, node goes well with mongodb (json objects front to back) ...
The device and service storage options can be different (and likely should be) separate by service interfaces (soap, rest/http, sockets etc...).
It's hard to have a one size fits all solution but often something light weight like sqlite on the device or client makes for ease of installation/config while scalability on the service side with something like sqlserver/mysql or couchdb/mongodb makes sense.
Some links to read:
http://www.mongodb.org/display/DOCS/Comparing+Mongo+DB+and+Couch+DB
http://www.sqlite.org/
http://blogs.msdn.com/b/sqlexpress/archive/2011/07/12/introducing-localdb-a-better-sql-express.aspx
You're question is pretty wide open and there's no one size fits all solution. Hopefully I provided some options to think about.
There's an interesting project out there called AppJs (http://appjs.com/), which packages Node.JS and Chrominium as a desktop environment. It's currently very fresh (very little documentation), but it appears to be straight forward enough (you'll be using the same tools as you would for your online application).
As for synchronising the offline and online environments. I doubt you can rely on CouchDB in the way that you envisage. CouchDB mobile support is not as comprehensive as some of the documentation suggests. So in this sense, it would be no different to using SQL/Mongo/Punchcards.
You might have more luck with designing a suitable serialisation scheme based on XML or JSON (or just plain text), and passing files between the online and offline installations.
Edit - Since writing this, Node Webkit - http://nwjs.io/ - is clearly the most obvious replacement for App.js. It has a very simple API, and some great features.
I want to write very simple app, witch monitors states of some sites. I also want to make it in Couchapp style without using any environment except CouchDB.
So the question is how I can make CouchApp send sites requests using some schedule by itself?
BTW, if I fail with this CouchApp, is there some way to make it not involving demon stuff (or cron) on PHP or on even Java? I want to keep it as simple as possible, but not simpler.
rsp is correct. Since CouchDB uses web protocols and Javascript, it has become a victim of its own success.
My rule of thumb is this: CouchDB is a database. It stores your data. I do not expect MySQL to automatically monitor external web sites. Why would I expect CouchDB to do that?
However I agree; CouchDB always benefits from some persistent processing to maintain the data.
Since CouchDB is completely web-based, you could start with a simple dedicated "worker" web browser. Fetch a password-protected HTML page from CouchDB. That page has the Javascript to make the browser query servers and update CouchDB. This could work in the short-term as a quick solution. However browsers impose security restrictions on your queries; and also a browser is not a long-term computing platform.
The traditional way is to run your own client software to do these things. You can either run a dedicated computer, or use PHP, NodeJS, or any other hosting services out there.
2. The
You can't do it in CouchDB alone (CouchApps can only have pure functions without side effects so they can be guaranteed to be cacheable) but you can do it using simple scripts that talk to CouchDB. See this talk by Mikeal Rogers for details on how to do it.