I am new to Distributed databases and the initial roadblock am facing is to have a distributed transaction across heterogeneous databases where the control goes to Transaction Manager which is mostly the OS implementation (MSDTC in Windows).
How are generally Distributed Transactions implemented on Linux?
And if they are implemented which major databases have support for it? (MSSQL, MYSQL, POSTGRES, ORACLE, SYBASE)
I don't think there is any service like MSDTC in Linux. I expect you will need to use some 3rd party transaction manager. Or maybe you could try to port MSDTC to Linux on Mono (no idea how hard it is or if possible).
I'm not much familiar with transaction managers other than for Java.
Related
We are migrating our Postgresql v10 (Linux) database to Azure and due to the timeframe involved would prefer not to change our current use of the npgsql driver in our apps.
Has anyone had any experience here? Can we utilize our use of that driver with very little or no changes to our code?
The one thing I'm aware of is that queries must use the shard column to help the Citus coordinator distribute to the right node. Anything else?
Thank you!
I have no personal experience with Citus, but as far as I know, Citus is supposed to be extremely compatible with PostgreSQL - definitely at the wire protocol level - and so general PostgreSQL drivers (like Npgsql) are supposed to work fine with it.
You may definitely need to change the way you're doing things at a higher level (e.g. sharding), but at the low-level database driver level Npgsql should be OK.
We are attempting to build a system that gets a list of task to execute from a Cassandra database and then through some kind of group consensus creates an execution plan (preferably on one node) which is then agreed on and executed by the entire cluster of servers. We really do not want to add any additional pieces of software such as Redis or a AMPQ system, rather have the consensus built directly into all of the servers running the jobs. So far we have found Skiff, an implementation of the Raft algorithm that looks like it could accomplish the task, but I was wondering if anyone has found an elegant solution to this problem in a pure Node.js way not involving external messaging systems.
Cassandra supports lightweight transactions, which is basically Paxos implementation that offers linearizable consistency and CAS operation (consensus). So you can use Cassandra itself to serialize the execution plan.
I have been using CouchDB on some prototype applications and it has been brilliant, very easy to use and extremely quick. I was wondering if anyone has been using it in production and have any views on it's reliability, performance suitability for operational management etc ?? I am considering using it to support a service layer and would make use of its replication functionality.
Any comments/experiences would be most welcome.
I've used CouchDB for a few small in-house applications - it's been very stable and I've had no serious complaints. Setting that aside, a few small gripes -
1) Databases can be synchronized, but not nodes. That is, if you have four servers and twenty databases, you have to specify each server, and each database to synchronize. A minor gripe, but I prefer less management to more.
2) Since databases are append only, a database with a bunch of activity gets really big really quickly. Compacting fixes this, but isn't exactly fast, especially on big (e.g. 20 gigabytes) database. Scheduling compact for the weekends solved this, but doing that is probably less of an option for high availability applications.
3) Javascript is the de facto view language. What is not well advertised is that since CouchDB is written in Erlang, it also supports Erlang views, which are faster as they are "native". For applications doing a lot of operations in views, Erlang probably makes more sense.
Setting those minor issues aside, I'd wholeheartedly recommend it.
CouchDB ships in Ubuntu and is a fundamental component of the Ubuntu One service.
I have to connect to a RDBMS (locally and remotely) via iOS. I have seen the solution ( http://odbcrouter.com/ipad), but for that we have to buy a server side program to run but its very expensive and not a mangeable solution as we have to buy it for more than 1000 stores and making updates etc for each store will be very difficult.
I have just checkout the following iOS app ( http://www.impathic.com/impathic/index.html), it is doing the same thing without any server side code, which make me think that, there will be a way to do this without any server side code... So I was wondering if these RDBMS expose themselves the way I am expecting.
Thanks.
That impathic program is very interesting, but they're likely using a 3rd party library.
The RDBMS wire protocols aren't common, and typically only ODBC and JDBC drivers have implementations of them (and the actual native drivers of course).
If your database is Postgres or MySQL (or some other OSS DB) you may well be able to simply port the native drivers source code to iOS and build it for ARM. I've not done it, but it might work. I don't know if any of the other DBs have ARM native drivers for iOS or note.
Otherwise, you would likely be better off with a middleware solution, depending on what kind of work you plan on doing. Webservices are pretty straight forward for basic cases.
I'm hoping someone can validate or correct my conclusions here.
I'm looking into writing a small side project. I want to create a desktop application for taking notes that will synchronise to a web-server so that multiple installations can be kept in step and data shared and also so that it can be accessed via a browser if necessary.
I've kind of been half-listening to the noises about CouchDB and I've heard mention of "offline functionality", of desktop-couchdb and of moves to utilise its ability to handle intermittent communications to enable distributed applications in the mobile market. This all led me to believe that it might be an interesting option to look at for providing my data storage and also handling my synchronisation needs, but after spending some time looking around for info on how to get started my conclusion is that I've got completely the wrong end of the stick and the reality is that:
There's no way of packaging up a CouchDB instance, distributing it as part of a desktop application and running it in the context of that application to provide local storage and synchronisation to a central database.
Am I correct here? If so is there any technology out there that does this sort of thing or am I left just rolling my own local storage and maybe still using CouchDB on the server?
Update (2012/05): check out the new TouchDB projects from Couchbase if you are targeting Mac OS X and/or iOS or Android. These actually use SQLite under the hood (at least for now) but can replicate to/from a "real" CouchDB server. Another clientside alternative that is finally starting to mature is PouchDB, which runs in IndexedDB capable browser engines. Using these or using them to inspire similar port to another desktop platform is now becoming a better-trod path.
Original answer:
There's no way of packaging up a
CouchDB instance, distributing it as
part of a desktop application and
running it in the context of that
application to provide local storage
and synchronisation to a central
database.
At this point in time, your statement is practically correct although it is possible to include CouchDB in an app — for an example see CouchDBX.app which is a thin wrapper around a prefixed bundle of CouchDB and all its dependencies.
The easiest way to build a CouchDB app is to assume that the user will already have a CouchDB server running. This is easier than it sounds, especially with Couchone's hosting or a prebuilt app like CouchDBX on OS X or DesktopCouch on Ubuntu. This latter is especially interesting, because if I understand correctly it is included by default with Ubuntu these days, and automatically spins up a CouchDB server per-user when you query its port via D-Bus. Something similar could (and should) be done on OS X using launchd and Bonjour.
So as you write, you either would design your app to store data in a local format and optionally sync with a CouchDB service you provide or you'd have to build and bundle all of Erlang, SpiderMonkey and CouchDB together with your app along with some scripts to make sure it was running when needed. This is possible but obviously neither of these are ideal, and believe me you're not the only one wanting a simpler solution for desktop-oriented apps!