Tonight in my daily tech Googling I came across couchDB, after seeing tons of presentations about how it perform ten to hundred times better then any RDBM, how it would save us from SQL languages, tables, primary keys and so much more. I decided myself to try it myself. Only problem it seems I am unable to figure out how it works.
Like for a start I would like to code a web contact manager using couchDB. The project would enable user to do basic stuff like
Create/ Edit / Delete contacts
see a list of their contact ordered
search them on various criteria
So how do I start ?
Here some of my thoughts
create a database per user like July, Ann
in those DB, add some document with type contact, the document would look like this at first place see code 1
create / edit / delete is straight forward just need to do the PUT, POST, DELETE in the good database
searching would be handled by couchdb-lucene like dnolen suggested
now here come the difficult part, I don't really understand the whole map/reduce concept and how I can use that to do the jobs I used to do with SQL. Also with views how do you handle paging, also grouping.
I would like to build a screen with a paging set of links something like this
John, Doe
Johny, Hallyday
Jon, Skeet
A B C D E F **J** etc .... <-- those are link to see persons with that first name
what view should I create to achieve that, if you can provide samples it would wonderful.
Contact document.
{
type: 'contact',
firstname: 'firstname',
lastname: 'lastname',
email: ['home': 'foobar#foobar.net', 'work': 'foobar#foobar-working.net'],
phone: ['home': '+81 00 0000 0000'],
address: []
... some other fields maybe ...
}
The upcoming book by O'Reilly is free to read online:
http://books.couchdb.org/relax/
Just install and play around - you can do straight http requests using curl on the command line, or use the built-in web interface called futon.
Storing and retrieving data is really easy, the hardest part is thinking in terms of map/reduce-views instead of sql queries.
IBM has a great tutorial, making use of curl to read/write via the REST interface.
Your application is quite easy to do with CouchDB. You would have a database per user. Contacts are simply documents in a particular user's database. CRUD is just talking to the database using HTTP. You could create views that emit keys (last name, first name) to allow for sorting.
For powerful search I would recommend couchdb-lucene.
Related
I'm fairly new to Mongo, and the documentation is still a little blurry for my understanding.
I'm working on a pharmacy app - where doctors are connected to practices. (medical practice) The details (name, address, etc.) of these practices can be changed at any given time, and so I cannot create the doctor in this format:
{ name: "Dr. John", practice: "Yellow Medical Centre" } - because if that name is changed, it will be a pain updating it everywhere.
Now, my question is, assuming that I am already inserting doctors with a practice_id field instead of a practice_name field - how do I resolve the practice_id into a practice_name, before sending it back to the client - because they would like to see the doctor's practice name, not the ID of it.
(I made a huge mess, initially, so I'll just leave the plain part here)
app.get('/doctors', function(req,res){
db.doctors.find().forEach(function(err,doc){
});
});
Am I supposed to loop through them all, or not? Mt goal is for each Doctor's->practice_id to actually contain practice_name - which obviously requires the assistence of the db.practices collection. But how do I make that translation in this stage? Or am I supposed to do it from the client-side controller?
Some guidance would be helpful, thanks.
We have a website where users put up ads for stuff they want to sell, with parameters such as price, location, title and description. These can then be searched for using sphinx and allowing users to specify min- and maxprice, a location with a searchradius (using google maps) etc. Users can choose to save these searches and get emails when new ads appear that fit their search. Herein lies the problem: We want to perform a reverse search every time an ad is posted. With the price, location, title and description as parameters we want to search through all the saved "searches" and get the ones that would have found the ad. The min- and maxprice should just be performed in a query i suppose, and some Quorom syntax to get all ads with at least 2 or mby just 1 occurance in the title/description. Our problem lies mostly in the geo-search. How do we find all searches where the "search-circles" would include our newly posted location without performing a search for every saved search?
That is the main-question, any comment on our suggested solution to the other problems is also very welcome. Thank you in advance / Jenny
The standard 'geo-search' support on sphinx should work just as well on a Prospective Index, as a normal retrospective search.
Having built a sphinx 'index' of all the saved searches...
And you run a query using the 'ad' as the search query:- rather than the 'filter' using a fixed radius, you just use the radius from the attribute (ie the radius stored on the particular query) - if using the API cant use setFilterRange directly, need to use setSelect, to make a new virtual attribute.
$cl->setSelect("*,IF(#geodist<radius,1,0) as myfilter");
$cl->setFilter('myfilter',array(1));
(and yes, the min/maxprice can just be done with normal filters too - just inverting the logic to that you would use in a retrospective search)
... the complication is in the 'full-text' query, if the saved search is anything more than a single keyword, but you appear to have already figured out that part.
I am working on Notes in MS CRM 2011.
I have many roles over many entities.
I want role based security to notes for any entity records.
let me explain what i want:
Suppose i have an entity namely E1.
Role R1 and R2 has read and write access to E1.
But i want that user having role R2 can only upload and view notes for any record of entity E1.
Hope now my requirement is clear to all of you.
Please suggest me how can i achieve it using MS CRM 2011.
I can think of two ways to do this.
You can create a plugin on create/update of the annotation(note) and check if the note is related to entity E1 and check the roles of the user making the change and see if they only have the R2 role. If that is the case you can throw an InvalidPluginExecutionException with a message like 'You do not have permissions to edit/create these records'.
You can try using role based forms or JS to hide the notes area for R1 users.
You probably want to use a combination of #1 & #2. The users can still access the notes via advanced find and thus will be able to edit those notes. The plugin will prevent that fringe case as well.
*Edit
There are a couple more things that you might be able to deal with the advanced find records. You can remove the annotation entity from advanced find via the unsupported method described here.
Otherwise there is one more thing you can do if you want to prevent those results showing up at all, and you want to stay supported. You can write a plugin on Post-RetrieveMultiple of the annotation entity to strip out the results directly from the return result. There are a couple downsides to this though.
You are executing your plug-in every time the retrieve multiple is called on the entity. So this code will need to be as efficient as possible since that delay will be noticeable by the end user whenever they retrieve these records.
Things like advanced find will display odd results. For example if your paging is set to 50 records and you strip out 10, they will only see 40 records on their page and the total record count will include the records you are stripping out.
Through roles i don't know a way to do that, because you configure the access to notes generic, so applies to all entities. You have to access with Javascript navigating in DOM. Check a example:
document.getElementById("notescontrol").contentWindow.document.getElementById("NotesTable")
You can check this with the help of a develeper tool in your browser.
Like from Microsoft - "KB2756872" or from National Vulnerability
Database - "CVE-2010-1428" or from Red Hat - "RHSA-2010:0376" or
from OIDs - "1.3.6.1.4.1.311" or from UUID/GUID
- "550e8400-e29b-41d4-a716-446655440000".
I want to put several jobs to UIDs. See next...
I develop blog software and have idea to put unique ID in body of
each post so can easily identify that copy from local storage is
correspond to remote published copy.
Also I want to post to many different blogging services so if one
is down articles will be accessible from another. So link can
dead but if I add UID - anyone can try web-search to find post on
another service!
Also this allow to gather some article spreading
statistics. Many sites just replicate content (copy-writing and
rewriting bots and people) to broke search engines. With UID I
easily can identify such sites...
So my question how is to make UIDs (in which form) so it would be
easily indexed by search engines (web, like Google/Yahoo, and
corporate, like Lucene/Solr/Sphinx/Xapian/etc).
I know about some limitation of search engine like:
only >= 3 chars for each search part
it was not indexed dust like gfh6wytrh6wu56he5gahj763
so this task s not easy...
Any advice is appreciated (books/blog articles/etc).
You could use Tag URIs, as defined by RFC 4151.
They are globally unique, and everyone who owned a domain name or an email address for at least a day can mint them.
Note that these URIs only identify, they don’t locate. So a Tag URI doesn’t say anything about where something is published.
Let’s say your site’s domain is "example.com". If you create a blog post, you could create the following Tag URI:
tag:example.com,2012-12:cute-cat
Note that the date in this URI is not a publication date! It must be a (past) date on which you owned the domain (resp. email address). If you registered your domain in 2003, you could always use Tag URIs starting with tag:example.com,2004: (not "2003", because "2003" would mean "2003-01-01", which might be a time where you didn’t own the domain yet), followed by a (unique) string under your control. However, if you like you could always use the publication date, of course. But don’t use future dates.
You can use year and number based article identifier just like CVE identifiers. Since you need revisions as well, you can append dot after the identifier to clarify the version. For example, for an AWesome Blog Service, AWBS-2012-1.0 would refer to original document, AWBS-2012-1.1 would refer to first revision etc.
However, you need to make sure that AWBSs are unique before you use them. CVEs are assigned manually from the pool. You would probably need some kind of service that assigns AWBS from pool. It could be a simple database query.
I'm doing a notification system for my website.
A notification systeme like facebook. Or stackoverflow.
I have 2 problems.
How store in db ? I can store ALL notifications in the user document ? or in a document apart (because i think monogdb is limited for size in a document ?) Or, store intelligently ? (using inc, or a value (see: true/false) in db, with query sophisticated)
How do for brought at the page ? For exemple, when i click in a link in my inbox for stackoverflow, i'm redirect to the page. But me, i have a system that is multipage for exemple: I have 100 friends. There are listed 30 per page. So when i click on the notification i can't redirect to the because it's impossible to know the good page (users can be removed).
Thank you very much !
And if you have another ideas, tell me. Thanks.
EDIT:
(sorry for my english, i'm french)
For the first problem, i realize that i have to wait the time comes to choose my structure. Because my notification is .. a little complicated, so advance to the feeling.
For the second, i solved the problem. I explain:
(I take the exemple of friends because it's easy to undestand.)
I stored my data like this:
{
friends: [
{_id: xxxxx, ts: xxxx},
{_id: xxxxx, ts: xxxx}
]
}
Imagine i display all friends: 30 per page.
The problems are:
when i want to display all friends i cant sort using mongo. (a little problem)
If i want to lead a user to this list (30 per page) at a special friend, always keeping the sort by ts. I can't know the page. The uniq solution is to take all document.
But: veryyy bad in performance.
So, i store like this:
{
friends: {
xxxx: {ts:xxx},
xxxx: {ts:xxx}
}
}
Know i can sort the document, with use skip and limit.
So if i want a portion, i do not need to take all documents.
To know the page, i just do the number of < or > to the ts, i have for exemple 11 friends who are > to the ts of the friends that i want, and do a count for all friends (ex: 50 friends) with 50 and 11, i can guess the page.
Is this solution is good ?
- i need a count
- a query to know the number of > or <
and i can take the page where is listed the friend, keeping the sort ts
You can don't understand why i use a count. I need because they are not store in the same docment.
2 EDIT:
The problem with this solution, is that i need to make query object and update object outside of the mongo query (ex: for do friends.xxxxxx: {$exists:true}
ps: And what advantages are to use ts instead of date for mongodb ?
I'm using ts but i think i will store date, and no ts.
3 EDIT
I will do like Sammaye. Store in separate document. Take a look at: http://mongly.com/Multiple-Collections-Versus-Embedded-Documents/#1 and http://openmymind.net/2012/1/30/MongoDB-Embedded-Documents-vs-Multiple-Collections/
#Stennie make a pretty complete answer.
However recently I did a similar thing in PHP for my website. The first thing to understand is whether you are doing a notification system or a wall (the two are very different), it seems unclear to me and I am not sure what you mean by:
How do for brought at the page ? For exemple, when i click in a link
in my inbox for stackoverflow, i'm redirect to the page. But me, i
have a system that is multipage for exemple: I have 100 friends. There
are listed 30 per page. So when i click on the notification i can't
redirect to the because it's impossible to know the good page (users
can be removed).
That is not very good English and is very confusing when I read it. If you can expand on that I am sure people can answer better.
For a notification system I found that a large collection of notification objects also worked. So I had a schema like:
{
_id: {},
to_user: ObjectId{},
user_id: ObjectId{}, // Originating user
custom_text: "has posted a new comment on your wall post",
read: false,
ts: MongoDate()
}
And this would literally be the document I have to produce notifications. Each time a user commits an action that generates a notification it writes a new row to the DB with to_user being populated each time with each user needing to be notified. As for multiple users commiting the same action I actually convert the user_id field in a list of OjbectId's so I can say:
Sam, Dan and Mike all commented on your wall post
I then query by ts storing the last ts that the user looked at in their row allowing me to do a range based query on the newest notifications each time. This works quite well for sharding and querying in my personal experience.
Hope it helps,
Whether to embed or link is a common question for data modelling in MongoDB. If your number of notifications is going to be unbounded, you are likely going to be better saving these in a separate collection.
The current 16Mb document limit actually isn't as much as an issue as some other considerations:
A performance issue you may encounter by including all notifications in a single document is that fast-growing documents may also need to be relocated in the database more frequently (see Padding Factor).
You may want to be applying multiple updates to a document (such as setting a "read" flag on notifications) in a very short period of time, which means more contention for updating the same document (see Atomic Operations).
In order to implement paging you can use limit() in combination with a range query or skip(). A range query (eg. based on an indexed notificationDate) will make more effective use of indexes and perform better than skip() as your collection grows.