I have an application in lotus domino with a field
UNIDID-number(computed)...
I want that every time a new entry is created, this field to increment by 1 and the new value should be stored in new record document..
I have a #dbcolumn formula which will get me the last entry in the UNIDID field-
mFind:=#DbColumn("" : "NoCache" ; #DbName ; "lkpEmpMasterbyOnlyUnidCode";1);
How do I increment mFind and submit it in form of 'UNIDXXXX'?
Employ document locking to assure number uniqueness in sequential document numbering solution
To answer your specific question:
lastEntry := #Subset(mFind;-1);
"UNID" + lastEntry;
But here are a couple of things you should think about:
"UNID" is a term that has a very specific meaning in Notes and Domino. It refers to the Universal ID that is automatically assigned to every document in every database. If you use this term in your application for another purpose, you are likely to cause confusion someday when some other Notes expert has to look at your application.
The best way to assign a sequential id is to let the server do it for you. I.e., save the document with an empty field, and create an agent that runs on new and edited documents, checks for the empty field, and assigns the next available id if necessary. Since only one agent can run in the database at a time, and it works on only one document at a time, this guarantees that you can't possibly have two documents that are saved simultaneously with the same unique id. The downside of this is that you can't show the user the sequential id before the agent gets a chance to run.
Read the comprehensive treatment of sequential numbering in Notes/Domino written by IBMer Andre Guirard. He considers the use of #DBColumn for sequential numbering a significant performance risk and provides algorithms and code for alternative approaches.
I recommend the approach under the heading "Number Generation On Demand" using the code in Listing 2. Document locking is overkill- merely detecting whether the numbering document has been modified since it was read and reacting accordingly is sufficient.
This formula is from my technical notes database, generates a sequentially increasing number using a computed when composed field on the form so user can see the number but it is only fixed when you save the document.
T_List:=#DbColumn("" : "NoCache"; ""; "RefNumView"; 1);
#If(#IsNewDoc & #Elements(T_List)=0;1;#IsNewDoc & !#IsError(T_List);#Subset(T_List;1) + 1;RefNumber)
You need a view first column sorted in descending order with the field that stores your number, if the view has no records the code above starts numbering at 1
The database was developed in Notes 4.5 but I'm still using it now with 8.5 notes client & designer and have never needed to change the formula, indeed its been reused many times over the years in all the later versions. It won't work if documents are created on multiple servers, for that the scheduled agent is the only way to get a truly unique sequential numbering.
Related
I have a view in my Xpage application that contains a lot of elements. From this view I need to build a table with custom rows (I can't just display the view, I need to build the rows to display myself because I need to compute data from other database, things that you can't do directly in a view).
In order to do so I know that I can use Dataview, Datatable or repeat control (other ideas maybe?). For sure I can't bring all the data on the client, it's way too much.
I am looking for a solution that will allow me to do paging (easy to do with the pager component) but more important sorting on header click. To be clear, I need sorting for all the entries of the view and not only for the current displayed page on the client.
What can be the more efficient way to do so ? I really have a lot of data to compute so I need the fastest way to do it.
(I can create several views with different sorting criteria if needed).
Any repeating control can have pagers applied to it. Also View Panels can include data not in the current view - just set the columnName property to blank and compute the value property. bear in mind you will not be able to sort on those column though - they're not columns, they're values computed at display time.
Any computed data is only computed for the entries currently shown. So if you have 5000 entries in the view but are only displaying 30 at a time, the computed data will only be computed for the current 30.
If your users need to be able to sort on all columns and you have a lot of data, basically they have to accept that they're requirements mean all that data needs computing when they enter the view...and whenever it's updated, by themselves or any other users. That's never going to be quick, and the requirements are the issue there, not the architecture. RDBMS may be better as a back-end, if that's a requirement, as long as the data doesn't go across multiple tables. Otherwise graph database structure may be a better alternative.
The bigger question is why the users need to sort on any column. Do the users really want to sort on the fifth column and then scroll down to entries beginning with a "H"? Do they want to sort on the fourth column and scroll down to entries for May 2014? On a Notes Client, that's a traditional approach, because it's easier than filtering. But usually users know what they're looking for - they don't want entries beginning "H", they want entries where the department is HR. If that's the case, sorting on all columns and paging is not the most efficient method either from a database design or a usability point of view.
To keep the processing faster and lightweight, I use JSON with JQuery DataTables.
Depending on the Data-size and usage, JSON could be generated on the fly or scheduled basis and saved in Lotus Notes Documents or ApplicationScope variables.
$.each(data, function(i, item) {
dataTable.row.add( [data[i].something1,data[i].something2,data[i].something3])
});
You can compute a viewColumn but if you have a lot going on I wouldn't go that route.
This is where Java in XPages SHINE!
Build a Java object to represent your row. So in java use backend logic to get all the data you need. Let's say you have a report of Sales Orders for a a company. And sales orders is pulling data from different places. Your company object would have a method like:
List<salesOrder> getOrders() {}
so in the repeat you call company.getOrders() and it returns all the rows that you worked out in java and populated. So your "rowData" collection name in the repeat can access all the data you want. Just build it into a table.
But now the sorting... We've been using jQuery DataTables to do just this.. It's all client side... your repeat comes down and then the DataTables kicks in and can make everything sortable... no need to rely on views.. works great...
Now it's all client side but supports paging and works pretty decent. If you're just pumping out LOTS of records - 6,000+ then you might want to look at outputting the data as json and taking advatange of some server cacheing... We're starting to use it with some really big output.. LOTS of rows and it's working well so far. Hopefully I'll have some examples on NotesIn9.com in the near future.
I'm currently playing with couchDB a bit and have the following scenario:
I'm implementing an issue tracker. Requirement is that each issue document has (besides it's document _id) a unique numerical sequential number in order to refer to it in a more appropriate way.
My first approach was to have a view which simply returns the count of unique issue documents currently stored. Increment that value on the client side by 1, assign it to my new issue and insert that.
Turned out to be a bad idea, when inserting multiple issues with ajax calls or having multiple clients adding issues at the same time. In latter case is wouldn't be even possible without communication between clients.
Ideally I want the sequential number to be generated on couch, which is afaik not possible due to conflicting states in distributed systems.
Is there any good pattern one could use (maybe on the client side) to approach this? I feel like this is a standard kind of use case (thinking of invoice numbers, etc).
Thanks in advance!
You could use a separate document which is empty, though it only consists of the id and rev. The rev prefix is always an integer, so you could use it as your auto incrementing number.
Just make a POST to your document, this will increase the rev and return it. Then you can use this generated value for your purpose.
Alternative way:
Create a separate document, consisting of value and lock. Then execute something like: "IF lock == true THEN return ELSE set lock = true AND increase value by 1", then do a GET to retrieve the new value and finally set lock = false.
I agree with you that using a view that gives you a document count is not a great idea. And it is the reason that couchdb uses a uuid's instead.
I'm not aware of a sequential id feature in couchdb, but think it's quite easy to write. I'd consider either:
An RPC (eg. with RabbitMQ) call to a single service to avoid concurrency issues. You can then store the latest number in a dedicated document on a specific non distributed couchdb or somewhere else. This may not scale particularly well, but you're writing a heck of an issue tracking system before this becomes an issue.
If you can allow missing numbers, set the uuid algorithm on your couch to sequential and you are at least good until the first buffer overflow. See more info at: http://couchdb.readthedocs.org/en/latest/config/misc.html#uuids-configuration
Im using CouchDB with node.js. Right now there is one node involved and even in remote future its not planned to changed that. While I can remove most of the cases where a short and auto-incremental-like (it can be sparse but not like random) ID is required there remains one place where the users actually needs to enter the ID of a product. I'd like to keep this ID as short as possible and in a more human readable format than something like '4ab234acde242349b' as it sometimes has to be typed by hand and so on.
However in the database it can be stored with whatever ID pleases CouchDB (using the default auto generated UUID) but it should be possible to give it a number that can be used to identify it as well. What I have thought about is creating a document that consists of an array with all the UUIDs from CouchDB. When in node I create a new product I would run an update handler that updates said document with the new unique ID at the end. To obtain the products ID I'd then query the array and client side using indexOf I could get the index as a short ID.
I dont know if this is feasible. From the performance point of view I can say the following: There are more queries that should do numerical ID -> uuid than uuid -> numerical ID. There will be at max 7000 new entries a year in the database. Also there is no use case where a product can be deleted yet I'd like not to rely on that.
Are there any other applicable ways to genereate a shorter and more human readable ID that can be associated with my document?
/EDIT
From a technical point of view: It seems to be working. I can do both conversions number <-> uuid and it seems go well. I dont now if this works well with replication and stuff but as there is said array i guess it should, right?
You have two choices here:
Set your human readable id as _id field. Basically you can just set in create document calls to DB, and it will accept it. This can be a more lightweight solution, but it comes with some limitations:
It has to be unique. You should also be careful about clients trying to create documents, but instead overwrite existing ones.
It can only contain alphanumeric or a few special characters. In my experience it is asking for trouble to have extra character types.
It cannot be longer than a theoretical string length limit(Couchdb doesn't define any, but you should). Long ids will increase your views(indexes) size really bad. And it might make it s lower.
If these things are no problem with you, then you should go with this solution.
As you said yourself, let the _id be a UUID, and set the human readable id to another field. To reach the document by the human readable id, you can just create a view emitting the human readable id as a key, and then either emit the document as value or get the document via include_docs=true option. Whenever the view is reached Couchdb will update the view incrementally and return you the list. This is really same as you creating a document with an array/object of ids inside it. Except with using a couchdb view, you get more performance.
This might be also slightly slower on querying and inserting. If the ids are inserted sequentially, it's fine, if not, CouchDB will slightly take more time to insert it at the right place. These don't work well with huge amounts of insert coming at the DB.
Querying shouldn't be more than 10% of total query time longer than first option. I think 10% is really a big number. It will be most probably less than 5%, I remember in my CouchDB application, I switched from reading by _id to reading from a view by a key and the slow down was very little that from user end point, when making 100 queries at the same time, it wasn't noticeable.
This is how people, query documents by other fields than id, for example querying a user document with email, when the user is logging in.
If you don't know how couchdb views work, you should read the views chapter of couchdb definite guide book.
Also make sure you stay away from documents with huge arrays inside them. I think CouchDB, has a limit of 4GB per document. I remember having many documents and it had really long querying times because the view had to iterate on each array item. In the end for each array item, instead I created one document. It was way faster.
good day every one.
what are the effect of re-cache if i use it to my updating data?
This is what i trying to do, but i need to know what will happen if i replace the no-cache to re-cache
i will explain the idea,
in this textbox it will call the last digit in my database and add 1, but i figure out that there is a problem in this, if multiple user using this form there are possibility that the number will duplicate.#If(#IsNewDoc;#Elements(#DbColumn("":"nocache";#DbName;"GPA";1))+1;#Return(GPnum))
my thought guide me in this path, if i use #dblookup to find if there is a duplication in my number but, i dont make it thru .
Recache will not help you avoid duplicates.
You are trying to increment a counter in Lotus Notes to create a unique sequential identifier for documents. This is a problem that has been discussed many times, by many people, for at least 20 years. You can find good information here in StackOverflow and in various other forums, blogs, and articles. The two approaches that work are
Store the last counter value in a config doc, and use document locking to assure that you don't have two users accessing and updating it at the same time.
Do not set the counter variable directly in user code. Write your code to put a "pending" value in the field, and rely on a scheduled or triggered background agent that runs on only one server to set the final value. (Since the Agent Manager guarantees that only one agent can run at a time in one database, you will not have conflicts.)
Don't use a sequential counter for your identifier. Use the #Unique function instead. Documents will have a unique code instead of a unique number.
Please see this answer, and this answer, and this article.
Just what it says on the tin. I have a bunch of (say) Member documents, I want each user to be able to set the Email field on his document to whatever he wants, example an Email that exists on another document. If I just do a check-insert, I'm vulnerable to a race condition. Is there some idiom for "locking" or inserting-then-checking?
The only sure way is to create a doc with the unique value as doc id.
As the other answer notes, the only field guaranteed to be unique in CouchDB is _id.
You might borrow a trick from the replicator here. In order to fast-forward a second replication between the same two hosts, it writes a checkpoint document which records the update sequence it last reached. But how does it find the checkpoint document in the future? Like so;
"_id": md5(source.host + source.port + target.host + target.port)
A general pattern can be extracted where your unique fields form part of the id itself. Running them through md5 guarantees a fixed length identifier.
In your case, you could just use email address as your id.
Changing one of these fields is a two step process, but one that still maintains the uniqueness property.
Copy the document to its new id (http://wiki.apache.org/couchdb/HTTP_Document_API#COPY)
If successful, delete the old one (http://wiki.apache.org/couchdb/HTTP_Document_API#DELETE)
A crash between steps 1 and 2 will leave the old document it place, so you may wish to add a reference to the old document in the new document. You can then create a view of those back references and perform a clean up sweep periodically.
All that said, CouchDB deliberately only supports only one unique field, as opposed to a typical RDBMS which can support elaborate relational constraints, in order for a solution to scale up cleanly in a cluster (c.f, BigCouch). In your case, where email address must be unique, much of what I've said should work (email addresses don't change that often), but obviously this is swimming upstream to a degree.
HTH,
B.