no duplication in database lotus note - lotus-notes

good day every one.
what are the effect of re-cache if i use it to my updating data?
This is what i trying to do, but i need to know what will happen if i replace the no-cache to re-cache
i will explain the idea,
in this textbox it will call the last digit in my database and add 1, but i figure out that there is a problem in this, if multiple user using this form there are possibility that the number will duplicate.#If(#IsNewDoc;#Elements(#DbColumn("":"nocache";#DbName;"GPA";1))+1;#Return(GPnum))
my thought guide me in this path, if i use #dblookup to find if there is a duplication in my number but, i dont make it thru .

Recache will not help you avoid duplicates.
You are trying to increment a counter in Lotus Notes to create a unique sequential identifier for documents. This is a problem that has been discussed many times, by many people, for at least 20 years. You can find good information here in StackOverflow and in various other forums, blogs, and articles. The two approaches that work are
Store the last counter value in a config doc, and use document locking to assure that you don't have two users accessing and updating it at the same time.
Do not set the counter variable directly in user code. Write your code to put a "pending" value in the field, and rely on a scheduled or triggered background agent that runs on only one server to set the final value. (Since the Agent Manager guarantees that only one agent can run at a time in one database, you will not have conflicts.)
Don't use a sequential counter for your identifier. Use the #Unique function instead. Documents will have a unique code instead of a unique number.
Please see this answer, and this answer, and this article.

Related

Couchbase retrieving relational docs in nodeJS

I am still debating which way to go and possibly store certain information in its own doc. so for example the customer can have addresses with each address would be its own doc and then in the customer doc there would be an array of ref keys stored under addresses. The benefit would be i could update these docs simply based on the key value vs having to get the customer doc first, finding the array index of the address and then either modify the whole doc or go and use subdoc to replace the content of the array with the index.
Where i am stuck is how to retrieve those referenced subdoc's. is N1QL the only way to go or does the KV API offer a way to do this short of retrieving the whole customer doc, then looping thru address array and retrieving all referenced docs that way. I know Ottoman offers something like that but i am having an issue with the latest version of SDK 2.6 and Ottoman as its not very well maintained. So hopefully someone can share some insight what and why its the best way.
If you want to rely on key/value, then you'll need to do the multiple lookup as you've described. I'm not very familiar with Ottoman: it might do this for you, but behind the scenes it will still be multiple key/value operations and/or N1QL.
With N1QL, you can perform JOINs, but again, behind the scenes it's going to eventually be pulling documents out by key/value. It just does those extra steps for you. Direct key/value is always going to be the fastest route.
If you are still in the process of deciding whether to split the data amongst multiple documents or "denormalize" the data into a single doc, one thing you should think about is how often you're going to access customer+addresses together and how often you're going to customer/access separately. If you're reading/writing customer+address often, consider putting it in one document. Otherwise, consider putting it in multiple documents.
The third option is to store it both places, or rather "cache" the address data in the customer document. This is tricky, because it could get out of sync if you're not careful. So make sure it's worth it before you go down that road.

How to insert an auto increment / sequential number value in CouchDB documents?

I'm currently playing with couchDB a bit and have the following scenario:
I'm implementing an issue tracker. Requirement is that each issue document has (besides it's document _id) a unique numerical sequential number in order to refer to it in a more appropriate way.
My first approach was to have a view which simply returns the count of unique issue documents currently stored. Increment that value on the client side by 1, assign it to my new issue and insert that.
Turned out to be a bad idea, when inserting multiple issues with ajax calls or having multiple clients adding issues at the same time. In latter case is wouldn't be even possible without communication between clients.
Ideally I want the sequential number to be generated on couch, which is afaik not possible due to conflicting states in distributed systems.
Is there any good pattern one could use (maybe on the client side) to approach this? I feel like this is a standard kind of use case (thinking of invoice numbers, etc).
Thanks in advance!
You could use a separate document which is empty, though it only consists of the id and rev. The rev prefix is always an integer, so you could use it as your auto incrementing number.
Just make a POST to your document, this will increase the rev and return it. Then you can use this generated value for your purpose.
Alternative way:
Create a separate document, consisting of value and lock. Then execute something like: "IF lock == true THEN return ELSE set lock = true AND increase value by 1", then do a GET to retrieve the new value and finally set lock = false.
I agree with you that using a view that gives you a document count is not a great idea. And it is the reason that couchdb uses a uuid's instead.
I'm not aware of a sequential id feature in couchdb, but think it's quite easy to write. I'd consider either:
An RPC (eg. with RabbitMQ) call to a single service to avoid concurrency issues. You can then store the latest number in a dedicated document on a specific non distributed couchdb or somewhere else. This may not scale particularly well, but you're writing a heck of an issue tracking system before this becomes an issue.
If you can allow missing numbers, set the uuid algorithm on your couch to sequential and you are at least good until the first buffer overflow. See more info at: http://couchdb.readthedocs.org/en/latest/config/misc.html#uuids-configuration

Azure DB Sequencial numbering

I'm having a few issues with designing a database in Azure at the moment, down to the following:
SQL2012 Auto-Increment keys can/will jump by 1000 fairly regularly, related to the new "feature" of SQL 2012, as documented here (link). This has been closed on "By Design" in Connect (link)
The recommendation is to use either a startup flag to avoid this behaviour (cannot do with Azure), or to use Sequences to generate incrementing numbers instead.
However, SEQUENCE is unsupported in Azure DB. It in fact alternates between being an active issue and a "won't fix" issue on Connect (link)
So, my question is how to actually go about having a field auto-increment by 1 on insert to an Azure DB table, whilst avoiding large gaps.
I did think about using a Trigger instead, and then using that to find the existing Max value. Didn't seem clean. I also thought it would cause concurrency issues.
I'm happy to have a surrogate key here, but without Sequences I am wondering what the recommended route would be to actually generate the value for the surrogate at insert time.
Any advice appreciated.
Edit: Please note I am using the older "Web/Business" type of DB rather than the new tiers; I don't know if that will make a difference to any answers.
The reason your feedback has been marked as it was is because auto-increment by 1 becomes a very hard problem to solve at a database layer at scale. If you "shard" your database (split it to cope with load) how would each shard know which number to increment to? This should be extracted from the database and built into your application logic.
Without knowing the background requirement to having an incremental number it's hard to advise of the right solution. Is it for uniqueness or for a sequence? If it's for a sequence does it really matter that it isn't by 1 each time as long as the number is (a) unique and (b) increases on each insert? Could you make the setting of this sequence an offline process? That is - use insert date / time via a batch process to assign a sequence number?
If it's for uniqueness use a GUID or similar.

CouchDB - human readable id

Im using CouchDB with node.js. Right now there is one node involved and even in remote future its not planned to changed that. While I can remove most of the cases where a short and auto-incremental-like (it can be sparse but not like random) ID is required there remains one place where the users actually needs to enter the ID of a product. I'd like to keep this ID as short as possible and in a more human readable format than something like '4ab234acde242349b' as it sometimes has to be typed by hand and so on.
However in the database it can be stored with whatever ID pleases CouchDB (using the default auto generated UUID) but it should be possible to give it a number that can be used to identify it as well. What I have thought about is creating a document that consists of an array with all the UUIDs from CouchDB. When in node I create a new product I would run an update handler that updates said document with the new unique ID at the end. To obtain the products ID I'd then query the array and client side using indexOf I could get the index as a short ID.
I dont know if this is feasible. From the performance point of view I can say the following: There are more queries that should do numerical ID -> uuid than uuid -> numerical ID. There will be at max 7000 new entries a year in the database. Also there is no use case where a product can be deleted yet I'd like not to rely on that.
Are there any other applicable ways to genereate a shorter and more human readable ID that can be associated with my document?
/EDIT
From a technical point of view: It seems to be working. I can do both conversions number <-> uuid and it seems go well. I dont now if this works well with replication and stuff but as there is said array i guess it should, right?
You have two choices here:
Set your human readable id as _id field. Basically you can just set in create document calls to DB, and it will accept it. This can be a more lightweight solution, but it comes with some limitations:
It has to be unique. You should also be careful about clients trying to create documents, but instead overwrite existing ones.
It can only contain alphanumeric or a few special characters. In my experience it is asking for trouble to have extra character types.
It cannot be longer than a theoretical string length limit(Couchdb doesn't define any, but you should). Long ids will increase your views(indexes) size really bad. And it might make it s lower.
If these things are no problem with you, then you should go with this solution.
As you said yourself, let the _id be a UUID, and set the human readable id to another field. To reach the document by the human readable id, you can just create a view emitting the human readable id as a key, and then either emit the document as value or get the document via include_docs=true option. Whenever the view is reached Couchdb will update the view incrementally and return you the list. This is really same as you creating a document with an array/object of ids inside it. Except with using a couchdb view, you get more performance.
This might be also slightly slower on querying and inserting. If the ids are inserted sequentially, it's fine, if not, CouchDB will slightly take more time to insert it at the right place. These don't work well with huge amounts of insert coming at the DB.
Querying shouldn't be more than 10% of total query time longer than first option. I think 10% is really a big number. It will be most probably less than 5%, I remember in my CouchDB application, I switched from reading by _id to reading from a view by a key and the slow down was very little that from user end point, when making 100 queries at the same time, it wasn't noticeable.
This is how people, query documents by other fields than id, for example querying a user document with email, when the user is logging in.
If you don't know how couchdb views work, you should read the views chapter of couchdb definite guide book.
Also make sure you stay away from documents with huge arrays inside them. I think CouchDB, has a limit of 4GB per document. I remember having many documents and it had really long querying times because the view had to iterate on each array item. In the end for each array item, instead I created one document. It was way faster.

How to create an auto incrementing field in lotus domino?

I have an application in lotus domino with a field
UNIDID-number(computed)...
I want that every time a new entry is created, this field to increment by 1 and the new value should be stored in new record document..
I have a #dbcolumn formula which will get me the last entry in the UNIDID field-
mFind:=#DbColumn("" : "NoCache" ; #DbName ; "lkpEmpMasterbyOnlyUnidCode";1);
How do I increment mFind and submit it in form of 'UNIDXXXX'?
Employ document locking to assure number uniqueness in sequential document numbering solution
To answer your specific question:
lastEntry := #Subset(mFind;-1);
"UNID" + lastEntry;
But here are a couple of things you should think about:
"UNID" is a term that has a very specific meaning in Notes and Domino. It refers to the Universal ID that is automatically assigned to every document in every database. If you use this term in your application for another purpose, you are likely to cause confusion someday when some other Notes expert has to look at your application.
The best way to assign a sequential id is to let the server do it for you. I.e., save the document with an empty field, and create an agent that runs on new and edited documents, checks for the empty field, and assigns the next available id if necessary. Since only one agent can run in the database at a time, and it works on only one document at a time, this guarantees that you can't possibly have two documents that are saved simultaneously with the same unique id. The downside of this is that you can't show the user the sequential id before the agent gets a chance to run.
Read the comprehensive treatment of sequential numbering in Notes/Domino written by IBMer Andre Guirard. He considers the use of #DBColumn for sequential numbering a significant performance risk and provides algorithms and code for alternative approaches.
I recommend the approach under the heading "Number Generation On Demand" using the code in Listing 2. Document locking is overkill- merely detecting whether the numbering document has been modified since it was read and reacting accordingly is sufficient.
This formula is from my technical notes database, generates a sequentially increasing number using a computed when composed field on the form so user can see the number but it is only fixed when you save the document.
T_List:=#DbColumn("" : "NoCache"; ""; "RefNumView"; 1);
#If(#IsNewDoc & #Elements(T_List)=0;1;#IsNewDoc & !#IsError(T_List);#Subset(T_List;1) + 1;RefNumber)
You need a view first column sorted in descending order with the field that stores your number, if the view has no records the code above starts numbering at 1
The database was developed in Notes 4.5 but I'm still using it now with 8.5 notes client & designer and have never needed to change the formula, indeed its been reused many times over the years in all the later versions. It won't work if documents are created on multiple servers, for that the scheduled agent is the only way to get a truly unique sequential numbering.

Resources