I believe the answer is no but I'd like confirmation.
With Fabric, endorsers simulate the transaction upon latest state and prepare the proposal adding the read and write set of keys.
At the commit phase, the peer will receive a block from the ordering service and the write set update is only applied if the read set has not been updated (versioning check).
So for the same block, the same key cannot be updated by 2 different transactions of the same block.
If it is the case, aggregating value and maintaining balance on-chain might be problematic for frequent transactions use-case. Such operation should be left for off-chain application layer.
So for the same block, the same key can not be updated by 2 different transactions of the same block.
The above is correct. Hyperledger Fabric uses an MVCC-like model in order to prevent collisions (or "double spend"). You'll want to wait for the previous state change transaction to commit before attempting to update the state again.
Related
This is a question about StateDB automatic fix after tampering.
We're wondering if manipulation to StateDB (CouchDB) could be detected and fixed automatically by peer.
The following documents states there is a a state reconciliation process synchronizing world state in Peer.
https://hyperledger-fabric.readthedocs.io/ja/latest/gossip.html#gossip-messaging
In addition to the automatic forwarding of received messages, a state
reconciliation process synchronizes world state across peers on each
channel. Each peer continually pulls blocks from other peers on the
channel, in order to repair its own state if discrepancies are
identified.
But when we test it as follows:
Step 4: modify value of key a in StateDB
Step 10: Wait for 15 minutes
Step 11: Create a new block
Step 12: Check value of a through chaincode and confirm it in StateDB directly
The tampered value is not fixed automatically by peer.
Can you help clarify the meaning of "state reconciliation process" in the above document and if peer would fix the tampering to StateDB.
Thank you.
Gossip protocol is to sync up legit data among peers, not tampered data in my view. What is legit data? Data whose computed hash at anypoint of time matches with the orignally computed hash, which will not be the case for the tampered data, and so I'd not expect Gossip protocol to sync such 'dirty data'. This defeats the purpose of Blockchain altogether as a technology, and hence this is a wrong test to be performed in my view.
Now, then what is Gossip protocol? Refer https://hyperledger-fabric.readthedocs.io/ja/latest/gossip.html#gossip-protocol
"Peers affected by delays, network partitions, or other causes
resulting in missed blocks will eventually be synced up to the current
ledger state by contacting peers in possession of these missing
blocks."
So, in cases where the peer should have comitted a block to ledger and would have missed due to some reasons like the ones said above, 'Gossip' is only a fall back strategy for the HLF to reconcile the ledger among the peers.
Now in your test case, I see you are using 'query', now query does not go via the orderer to all the peers, it just goes to one peer and returns the value, you need to do a 'getStringState' as a 'transaction', for the 'enderosement' to run, and that is when would the endorsement fail citing the mismatch between the values for the same key among the peers is what I'd expect.
# Gossip state transfer related configuration
state:
# indicates whenever state transfer is enabled or not
# default value is true, i.e. state transfer is active
# and takes care to sync up missing blocks allowing
# lagging peer to catch up to speed with rest network
enabled: false
.......................................................................................................
# the process of reconciliation is done in an endless loop, while in each iteration reconciler tries to
# pull from the other peers the most recent missing blocks with a maximum batch size limitation.
# reconcileBatchSize determines the maximum batch size of missing private data that will be reconciled in a
# single iteration.
reconcileBatchSize: 10
# reconcileSleepInterval determines the time reconciler sleeps from end of an iteration until the beginning
# of the next reconciliation iteration.
reconcileSleepInterval: 1m
# reconciliationEnabled is a flag that indicates whether private data reconciliation is enable or not.
reconciliationEnabled: true
Link: https://github.com/hyperledger/fabric/blob/master/sampleconfig/core.yaml
In addition to the automatic forwarding of received messages, a state reconciliation process synchronizes world state across peers on each channel. Each peer continually pulls blocks from other peers on the channel, in order to repair its own state if discrepancies are identified. Because fixed connectivity is not required to maintain gossip-based data dissemination, the process reliably provides data consistency and integrity to the shared ledger, including tolerance for node crashes.
In the documentation for the transaction flow of hyperledger fabric it is mentioned that
"The ordering service does not need to inspect the entire content of a transaction in order to perform its operation, it simply receives transactions from all channels in the network, orders them chronologically by channel, and creates blocks of transactions per channel."
I have a couple of questions here
What does "chronological ordering" mean?. Does it mean that the transactions for a channel are ordered depending on the time they are received at the Ordering service node (Leader) ?
If two client applications are submitting an update transaction for the same key on the ledger almost at the same time [Let us call them tx1 (updating key x to value p) , tx2 (updating key x to value q)], all the endorsing peers will simulate the update transaction proposal and return the write set in the transaction proposal response. When the clients send these endorsement proposal requests to ordering service nodes , in which order will these update transactions be ordered in a block ?.
The order of transactions in the block can be tx1,tx2 OR tx2,tx1 , assuming the update transaction has only the write set and no read set , both the transactions in either orders are valid. What will be the final value of the key on the ledger [p or q] ?
I am trying to understand if the final value of x is deterministic , and what factors would influence it.
What does "chronological ordering" mean?. Does it mean that the transactions for a channel are ordered depending on the time they are received at the Ordering service node (Leader)?
In general, the orderer makes no guarantees about the order in which messages will be delivered just that messages will be delivered in the same order to all peer nodes.
In practice, the following generally holds true for the current orderer implementations:
Solo - messages should be delivered in the order in which they were received
Kafka - messages should be delivered in the order in which they were received by each orderer node and generally even in the order they are received across multiple ordering nodes.
This holds true for the latest fabric version as well.
If two client applications are submitting an update transaction for the same key on the ledger almost at the same time [Let us call them tx1 (updating key x to value p) , tx2 (updating key x to value q)], all the endorsing peers will simulate the update transaction proposal and return the write set in the transaction proposal response. When the clients send these endorsement proposal requests to ordering service nodes, in which order will these update transactions be ordered in a block ?.
When you submit a transaction, the peer generates a read and write set. This read/write set is then used when the transaction is committed to the ledger. It contains the name of the variables to be read/written and their version when they were read. If, during the time between set creation and committing, a different transaction was committed and changed the version of the variable, the original transaction will be rejected during committal because the version when read is not the current version.
To address this, you will have to create data and transaction structures that avoid concurrently editing the same key, other you might get MVCC_READ_CONFLICT error.
In Bitcoin blockchain, transactions are only of the type "transferring some amount from X to Y", so we always have an actual change the blockchain.
In Hyperledger Fabric transaction are like:
Instatiate some chaincode in a channel
Invoke some chaincode (that may be just a query to show a certain
asset in the ledger / world state)
I'm sure that instantiating chaincode in some channel or running something that changes the world state are transaction that qualify to fit in a block.
What I don't understand is exactly which transactions can enter in the next block. If some chaincode does not write or update the world state (in RW sets, the Read set is the same of the Write one) can still be included in the next block? Or can be safely "discarded"?
All endorsing peer must have instantiated chaincode so that they can simulate transactions and create Read/Write(RW) set. Installing and instantiating chaincode are different transaction than regular invoke.
Invoking some transaction meansa it can be one of
Query
Adding Asset, Updating Asset, Deleting Asset
HF maintains versioning of each unique key and it increases chronologically when we update same asset. This versioning (MVCC - Multiversion concurrency control) avoid double-spending problem as well.
HF dont care about whatever value we puts for key(Value could be anything).
Lets assume we have to add car asset with key car1 and value is {"name":"Audi", "owner":"ABC"}.
When we send the transaction to all endorsing peer, they create RW set as below
Read Set: NA
Write set: Key-car1, version-1, value-{"name":"Audi", "owner":"ABC"}
When the tx gets committed to blockchain, one of the block will have this transaction and the current state database (Couch db or level db) will heve this latest value for that key : Key-car1, version-1, value-{ "name": "Audi", "owner": "ABC"}
Let's assume if we updating this same asset(car1)
car1 - { "name": "Audi", "owner": "PQR"}
Here we are changing owner from ABC to PQR
This time Endorsing peer create the following RW set
Read Set: Key-car1, version-1
Write set: Key-car1, version-2, value-{"name":"Audi", "owner":"PQR"}
Once the tx gets committed to the blockchain. again this transaction gets added into one block and most importantly, the current state database will get updated with this latest value and old value will be updated and the version also get changed to 2.
Just a general question, if I'm building a blockchain for a business I want to store 3 years of transactions but anything older than that I won't need and don't want actively in the working database. Is there a way to backup and purge a blockchain or delete items older than some time frame? I'm more interested in the event logic than the forever memory aspect.
I'm not aware of any blockchain technology capable of this yet, but Hyperledger Fabric in particular is planning to support data archiving (checkpointing). Simply put, participants need to agree on a block height, so that older blocks can be discarded. This new block then becomes the source of trust, similar to original genesis block. Also, snapshot needs to be taken and consented, which captures current state.
From serviceability point of view, it's slightly more complicated, i.e. you may have nodes that are down while snapshotting, etc.
If you just want to purge the data after a while, Fabric Private Data has an option which could satisfy your desire.
blockToLive Represents how long the data should live on the private database in
terms of blocks. The data will live for this specified number of
blocks on the private database and after that it will get purged,
making this data obsolete from the network so that it cannot be
queried from chaincode, and cannot be made available to requesting
peers
You can read more here.
Personally, I don't think there is a way to remove a block from the chain. It might destroy the Immutable property of blockchain.
There are 2 concepts which help you achieve your goals.
The one thing is already mentioned. It is about Private Data. Private data gives you the possibility to 'label' data with a time to live. Then only the private data hashes are stored on the chain (to be able to verify this transaction) but the data itself is stored in so called SideDBs and gets fully pruned (except the hashes on the chain of course). This is kind of the basis for using Fabric without workarounds and achieving GDPR.
The other thing, which was not mentioned yet and kind of is very helpful to this question
Is there a way to backup and purge a blockchain or delete items older than some time frame?
Every peer only stores the 'current state' of the ledger in his StateDB. The current state could be described as the data which is labeled 'active' and probably soon to be used again. You can think of the StateDB being like a Cache. Every Data is comes into this Cache by creating or updating a new key (invoking). To remove a key from the Cache you can use 'DelState'. So it is labeled 'deleted' and not in the Cache anymore. BUT it is still on the ledger! and you can retrieve the history and data to that key.
Conclusion: For 'real' deleting of data you have to use the concept of Private Data and for managing data in your StateDB (think of the 'Cache' analogy) you can simply use built in functions.
I have a scenario where i have to update multiple transactions in ledger at the same time.
In simple case, two transactions has to executed at the same time in order to make the use case valid. if anyone of them fails the other one should revert back.
err = stub.PutState(key, tradeJSONasBytes)
using hyperledger 1.1 and golang smart contract.
If you want to save multiple transaction you can call multiple PutState() but there is nothing like reverting the transaction, even if the transaction fails it is still stored as a block, as it is the part of immutability condition.