An endorsement is the process where endorsing peers execute a transaction and agree on the result. How is it different from the consensus in Fabric?
One key difference between Hyperledger Fabric and many other blockchain platforms is the lifecycle of a transaction.
In other platforms, the lifecycle of a transaction is usually Order-execute in which:
Order: transactions are added to the ledger in some order and disseminated to all peers.
Execute: transactions are sequentially executed (e.g. using smart contract code) on all peers.
While in Hyperledger Fabric, the lifecycle of a transaction is different as it is a Execute-order-validate model:
Execute: Transactions are executed (using chaincode) in any order, possibly even in parallel.
Order: When enough peers agree on the results of a transaction, it’s added to the ledger and disseminated to all peers. This step is where the transactions are first given an ordering — until transactions are added to the ledger.
Validate: Each peer validates and applies the ledger’s transactions in sequence. Now that the transactions have an ordering, the peers can check whether a later transaction was invalidated by an earlier transaction. For example, this prevents one item from being sold two times (called double-spending).
Endorsement in Hyperledger Fabric basically allows users to define policies around the execution of chaincode. These endorsement policies define which peers need to agree on the results of a transaction before it can be added to the ledger.
Now let's see how endorsement works:
Fabric starts with a transaction proposal. It’s a bundle of information used to trigger a specific chaincode. The transaction proposal is sent to some peers for endorsement. An endorsing peer executes the chaincode, which (if it succeeds) yields an actual transaction for the ledger. The endorsing peer then signs the transaction and returns it to the proposer. This is the Execute step in execute-order-validate.
Once the creator of the proposal receives enough signatures to satisfy the endorsement policy, it can submit the transaction (and the signatures) to be added to the ledger. This is the Order step.
Consensus or Validation is the last step in which all the peers verify if there are any changes in the world state in between this whole transaction process and then validates only those transactions that are valid and marks other transactions as invalid.
Related
Given the fact that all peers are running the same chaincode and have same ledger, does it make any difference whether one or multiple peers are chosen for endorsement? Isn't getting endorsement from multiple peers redundant?
Hyperledger fabric uses order-execute model by separating the transaction flow into three steps:
execute a transaction and check its correctness, thereby endorsing it,
order transactions via a (pluggable) consensus protocol, and
validate transactions against an application-specific endorsement policy before committing them to the ledger.
When a peer gets a transaction, it executes transactions against the current state data held by it to simulate the transaction.
The most probable reason for multiple peer endorsement is to ensure concurrent states across peers upon execution of the transaction and ensuring correct transaction execution, This is necessary to ensure data that was read during chaincode execution has not changed, and therefore the execution results are still valid and can be committed to the ledger state database.
I don't know some details about Fabric in this document.
Document: The application verifies the endorsing peer signatures and compares the proposal responses.
Is it necessary to have all the responses the same when verifying， or most of them？
If it is part of them, what is the ratio?
Whether it can be controlled by the Fabric SDK?
Document: The ordering service does not need to inspect the entire content of a transaction in order to perform its operation, it simply receives transactions from all channels in the network, orders them chronologically by channel, and creates blocks of transactions per channel.
Raft and Kafka are only used to guarantee availability, not for voting. Is this correct?
Document: Transaction is validated and committed.
When committing, is there a process of voting verification?
The application does not actually need to check anything (it's
optional but a good way to avoid sending invalid transactions).
When you instantiate chaincode on a channel, you set the endorsement policy for that chaincode. The endorsement policy specifies how many organization's peers must sign the response. This is actually enforced during validation on the peers (after the peer receives blocks/transaction from the orderer but prior to committing the data). If there are not enough signatures to satisfy the policy, transactions are marked invalid and state is not committed.
With Fabric v1.2 and later, clients can use the discovery service to obtain the minimum list of peers required to meet the endorsement policy for chaincode on each channel.
Kafka / Raft are used to ensure that all ordering service nodes process transactions and deliver blocks in the same order. With Fabric v1.4.1 you will have the option to use Raft consensus rather than Kafka.
See comment in 2. about enforcing endorsement policies during validation.
I have simple question about Hyperledger Fabric. so here is my question:
I know that for committing peers (ordinary peers) it's not 100% necessary to have chaincode installed, but every peer should have ledger and that's 100% necessary.
so suppose I am committing peer and I have not chaincode installed, but I have ledger of course. Now new block arrives from ordering peer.
1) I should check blocks validity, so I need to query ledger
2) I should add this block, so I need to update ledger.
So how can I do this two above without chaincode? Isn't chaincode necessary for above operations? (query and update)
To check the transaction/block's validity, the peer does not need to have the chaincode, the readwrite set available in the transaction is compared against the ledger (available in all peer) to decide the validity.
And adding the block to the ledger is not dependent upon the validity check. If it was found invalid, it will still be added to the ledger but with an invalid tag.
You cannot query a peer without having a chaincode. Must need chaincode installed in the peer in order to query | invoke.
Hyperledger Fabric has two types of peers
1) Endorsing peers
2) Committing Peers
Endorsing peers must need a chaincode which means you need to install chaincode on endorsing peers because its duty is to make sure the transaction owner has sufficient rights and it simulates that transaction against the ledger for that it needs a chaincode interface in order to complete simulation.
Once the simulation is done it sends back the R/W sets and simulation result to client and client will send this to the orderer
Orderer then distributes to committing peers ( No need to install chaincode)
Whole: Applications generate a transaction proposal which they send to each of the required set of peers for endorsement. Each of these endorsing peers then independently executes a chaincode using the transaction proposal to generate a transaction proposal response. It does not apply this update to the ledger, but rather simply signs it and returns it to the application.
According to hyperledger fabric documentation, the chaincode should only be deployed in endorsing peers, and it says still the non endorsing peers can validate and update the ledger. Now I am bit confused if non endorsing peers don't have a chaincode, how can they generate R/W sets. How the non endorsing peers will be able to create new state for the asset, if they aren't aware of the logic (chaincode) behind it ?
If you look at https://hyperledger-fabric.readthedocs.io/en/release-1.1/txflow.html#, you'll find a section that states:
The blocks of transactions are “delivered” to all peers on the channel. The transactions within the block are validated to ensure endorsement policy is fulfilled and to ensure that there have been no changes to ledger state for read set variables since the read set was generated by the transaction execution. Transactions in the block are tagged as being valid or invalid.
A block is an ordered set of transactions and transactions include the state transitions in the form of read/write sets. The output of endorsement is actually the read/write set and these are what are ordered and delivered to all peers in the channel.
In order to validate a transaction, a peer needs to check the following:
Is the transaction well-formed
Was the endorsement policy met (the
endorsement policy is distributed to all peers in a channel when
chaincode is instantiated even if the peer does not have the
chaincode bytes)
MVCC check
In order to do the above, peers do not need to execute the chaincode itself.
NB: I am seeking to understand how endorsements works in general. This will help me determine how to design applications when using Hyperledger Composer.
When I read the links here and here, I came across this statement: "Transactions have to be “endorsed” and only endorsed transactions may be committed and have an effect on the state". The statement is clear. However, let's consider the composer developer tutorial here. We have a commodity that is currently owned by an owner(Trader1) who could sell it to somebody else(Trader 2). Currently, how many endorsements are needed for the transaction to be put on the blockchain? Because, when running the application, I only submit a transaction Trade and I get results. I only deal with one function, and I get results. The following things are transparent to me as a programmer:
Creation of a transaction proposal,
When the transaction proposal is endorsed and by whom,
Whether an endorsement is performed explicitly by a human on the other end or it's programmatically done by code
That there is a proposal response from the endorser and how many they are,
When the application verifies the endorsing peer signatures
When the application creates a transaction message from the transaction proposal and response
etc.
All I do is submit one transaction and get a result.
So it becomes hard for me to assess the value of endorsement policies besides the theory in the text. And thus, the difficulty in designing a program to utilize the same. For example, consider two scenarios which we could use to handle a Trade:
We need 2 endorsements from the seller and the buyer before a transaction is commited. This would effectively be one transaction (This is what is transparent to me)
We need 2 authorizations from seller and the buyer before a transaction is commited. These authorizations could update states in the commodity such that we capture the approval from both the seller and the buyer. This could be 2 transactions i.e. sellerTradeRequest, buyerTradeApproval. The sellerTradeRequest could update commodity.sellerApproval=true while the buyerTradeApproval could update commodity.buyerApproval=true. Then, a final trade transaction that checks that the states on the commodity are OK i.e. commodity.sellerApproval=true and commodity.buyerApproval=true before commiting the transaction.
If I get a clear distinction between 1 and 2, especially how composer enables 1 above. Then maybe I will start understanding how to use endorsements.
Could anyone help?
The endorsement process is described in the docs. That said, in simple terms, the process of endorsement involves an endorsing peer signing the read/write set of a transaction proposal with its certificate. This basically says: the peer identified by the signing certificate asserts that these are the read/write sets of the proposed transaction simulation.
A client that invokes a proposed transaction will need knowledge of the endorsement policy for that channel/chaincode so that it can transmit the proposed transaction to each of the endorsing peers (or to a sufficient subset of endorsing peers to satisfy the policy). When it has received the responses from each endorsing peer, it will check that the transaction is valid and then broadcast the endorsed transaction to the ordering service, which will add it to a block and broadcast to the validating peers for that channel.
Validating peers will then validate the endorsement policy of the channel/chaincode against the transactions in a block to ensure that:
all endorsements are valid (i.e. they are valid signatures from valid certificates over the expected message)
there are an appropriate number of endorsements
endorsements come from the expected source(s)
If the endorsement policy is satisfied, then the transaction is committed and the world state updated with the read/write set.
At present, Composer cannot manage the endorsement policy, but this is definitely on the development roadmap.
The endorsement policy enables you the chance to add an extra verification layer to your Blockchain. You define it when you create the channel.
When a (Initial) Peer sends a proposal, firts of all the proposal is send to all of the Peers that you have defined in your Endorsement policy.
Then, each Endorser Peer executes the proposal against its ledger. The Endorser Peer will send the result signed to the Initial Peer.
The Initial Peer will receive more than one respone. Then, the Initial Peer will verify all the signatures and will compare the result of each response. The result sent by each Endorse Peer have to be the same.
In that case, the Initial Peer will send the transaction to the Orderers. That transaction will wraper all the responses all of the Endorses Peers.
So, you should define your Endorsement policy according to your requirements