In case of Bitcoin (and Proof of Work in general) there is a simple rule that provides automatic conflict resolution when there are 2 differing copies of the ledger for any reason. And that rule is the longer chain wins.
But how are conflicts resolved in case of Fabric? Saying that conflicts never occur in Fabric is not an answer because if conflicts never occur, then what is the purpose of getting endorsement from more than one peer?
Stating alternatively, if your answer is that conflicts never occur in Fabric, then please explain why would someone want to get endorsement from more than one peer?
Yet another way to frame this question: let's say the copy of the ledger hosted on the peer of another organization got hacked unbeknownst to them. Now your organization and the other organization have different records. How will the conflict be reconciled? And don't forget the havoc this will cause - all transactions submitted by users will now fail to get endorsement until the conflict is resolved. A hack on another organization disrupted your business even though your ledger was not compromised.
In Proof of work:
Client will submit a transaction and it will be in pool, any miner can take and validate the transaction and then do mining, if he get solved quickly then he will publish to other miners. Here many nodes involved in Orderer to create a block
In Hyperledger Fabric [HighLevel]:
Client send transaction to endorsing peers(more than one), endorsing peers will sent back R/W and signatures to client, if endorsement failed means data in consistency then it will mark as failed and sent back to client.
Client will send entire payload to Orderer. Orderer simply creates a block and ship to committing peers, committing peers simply very endorsements (more than one ) and commit to the ledger
No tell me by comparing both, do u think still conflicts will come in hyperledger fabric?
Purpose of getting endorsement: In order to authentic more endorsements more accurate.
LowLevel:
Stage 1: [CLient Initiate Tx]
Client A is sending a request to update the ledger.
This request targets peerA and peerB, who are respectively
representative of Client A and Client B. The endorsement policy states
that both peers must endorse any transaction, therefore the request
goes to peerA and peerB.
Stage 2: Endorsing peers verify signature & execute the transaction
The endorsing peers verify (1) that the transaction proposal is well
formed, (2) it has not been submitted already in the past
(replay-attack protection), (3) the signature is valid (using the
MSP), and (4) that the submitter (Client A, in the example) is
properly authorized to perform the proposed operation on that channel
(namely, each endorsing peer ensures that the submitter satisfies the
channel’s Writers policy). The endorsing peers take the transaction
proposal inputs as arguments to the invoked chaincode’s function. The
chaincode is then executed against the current state database to
produce transaction results including a response value, read set, and
write set (i.e. key/value pairs representing an asset to create or
update). No updates are made to the ledger at this point. The set of
these values, along with the endorsing peer’s signature is passed back
as a “proposal response” to the SDK which parses the payload for the
application to consume.
Stage 3: Client Proposal responses are inspected
The application verifies the endorsing peer signatures and compares
the proposal responses to determine if the proposal responses are the
same. If the chaincode is only queried the ledger, the application
would inspect the query response and would typically not submit the
transaction to the ordering service. If the client application intends
to submit the transaction to the ordering service to update the
ledger, the application determines if the specified endorsement policy
has been fulfilled before submitting (i.e. did peerA and peerB both
endorse). The architecture is such that even if an application chooses
not to inspect responses or otherwise forwards an unendorsed
transaction, the endorsement policy will still be enforced by peers
and upheld at the commit validation phase.
Stage 4 : Client assembles endorsements into a transaction and broadcast
The application “broadcasts” the transaction proposal and response
within a “transaction message” to the ordering service. The
transaction will contain the read/write sets, the endorsing peers
signatures and the Channel ID. The ordering service does not need to
inspect the entire content of a transaction in order to perform its
operation, it simply receives transactions from all channels in the
network, orders them chronologically by channel, and creates blocks of
transactions per channel.
Stage 5: Transaction is validated and committed
The blocks of transactions are “delivered” to all peers on the
channel. The transactions within the block are validated to ensure
endorsement policy is fulfilled and to ensure that there have been no
changes to ledger state for read set variables since the read set was
generated by the transaction execution. Transactions in the block are
tagged as being valid or invalid.
Stage 6: Ledger updated
Each peer appends the block to the channel’s chain, and for each valid
transaction the write sets are committed to current state database. An
event is emitted, to notify the client application that the
transaction (invocation) has been immutably appended to the chain, as
well as notification of whether the transaction was validated or
invalidated.
Related
I am currently working in projects using Fabric 1.4 I am able to get a decent idea of how things seems to work out but I am quite confused with the way smart contracts and endorsement policies work together.
What i have understood is that endorsement policy is defined during chaincode instantiation where say two or three orgs will sign a transaction in order for it to be valid. This kind of an endorsement just verifies the transaction signatures right ? no data level checks.
Like say I have this kind of a scenario: I have three orgs(Org1,Org2,Org3) each with a peer. Now through a client web app each peer puts data into the blockchain. Now how do I verify if the data (the information that I submit in the blockchain) say for example the name and id are validated by another peer properly and only then added to the world state.
Example: If peer0 in org1 adds data, name and id and the ID is wrong. Org2 has a list of IDs and it should check if the ID that org1 added matches with it and validates it. If it validates then it is fine and data can be put it in the world state
How to define this kind of a transaction state level validation (more of a data level validation rather than just signature verification)? Can this be done in the Go smart contract.
Any help and suggestions would help.
Thanks
During the endorsement, each selected endorser executes (or simulates) the transaction and returns its response, read set and write set signed. The client checks signatures and that responses from different endorsers match (or at least it should do it, anyway this check is performed later again by committers), so there are data level checks (your premise is wrong). The client assembles all the endorsements into a transaction and broadcasts it so it reaches to the ordering service. The ordering service adds the transaction to a block and the block is send to every (committer) peer joined to the channel. The committers perform their checks again and commit the transaction into the state.
It is perfectly explained here: https://hyperledger-fabric.readthedocs.io/en/release-1.4/txflow.html.
Given the fact that all peers are running the same chaincode and have same ledger, does it make any difference whether one or multiple peers are chosen for endorsement? Isn't getting endorsement from multiple peers redundant?
Hyperledger fabric uses order-execute model by separating the transaction flow into three steps:
execute a transaction and check its correctness, thereby endorsing it,
order transactions via a (pluggable) consensus protocol, and
validate transactions against an application-specific endorsement policy before committing them to the ledger.
When a peer gets a transaction, it executes transactions against the current state data held by it to simulate the transaction.
The most probable reason for multiple peer endorsement is to ensure concurrent states across peers upon execution of the transaction and ensuring correct transaction execution, This is necessary to ensure data that was read during chaincode execution has not changed, and therefore the execution results are still valid and can be committed to the ledger state database.
I don't know some details about Fabric in this document.
Document: The application verifies the endorsing peer signatures and compares the proposal responses.
Is it necessary to have all the responses the same when verifying， or most of them？
If it is part of them, what is the ratio?
Whether it can be controlled by the Fabric SDK?
Document: The ordering service does not need to inspect the entire content of a transaction in order to perform its operation, it simply receives transactions from all channels in the network, orders them chronologically by channel, and creates blocks of transactions per channel.
Raft and Kafka are only used to guarantee availability, not for voting. Is this correct?
Document: Transaction is validated and committed.
When committing, is there a process of voting verification?
The application does not actually need to check anything (it's
optional but a good way to avoid sending invalid transactions).
When you instantiate chaincode on a channel, you set the endorsement policy for that chaincode. The endorsement policy specifies how many organization's peers must sign the response. This is actually enforced during validation on the peers (after the peer receives blocks/transaction from the orderer but prior to committing the data). If there are not enough signatures to satisfy the policy, transactions are marked invalid and state is not committed.
With Fabric v1.2 and later, clients can use the discovery service to obtain the minimum list of peers required to meet the endorsement policy for chaincode on each channel.
Kafka / Raft are used to ensure that all ordering service nodes process transactions and deliver blocks in the same order. With Fabric v1.4.1 you will have the option to use Raft consensus rather than Kafka.
See comment in 2. about enforcing endorsement policies during validation.
The transaction flow in the Fabric states that:
Client proposes a transaction
This proposal is passed to all the endorsing peers of the network
Endorsing peer simulates the chaincode, signs the proposal and sends it back to the client.
The client collects the signature and waits until the total number of signature matches the one specified in the endorsement policy
Once the client has enough signatures, it sends this request to the Orderer
Orderer creates blocks and sends it to the anchor peers who validates and commits into their ledger
If we are using the Composer REST server, how are all these steps handled? Are they abstracted out from the client? Or do we need to explicitly write logic in the TP function or client to handle the proposal responses?
My confusion comes from the fact that when we use the node SDK, we have to explicitly implement the logic for handling all this. As we do it over here.
Composer implements a Submit/Notify model and that includes the Rest Server. It will submit the proposal, collect the endorsements, send them to the orderer and wait for a block with the transaction in to be committed before informing the caller of the completion of the transaction.
As Composer has no visibility of the endorsement policy it will send proposals to all peers in the channel (that have been defined as having the endorsingPeer role in the connection profile, which if not specified defaults to true) and collect as many responses as possible before sending them to be ordered. The Peer(s) will then decide whether those endorsements satisfy the endorsement policy.
Composer also waits for all peers (that have been defined in the connection profile as having the eventSource role which defaults to true if not specified) to notify that the transaction has been committed before notifying the client
NB: I am seeking to understand how endorsements works in general. This will help me determine how to design applications when using Hyperledger Composer.
When I read the links here and here, I came across this statement: "Transactions have to be “endorsed” and only endorsed transactions may be committed and have an effect on the state". The statement is clear. However, let's consider the composer developer tutorial here. We have a commodity that is currently owned by an owner(Trader1) who could sell it to somebody else(Trader 2). Currently, how many endorsements are needed for the transaction to be put on the blockchain? Because, when running the application, I only submit a transaction Trade and I get results. I only deal with one function, and I get results. The following things are transparent to me as a programmer:
Creation of a transaction proposal,
When the transaction proposal is endorsed and by whom,
Whether an endorsement is performed explicitly by a human on the other end or it's programmatically done by code
That there is a proposal response from the endorser and how many they are,
When the application verifies the endorsing peer signatures
When the application creates a transaction message from the transaction proposal and response
etc.
All I do is submit one transaction and get a result.
So it becomes hard for me to assess the value of endorsement policies besides the theory in the text. And thus, the difficulty in designing a program to utilize the same. For example, consider two scenarios which we could use to handle a Trade:
We need 2 endorsements from the seller and the buyer before a transaction is commited. This would effectively be one transaction (This is what is transparent to me)
We need 2 authorizations from seller and the buyer before a transaction is commited. These authorizations could update states in the commodity such that we capture the approval from both the seller and the buyer. This could be 2 transactions i.e. sellerTradeRequest, buyerTradeApproval. The sellerTradeRequest could update commodity.sellerApproval=true while the buyerTradeApproval could update commodity.buyerApproval=true. Then, a final trade transaction that checks that the states on the commodity are OK i.e. commodity.sellerApproval=true and commodity.buyerApproval=true before commiting the transaction.
If I get a clear distinction between 1 and 2, especially how composer enables 1 above. Then maybe I will start understanding how to use endorsements.
Could anyone help?
The endorsement process is described in the docs. That said, in simple terms, the process of endorsement involves an endorsing peer signing the read/write set of a transaction proposal with its certificate. This basically says: the peer identified by the signing certificate asserts that these are the read/write sets of the proposed transaction simulation.
A client that invokes a proposed transaction will need knowledge of the endorsement policy for that channel/chaincode so that it can transmit the proposed transaction to each of the endorsing peers (or to a sufficient subset of endorsing peers to satisfy the policy). When it has received the responses from each endorsing peer, it will check that the transaction is valid and then broadcast the endorsed transaction to the ordering service, which will add it to a block and broadcast to the validating peers for that channel.
Validating peers will then validate the endorsement policy of the channel/chaincode against the transactions in a block to ensure that:
all endorsements are valid (i.e. they are valid signatures from valid certificates over the expected message)
there are an appropriate number of endorsements
endorsements come from the expected source(s)
If the endorsement policy is satisfied, then the transaction is committed and the world state updated with the read/write set.
At present, Composer cannot manage the endorsement policy, but this is definitely on the development roadmap.
The endorsement policy enables you the chance to add an extra verification layer to your Blockchain. You define it when you create the channel.
When a (Initial) Peer sends a proposal, firts of all the proposal is send to all of the Peers that you have defined in your Endorsement policy.
Then, each Endorser Peer executes the proposal against its ledger. The Endorser Peer will send the result signed to the Initial Peer.
The Initial Peer will receive more than one respone. Then, the Initial Peer will verify all the signatures and will compare the result of each response. The result sent by each Endorse Peer have to be the same.
In that case, the Initial Peer will send the transaction to the Orderers. That transaction will wraper all the responses all of the Endorses Peers.
So, you should define your Endorsement policy according to your requirements