How can be a chaincode executed in parallel? - hyperledger-fabric

I've been studying Hyperledger Fabric for about 1 year.
But I still can't tell how a chaincode works and is dealt with exactly, especially in terms of its implementation and process.
This is how I understand a chaincode below.
A chaincode is running and is isolated from peers, as one of docker containers for its integrity in the network. The chaincode is a program which defines how functions in transactions(e.g., AddTwoIntegerValues()) should update the ledger
Also, endorsing peers access the chaincode for executing functions in the transaction proposals from clients or other peers to respond with endorsement.(This process is not clear)
In that case, I'd like to ask you how those endorsing peers can execute or access the chaincode container in parallel? I heard 'chaincode execution' means simulation of the chaincode logic computation. But I can't get that.
In other words, my question is how can they execute or access that at the same time? Do they have copies of that chaincode? How do peers know the chaincode logic? Could you please correct me or explain chaincode process if I'm wrong?
Please tell me if you have any idea about my question. I'm looking forward to your answer!

Each peer has its own chaincode container for each version of a given chaincode code.
When the chaincode is installed on a peer, the code package of the chaincode's code is written to the file system.
Later on, when the peer receives a request to invoke a function on the chaincode (Init() or Invoke(), when Init() runs on instantiate), the peer checks to see if the chaincode is already running, and if not, it:
Spins a container to compile the chaincode code package to produce the chaincode shim binary and afterwards destroys it
Spins a container that would actually run the chaincode shim binary.
The chaincode shim binary:
Connects to the peer via gRPC and registers itself to be the name it is installed with.
Runs an endless loop in which it waits for commands to simulate transactions from the peer.
Whenever the peer receives a proposal from a client, it:
Locates the chaincode container gRPC stream and forwards the proposal from the client to the chaincode
Prepares a map of key reads and writes that the chaincode shim would ask the peer during its execution, which is called a Read-Write set.
Then, the chaincode shim, running inside the container - extracts the arguments from the proposal and starts running the chaincode logic (part of its binary).
If the chaincode logic contains data access operations such as GetState or PutState, it sends a request to the peer down the same gRPC stream it is connected to the peer, and then the peer does the following:
If the data operation is a read operation, it adds to the read-write set the key and the version read from the key, and sends back the key and the value to the chaincode shim container over the gRPC stream.
If the data operation is a write operation, it adds to the read-write set the key and the value and the version, and then sends back an "OK" to the chaincode shim(*).
(*) I personally think this step is not necessary... it just prolongs the execution time
After the chaincode shim finished computing the transaction, it sends back the result (i.e "OK") to the peer, along with a marker that denotes that the transaction finished executing.
Since the values that are written, are not really written to the DB but just added to an in-memory map, this is called a "transaction simulation".
The peer then proceeds to sign the transaction simulation's results (the read-write set map and the result from the chaincode shim) and then returns the signed transaction simulation (also called an "endorsement") back to the client as a response.
In that case, I'd like to ask you how those endorsing peers can execute or access the chaincode container in parallel?
You can do that in parallel because when the chaincode shim invokes the transaction, it does so on a separate goroutine.

Related

What is the difference between Install and Instantiate in Hyperledger fabric?

What happens during chaincode install and instantiate in Hyperledger fabric?
A common misunderstanding when interacting with chaincode on the network is the difference between chaincode installation and instantiation. It is important that all peers on the network MUST have chaincode installed, but not instantiated.
Chaincode installation means that we are putting the source code (of our chaincode) on a specific peer.
Chaincode instantiation means that we are initializing the chaincode source code. This is done by passing through a set of initialization arguments attached to the instantiate command.
Please note that, even though the chaincode is installed on the peer, when chaincode gets instantiated, it is instantiated on the channel.
Chaincode installation means keeping chaincode on the peers of the ledger.
chaincode instantiation means initializing chaincode with the set of parameters with we pass through chaincode command.
Installing the chaincode on the peers is required and instantiating the chaincode is not necessary.
Install:
The process of placing a chaincode on a peer’s file system.
Instantiate:
The process of starting and initializing a chaincode application on a specific channel. After instantiation, peers that have the chaincode installed can accept chaincode invocations. As it's related to channel, you do not need to instantiate from every peer on this channel, once it's instantiate by maintaining some valid process, the rules will be same for each participating node.
NOTE: This method i.e. Instantiate was used in the 1.4.x and older versions of the chaincode lifecycle. For the current procedure used to start a chaincode on a channel with the new Fabric chaincode lifecycle introduced as part of Fabric v2.0, see Chaincode-definition_.
So from Fabric v2.0 or later, you have to commit the chaincode after proper approval process instead of Instantiate.

Hyperledger Fabric: Error: could not assemble transaction: ProposalResponsePayloads do not match

what are the steps to troubleshoot below error when trying to invoke a chaincode?
Error: could not assemble transaction: ProposalResponsePayloads do not match - proposal response: version:1 response:<status:200 payload:"[\"00000\"]" > ...
we get this error when trying to invoke a chaincode using peer chaincode invoke
#morpheus: Has answered it excellently:
So I thought I will add to the above list of possible reasons:
I had by mistake added something like getting the current timestamp, and was using this for capturing the event date. This led to the different transaction responses by the endorsers, thus leading to the Response Payload not matching. The whole point to remember is that the result of execution should be deterministic as it is going to be run on all of the selected endorsing peers.
So use ctx.GetStub().GetTxTimestamp() for capturing the event time. This is the time when the transaction began and it will be constant across the endorser executions.
Check that you have installed the chaincode on all the peers your peer chaincode invoke command is targetting. That is the most likely cause of this error.
Other ways this error can occur:
You modified your chaincode and instead of installing a new version and upgrading the chaincode, you tried to be smart and overwrite the chaincode with the new file thinking that Fabric would not notice.
It can also happen if there is no chaincode container running on target peer and Docker daemon cannot be found on the peer node when it tries to instantiate a container or instantiation fails for some other reason
Another reason why this error can happen is if some peer nodes are using LevelDB and others are using CouchDB
The error itself originates from here. The first step to debug this error is to invoke the chaincode individually one-by-one on one peer node at a time.
Another reason that I forgot to check is using storing randomly generated values.
I never used random "Id" until now and didn't notice that it leads to ENDORSEMENT_MISMATCH

What install and instantiation chaincode really mean in hyperledger fabric? and what are the differences between them?

In the hyperledger fabric documentation, there are 2 terms used
1. Install the chaincode on peers and
2. Instantiate the chaincode on the channel
What are the major differences between these two?
In the documentation it said that a chaincode can be installed on multiple peers but can be instantiated once. I understood this point as a channel only needs the information about the channel.
I was following the balance-transfer example, so after channel creation, peers need to be joined to that channel.
There are 2 peers which joined the channel ["peer0.org1.example.com", "peer0.org1.example.com"], so when I am instantiating the chaincode it is creating 2 docker images of chaincode
dev-peer0.org1.example.com-chaincode-v0
dev-peer1.org1.example.com-chaincode-v0
What these 2 images really mean?
Isn't initializing the chaincode means for the channel?
Or channel initialize it on all the peers who joined it?
Where actually this initialization is happening?
Thanks!
Thanks to #PaulO'Mahony and #kajuken for the resources and explanation.
Following are the summary of my doubts:
A chaincode runs in a Docker container that is associated with any peer that needs to interact with it.
Chaincode is installed on a peer, then instantiated on a channel.
All members that want to submit transactions or read data by using a chaincode need to install the chaincode on their peer.
Instantiation will input the initial data used by the chaincode, and then start the chaincode containers on peers joined to the channel with the chaincode installed.
Note that only one network member needs to instantiate a chaincode. If a peer with a chaincode installed joins a channel where it has already been instantiated, the chaincode container will start automatically.
a chaincode is installed onto the file system of every peer that joins a channel, the chaincode must then be instantiated on the channel so that peers can interact with the ledger via the chaincode container. The instantiation performs any necessary initialization of the chaincode. This will often involve setting the key value pairs that comprise a chaincode's initial world state.
A peer can the install the chaincode once, and then use the same chaincode container on any channel where it has been instantiated.
References:
install and instantiate the chaincode
instantiate the chaincode
What these 2 images really mean?
Isn't initializing the chaincode means for the channel?
Yes and no. Every peer needs the same version of the chaincode installed on itself since everybody needs to be able to execute and verify incoming queries/invokes.
So there are 2 steps to do.
install the chaincode on every peer on the channel
instantiate the chaincode on the channel
Where actually this initialization is happening?
So the instantiating of chaincode is happening last after every peer has "knowledge" of the chaincode and it can be verified.

How to instantiate all chaincodes at once in the channel?

I have a question about chaincode instantiation.
I think all same chaincode in a channel will be instantiated all at once by one request from this doc.
http://hyperledger-fabric.readthedocs.io/en/release-1.1/install_instantiate.html
Note: The initial instantiation applies to all peers in the channel, and is affected upon any peer that has the chaincode installed.
but in my vagrant environment with v1.0.6 fabric, always only one (of three installed cc on endorsers) chaincode is instantiated by my Instantiate request from Node SDK.
then it seems that if any other proposal request is received, that endorser start to instantiate other chaincode. so if my endorsement policy needs that endorsement, first invoke request is failed...
Initially, chaincode will only be launched on peers to which you send the actual instantiate proposal. It will then be launched on other peers in the channel upon the first invoke request. If you want to have the chaincode launched at instantiate time, you need to send the instantiate request to all of the endorsing peers. The channel.sendInstantiateProposal takes a ChaincodeInstantiateUpgradeRequest which allows you to specify an array of peers in it's target property.

When transaction is finalized in HLF v1?

According to architecture explained (http://hyperledger-fabric.readthedocs.io/en/latest/arch-deep-dive.html), ordering service collects transactions (RWSets) into block for distribution to committing peers. Then, committing peer validates endorsement policy and RWsets then apply the transaction to ledger.
To verify the transaction was succeeded, should client application wait until all committing peers returned "Success" event ? Or just need to verify only one "Success" event ?
To verify the transaction was succeeded, should client application
wait until all committing peers returned "Success" event ? Or just
need to verify only one "Success" event ?
Tanaka, that's a very good question!
The short answer is No.
The reason is that in contrast to existing popular blockchains, HLF has a unique transaction lifecycle which does:
A transaction is simulated on some endorser or a few endorsers
It is sent to the ordering service and is cut into some block
The block is sent to peers, and they all execute the same validation code and all validation code for a specific transaction is guaranteed to reach the same conclusion in all peers, because they run it in the same order across all of them.
Therefore, if a transaction is validated on some peer - when other peers will receive the block the transaction resides in - they will too consider it as valid.
However - a very important aspect you should consider is data availability and synchronization.
For example, if you have an application that uses 10 peers and only 1 peer got the event and the rest didn't, and you invoke another transaction on the other peers, it might be that the endorsements that the other peers will compute will be turned into an invalid transaction, because they will simulate on old data (the fact that they didn't get the event yet proves that they have not processed the block for that transaction), so you need to keep that in mind.

Resources