I am setting up the Key-based endorsement policy on Hyperledger fabric v2.3.3.
My chaincode endorsement policy is "OR('org1.peer','org2.peer','org3.peer','org4.peer')" and for a very specific key, the endorsement policy is "AND('org1.peer','org2.peer')". When a transaction is submitted very first time, it works fine because at that time chaincode endorsement policy works, but when I try to update the same key, it does not work. The node fabric-sdk sends the endorsement request to only 1 org i.e org1 (invoker) and the peer rejects the block and throws "Endorsement Policy Failure".
When I verified the block, I saw that the transaction was endorsed by only org1 not by both org1 and org2. Seems like the fabric-sdk did not send the transaction to the other org.
However, if I change my chaincode-endorsement policy to "AND('org1.peer','org2.peer','org3.peer','org4.peer')", everything works fine because the fabric-sdk is sending the transaction to every org's peer for endorsement.
The official document says
In practice, this means that the key-level endorsement policy can be either less restrictive or more restrictive than the chaincode-level or collection-level endorsement policies.
but from the above scenario, it seems like it can only be less-restrictive.
Peer logs:
2021-10-14 11:05:24.443 UTC [gossip.privdata] StoreBlock -> INFO 36d Received block [207] from buffer channel=assetschannel
2021-10-14 11:05:24.450 UTC [vscc] Validate -> ERRO 36e VSCC error: stateBasedValidator.Validate failed, err validation of key PUB-TEST-6 (coll'':ns'assets') in tx 207:0 failed: signature set did not satisfy policy
2021-10-14 11:05:24.454 UTC [committer.txvalidator] validateTx -> ERRO 36f Dispatch for transaction txId = b13b21ddd0cb351b93b47ccc556ea4b62bf41abd3a23a3734d56304c91e6066e returned error: validation of key PUB-TEST-6 (coll'':ns'assets') in tx 207:0 failed: signature set did not satisfy policy
2021-10-14 11:05:24.454 UTC [committer.txvalidator] Validate -> INFO 370 [assetschannel] Validated block [207] in 5ms
2021-10-14 11:05:24.459 UTC [validation] preprocessProtoBlock -> WARN 371 Channel [assetschannel]: Block [207] Transaction index [0] TxId [b13b21ddd0cb351b93b47ccc556ea4b62bf41abd3a23a3734d56304c91e6066e] marked as invalid by committer. Reason code [ENDORSEMENT_POLICY_FAILURE]
2021-10-14 11:05:24.495 UTC [kvledger] commit -> INFO 372 [assetschannel] Committed block [207] with 1 transaction(s) in 36ms (state_validation=0ms block_and_pvtdata_commit=6ms state_commit=29ms) commitHash=[35570558fc04d3dfa400f053f2a0f9bed92ea227c894ae4536990627a5a19036]
fabric-sdk logs:
2021-10-14T11:05:24.491Z - warn: [TransactionEventHandler]: strategyFail: commit failure for transaction "b13b21ddd0cb351b93b47ccc556ea4b62bf41abd3a23a3734d56304c91e6066e": TransactionError: Commit of transaction b13b21ddd0cb351b93b47ccc556ea4b62bf41abd3a23a3734d56304c91e6066e failed on peer peer1.org1.com:7051 with status ENDORSEMENT_POLICY_FAILURE
2021-10-14T11:05:24.491Z - warn: [TransactionEventHandler]: strategyFail: commit failure for transaction "b13b21ddd0cb351b93b47ccc556ea4b62bf41abd3a23a3734d56304c91e6066e": TransactionError: Commit of transaction b13b21ddd0cb351b93b47ccc556ea4b62bf41abd3a23a3734d56304c91e6066e failed on peer peer1.org1.com:7051 with status ENDORSEMENT_POLICY_FAILURE
The problem here is that during endorsement the client does not know what ledger keys your transaction invocation will read/write or what key-based endorsement policies might be applied. All it knows about (using the discovery service) is the endorsement policy for the chaincode being invoked, and so it will pick endorsing peers to satisfy that policy.
If your transaction invocation is going to access private data collections or make chaincode-to-chaincode calls, you can indicate that to the client so it can (again using the discovery service) select an appropriate set of endorsing peers to satisfy the effective endorsement policy considering those collections and/or chaincode invocations. This tutorial page describes how this is done:
https://hyperledger.github.io/fabric-sdk-node/release-2.2/tutorial-discovery-fabric-network.html
This still doesn't cover the scenario of key-based endorsement policies since the ledger keys accessed are a determined by the business logic of your transaction function and parameters passed in by the client application. So you need to provide the knowledge of organizations that must endorse explicitly using Transaction.setEndorsingOrganizations(). This will override the normal mechanism for selecting endorsing peers and use the organizations you have specified.
At the time of writing, a newer Fabric Gateway client API (Node, Go and Java) is under development, which moves much of the logic that exists in the client-side SDKs today into a Gateway peer. It is targeted for release with Fabric v2.4. This Fabric Gateway implementation will inspect the read/write sets generated by simulation of the transaction invocation during endorsement, and use this to dynamically detect the endorsement requirements for a given transaction invocation. So you should no longer need to specify the required organizations, collections or chaincode-to-chaincode calls when making client invocations, and it should work out-of-the-box with key-based endorsement policies. The exception to this general rule is when passing private data, where you should specify the organizations that you will allow to receive the data to prevent accidental leakage of private information.
Related
The use case is:
I created a network with Raft Ordering service having one channel say, channel1 with three Organisations (Org1, Org2 and Org3).
Org1 and Org2 peers have joined the channel channel1.
Org3 is just present in the channel config but not joined the channel yet.
Now I added the new orderer endpoints in the system channel and channel1 config and removed the old orderer endpoints.
My new orderers are working fine, able to fetch the config for both channels.
I fetched the 0 block of channel1 from Org3 peer and issued the join command.
The command works fine, but in the peer I am getting these error:
2022-04-20 05:28:18.210 UTC 006b WARN [peer.blocksprovider] func1 -> Encountered an error reading from deliver stream: EOF channel=channel1 orderer-address=orderer.example.com:7050
2022-04-20 05:28:18.210 UTC 006c WARN [peer.blocksprovider] DeliverBlocks -> Got error while attempting to receive blocks: received bad status SERVICE_UNAVAILABLE from orderer channel=channel1 orderer-address=orderer.example.com:7050
The Org3 peer is still trying to connect with the older orderer endpoints (as they were defined in the 0 block initially).
So how to sort out this problem?
One way I can think is to use the snapshot of the Org1/Org2 peer maybe.
What are your thoughts?
Thanks
I found the solution, so if anyone in the future needs it, can use this answer to join the channel when the orderer endpoints are updated in the channel configuration.
Method 1:
In the peer configuration you can override some of the variables to override the old orderer endpoint with the new orderer endpoints.
The link to the config file and params is:
https://github.com/hyperledger/fabric/blob/main/sampleconfig/core.yaml#L382-L388
This method will allow you to join the peer using genesis block and is supported by older versions of Fabric (<=v2.2) which does not support joining by snapshot.
Method 2:
If you are on Fabric v2.2+, then you can either use the Method 1 or join the channel using a snapshot from another peer.
There is tutorial in the official docs for that, please check it:
https://hyperledger-fabric.readthedocs.io/en/release-2.3/peer_ledger_snapshot.html
Thanks,
Sahil
From HLF docs
joining by snapshot will provide a peer with the latest channel configuration, which may be important if the channel configuration has changed since the genesis block. For example, the peer may need the orderer endpoints or CA certificates from the latest channel configuration before it can successfully pull blocks from the ordering service.
Taking ledger snapshots and using them to join channels
When I try to create a channel using Hyperledger Fabric, the request fails and returns the following error:
Client logs:
Error: got unexpected status: BAD_REQUEST -- error validating channel creation transaction for new
channel 'testchannel', could not succesfully apply update to template configuration: error
authorizing update: error validating DeltaSet: policy for [Group] /Channel/Application not
satisfied: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy
requires 1 of the 'Admins' sub-policies to be satisfied
This error occurs when there is a problem with the identity (consisting of MSP ID, certificate, and keys) that submitted the request. If you use the default Fabric configuration policies, channels need to be created by organization administrators. The error is produced by your identity not being able to satisfy default policy on the /Channel/Application/Admins path.
There are several reasons why the policy would reject your identity, including the use of invalid or expired certificates. You can learn more about why the request failed by looking at your orderer logs. The Ordering Service is the node that enforces the policies that create or update channels.
When you examine your orderer logs, look for an error that is similar to what was returned to your client. You may find an error from a certificate check immediately preceding the policy error (Principal deserialization failure). This implies that the channel creation was rejected because the MSP ID was not recognized as valid.
Ordering Service logs:
2019-08-06 15:31:43.589 UTC [cauthdsl] deduplicate -> ERRO 021 Principal deserialization failure
(MSP SampleOrg is unknown) for identity 0
2019-08-06 15:31:43.589 UTC [orderer.common.broadcast] ProcessMessage -> WARN 022 [channel:
testchannel] Rejecting broadcast of config message from 172.27.0.7:34750 because of error: error
validating channel creation transaction for new channel 'testchannel', could not succesfully apply
update to template configuration: error authorizing update: error validating DeltaSet: policy for
[Group] /Channel/Application not satisfied: implicit policy evaluation failed - 0 sub-policies
were satisfied, but this policy requires 1 of the 'Admins' sub-policies to be satisfied
The error before the policy warning, ERRO 021 Principal deserialization failure (MSP SampleOrg is unknown) for identity 0, indicates that the MSP ID that was passed as a parameter with the request was not recognized by the ordering service. This could be a result of passing the wrong MSP ID to the command. This error may also indicate that your organization has not joined the consortium hosted by the ordering service system channel. If you are updating an application channel, this error could occur if your organization is not yet a member of the channel you are trying to update.
If the MSP ID of the identity is valid, you may encounter the following certificate validation error:
Ordering Service logs:
2019-08-06 15:34:45.730 UTC [cauthdsl] deduplicate -> ERRO 02d Principal deserialization failure
(the supplied identity is not valid: x509: certificate signed by unknown authority) for identity 0
2019-08-06 15:34:45.730 UTC [orderer.common.broadcast] ProcessMessage -> WARN 02e [channel:
testchannel] Rejecting broadcast of config message from 172.27.0.7:36214 because of error: error
validating channel creation transaction for new channel 'testchannel', could not succesfully apply
update to template configuration: error authorizing update: error validating DeltaSet: policy for
[Group] /Channel/Application not satisfied: implicit policy evaluation failed - 0 sub-policies
were satisfied, but this policy requires 1 of the 'Admins' sub-policies to be satisfied
In this case, the ordering service recognized your MSP ID, but could not validate that your certificate was issued by one of your organization's certificate authorities. If you are managing multiple organizations, this error could be the result of you using a mismatched MSP ID and certificate to submit the request. This error could also occur if your admin certificates have expired. If this is a test network that has been launched recently, you may be issuing the request from an identity that was created by a certificate authority on an older incarnation of your network.
It will be more common that your certificate has passed the validation check, but could not fulfill the channel creation policy. If that is the case, the error in your orderer logs would look like the following:
Ordering Service logs:
2019-08-06 15:36:52.307 UTC [orderer.common.broadcast] ProcessMessage -> WARN 032 [channel:
testchannel] Rejecting broadcast of config message from 172.27.0.7:37236 because of error: error
validating channel creation transaction for new channel 'testchannel', could not succesfully apply
update to template configuration: error authorizing update: error validating DeltaSet: policy for
[Group] /Channel/Application not satisfied: implicit policy evaluation failed - 0 sub-policies
were satisfied, but this policy requires 1 of the 'Admins' sub-policies to be satisfied
The identity that submitted the request is a valid member of your organization, and your organization is recognized to be a member of the system channel or application channel. However, the identity did not have the permission to create a channel. You may have used the wrong identity to submit the request, and used an identity that was not your organization administrator. Submit the request from your admin identity, or create a new admin, and have the channel administrator update your organization MSP.
If you encounter this error for operations other then channel creation, your certificate may not be authorized for the correct role. Check that your client certificates and peer certificates have the client and peer OU respectively.
Additional debugging techniques
If you need additional help debugging a policy or certificate related error, you can increase the logging level related to those components:
FABRIC_LOGGING_SPEC=”WARN:cauthdsl=debug:policies=debug:msp=debug
You can also manually pull your organizations root certificate from an application or system channel and use them to verify your client side certs. Use the following command to pull the latest configuration block from your channel.
peer channel fetch config ./configupdate/config_block.pb -o <orderer_endpoint> -c <my_channel> --tls --cafile <PATH_TO_ORDERER_TLS_CERT>
Then use the following command to convert the configuration block into JSON.
configtxlator proto_decode –type=common.Block --input=config_block.pb --output=config_block.json
This allows you to pull the root certificate from the block using the following command. Replace with the MSP ID of your organization.
jq -r .data.data[0].payload.data.config.channel_group.groups.Application.groups.<MSPID>\
.values.MSP.value.config.root_certs[0] config_block.json | base64 –decode > root.pem
If your MSP defines multiple root certificates or uses intermediate certificates, you will need to adjust the jq command above to properly extract them.
You can then use tools such as OpenSSL to validate your client side admin certificate against the root certificate.
openssl verify -CAFile <root.pem> <admincert.pem>
You can also use the following command to open the certificate and examine it in plaintext. This allows you to check fields such as the expiration date, the node OU, or the issuing CA.
openssl x509 -in <admincert.pem> -text
UTC [orderer.common.broadcast] ProcessMessage -> WARN 009 [channel: orgchannel] Rejecting broadcast of config message from 172.20.20.22:45668 because of error: error validating channel creation transaction for new channel 'orgchannel', could not succesfully apply update to template configuration: error authorizing update: error validating DeltaSet: policy for [Group] /Channel/Application not satisfied: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Admins' sub-policies to be satisfied
If you are getting aforementioned error then check your configtx file and verify below mentioned attributes. Copy paste the below section and replace capabilities section.
Capabilities:
# Channel capabilities apply to both the orderers and the peers and must be
# supported by both.
# Set the value of the capability to true to require it.
Channel: &ChannelCapabilities
# V1.4.3 for Channel is a catchall flag for behavior which has been
# determined to be desired for all orderers and peers running at the v1.4.3
# level, but which would be incompatible with orderers and peers from
# prior releases.
# Prior to enabling V1.4.3 channel capabilities, ensure that all
# orderers and peers on a channel are at v1.4.3 or later.
V1_4_3: true
# V1.3 for Channel enables the new non-backwards compatible
# features and fixes of fabric v1.3
V1_3: false
# V1.1 for Channel enables the new non-backwards compatible
# features and fixes of fabric v1.1
V1_1: false
# Orderer capabilities apply only to the orderers, and may be safely
# used with prior release peers.
# Set the value of the capability to true to require it.
Orderer: &OrdererCapabilities
# V1.4.2 for Orderer is a catchall flag for behavior which has been
# determined to be desired for all orderers running at the v1.4.2
# level, but which would be incompatible with orderers from prior releases.
# Prior to enabling V1.4.2 orderer capabilities, ensure that all
# orderers on a channel are at v1.4.2 or later.
V1_4_2: true
# V1.1 for Orderer enables the new non-backwards compatible
# features and fixes of fabric v1.1
V1_1: false
# Application capabilities apply only to the peer network, and may be safely
# used with prior release orderers.
# Set the value of the capability to true to require it.
Application: &ApplicationCapabilities
# V1.4.2 for Application enables the new non-backwards compatible
# features and fixes of fabric v1.4.2.
V1_4_2: true
# V1.3 for Application enables the new non-backwards compatible
# features and fixes of fabric v1.3.
V1_3: false
# V1.2 for Application enables the new non-backwards compatible
# features and fixes of fabric v1.2 (note, this need not be set if
# later version capabilities are set)
V1_2: false
# V1.1 for Application enables the new non-backwards compatible
# features and fixes of fabric v1.1 (note, this need not be set if
# later version capabilities are set).
V1_1: false
Also check your channel name
export SYS_CHANNEL=orgchannel #should be your own system channel name
export CHANNEL_NAME=org1orgchannel #should be your own system channel name
export CHANNEL_ID=org1orgchannel #should be your own system channel name
Also remember SYS_CHANNEL value should be different from CHANNEL_NAME. CHANNEL_NAME and CHANNEL_ID should be same.
If still getting issue mail me on actachieverepeat#gmail.com.
Transaction is not endorsed by the endorsers in the multiple organisations
Tried changing the endorsement policy from 3 orgs to 2 orgs to verify whether the transaction is working or not.
Unhandled error for request POST /api/TransferCommodityPossession:
Error: Error trying invoke business network with transaction id d5334feb70ba03629859f1e4b87d2a687c14f5a5300b3219ecf330d6d22ae5d9.
Error: Peer localhost:7051 has rejected transaction 'd5334feb70ba03629859f1e4b87d2a687c14f5a5300b3219ecf330d6d22ae5d9' with code ENDORSEMENT_POLICY_FAILURE
Transaction should get executed which is being endorsed by the endorsers in the three different organizations in a single instance system such that the data gets updated in the couch db.
Am working on hyperledger fabric, i have network setup of 2 org,1 channel and 1 orderer.
I installed chaincode(Say CH1) in Org1, with endoserment policy "AND(Org1.member,Org2,member)". Now i need to install same chaincode(CH1) in Org2 to validate and endorse transaction.
Since chaincode(CH1) installed in Org2(for endorsement purpose), also permitted to execute transaction using same chaincode(CH1), which will affect state, created by Org1. How to prevent this?
You need to separate 3 concepts here:
Installing chaincode
Instantiating chaincode on a channel
Endorsing chaincode
Installing chaincode simply makes the bytes of the chaincode available on the peer on which it is installed.
Instantiating chaincode on a channel makes that chaincode available for execution on the channel.
The endorsement policy determines which peers need to successfully execute and sign a transaction (technically peers sign the endorsement response).
In your case, installing the chaincode on the peers for both Org1 and Org2 makes the chaincode bytes available to the peers.
Instantiating the chaincode on a channel will make it available for execution.
The endorsement policy you set requires that a peer from Org1 AND a peer from Org2 must execute and endorse the transaction.
Once the client has collected the endorsements, the transaction will be send to the ordering service and then delivered to all peers in the channel. When the peers in the channel receive a transaction which involves CH1, they will check to make sure that the endorsement policy has been met (in this case that a peer from both Org1 and Org2 signed the transaction) and only then can it be committed (after the other validation checks)
I referenced the web page : http://hyperledger-fabric.readthedocs.io/en/latest/arch-deep-dive.html
And I want to ask you about this picture and Endorsement Policy.
In section 2.3 of the website, the following statement appears. And that is related to ③ of the picture.
"The exact number of “enough” depend on the chaincode endorsement
policy"
And in section 2.4 of this web site, the following statement appears for a peer. that is related to ④ of the picture.
It checks that the blob.endorsement is valid according to the policy
of the chaincode (blob.tran-proposal.chaincodeID) to which it refers.
Question
What is the difference between an ③ endorsement policy and ④ a policy of
the chaincode?
Is ESCC related to ③, and ④ related to VSCC?
Question
What is the difference between an ③ endorsement policy and ④ a policy of the chaincode?
At stage 3, the peer actually proceed with endorsement, meaning literally signing the results of the chaincode invocation. While at 4 this is happens at commit time, where peer ensures whenever endorsement policy satisfied.
NOTE: It's up to the client to collect required number of endorsements.
Is ESCC related to ③, and ④ related to VSCC?
Yes, ESCC or Endorsement System ChainCode is the actually a system chaincode which responsible to "endorse" transaction, where VSCC or Validation System ChainCode is a system chaincode to check whenever endorsement policy satisfied.