I am running a custom Hyperledger Fabric network with TLS enabled in all communications. Everyting looks great inside of an organizations and CLI can connect to its org peers with TLS and CLIENTAUTH enabled. This is the status of the network:
org1
- peer0
anchor: true
tls_enabled: true
tls_clientauth_enabled: true
- peer1
tls_enabled: true
tls_clientauth_enabled: true
- orderer0
tls_enabled: true
tls_clientauth_enabled: false
org2
- peer0
anchor: true
tls_enabled: true
tls_clientauth_enabled: true
- peer1
tls_enabled: true
tls_clientauth_enabled: true
- orderer0
tls_enabled: true
tls_clientauth_enabled: false
org3
- orderer0
tls_enabled: true
tls_clientauth_enabled: false
However, once I start a channel than involves multiple organizations, I get Gossip and TLS hanshake related errors. This is an example (log from peer0.org1):
2019-10-07 10:01:45.615 UTC [gossip.service] func1 -> INFO 046 Elected as a leader, starting delivery service for channel global
2019-10-07 10:01:48.620 UTC [ConnProducer] NewConnection -> ERRO 047 Failed connecting to {orderer0.org2:7050 [org1MSP org2MSP org3MSP]} , error: context deadline exceeded
2019-10-07 10:01:51.622 UTC [ConnProducer] NewConnection -> ERRO 048 Failed connecting to {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} , error: context deadline exceeded
2019-10-07 10:01:54.638 UTC [ConnProducer] NewConnection -> ERRO 049 Failed connecting to {orderer0.org3:7050 [org1MSP org2MSP org3MSP]} , error: context deadline exceeded
2019-10-07 10:01:54.638 UTC [ConnProducer] NewConnection -> ERRO 04a Could not connect to any of the endpoints: [{orderer0.org2:7050 [org1MSP org2MSP org3MSP]} {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} {orderer0.org3:7050 [org1MSP org2MSP org3MSP]}]
2019-10-07 10:01:54.638 UTC [deliveryClient] connect -> ERRO 04b Failed obtaining connection: could not connect to any of the endpoints: [{orderer0.org2:7050 [org1MSP org2MSP org3MSP]} {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} {orderer0.org3:7050 [org1MSP org2MSP org3MSP]}]
2019-10-07 10:01:54.638 UTC [deliveryClient] try -> WARN 04c Got error: could not connect to any of the endpoints: [{orderer0.org2:7050 [org1MSP org2MSP org3MSP]} {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} {orderer0.org3:7050 [org1MSP org2MSP org3MSP]}] , at 1 attempt. Retrying in 1s
2019-10-07 10:01:58.640 UTC [ConnProducer] NewConnection -> ERRO 04d Failed connecting to {orderer0.org2:7050 [org1MSP org2MSP org3MSP]} , error: context deadline exceeded
2019-10-07 10:01:59.833 UTC [core.comm] ServerHandshake -> ERRO 04e TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=172.17.0.24:54590
2019-10-07 10:02:00.828 UTC [core.comm] ServerHandshake -> ERRO 04f TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=172.17.0.24:54604
2019-10-07 10:02:01.642 UTC [ConnProducer] NewConnection -> ERRO 050 Failed connecting to {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} , error: context deadline exceeded
2019-10-07 10:02:02.609 UTC [core.comm] ServerHandshake -> ERRO 051 TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=172.17.0.24:54624
2019-10-07 10:02:04.647 UTC [ConnProducer] NewConnection -> ERRO 052 Failed connecting to {orderer0.org3:7050 [org1MSP org2MSP org3MSP]} , error: context deadline exceeded
2019-10-07 10:02:04.647 UTC [ConnProducer] NewConnection -> ERRO 053 Could not connect to any of the endpoints: [{orderer0.org2:7050 [org1MSP org2MSP org3MSP]} {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} {orderer0.org3:7050 [org1MSP org2MSP org3MSP]}]
2019-10-07 10:02:04.647 UTC [deliveryClient] connect -> ERRO 054 Failed obtaining connection: could not connect to any of the endpoints: [{orderer0.org2:7050 [org1MSP org2MSP org3MSP]} {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} {orderer0.org3:7050 [org1MSP org2MSP org3MSP]}]
2019-10-07 10:02:04.647 UTC [deliveryClient] try -> WARN 055 Got error: could not connect to any of the endpoints: [{orderer0.org2:7050 [org1MSP org2MSP org3MSP]} {orderer0.org1:7050 [org1MSP org2MSP org3MSP]} {orderer0.org3:7050 [org1MSP org2MSP org3MSP]}] , at 2 attempt. Retrying in 2s
2019-10-07 10:02:04.755 UTC [core.comm] ServerHandshake -> ERRO 056 TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=172.17.0.19:48822
2019-10-07 10:02:05.755 UTC [core.comm] ServerHandshake -> ERRO 057 TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=172.17.0.19:48836
2019-10-07 10:02:07.118 UTC [core.comm] ServerHandshake -> ERRO 058 TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=172.17.0.19:48854
2019-10-07 10:02:09.651 UTC [ConnProducer] NewConnection -> ERRO 059 Failed connecting to {orderer0.org2:7050 [org1MSP org2MSP org3MSP]} , error: context deadline exceeded
2019-10-07 10:02:10.595 UTC [gossip.gossip] func1 -> WARN 05a Deep probe of peer0.org2:7051 failed: context deadline exceeded
github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).learnAnchorPeers.func1
/opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:251
github.com/hyperledger/fabric/gossip/discovery.(*gossipDiscoveryImpl).Connect.func1
/opt/gopath/src/github.com/hyperledger/fabric/gossip/discovery/discovery_impl.go:153
runtime.goexit
/opt/go/src/runtime/asm_amd64.s:1333
2019-10-07 10:02:10.595 UTC [gossip.discovery] func1 -> WARN 05b Could not connect to Endpoint: peer0.org2:7051, InternalEndpoint: peer0.org2:7051, PKI-ID: <nil>, Metadata: : context deadline exceeded
How can I solve it? I tried to share CA Pems between orgs and set CORE_PEER_TLS_CLIENTROOTCAS_FILES to "/public/org1/ca-chain.pem /public/org2/ca-chain.pem /public/org3/ca-chain.pem", but it does not work. Do I have to add the client cert for each peer/orderer to every one of my nodes? Where? Is this problem not related to TLS? I am really lost.
Thank you.
Have you specified CORE_PEER_TLS_CERT_FILE, CORE_PEER_TLS_KEY_FILE and CORE_PEER_TLS_ROOTCERT_FILE (this one is for the CA) for each peer?
And if you run your CA Server with TLS you have to specify the cert and keyfiles for TLS there as well.
Related
I have a Hyperlegder Fabric network set-up which is operating fine as long as I don't use new Fabric-Gateway SDK (https://hyperledger-fabric.readthedocs.io/en/release-2.4/gateway.html).
I upgraded my network from 2.3.1 to 2.4.1 and wanted to try the new SDK, but cannot connect to the Peer. Below I give some details of my configuration.
Peer-base docker service:
peer-base:
image: hyperledger/fabric-peer
environment:
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=${COMPOSE_PROJECT_NAME}_basic
- FABRIC_LOGGING_SPEC=info:gateway,comm,comm.grpc,comm.grpc.server=debug
- CORE_CHAINCODE_LOGGING_LEVEL=info
- CORE_PEER_LISTENADDRESS=0.0.0.0:7051
- CORE_PEER_CHAINCODELISTENADDRESS=0.0.0.0:7052
- CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/peer/msp
- CORE_PEER_GOSSIP_USELEADERELECTION=true
- CORE_PEER_GOSSIP_ORGLEADER=false
- CORE_PEER_PROFILE_ENABLED=true
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/peer/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/peer/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/peer/tls/ca.crt
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=***
- CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=***
- CORE_METRICS_PROVIDER=prometheus
- CORE_OPERATIONS_LISTENADDRESS=0.0.0.0:7055
- CORE_PEER_GATEWAY_ENABLED=true
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: peer node start
volumes:
- ./config:/etc/hyperledger/configtx
- /var/run/:/host/var/run/
networks:
- basic
restart: always
After migrating to 2.4.1, I added CORE_PEER_GATEWAY_ENABLED=true.
The peer docker service, which extends the peer-base:
peer0.org1.tcash.com:
container_name: peer0.org1.tcash.com
extends:
file: docker-compose-org1-base.yaml
service: peer-base
environment:
- CORE_PEER_ID=peer0.org1.tcash.com
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_ADDRESS=peer0.org1.tcash.com:7051
- CORE_PEER_CHAINCODEADDRESS=peer0.org1.tcash.com:7052
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=test2.tcash.sigmacomp.pl:7051
- CORE_PEER_GOSSIP_ENDPOINT=test2.tcash.sigmacomp.pl:7051
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb0.org1.tcash.com:5984
ports:
- 7051:7051
- 7053:7053
- 7055:7055
volumes:
- ./crypto-config/peerOrganizations/org1.tcash.com/peers/peer0.org1.tcash.com:/etc/hyperledger/peer
- ./persistence/peer0.org1.tcash.com/:/var/hyperledger/production
depends_on:
- couchdb0.org1.tcash.com
extra_hosts:
- orderer0.tcash.com:146.59.17.169
- orderer1.tcash.com:146.59.17.169
- orderer2.tcash.com:146.59.17.169
- orderer3.tcash.com:146.59.17.169
- orderer4.tcash.com:146.59.17.169
- peer2.org1.tcash.com:51.195.202.90
- peer3.org1.tcash.com:51.195.202.90
- peer4.org1.tcash.com:51.68.172.244
- peer5.org1.tcash.com:51.68.172.244
No changes have been made here during migration to 2.4.1.
I can see in the Peer logs, that new gateway service has been started:
2022-01-21 12:34:09.177 UTC 0023 INFO [nodeCmd] serve -> Starting peer with Gateway enabled
2022-01-21 12:34:09.177 UTC 0024 INFO [nodeCmd] serve -> Starting peer with ID=[peer0.org1.tcash.com], network ID=[dev], address=[peer0.org1.tcash.com:7051]
2022-01-21 12:34:09.177 UTC 0025 INFO [nodeCmd] func7 -> Starting profiling server with listenAddress = 0.0.0.0:6060
2022-01-21 12:34:09.177 UTC 0026 INFO [nodeCmd] serve -> Started peer with ID=[peer0.org1.tcash.com], network ID=[dev], address=[peer0.org1.tcash.com:7051]
After deploying the network, I try to run the transaction with the following code (NodeJS):
'use strict';
const fs = require('fs');
const crypto = require('crypto');
const grpc =require('#grpc/grpc-js');
const { connect, signers } = require('#hyperledger/fabric-gateway');
async function main() {
// Main try/catch block
try {
const credentials = fs.readFileSync('walletOffline/user.cert.pem');
const identity = { mspId: 'Org1MSP', credentials };
const privateKeyPem = fs.readFileSync('walletOffline/user.key.pem');
const privateKey = crypto.createPrivateKey(privateKeyPem);
const signer = signers.newPrivateKeySigner(privateKey);
const ccpJSON = fs.readFileSync('connection.json');
const ccp = JSON.parse(ccpJSON);
const peerName = ccp.organizations.org1.peers[0];
const peerAddress = ccp.peers[peerName].url.replace('grpcs://', '');
const tlsCACert = ccp.peers[peerName].tlsCACerts.pem;
const grpcOptions = ccp.peers[peerName].grpcOptions;
const tlsRootCert = Buffer.from(tlsCACert);
const tlsCredentials = grpc.credentials.createSsl(tlsRootCert);
const client = new grpc.Client(peerAddress, tlsCredentials, grpcOptions);
const gateway = connect({identity, signer, client});
const network = gateway.getNetwork('tcashchannel');
const contract = network.getContract('tcash');
const result = await contract.evaluateTransaction('queryAccountState', '100', '');
console.log('result: ' + result);
} catch (error) {
console.log('Error: ' + error);
console.log(error.stack);
}
}
main();
As you can see, I am extracting connection parameters from the JSON connection profile. This connection profile I use with the 'old' HF Node SDK and it's working without issues. However running this code gives me the following error from contract.evaluateTransaction() after 120 seconds timeout:
GatewayError: 14 UNAVAILABLE: failed to create new connection: context deadline exceeded
at newGatewayError (/Users/michaliwanicki/git/tcash/tcash-application/node_modules/#hyperledger/fabric-gateway/dist/gatewayerror.js:40:12)
at Object.callback (/Users/michaliwanicki/git/tcash/tcash-application/node_modules/#hyperledger/fabric-gateway/dist/client.js:81:67)
at Object.onReceiveStatus (/Users/michaliwanicki/git/tcash/tcash-application/node_modules/#grpc/grpc-js/build/src/client.js:180:36)
at Object.onReceiveStatus (/Users/michaliwanicki/git/tcash/tcash-application/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/Users/michaliwanicki/git/tcash/tcash-application/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /Users/michaliwanicki/git/tcash/tcash-application/node_modules/#grpc/grpc-js/build/src/call-stream.js:182:78
at processTicksAndRejections (internal/process/task_queues.js:77:11)
I can also see the corresponding entry in the peer logs:
2022-01-21 14:24:14.961 UTC 007e INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Evaluate grpc.peer_address=178.183.68.178:54151 error="rpc error: code = Unavailable desc = failed to create new connection: context deadline exceeded" grpc.code=Unavailable grpc.call_duration=2m0.00087636s
There are no errors or warnings in the peer log.
EDIT:
After switching logging level to DEBUG and filtering it out, I came across the following part:
2022-01-27 13:38:19.217 UTC 67af DEBU [core.comm] ServerHandshake -> Server TLS handshake completed in 69.892651ms server=PeerServer remoteaddress=178.183.68.178:58755
2022-01-27 13:38:19.356 UTC 67b0 DEBU [lockbasedtxmgr] newQueryExecutor -> constructing new query executor txid = [407898ef-0004-4f25-be10-b603a2aaf919]
2022-01-27 13:38:19.357 UTC 67b1 DEBU [statecouchdb] GetState -> GetState(). ns=, key=CHANNEL_CONFIG_ENV_BYTES
2022-01-27 13:38:19.358 UTC 67b2 DEBU [lockbasedtxmgr] Done -> Done with transaction simulation / query execution [407898ef-0004-4f25-be10-b603a2aaf919]
2022-01-27 13:38:19.358 UTC [grpc] WarningDepth -> DEBU 02f [core]Adjusting keepalive ping interval to minimum period of 10s
2022-01-27 13:38:19.359 UTC [grpc] InfoDepth -> DEBU 030 [core]parsed scheme: ""
2022-01-27 13:38:19.359 UTC [grpc] InfoDepth -> DEBU 031 [core]scheme "" not registered, fallback to default scheme
2022-01-27 13:38:19.359 UTC [grpc] InfoDepth -> DEBU 032 [core]ccResolverWrapper: sending update to cc: {[{test2.tcash.sigmacomp.pl:8051 <nil> 0 <nil>}] <nil> <nil>}
2022-01-27 13:38:19.360 UTC [grpc] InfoDepth -> DEBU 033 [core]ClientConn switching balancer to "pick_first"
2022-01-27 13:38:19.360 UTC [grpc] InfoDepth -> DEBU 034 [core]Channel switches to new LB policy "pick_first"
2022-01-27 13:38:19.360 UTC [grpc] InfoDepth -> DEBU 035 [core]Subchannel Connectivity change to CONNECTING
2022-01-27 13:38:19.360 UTC [grpc] InfoDepth -> DEBU 036 [core]pickfirstBalancer: UpdateSubConnState: 0xc002ed2b30, {CONNECTING <nil>}
2022-01-27 13:38:19.361 UTC [grpc] InfoDepth -> DEBU 037 [core]Channel Connectivity change to CONNECTING
2022-01-27 13:38:19.360 UTC [grpc] InfoDepth -> DEBU 038 [core]Subchannel picks a new address "test2.tcash.sigmacomp.pl:8051" to connect
2022-01-27 13:38:19.370 UTC [grpc] InfoDepth -> DEBU 039 [core]Subchannel Connectivity change to TRANSIENT_FAILURE
2022-01-27 13:38:19.370 UTC [grpc] InfoDepth -> DEBU 03a [core]pickfirstBalancer: UpdateSubConnState: 0xc002ed2b30, {TRANSIENT_FAILURE connection closed}
2022-01-27 13:38:19.370 UTC [grpc] InfoDepth -> DEBU 03b [core]Channel Connectivity change to TRANSIENT_FAILURE
2022-01-27 13:38:19.370 UTC [grpc] InfoDepth -> DEBU 03c [transport]transport: loopyWriter.run returning. connection error: desc = "transport is closing"
EDIT 2:
I noticed that there are some errors in peer logs belonging to the other peers in the network (not the one which is called by the client application and running the Gateway service). It seems that there is a problem with establishment of TLS between peers when using Gateway SDK:
2022-02-10 14:36:24.934 UTC 24b0 DEBU [gossip.comm] func1 -> Got message: GossipMessage: Channel: , nonce: 0, tag: CHAN_OR_ORG state_info_pull_req: Channel MAC:23b92135be842b052b823a7c87853436fb579040416405d4fdfd0b6db0aa02d9, Envelope: 39 bytes, Signature: 0 bytes
2022-02-10 14:36:24.934 UTC 24b1 DEBU [gossip.gossip] handleMessage -> Entering, 54.37.226.59:7051 5c2af6d536100ada4e7f1829978c7f0163a6589f47f44207aa51a84987fe6a5b sent us GossipMessage: Channel: , nonce: 0, tag: CHAN_OR_ORG state_info_pull_req: Channel MAC:23b92135be842b052b823a7c87853436fb579040416405d4fdfd0b6db0aa02d9, Envelope: 39 bytes, Signature: 0 bytes
2022-02-10 14:36:24.935 UTC 24b2 DEBU [gossip.gossip] handleMessage -> Exiting
2022-02-10 14:36:24.942 UTC 24b3 ERRO [core.comm] ServerHandshake -> Server TLS handshake failed in 15.541µs with error tls: first record does not look like a TLS handshake server=PeerServer remoteaddress=172.24.0.1:36394
2022-02-10 14:36:24.942 UTC [grpc] WarningDepth -> DEBU 04e [core]grpc: Server.Serve failed to complete security handshake from "172.24.0.1:36394": tls: first record does not look like a TLS handshake
I suspect that there is some piece of configuration which is required for this feature to work, which I am missing. I will appreciate if anyone can help me find it.
It looks like the gateway peer is failing to connect to another endorsing peer in the network. Are you seeing any gossip communication between the peers in the logs?
Try reducing the dialTimeout to something less than the endorsementTimeout in the core.yaml and see if it connects to the other peers.
I know there are several errors like this one, but I see most of them are with the test-network, using a local environment and my case is different.
I have a blockchain running using k8s it used to be running in V1.4 and we migrated it to V2.3 so the certificates were generated without a SAN but to make them work with the new version we are using the env variable GODEBUG="x509ignoreCN=0" I can connect to it using a console client to do changes in the channel and configuration, and I can connect with a node js client and execute contracts, I want to connect using a Go client for a different app but I'm having trouble with the config.
I'm getting a TRANSIENT_FAILURE error in the Go Client and a tls: bad certificate in the peer but the certificates are the same that work for the other environments, the config.yaml I'm using is based on my configuration from the test-network that's also working.
Something weird though, is that the configPath doesn't seem to work and I need to add the users individualy to make it run.
So this is my config.yaml
---
name: network-company
version: 1.0.0
client:
organization: company
logging:
level: debug
connection:
timeout:
peer:
endorser: "300"
cryptoconfig:
path: /home/company
credentialStore:
path: /home/company/users
cryptoStore:
path: /home/company/users/admin#mycompany.com/msp
BCCSP:
security:
enabled: true
default:
provider: "SW"
hashAlgorithm: "SHA2"
softVerify: true
level: 256
tlsCerts:
systemCertPool: false
client:
cert:
path: /home/company/users/admin#mycompany.com/tls/cert.pem
key:
path: /home/company/users/admin#mycompany.com/tls/key.pem
organizations:
company:
mspid: company
cryptoPath: /home/company/users/admin#mycompany.com/msp
users:
admin:
cert:
path: /home/company/users/admin#mycompany.com/msp/signcerts/cert.pem
key:
path: /home/company/users/admin#mycompany.com/msp/keystore/key.pem
peers:
- peer1.mycompany.com
- peer2.mycompany.com
peers:
peer1.mycompany.com:
url: grpcs://peer1.mycompany.com:443
tlsCACerts:
path: /home/company/users/admin#mycompany.com/tls/ca.pem
grpcOptions:
ssl-target-name-override: peer1.mycompany.com
hostnameOverride: peer1.mycompany.com
keep-alive-time: 10s
keep-alive-timeout: 20s
keep-alive-permit: false
fail-fast: false
httpOptions:
verify: false
peer2.mycompany.com:
url: grpcs://peer2.mycompany.com:443
tlsCACerts:
path: /home/company/users/admin#mycompany.com/tls/ca.pem
grpcOptions:
ssl-target-name-override: peer2.mycompany.com
hostnameOverride: peer2.mycompany.com
keep-alive-time: 10s
keep-alive-timeout: 20s
keep-alive-permit: false
fail-fast: false
allow-insecure: true
httpOptions:
verify: false
channels:
mychannel:
peers:
peer1.mycompany.com:
endorsingPeer: true
chaincodeQuery: true
ledgerQuery: true
eventSource: true
peer2.mycompany.com:
endorsingPeer: true
chaincodeQuery: true
ledgerQuery: true
eventSource: true
policies:
queryChannelConfig:
minResponses: 1
maxTargets: 1
retryOpts:
attempts: 5
initialBackoff: 500ms
maxBackoff: 5s
backoffFactor: 2.0
entityMatchers:
peer:
- pattern: peer1.mycompany.com
urlSubstitutionExp: peer1.mycompany.com:443
sslTargetOverrideUrlSubstitutionExp: peer1.mycompany.com
mappedHost: peer1.mycompany.com
- pattern: peer2.mycompany.com
urlSubstitutionExp: peer2.mycompany.com:443
sslTargetOverrideUrlSubstitutionExp: peer2.mycompany.com
mappedHost: peer2.mycompany.com
These are my certificates paths:
/home/company/users/admin#mycompany.com
+ msp
+ admincerts/<empty>
+ signcerts/cert.pem
+ cacerts/ca.pem
+ intermediatecerts/ca.pem
+ tlscacerts/ca.pem
+ tlsintermediatecerts/ca.pem
+ keystore/key.pem
+ tls
+ ca.pem
+ cert.pem
+ key.pem
These are the logs from the client:
[fabsdk/core] 2022/01/04 14:34:09 UTC - sw.getOptsByConfig -> DEBU Initialized SW cryptosuite
[fabsdk/fab] 2022/01/04 14:34:09 UTC - n/a -> DEBU KeyStore opened at [/home/company/users/admin#mycompany.com/msp/keystore]...done
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).loadEndpointConfiguration -> DEBU Client is: {Organization:company TLSCerts:{Client:{Key:{Path:/home/company/users/admin-tls#mycompany.com/tls/key.pem Pem: bytes:[]} Cert:{Path:/home/company/users/admin-tls#mycompany.com/tls/cert.pem Pem: bytes:[]}}}}
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).loadEndpointConfiguration -> DEBU channels are: map[mychannel:{Orderers:[] Peers:map[peer1.mycompany.com:{EndorsingPeer:true ChaincodeQuery:true LedgerQuery:true EventSource:true} peer2.mycompany.com:{EndorsingPeer:true ChaincodeQuery:true LedgerQuery:true EventSource:true}] Policies:{QueryChannelConfig:{MinResponses:1 MaxTargets:1 RetryOpts:{Attempts:5 InitialBackoff:500ms MaxBackoff:5s BackoffFactor:2 RetryableCodes:map[]}} Discovery:{MinResponses:0 MaxTargets:0 RetryOpts:{Attempts:0 InitialBackoff:0s MaxBackoff:0s BackoffFactor:0 RetryableCodes:map[]}} Selection:{SortingStrategy: Balancer: BlockHeightLagThreshold:0} EventService:{ResolverStrategy: MinBlockHeightResolverMode: Balancer: BlockHeightLagThreshold:0 PeerMonitor: ReconnectBlockHeightLagThreshold:0 PeerMonitorPeriod:0s}}}]
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).loadEndpointConfiguration -> DEBU organizations are: map[company:{MSPID:company CryptoPath:/home/company/users/admin#mycompany.com/msp Users:map[admin:{Key:{Path:/home/company/users/admin#mycompany.com/msp/keystore/key.pem Pem: bytes:[]} Cert:{Path:/home/company/users/admin#mycompany.com/msp/signcerts/cert.pem Pem: bytes:[]}}] Peers:[peer1.mycompany.com peer2.mycompany.com] CertificateAuthorities:[]}]
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).loadEndpointConfiguration -> DEBU orderers are: map[]
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).loadEndpointConfiguration -> DEBU peers are: map[peer1.mycompany.com:{URL:grpcs://peer1.mycompany.com:443 GRPCOptions:map[allow-insecure:true fail-fast:false hostnameoverride:peer1.mycompany.com keep-alive-permit:false keep-alive-time:10s keep-alive-timeout:20s ssl-target-name-override:peer1.mycompany.com] TLSCACerts:{Path:/home/company/users/admin#mycompany.com/tls/ca.pem Pem: bytes:[]}} peer2.mycompany.com:{URL:grpcs://peer2.mycompany.com:443 GRPCOptions:map[allow-insecure:true fail-fast:false hostnameoverride:peer2.mycompany.com keep-alive-permit:false keep-alive-time:10s keep-alive-timeout:20s ssl-target-name-override:peer2.mycompany.com] TLSCACerts:{Path:/home/company/users/admin#mycompany.com/tls/ca.pem Pem: bytes:[]}}]
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).compileMatchers -> DEBU Matchers are: {matchers:map[]}
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).compileMatchers -> DEBU Entity matchers are not configured
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).loadTLSClientCerts -> DEBU Reading pk from config, unable to retrieve from cert: Could not find matching key for SKI: Failed getting key for SKI [[91 7 147 74 144 157 11 46 234 18 159 199 54 134 208 140 16 147 185 225 158 7 112 91 182 170 49 46 139 190 194 99]]: key with SKI 5b07934a909d0b2eea129fc73686d08c1093b9e19e07705bb6aa312e8bbec263 not found in /home/company/users/admin#mycompany.com/msp/keystore
[fabsdk/fab] 2022/01/04 14:34:09 UTC - fab.(*EndpointConfig).loadPrivateKeyFromConfig -> DEBU pk read from config successfully
[fabsdk/msp] 2022/01/04 14:34:09 UTC - msp.(*IdentityConfig).loadIdentityConfigEntities -> DEBU Client is: {Organization:company Logging:{Level:debug} CryptoConfig:{Path:/home/company} TLSCerts:{Client:{Key:{Path:/home/company/users/admin-tls#mycompany.com/tls/key.pem Pem: bytes:[]} Cert:{Path:/home/company/users/admin-tls#mycompany.com/tls/cert.pem Pem: bytes:[]}} SystemCertPool:false} CredentialStore:{Path:/home/company/users CryptoStore:{Path:/home/company/users/admin#mycompany.com/msp}}}
[fabsdk/msp] 2022/01/04 14:34:09 UTC - msp.(*IdentityConfig).loadIdentityConfigEntities -> DEBU organizations are: map[company:{MSPID:company CryptoPath:/home/company/users/admin#mycompany.com/msp Users:map[admin:{Key:{Path:/home/company/users/admin#mycompany.com/msp/keystore/key.pem Pem: bytes:[]} Cert:{Path:/home/company/users/admin#mycompany.com/msp/signcerts/cert.pem Pem: bytes:[]}}] Peers:[peer1.mycompany.com peer2.mycompany.com] CertificateAuthorities:[]}]
[fabsdk/msp] 2022/01/04 14:34:09 UTC - msp.(*IdentityConfig).loadIdentityConfigEntities -> DEBU certificateAuthorities are: map[]
[fabsdk/msp] 2022/01/04 14:34:09 UTC - msp.(*IdentityConfig).compileMatchers -> DEBU Matchers are: {matchers:map[]}
[fabsdk] 2022/01/04 14:34:09 UTC - fabsdk.initSDK -> DEBU SDK initialized successfully
SDK created
resourceManagerClient created
[fabsdk/fab] 2022/01/04 14:34:09 UTC - chconfig.(*params).SetChConfigRefreshInterval -> DEBU RefreshInterval: 1m30s
[fabsdk/util] 2022/01/04 14:34:09 UTC - lazyref.WithRefreshInterval.func1 -> DEBU Checking refreshIntervalSetter
[fabsdk/fab] 2022/01/04 14:34:09 UTC - peer.(*peerEndorser).ProcessTransactionProposal -> DEBU Processing proposal using endorser: peer2.mycompany.com:443
[fabsdk/fab] 2022/01/04 14:34:09 UTC - comm.(*CachingConnector).DialContext -> DEBU DialContext: peer2.mycompany.com:443
[fabsdk/fab] 2022/01/04 14:34:09 UTC - comm.(*CachingConnector).createConn -> DEBU creating connection [peer2.mycompany.com:443]
[fabsdk/fab] 2022/01/04 14:34:09 UTC - comm.(*CachingConnector).createConn -> DEBU storing connection [peer2.mycompany.com:443]
[fabsdk/fab] 2022/01/04 14:34:09 UTC - comm.(*CachingConnector).removeConn -> DEBU removing connection [peer2.mycompany.com:443]
[fabsdk/fab] 2022/01/04 14:34:09 UTC - txn.SendProposal.func1 -> DEBU Received error response from txn proposal processing: Transaction processing for endorser [peer2.mycompany.com:443]: Endorser Client Status Code: (2) CONNECTION_FAILED. Description: dialing connection on target [peer2.mycompany.com:443]: connection is in TRANSIENT_FAILURE
[fabsdk/common] 2022/01/04 14:34:09 UTC - retry.(*RetryableInvoker).Invoke -> DEBU Failed with err [queryChaincode failed: Transaction processing for endorser [peer2.mycompany.com:443]: Endorser Client Status Code: (2) CONNECTION_FAILED. Description: dialing connection on target [peer2.mycompany.com:443]: connection is in TRANSIENT_FAILURE] on attempt #1. Checking if retry is warranted...
[fabsdk/common] 2022/01/04 14:34:09 UTC - retry.(*RetryableInvoker).Invoke -> DEBU ... retry for err [queryChaincode failed: Transaction processing for endorser [peer2.mycompany.com:443]: Endorser Client Status Code: (2) CONNECTION_FAILED. Description: dialing connection on target [peer2.mycompany.com:443]: connection is in TRANSIENT_FAILURE] is NOT warranted after 1 attempt(s).
2022/01/04 14:34:09 error inicializando el SDK: error creando el cliente del canal: event service creation failed: could not get chConfig cache reference: QueryBlockConfig failed: QueryBlockConfig failed: queryChaincode failed: Transaction processing for endorser [peer2.mycompany.com:443]: Endorser Client Status Code: (2) CONNECTION_FAILED. Description: dialing connection on target [peer2.mycompany.com:443]: connection is in TRANSIENT_FAILURE
These are the logs from the peer:
2022-01-04 15:23:56.740 UTC [core.comm] ServerHandshake -> ERRO 335 Server TLS handshake failed in 7.869808ms with error remote error: tls: bad certificate server=PeerServer remoteaddress=10.250.37.211:33666
2022-01-04 15:23:56.740 UTC [grpc] WarningDepth -> DEBU 336 [core]grpc: Server.Serve failed to complete security handshake from "10.250.37.211:33666": remote error: tls: bad certificate
A couple of weird things: I need to add the users to the organization, some how it's not able to extract the users from the configPath and I get an error user not found.
It's looking for the tls private key in the msp/keystore folder and of course that isn't there
I tried adding the GODEBUG="x509ignoreCN=0" but it doesn't change the result
I generated an extra TLS certificates just for the client adding the SAN to them as "email:admin-tls#mycompany.com" and mapped it to the configuration with the same result.
I have tried adding and removing the entityMatchers with the same result.
I also did a tcp dump on it server answers to the Client hello with Server hello and then fails with a 400 bad request error.
I have used the configurations referenced from here and here but everything looks good to me.
I'm running out of ideas, is my configuration right? why do I need to force the clients and configPath doesn't work? why is it looking for the tls key in the msp keystore? but mostly why do I get a tls bad certificate if the certificates are good and pass an openssl verify against the ca.pem?
Thanks
UPDATE
This is my certificate info using openssl x509 -in certificate.pem -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
56:00:01:ba:18:d9:cf:ed:a7:00:b9:af:b3:00:01:00:01:ba:18
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = CO, L = Medellin, O = mycompany, CN = CA My Company, emailAddress = admin#mycompany.com
Validity
Not Before: Jan 5 12:36:08 2022 GMT
Not After : Jan 5 12:36:08 2023 GMT
Subject: C = CO, ST = ANTIOQUIA, L = MEDELLIN, O = mycompany, OU = admin, CN = admin-tls#mycompany.com
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:d0:c8:3b:20:2f:8f:3b:91:72:f8:71:27:30:cf:
b5:6e:38:e3:5f:e6:1b:42:77:fd:00:f2:8c:1b:ae:
6c:2e:7d:50:03:75:d5:03:db:72:d5:33:27:f3:65:
c1:2a:09:72:6a:d5:cc:c5:45:ca:e5:e5:8c:65:4b:
f2:51:dc:dd:ac
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Key Usage:
Digital Signature
X509v3 Extended Key Usage:
TLS Web Server Authentication, Microsoft Encrypted File System, E-mail Protection, TLS Web Client Authentication, Any Extended Key Usage
X509v3 Subject Alternative Name:
DNS:mycompany.com
X509v3 Subject Key Identifier:
9E:C5:4B:E8:67:B1:84:BF:67:E9:BE:A3:DE:B5:EB:E3:3B:4E:D3:D0
X509v3 Authority Key Identifier:
keyid:40:54:B4:93:16:92:20:E3:DB:AA:7A:A3:13:06:EC:65:4F:83:B3:58
X509v3 CRL Distribution Points:
Full Name:
URI:http://crl.epm.com.co/CRL/CAEPMClaseI.crl
URI:http://crl2.epm.com.co/CRL/CAEPMClaseI.crl
Authority Information Access:
OCSP - URI:http://ocsp.epm.com.co/OCSP
CA Issuers - URI:http://crl.epm.com.co/CRL/CAEPMClaseI.crt
1.3.6.1.4.1.311.21.7:
0-.%+.....7.....-...K..=...>...k8...,...A..d..
S/MIME Capabilities:
......0...+....0050...*.H..
..*.H..
Signature Algorithm: sha256WithRSAEncryption
6f:d6:85:b1:08:cd:07:78:3d:92:50:85:14:3a:f0:2e:1f:84:
10:7c:1d:15:10:0f:9f:66:e7:1a:41:63:12:83:0c:7c:04:c7:
c8:f4:8b:97:d9:6e:28:78:cb:9b:af:89:6b:1a:f3:06:40:1e:
6f:90:81:8f:f9:03:ff:92:5a:de:8c:7c:35:f9:21:00:fd:85:
c0:73:48:71:c0:82:7a:17:6d:09:ce:b6:03:5a:c5:1f:4f:47:
6d:95:94:a3:e2:cd:19:03:3d:85:a0:7e:5e:cf:a5:55:32:9f:
28:cf:d4:10:c8:42:57:a1:9b:cc:9a:4b:82:75:86:15:92:b9:
ef:d2:b2:1c:5d:a9:fe:ff:d4:f2:83:9d:31:46:50:26:ea:78:
88:9c:72:0a:1e:81:42:88:7b:04:4b:4b:2b:86:3d:ed:f9:4d:
8f:5e:47:1b:0b:8e:a9:8a:ed:ed:8a:09:92:58:6b:14:6e:aa:
50:c1:ae:2f:75:41:5b:e9:28:2b:85:9f:70:0a:d3:27:64:44:
f4:f6:61:99:a0:db:90:05:8c:af:09:fa:36:a5:67:3a:b8:f5:
b3:06:fe:ba:50:aa:1f:2e:43:af:a6:3a:81:01:d1:26:cc:61:
3b:c3:d9:b5:50:d7:27:61:f6:3e:a3:1d:99:0f:f1:37:20:d4:
ea:6d:59:a9:56:09:09:74:90:d4:52:64:69:b5:ad:09:b8:67:
74:ca:52:76:0a:37:0e:29:e0:5b:5a:bc:24:42:a1:14:77:1f:
72:f5:e9:f8:64:17:2e:fc:57:e1:d3:74:3a:1d:a5:fb:f4:c3:
a3:1e:97:f3:ba:d6:33:6a:24:d5:79:d7:9c:07:b4:9d:d0:52:
24:dc:0d:d5:a1:37:42:25:09:3b:96:a5:91:19:6c:7e:58:71:
99:ff:04:14:4d:ea:25:57:23:58:2b:44:84:f9:c3:c4:f6:a6:
ed:81:75:0a:07:e4:ab:bf:74:ec:e6:8c:9e:b8:75:82:47:20:
75:64:94:45:93:2f:ec:8b:8f:2c:19:c8:b0:29:a0:4c:f7:05:
6d:c8:3d:54:78:9e:ac:17:1c:19:5e:3c:1a:ba:93:db:1c:41:
ab:09:e2:d7:e3:11:87:eb:b7:38:e6:cf:85:1e:8a:75:9d:8c:
68:44:5d:d6:60:e7:91:52:ec:08:5b:26:7b:a4:42:d6:49:fc:
3c:74:6c:96:23:99:86:6b:6b:57:89:94:ad:08:98:35:1d:38:
07:09:ae:c3:8c:1f:3c:36:94:a9:da:74:73:ff:5e:c1:0d:f7:
45:a6:85:94:1d:34:46:56:fd:82:3f:8d:c0:d0:a1:20:1b:76:
d3:2e:f8:a7:6c:3f:25:f4
So the issue seemed to be with the TLS certificates from the peers, they where generated before go 1.16 and didn't had the SANs in them, and the error seemed to be getting reflected to the go SDK. #yacovm thank you for your help figuring it out.
I've been following the tutorial for the Hyperledger Fabric development
but i have this error message while trying the command ./network.sh createChannel:
Error: failed to create deliver client for orderer: orderer client failed to connect to localhost:7050: failed to create new connection: connection error: desc = "transport: error while dialing: dial tcp [::1]:7050: connectex: Aucune connexion n’a pu être établie car l’ordinateur cible l’a expressément refusée."
!!!!!!!!!!!!!!! Channel creation failed !!!!!!!!!!!!!!!!
The French part means that no connection can be established cause the targeted computer has refused it.
After searching a bit there might be an issue with the ports or ip addresses but not sure.
My logs for the different components :
Orderer service :
2020-05-27 09:13:16.388 UTC [localconfig] completeInitialization -> WARN 001 General.GenesisFile should be replaced by General.BootstrapFile
2020-05-27 09:13:16.389 UTC [localconfig] completeInitialization -> INFO 002 Kafka.Version unset, setting to 0.10.2.0
2020-05-27 09:13:16.389 UTC [orderer.common.server] prettyPrintStruct -> INFO 003 Orderer config values:
General.ListenAddress = "0.0.0.0"
General.ListenPort = 7050
General.TLS.Enabled = true
General.TLS.PrivateKey = "/var/hyperledger/orderer/tls/server.key"
General.TLS.Certificate = "/var/hyperledger/orderer/tls/server.crt"
General.TLS.RootCAs = [/var/hyperledger/orderer/tls/ca.crt]
General.TLS.ClientAuthRequired = false
General.TLS.ClientRootCAs = []
General.Cluster.ListenAddress = ""
General.Cluster.ListenPort = 0
General.Cluster.ServerCertificate = ""
General.Cluster.ServerPrivateKey = ""
General.Cluster.ClientCertificate = "/var/hyperledger/orderer/tls/server.crt"
General.Cluster.ClientPrivateKey = "/var/hyperledger/orderer/tls/server.key"
General.Cluster.RootCAs = [/var/hyperledger/orderer/tls/ca.crt]
General.Cluster.DialTimeout = 5s
General.Cluster.RPCTimeout = 7s
General.Cluster.ReplicationBufferSize = 20971520
General.Cluster.ReplicationPullTimeout = 5s
General.Cluster.ReplicationRetryTimeout = 5s
General.Cluster.ReplicationBackgroundRefreshInterval = 5m0s
General.Cluster.ReplicationMaxRetries = 12
General.Cluster.SendBufferSize = 10
General.Cluster.CertExpirationWarningThreshold = 168h0m0s
General.Cluster.TLSHandshakeTimeShift = 0s
General.Keepalive.ServerMinInterval = 1m0s
General.Keepalive.ServerInterval = 2h0m0s
General.Keepalive.ServerTimeout = 20s
General.ConnectionTimeout = 0s
General.GenesisMethod = "file"
General.GenesisFile = "/var/hyperledger/orderer/orderer.genesis.block"
General.BootstrapMethod = "file"
General.BootstrapFile = "/var/hyperledger/orderer/orderer.genesis.block"
General.Profile.Enabled = false
General.Profile.Address = "0.0.0.0:6060"
General.LocalMSPDir = "/var/hyperledger/orderer/msp"
General.LocalMSPID = "OrdererMSP"
General.BCCSP.ProviderName = "SW"
General.BCCSP.SwOpts.SecLevel = 256
General.BCCSP.SwOpts.HashFamily = "SHA2"
General.BCCSP.SwOpts.Ephemeral = true
General.BCCSP.SwOpts.FileKeystore.KeyStorePath = ""
General.BCCSP.SwOpts.DummyKeystore =
General.BCCSP.SwOpts.InmemKeystore =
General.Authentication.TimeWindow = 15m0s
General.Authentication.NoExpirationChecks = false
FileLedger.Location = "/var/hyperledger/production/orderer"
FileLedger.Prefix = "hyperledger-fabric-ordererledger"
Kafka.Retry.ShortInterval = 5s
Kafka.Retry.ShortTotal = 10m0s
Kafka.Retry.LongInterval = 5m0s
Kafka.Retry.LongTotal = 12h0m0s
Kafka.Retry.NetworkTimeouts.DialTimeout = 10s
Kafka.Retry.NetworkTimeouts.ReadTimeout = 10s
Kafka.Retry.NetworkTimeouts.WriteTimeout = 10s
Kafka.Retry.Metadata.RetryMax = 3
Kafka.Retry.Metadata.RetryBackoff = 250ms
Kafka.Retry.Producer.RetryMax = 3
Kafka.Retry.Producer.RetryBackoff = 100ms
Kafka.Retry.Consumer.RetryBackoff = 2s
Kafka.Verbose = true
Kafka.Version = 0.10.2.0
Kafka.TLS.Enabled = false
Kafka.TLS.PrivateKey = ""
Kafka.TLS.Certificate = ""
Kafka.TLS.RootCAs = []
Kafka.TLS.ClientAuthRequired = false
Kafka.TLS.ClientRootCAs = []
Kafka.SASLPlain.Enabled = false
Kafka.SASLPlain.User = ""
Kafka.SASLPlain.Password = ""
Kafka.Topic.ReplicationFactor = 1
Debug.BroadcastTraceDir = ""
Debug.DeliverTraceDir = ""
Consensus = map[SnapDir:/var/hyperledger/production/orderer/etcdraft/snapshot WALDir:/var/hyperledger/production/orderer/etcdraft/wal]
Operations.ListenAddress = "127.0.0.1:8443"
Operations.TLS.Enabled = false
Operations.TLS.PrivateKey = ""
Operations.TLS.Certificate = ""
Operations.TLS.RootCAs = []
Operations.TLS.ClientAuthRequired = false
Operations.TLS.ClientRootCAs = []
Metrics.Provider = "disabled"
Metrics.Statsd.Network = "udp"
Metrics.Statsd.Address = "127.0.0.1:8125"
Metrics.Statsd.WriteInterval = 30s
Metrics.Statsd.Prefix = ""
2020-05-27 09:13:16.400 UTC [msp] loadCertificateAt -> WARN 004 Failed loading ClientOU certificate at [/var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem]: [could not read file /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: open /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.400 UTC [msp] loadCertificateAt -> WARN 005 Failed loading PeerOU certificate at [/var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem]: [could not read file /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: open /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.400 UTC [msp] loadCertificateAt -> WARN 006 Failed loading AdminOU certificate at [/var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem]: [could not read file /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: open /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.400 UTC [msp] loadCertificateAt -> WARN 007 Failed loading OrdererOU certificate at [/var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem]: [could not read file /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: open /var/hyperledger/orderer/msp/cacerts\ca.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.434 UTC [orderer.common.server] initializeServerConfig -> INFO 008 Starting orderer with TLS enabled
2020-05-27 09:13:16.441 UTC [fsblkstorage] NewProvider -> INFO 009 Creating new file ledger directory at /var/hyperledger/production/orderer/chains
2020-05-27 09:13:16.464 UTC [orderer.common.server] extractSysChanLastConfig -> INFO 00a Bootstrapping because no existing channels
2020-05-27 09:13:16.483 UTC [orderer.common.server] Main -> INFO 00b Setting up cluster for orderer type etcdraft
2020-05-27 09:13:16.490 UTC [orderer.common.server] reuseListener -> INFO 00c Cluster listener is not configured, defaulting to use the general listener on port 7050
2020-05-27 09:13:16.490 UTC [fsblkstorage] newBlockfileMgr -> INFO 00d Getting block information from block storage
2020-05-27 09:13:16.529 UTC [orderer.consensus.etcdraft] HandleChain -> INFO 00e EvictionSuspicion not set, defaulting to 10m0s
2020-05-27 09:13:16.530 UTC [orderer.consensus.etcdraft] createOrReadWAL -> INFO 00f No WAL data found, creating new WAL at path '/var/hyperledger/production/orderer/etcdraft/wal/system-channel' channel=system-channel node=1
2020-05-27 09:13:16.536 UTC [orderer.commmon.multichannel] Initialize -> INFO 010 Starting system channel 'system-channel' with genesis block hash 22c93e29c38e9681f960d390fda12c72869fc9ebfebf0a6d1c15f60198b13119 and orderer type etcdraft
2020-05-27 09:13:16.537 UTC [orderer.consensus.etcdraft] Start -> INFO 011 Starting Raft node channel=system-channel node=1
2020-05-27 09:13:16.537 UTC [orderer.common.cluster] Configure -> INFO 012 Entering, channel: system-channel, nodes: []
2020-05-27 09:13:16.537 UTC [orderer.common.cluster] Configure -> INFO 013 Exiting
2020-05-27 09:13:16.537 UTC [orderer.consensus.etcdraft] start -> INFO 014 Starting raft node as part of a new channel channel=system-channel node=1
2020-05-27 09:13:16.537 UTC [orderer.consensus.etcdraft] becomeFollower -> INFO 015 1 became follower at term 0 channel=system-channel node=1
2020-05-27 09:13:16.538 UTC [orderer.consensus.etcdraft] newRaft -> INFO 016 newRaft 1 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0] channel=system-channel node=1
2020-05-27 09:13:16.538 UTC [orderer.consensus.etcdraft] becomeFollower -> INFO 017 1 became follower at term 1 channel=system-channel node=1
2020-05-27 09:13:16.538 UTC [orderer.common.server] Main -> INFO 018 Starting orderer:
Version: 2.1.0
Commit SHA: 1bdf975
Go version: go1.14.1
OS/Arch: linux/amd64
2020-05-27 09:13:16.538 UTC [orderer.common.server] Main -> INFO 019 Beginning to serve requests
2020-05-27 09:13:16.538 UTC [orderer.consensus.etcdraft] run -> INFO 01a This node is picked to start campaign channel=system-channel node=1
2020-05-27 09:13:16.539 UTC [orderer.consensus.etcdraft] apply -> INFO 01b Applied config change to add node 1, current nodes in channel: [1] channel=system-channel node=1
2020-05-27 09:13:17.539 UTC [orderer.consensus.etcdraft] Step -> INFO 01c 1 is starting a new election at term 1 channel=system-channel node=1
2020-05-27 09:13:17.540 UTC [orderer.consensus.etcdraft] becomePreCandidate -> INFO 01d 1 became pre-candidate at term 1 channel=system-channel node=1
2020-05-27 09:13:17.540 UTC [orderer.consensus.etcdraft] poll -> INFO 01e 1 received MsgPreVoteResp from 1 at term 1 channel=system-channel node=1
2020-05-27 09:13:17.540 UTC [orderer.consensus.etcdraft] becomeCandidate -> INFO 01f 1 became candidate at term 2 channel=system-channel node=1
2020-05-27 09:13:17.541 UTC [orderer.consensus.etcdraft] poll -> INFO 020 1 received MsgVoteResp from 1 at term 2 channel=system-channel node=1
2020-05-27 09:13:17.541 UTC [orderer.consensus.etcdraft] becomeLeader -> INFO 021 1 became leader at term 2 channel=system-channel node=1
2020-05-27 09:13:17.541 UTC [orderer.consensus.etcdraft] run -> INFO 022 raft.node: 1 elected leader 1 at term 2 channel=system-channel node=1
2020-05-27 09:13:17.542 UTC [orderer.consensus.etcdraft] run -> INFO 023 Leader 1 is present, quit campaign channel=system-channel node=1
2020-05-27 09:13:17.543 UTC [orderer.consensus.etcdraft] run -> INFO 024 Raft leader changed: 0 -> 1 channel=system-channel node=1
2020-05-27 09:13:17.543 UTC [orderer.consensus.etcdraft] run -> INFO 025 Start accepting requests as Raft leader at block [0] channel=system-channel node=1
Peer 1 :
2020-05-27 09:13:16.435 UTC [msp] loadCertificateAt -> WARN 001 Failed loading ClientOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.435 UTC [msp] loadCertificateAt -> WARN 002 Failed loading PeerOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.435 UTC [msp] loadCertificateAt -> WARN 003 Failed loading AdminOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.436 UTC [msp] loadCertificateAt -> WARN 004 Failed loading OrdererOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org1.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.441 UTC [nodeCmd] serve -> INFO 005 Starting peer:
Version: 2.1.0
Commit SHA: 1bdf975
Go version: go1.14.1
OS/Arch: linux/amd64
Chaincode:
Base Docker Label: org.hyperledger.fabric
Docker Namespace: hyperledger
2020-05-27 09:13:16.442 UTC [peer] getLocalAddress -> INFO 006 Auto-detected peer address: 172.18.0.3:7051
2020-05-27 09:13:16.442 UTC [peer] getLocalAddress -> INFO 007 Returning peer0.org1.example.com:7051
2020-05-27 09:13:16.469 UTC [nodeCmd] initGrpcSemaphores -> INFO 008 concurrency limit for endorser service is 2500
2020-05-27 09:13:16.470 UTC [nodeCmd] initGrpcSemaphores -> INFO 009 concurrency limit for deliver service is 2500
2020-05-27 09:13:16.470 UTC [nodeCmd] serve -> INFO 00a Starting peer with TLS enabled
2020-05-27 09:13:16.500 UTC [ledgermgmt] NewLedgerMgr -> INFO 00b Initializing LedgerMgr
2020-05-27 09:13:16.513 UTC [leveldbhelper] openDBAndCheckFormat -> INFO 00c DB is empty Setting db format as 2.0
2020-05-27 09:13:16.514 UTC [fsblkstorage] NewProvider -> INFO 00d Creating new file ledger directory at /var/hyperledger/production/ledgersData/chains/chains
2020-05-27 09:13:16.521 UTC [leveldbhelper] openDBAndCheckFormat -> INFO 00e DB is empty Setting db format as 2.0
2020-05-27 09:13:16.535 UTC [leveldbhelper] openDBAndCheckFormat -> INFO 00f DB is empty Setting db format as 2.0
2020-05-27 09:13:16.536 UTC [ledgermgmt] NewLedgerMgr -> INFO 010 Initialized LedgerMgr
2020-05-27 09:13:16.547 UTC [gossip.service] New -> INFO 011 Initialize gossip with endpoint peer0.org1.example.com:7051
2020-05-27 09:13:16.549 UTC [gossip.gossip] New -> INFO 012 Creating gossip service with self membership of Endpoint: peer0.org1.example.com:7051, InternalEndpoint: peer0.org1.example.com:7051, PKI-ID: 58df3c0a908cbbd073a6b4138ef676c652aaab118fb99179d7304206f63a0207, Metadata:
2020-05-27 09:13:16.550 UTC [lifecycle] InitializeLocalChaincodes -> INFO 013 Initialized lifecycle cache with 0 already installed chaincodes
2020-05-27 09:13:16.550 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 014 Entering computeChaincodeEndpoint with peerHostname: peer0.org1.example.com
2020-05-27 09:13:16.550 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 015 Exit with ccEndpoint: peer0.org1.example.com:7052
2020-05-27 09:13:16.550 UTC [gossip.gossip] start -> INFO 016 Gossip instance peer0.org1.example.com:7051 started
2020-05-27 09:13:16.560 UTC [sccapi] DeploySysCC -> INFO 017 deploying system chaincode 'lscc'
2020-05-27 09:13:16.560 UTC [sccapi] DeploySysCC -> INFO 018 deploying system chaincode 'cscc'
2020-05-27 09:13:16.560 UTC [sccapi] DeploySysCC -> INFO 019 deploying system chaincode 'qscc'
2020-05-27 09:13:16.560 UTC [sccapi] DeploySysCC -> INFO 01a deploying system chaincode '_lifecycle'
2020-05-27 09:13:16.560 UTC [nodeCmd] serve -> INFO 01b Deployed system chaincodes
2020-05-27 09:13:16.560 UTC [discovery] NewService -> INFO 01c Created with config TLS: true, authCacheMaxSize: 1000, authCachePurgeRatio: 0.750000
2020-05-27 09:13:16.560 UTC [nodeCmd] registerDiscoveryService -> INFO 01d Discovery service activated
2020-05-27 09:13:16.560 UTC [nodeCmd] serve -> INFO 01e Starting peer with ID=[peer0.org1.example.com], network ID=[dev], address=[peer0.org1.example.com:7051]
2020-05-27 09:13:16.560 UTC [nodeCmd] serve -> INFO 01f Started peer with ID=[peer0.org1.example.com], network ID=[dev], address=[peer0.org1.example.com:7051]
2020-05-27 09:13:16.560 UTC [kvledger] LoadPreResetHeight -> INFO 020 Loading prereset height from path [/var/hyperledger/production/ledgersData/chains]
2020-05-27 09:13:16.561 UTC [fsblkstorage] preResetHtFiles -> INFO 021 No active channels passed
2020-05-27 09:13:16.561 UTC [nodeCmd] func6 -> INFO 022 Starting profiling server with listenAddress = 0.0.0.0:6060
Peer 2 :
2020-05-27 09:13:16.409 UTC [msp] loadCertificateAt -> WARN 001 Failed loading ClientOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.410 UTC [msp] loadCertificateAt -> WARN 002 Failed loading PeerOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.410 UTC [msp] loadCertificateAt -> WARN 003 Failed loading AdminOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.410 UTC [msp] loadCertificateAt -> WARN 004 Failed loading OrdererOU certificate at [/etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem]: [could not read file /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: open /etc/hyperledger/fabric/msp/cacerts\ca.org2.example.com-cert.pem: no such file or directory]
2020-05-27 09:13:16.420 UTC [nodeCmd] serve -> INFO 005 Starting peer:
Version: 2.1.0
Commit SHA: 1bdf975
Go version: go1.14.1
OS/Arch: linux/amd64
Chaincode:
Base Docker Label: org.hyperledger.fabric
Docker Namespace: hyperledger
2020-05-27 09:13:16.421 UTC [peer] getLocalAddress -> INFO 006 Auto-detected peer address: 172.18.0.2:9051
2020-05-27 09:13:16.421 UTC [peer] getLocalAddress -> INFO 007 Returning peer0.org2.example.com:9051
2020-05-27 09:13:16.433 UTC [nodeCmd] initGrpcSemaphores -> INFO 008 concurrency limit for endorser service is 2500
2020-05-27 09:13:16.434 UTC [nodeCmd] initGrpcSemaphores -> INFO 009 concurrency limit for deliver service is 2500
2020-05-27 09:13:16.434 UTC [nodeCmd] serve -> INFO 00a Starting peer with TLS enabled
2020-05-27 09:13:16.472 UTC [ledgermgmt] NewLedgerMgr -> INFO 00b Initializing LedgerMgr
2020-05-27 09:13:16.492 UTC [leveldbhelper] openDBAndCheckFormat -> INFO 00c DB is empty Setting db format as 2.0
2020-05-27 09:13:16.493 UTC [fsblkstorage] NewProvider -> INFO 00d Creating new file ledger directory at /var/hyperledger/production/ledgersData/chains/chains
2020-05-27 09:13:16.501 UTC [leveldbhelper] openDBAndCheckFormat -> INFO 00e DB is empty Setting db format as 2.0
2020-05-27 09:13:16.528 UTC [leveldbhelper] openDBAndCheckFormat -> INFO 00f DB is empty Setting db format as 2.0
2020-05-27 09:13:16.528 UTC [ledgermgmt] NewLedgerMgr -> INFO 010 Initialized LedgerMgr
2020-05-27 09:13:16.542 UTC [gossip.service] New -> INFO 011 Initialize gossip with endpoint peer0.org2.example.com:9051
2020-05-27 09:13:16.547 UTC [gossip.gossip] New -> INFO 012 Creating gossip service with self membership of Endpoint: peer0.org2.example.com:9051, InternalEndpoint: peer0.org2.example.com:9051, PKI-ID: c7429efa7a899a8b3644235bc56251ffbfb45fe3f55fc0a4d199fd03b1521df4, Metadata:
2020-05-27 09:13:16.547 UTC [lifecycle] InitializeLocalChaincodes -> INFO 013 Initialized lifecycle cache with 0 already installed chaincodes
2020-05-27 09:13:16.548 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 014 Entering computeChaincodeEndpoint with peerHostname: peer0.org2.example.com
2020-05-27 09:13:16.548 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 015 Exit with ccEndpoint: peer0.org2.example.com:9052
2020-05-27 09:13:16.549 UTC [gossip.gossip] start -> INFO 016 Gossip instance peer0.org2.example.com:9051 started
2020-05-27 09:13:16.555 UTC [sccapi] DeploySysCC -> INFO 017 deploying system chaincode 'lscc'
2020-05-27 09:13:16.558 UTC [sccapi] DeploySysCC -> INFO 018 deploying system chaincode 'cscc'
2020-05-27 09:13:16.558 UTC [sccapi] DeploySysCC -> INFO 019 deploying system chaincode 'qscc'
2020-05-27 09:13:16.559 UTC [sccapi] DeploySysCC -> INFO 01a deploying system chaincode '_lifecycle'
2020-05-27 09:13:16.559 UTC [nodeCmd] serve -> INFO 01b Deployed system chaincodes
2020-05-27 09:13:16.559 UTC [discovery] NewService -> INFO 01c Created with config TLS: true, authCacheMaxSize: 1000, authCachePurgeRatio: 0.750000
2020-05-27 09:13:16.559 UTC [nodeCmd] registerDiscoveryService -> INFO 01d Discovery service activated
2020-05-27 09:13:16.559 UTC [nodeCmd] serve -> INFO 01e Starting peer with ID=[peer0.org2.example.com], network ID=[dev], address=[peer0.org2.example.com:9051]
2020-05-27 09:13:16.559 UTC [nodeCmd] serve -> INFO 01f Started peer with ID=[peer0.org2.example.com], network ID=[dev], address=[peer0.org2.example.com:9051]
2020-05-27 09:13:16.559 UTC [kvledger] LoadPreResetHeight -> INFO 020 Loading prereset height from path [/var/hyperledger/production/ledgersData/chains]
2020-05-27 09:13:16.559 UTC [fsblkstorage] preResetHtFiles -> INFO 021 No active channels passed
2020-05-27 09:13:16.559 UTC [nodeCmd] func6 -> INFO 022 Starting profiling server with listenAddress = 0.0.0.0:6060
There are also several warnings but don't really know their meaning.
I've tried to relaunch docker as admin , also tried to relaunch the network.
I want to use chaincode-docker-devmode in fabric-samples to test my chaincode, but when I add couchdb in docker-compose-simple.yaml like this:
version: '2'
services:
orderer:
container_name: orderer
image: hyperledger/fabric-orderer
environment:
- FABRIC_LOGGING_SPEC=debug
- ORDERER_GENERAL_LISTENADDRESS=orderer
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_GENESISFILE=orderer.block
- ORDERER_GENERAL_LOCALMSPID=DEFAULT
- ORDERER_GENERAL_LOCALMSPDIR=/etc/hyperledger/msp
- GRPC_TRACE=all=true,
- GRPC_VERBOSITY=debug
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: orderer
volumes:
- ./msp:/etc/hyperledger/msp
- ./orderer.block:/etc/hyperledger/fabric/orderer.block
ports:
- 7050:7050
couchdb:
container_name: couchdb
image: hyperledger/fabric-couchdb
environment:
- COUCHDB_USER=
- COUCHDB_PASSWORD=
ports:
- 5984:5984
peer:
container_name: peer
image: hyperledger/fabric-peer
dns_search: .
environment:
- CORE_PEER_ID=peer
- CORE_PEER_ADDRESS=peer:7051
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer:7051
- CORE_PEER_LOCALMSPID=DEFAULT
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- FABRIC_LOGGING_SPEC=DEBUG
- CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/msp
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb:5984
- CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
- CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
volumes:
- /var/run/:/host/var/run/
- ./msp:/etc/hyperledger/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: peer node start --peer-chaincodedev=true
ports:
- 7051:7051
- 7053:7053
depends_on:
- orderer
- couchdb
cli:
container_name: cli
image: hyperledger/fabric-tools
tty: true
environment:
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- FABRIC_LOGGING_SPEC=DEBUG
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer:7051
- CORE_PEER_LOCALMSPID=DEFAULT
- CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/msp
working_dir: /opt/gopath/src/chaincodedev
command: /bin/bash -c './script.sh ; sleep 20'
volumes:
- /var/run/:/host/var/run/
- ./msp:/etc/hyperledger/msp
- ./../chaincode:/opt/gopath/src/chaincodedev/chaincode
- ./:/opt/gopath/src/chaincodedev/
depends_on:
- orderer
- peer
chaincode:
container_name: chaincode
image: hyperledger/fabric-ccenv
tty: true
environment:
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- FABRIC_LOGGING_SPEC=DEBUG
- CORE_PEER_ID=example02
- CORE_PEER_ADDRESS=peer:7051
- CORE_PEER_LOCALMSPID=DEFAULT
- CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/msp
working_dir: /opt/gopath/src/chaincode
command: /bin/bash -c 'sleep 6000000'
volumes:
- /var/run/:/host/var/run/
- ./msp:/etc/hyperledger/msp
- ./../chaincode:/opt/gopath/src/chaincode
depends_on:
- orderer
- peer
When I start the containers, using
docker-compose -f docker-compose-simple.yaml up
Container cil will return the error:
Error: error getting endorser client for channel: endorser client failed to connect to peer:7051: failed to create new connection: connection error: desc = "transport: error while dialing: dial tcp 172.29.0.4:7051: connect: connection refused"
some same errors are solve by add command:sleep in script,
command: /bin/bash -c 'sleep 6000000'
but my Container still return same error.
peer container's logs
2020-04-13 09:01:28.850 UTC [inproccontroller] getInstance -> DEBU 0f5 chaincode instance created for qscc-1.4.1
2020-04-13 09:01:28.850 UTC [inproccontroller] func2 -> DEBU 0f6 chaincode-support started for qscc-1.4.1
2020-04-13 09:01:28.850 UTC [inproccontroller] func1 -> DEBU 0f7 chaincode started for qscc-1.4.1
2020-04-13 09:01:28.851 UTC [chaincode] handleMessage -> DEBU 0f8 [] Fabric side handling ChaincodeMessage of type: REGISTER in state created
2020-04-13 09:01:28.851 UTC [chaincode] HandleRegister -> DEBU 0f9 Received REGISTER in state created
2020-04-13 09:01:28.851 UTC [chaincode] Register -> DEBU 0fa registered handler complete for chaincode qscc:1.4.1
2020-04-13 09:01:28.851 UTC [chaincode] HandleRegister -> DEBU 0fb Got REGISTER for chaincodeID = name:"qscc:1.4.1" , sending back REGISTERED
2020-04-13 09:01:28.851 UTC [chaincode] HandleRegister -> DEBU 0fc Changed state to established for name:"qscc:1.4.1"
2020-04-13 09:01:28.851 UTC [chaincode] sendReady -> DEBU 0fd sending READY for chaincode name:"qscc:1.4.1"
2020-04-13 09:01:28.851 UTC [chaincode] sendReady -> DEBU 0fe Changed to state ready for chaincode name:"qscc:1.4.1"
2020-04-13 09:01:28.851 UTC [chaincode] Launch -> DEBU 0ff launch complete
2020-04-13 09:01:28.851 UTC [chaincode] Execute -> DEBU 100 Entry
2020-04-13 09:01:28.851 UTC [qscc] Init -> INFO 101 Init QSCC
2020-04-13 09:01:28.851 UTC [chaincode] handleMessage -> DEBU 102 [91196774] Fabric side handling ChaincodeMessage of type: COMPLETED in state ready
2020-04-13 09:01:28.852 UTC [chaincode] Notify -> DEBU 103 [91196774] notifying Txid:91196774-4c1f-4369-b977-965de44d3d1b, channelID:
2020-04-13 09:01:28.852 UTC [chaincode] Execute -> DEBU 104 Exit
2020-04-13 09:01:28.852 UTC [sccapi] deploySysCC -> INFO 105 system chaincode qscc/(github.com/hyperledger/fabric/core/scc/qscc) deployed
2020-04-13 09:01:28.852 UTC [sccapi] deploySysCC -> INFO 106 system chaincode (+lifecycle,github.com/hyperledger/fabric/core/chaincode/lifecycle) disabled
2020-04-13 09:01:28.852 UTC [nodeCmd] serve -> INFO 107 Deployed system chaincodes
2020-04-13 09:01:28.855 UTC [ccprovider] ListInstalledChaincodes -> DEBU 108 Returning []
2020-04-13 09:01:28.850 UTC [container] unlockContainer -> DEBU 109 container lock deleted(qscc-1.4.1)
2020-04-13 09:01:28.870 UTC [container] lockContainer -> DEBU 10a waiting for container(qscc-1.4.1) lock
2020-04-13 09:01:28.870 UTC [container] lockContainer -> DEBU 10b got container (qscc-1.4.1) lock
2020-04-13 09:01:28.870 UTC [container] unlockContainer -> DEBU 10c container lock deleted(qscc-1.4.1)
2020-04-13 09:01:28.880 UTC [discovery] NewService -> INFO 10d Created with config TLS: false, authCacheMaxSize: 1000, authCachePurgeRatio: 0.750000
2020-04-13 09:01:28.880 UTC [nodeCmd] registerDiscoveryService -> INFO 10e Discovery service activated
2020-04-13 09:01:28.881 UTC [nodeCmd] serve -> INFO 10f Starting peer with ID=[name:"peer" ], network ID=[dev], address=[peer:7051]
2020-04-13 09:01:28.882 UTC [nodeCmd] serve -> INFO 110 Started peer with ID=[name:"peer" ], network ID=[dev], address=[peer:7051]
2020-04-13 09:01:32.819 UTC [msp] GetDefaultSigningIdentity -> DEBU 111 Obtaining default signing identity
2020-04-13 09:01:32.819 UTC [msp.identity] Sign -> DEBU 112 Sign: plaintext: 18012A86070A2D0A09706565723A3730...455254494649434154452D2D2D2D2D0A
2020-04-13 09:01:32.819 UTC [msp.identity] Sign -> DEBU 113 Sign: digest: E3138D4C867225B5BEF15EC07062AABC2AF920CCEF7ED0545D3BB223B79F06FF
2020-04-13 09:01:32.820 UTC [msp] GetDefaultSigningIdentity -> DEBU 114 Obtaining default signing identity
2020-04-13 09:01:32.821 UTC [msp.identity] Sign -> DEBU 115 Sign: plaintext: 0A09706565723A37303531
2020-04-13 09:01:32.821 UTC [msp.identity] Sign -> DEBU 116 Sign: digest: D18B553283AFC791A71A1AB3F379E85F4599DAAE3CE5B43F6A912E5BFD23E73D
2020-04-13 09:01:33.779 UTC [msp] GetDefaultSigningIdentity -> DEBU 117 Obtaining default signing identity
2020-04-13 09:01:33.779 UTC [msp.identity] Sign -> DEBU 118 Sign: plaintext: 18012A86070A2D0A09706565723A3730...455254494649434154452D2D2D2D2D0A
2020-04-13 09:01:33.780 UTC [msp.identity] Sign -> DEBU 119 Sign: digest: EA29BD51F7F58B547912B79712CC7AE73D9CBFA0FF80817409EA7BFB1F95BF5B
2020-04-13 09:01:33.780 UTC [msp] GetDefaultSigningIdentity -> DEBU 11a Obtaining default signing identity
2020-04-13 09:01:33.780 UTC [msp.identity] Sign -> DEBU 11b Sign: plaintext: 0A09706565723A37303531
2020-04-13 09:01:33.781 UTC [msp.identity] Sign -> DEBU 11c Sign: digest: D18B553283AFC791A71A1AB3F379E85F4599DAAE3CE5B43F6A912E5BFD23E73D
2020-04-13 09:01:33.782 UTC [gossip.discovery] periodicalSendAlive -> DEBU 11d Sleeping 5s
It's part of peer container's logs, it's still running and no error in it.
Is any one have any idea?
Thanks.
I created my network with a script like:
docker-compose -f $COMPOSE_FILE up -d $CA
docker-compose -f $COMPOSE_FILE up -d $ORDERER1 $PEER0 $PEER1
docker-compose -f $COMPOSE_FILE up -d $CLI
docker exec cli peer channel create -o orderer.example.com:7050 -c $CHANNEL_NAME -f /etc/hyperledger/config/channel.tx
The channel is created and if I enter the cli, inside the working directory, I can find with ls the new generated file beerchannel.block. In this directory I also have crypto, which contains genesis block and other config files, and crypto-config that contains msp and certificates.
At this point containers logs seems good.
Now I want to join peer0 to the channel with:
docker exec -e $ENV_ADDRESSP0 $CLI peer channel join -b $CHANNEL_NAME.block
As soon as I do this command, I cannot join peer0 to the channel.
The strange thing is that running the script return this message:
2019-11-22 10:04:00.868 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2019-11-22 10:04:00.922 UTC [channelCmd] executeJoin -> INFO 002 Successfully submitted proposal to join channel
So, everything seems fine.
But when I enter the logs of orderer, I get this message repeated:
2019-11-22 09:59:07.429 UTC [fsblkstorage] newBlockfileMgr -> INFO 009 Getting block information from block storage
2019-11-22 09:59:07.438 UTC [orderer.commmon.multichannel] newChain -> INFO 00a Created and starting new chain beerchannel
2019-11-22 09:59:07.440 UTC [comm.grpc.server] 1 -> INFO 00b streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Deliver grpc.peer_address=172.29.0.6:41778 grpc.code=OK grpc.call_duration=25.385144ms
2019-11-22 10:04:06.923 UTC [common.deliver] deliverBlocks -> WARN 00c [channel: beerchannel] Client authorization revoked for deliver request from 172.29.0.4:48406: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied: permission denied
2019-11-22 10:04:06.923 UTC [comm.grpc.server] 1 -> INFO 00d streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Deliver grpc.peer_address=172.29.0.4:48406 grpc.code=OK grpc.call_duration=1.001442ms
2019-11-22 10:04:07.026 UTC [common.deliver] deliverBlocks -> WARN 00e [channel: beerchannel] Client authorization revoked for deliver request from 172.29.0.4:48408: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied: permission denied
2019-11-22 10:04:07.026 UTC [comm.grpc.server] 1 -> INFO 00f streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Deliver grpc.peer_address=172.29.0.4:48408 grpc.code=OK grpc.call_duration=582.912µs
Since the peer involved is peer0, I also entered in peer0 logs and found this errors:
2019-11-22 10:04:00.870 UTC [endorser] callChaincode -> INFO 029 [][ec4f5097] Entry chaincode: name:"cscc"
2019-11-22 10:04:00.870 UTC [ledgermgmt] CreateLedger -> INFO 02a Creating ledger [beerchannel] with genesis block
2019-11-22 10:04:00.874 UTC [fsblkstorage] newBlockfileMgr -> INFO 02b Getting block information from block storage
2019-11-22 10:04:00.896 UTC [kvledger] CommitWithPvtData -> INFO 02c [beerchannel] Committed block [0] with 1 transaction(s) in 16ms (state_validation=0ms block_and_pvtdata_commit=10ms state_commit=2ms) commitHash=[]
2019-11-22 10:04:00.899 UTC [ledgermgmt] CreateLedger -> INFO 02d Created ledger [beerchannel] with genesis block
2019-11-22 10:04:00.902 UTC [gossip.gossip] JoinChan -> INFO 02e Joining gossip network of channel beerchannel with 1 organizations
2019-11-22 10:04:00.902 UTC [gossip.gossip] learnAnchorPeers -> INFO 02f No configured anchor peers of Org1MSP for channel beerchannel to learn about
2019-11-22 10:04:00.917 UTC [gossip.state] NewGossipStateProvider -> INFO 030 Updating metadata information, current ledger sequence is at = 0, next expected block is = 1
2019-11-22 10:04:00.919 UTC [sccapi] deploySysCC -> INFO 031 system chaincode lscc/beerchannel(github.com/hyperledger/fabric/core/scc/lscc) deployed
2019-11-22 10:04:00.919 UTC [cscc] Init -> INFO 032 Init CSCC
2019-11-22 10:04:00.920 UTC [sccapi] deploySysCC -> INFO 033 system chaincode cscc/beerchannel(github.com/hyperledger/fabric/core/scc/cscc) deployed
2019-11-22 10:04:00.920 UTC [qscc] Init -> INFO 034 Init QSCC
2019-11-22 10:04:00.920 UTC [sccapi] deploySysCC -> INFO 035 system chaincode qscc/beerchannel(github.com/hyperledger/fabric/core/scc/qscc) deployed
2019-11-22 10:04:00.920 UTC [sccapi] deploySysCC -> INFO 036 system chaincode (+lifecycle,github.com/hyperledger/fabric/core/chaincode/lifecycle) disabled
2019-11-22 10:04:00.921 UTC [endorser] callChaincode -> INFO 037 [][ec4f5097] Exit chaincode: name:"cscc" (51ms)
2019-11-22 10:04:00.921 UTC [comm.grpc.server] 1 -> INFO 038 unary call completed grpc.service=protos.Endorser grpc.method=ProcessProposal grpc.peer_address=172.29.0.6:42736 grpc.code=OK grpc.call_duration=51.473337ms
2019-11-22 10:04:06.919 UTC [gossip.election] beLeader -> INFO 039 42a5181dbddcff9d15ae32b05300e849fbcad1cf138e62f3d8b726d7b5db25d3 : Becoming a leader
2019-11-22 10:04:06.919 UTC [gossip.service] func1 -> INFO 03a Elected as a leader, starting delivery service for channel beerchannel
2019-11-22 10:04:06.923 UTC [blocksProvider] DeliverBlocks -> ERRO 03b [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:07.026 UTC [blocksProvider] DeliverBlocks -> ERRO 03c [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:07.239 UTC [blocksProvider] DeliverBlocks -> ERRO 03d [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:07.643 UTC [blocksProvider] DeliverBlocks -> ERRO 03e [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:08.445 UTC [blocksProvider] DeliverBlocks -> ERRO 03f [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:10.051 UTC [blocksProvider] DeliverBlocks -> ERRO 040 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:13.254 UTC [blocksProvider] DeliverBlocks -> ERRO 041 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:19.657 UTC [blocksProvider] DeliverBlocks -> ERRO 042 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:29.662 UTC [blocksProvider] DeliverBlocks -> ERRO 043 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:39.668 UTC [blocksProvider] DeliverBlocks -> ERRO 044 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:49.671 UTC [blocksProvider] DeliverBlocks -> ERRO 045 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:04:49.671 UTC [blocksProvider] DeliverBlocks -> ERRO 046 [beerchannel] Wrong statuses threshold passed, stopping block provider
2019-11-22 10:04:49.671 UTC [gossip.election] stopBeingLeader -> INFO 047 42a5181dbddcff9d15ae32b05300e849fbcad1cf138e62f3d8b726d7b5db25d3 Stopped being a leader
2019-11-22 10:04:49.671 UTC [gossip.service] func1 -> INFO 048 Renounced leadership, stopping delivery service for channel beerchannel
2019-11-22 10:05:56.924 UTC [gossip.election] beLeader -> INFO 049 42a5181dbddcff9d15ae32b05300e849fbcad1cf138e62f3d8b726d7b5db25d3 : Becoming a leader
2019-11-22 10:05:56.924 UTC [gossip.service] func1 -> INFO 04a Elected as a leader, starting delivery service for channel beerchannel
2019-11-22 10:05:56.929 UTC [blocksProvider] DeliverBlocks -> ERRO 04b [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:05:57.032 UTC [blocksProvider] DeliverBlocks -> ERRO 04c [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:05:57.235 UTC [blocksProvider] DeliverBlocks -> ERRO 04d [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:05:57.638 UTC [blocksProvider] DeliverBlocks -> ERRO 04e [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:05:58.441 UTC [blocksProvider] DeliverBlocks -> ERRO 04f [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:06:00.044 UTC [blocksProvider] DeliverBlocks -> ERRO 050 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:06:03.247 UTC [blocksProvider] DeliverBlocks -> ERRO 051 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:06:09.652 UTC [blocksProvider] DeliverBlocks -> ERRO 052 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:06:19.656 UTC [blocksProvider] DeliverBlocks -> ERRO 053 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:06:29.659 UTC [blocksProvider] DeliverBlocks -> ERRO 054 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:06:39.662 UTC [blocksProvider] DeliverBlocks -> ERRO 055 [beerchannel] Got error &{FORBIDDEN}
2019-11-22 10:06:39.662 UTC [blocksProvider] DeliverBlocks -> ERRO 056 [beerchannel] Wrong statuses threshold passed, stopping block provider
2019-11-22 10:06:39.662 UTC [gossip.election] stopBeingLeader -> INFO 057 42a5181dbddcff9d15ae32b05300e849fbcad1cf138e62f3d8b726d7b5db25d3 Stopped being a leader
2019-11-22 10:06:39.662 UTC [gossip.service] func1 -> INFO 058 Renounced leadership, stopping delivery service for channel beerchannel
It seems something related to permissions but I cannot understand what's wrong in here.
The cli contains the beerchannel.block file, successfully generated it and now I just want to add peer0 to the channel.
Adding configtx.yaml
Organizations:
- &OrdererOrg
Name: OrdererOrg
ID: OrdererMSP
MSPDir: crypto-config/ordererOrganizations/c.com/msp
Policies:
Readers:
Type: Signature
Rule: "OR('OrdererMSP.member')"
Writers:
Type: Signature
Rule: "OR('OrdererMSP.member')"
Admins:
Type: Signature
Rule: "OR('OrdererMSP.admin')"
- &s
Name: sMSP
ID: sMSP
MSPDir: crypto-config/peerOrganizations/s.c.com/msp
Policies:
Readers:
Type: Signature
Rule: "OR('sMSP.admin', 'sMSP.peer', 'sMSP.client')"
Writers:
Type: Signature
Rule: "OR('sMSP.admin', 'sMSP.client')"
Admins:
Type: Signature
Rule: "OR('sMSP.admin')"
AnchorPeers:
- Host: peer1.s.c.com
Port: 7051
- Host: peer2.s.c.com
Port: 8051
Capabilities:
Channel: &ChannelCapabilities
V1_4_3: true
V1_3: false
V1_1: false
Orderer: &OrdererCapabilities
V1_4_2: true
V1_1: false
Application: &ApplicationCapabilities
V1_4_2: true
V1_3: false
V1_2: false
V1_1: false
Application: &ApplicationDefaults
Organizations:
Policies:
Readers:
Type: ImplicitMeta
Rule: "ANY Readers"
Writers:
Type: ImplicitMeta
Rule: "ANY Writers"
Admins:
Type: ImplicitMeta
Rule: "MAJORITY Admins"
Capabilities:
<<: *ApplicationCapabilities
Orderer: &OrdererDefaults
OrdererType: solo
Addresses:
- orderer1.c.com:7050
BatchTimeout: 500ms
BatchSize:
MaxMessageCount: 15
AbsoluteMaxBytes: 99 MB
PreferredMaxBytes: 512 kb
Policies:
Readers:
Type: ImplicitMeta
Rule: "ANY Readers"
Writers:
Type: ImplicitMeta
Rule: "ANY Writers"
Admins:
Type: ImplicitMeta
Rule: "MAJORITY Admins"
BlockValidation:
Type: ImplicitMeta
Rule: "ANY Writers"
Capabilities:
<<: *OrdererCapabilities
Channel: &ChannelDefaults
Policies:
Readers:
Type: ImplicitMeta
Rule: "ANY Readers"
Writers:
Type: ImplicitMeta
Rule: "ANY Writers"
Admins:
Type: ImplicitMeta
Rule: "MAJORITY Admins"
Capabilities:
<<: *ChannelCapabilities
Profiles:
OneOrgOrdererGenesis:
<<: *ChannelDefaults
Orderer:
<<: *OrdererDefaults
Organizations:
- *OrdererOrg
Consortiums:
SampleConsortium:
Organizations:
- *s
OneOrgChannel:
<<: *ChannelDefaults
Consortium: SampleConsortium
Application:
<<: *ApplicationDefaults
Organizations:
- *s
SampleMultiNodeEtcdRaft:
<<: *ChannelDefaults
Capabilities:
<<: *ChannelCapabilities
Orderer:
<<: *OrdererDefaults
OrdererType: etcdraft
EtcdRaft:
Consenters:
- Host: orderer1.c.com
Port: 7050
ClientTLSCert: crypto-config/ordererOrganizations/c.com/orderers/orderer1.c.com/tls/server.crt
ServerTLSCert: crypto-config/ordererOrganizations/c.com/orderers/orderer1.c.com/tls/server.crt
- Host: orderer2.c.com
Port: 7050
ClientTLSCert: crypto-config/ordererOrganizations/c.com/orderers/orderer2.c.com/tls/server.crt
ServerTLSCert: crypto-config/ordererOrganizations/c.com/orderers/orderer2.c.com/tls/server.crt
- Host: orderer3.c.com
Port: 7050
ClientTLSCert: crypto-config/ordererOrganizations/c.com/orderers/orderer3.c.com/tls/server.crt
ServerTLSCert: crypto-config/ordererOrganizations/c.com/orderers/orderer3.c.com/tls/server.crt
Addresses:
- orderer1.c.com:7050
- orderer2.c.com:7050
- orderer3.c.com:7050
Organizations:
- *OrdererOrg
Capabilities:
<<: *OrdererCapabilities
Application:
<<: *ApplicationDefaults
Organizations:
- <<: *OrdererOrg
Consortiums:
SampleConsortium:
Organizations:
- *s
Check for the Reader policies that you have defined in your configtx.yaml this error is generated because of the policy mismatch. You have defined some specific user type(admin, peer, client) in your Reader policies but this specific user type is not passed into certificates that you have generated for your peer.
Edited:
If you want to make it generic and not specific to the identity type then you can edit the s org policies like this:
- &s
Name: sMSP
ID: sMSP
MSPDir: crypto-config/peerOrganizations/s.c.com/msp
Policies:
Readers:
Type: Signature
Rule: "OR('sMSP.member')"
Writers:
Type: Signature
Rule: "OR('sMSP.member')"
Admins:
Type: Signature
Rule: "OR('sMSP.admin')"
Check your crypto-config.yaml under peerOrgs section and add EnableNodeOUs property if missing then regenerate the crypto materials. Config example:
PeerOrgs:
- Name: Org1
Domain: org1.example.com
EnableNodeOUs: true