I have a nodejs REST api using fabric API. everything is working fine but after a given time (2days) we got an endless number of these errors. And the story repeats, BUT ONCE THE API IS RESTARTED THE ERROR DISAPEARS.
[ServiceEndpoint]: ServiceEndpoint grpcs://192.168.1.3:7051 reset connection failed :: Error: Failed to connect before the deadline on Discoverer- name: peer0.org1.example.com, url:grpcs://192.168.1.3:7051, connected:false, connectAttempted:true
[DiscoveryService]: send[mychannel] - no discovery results
[ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com, url:grpcs://192.168.1.3:7051, connected:false, connectAttempted:true
[ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.example.com url:grpcs://192.168.1.3:7051 timeout:3000
[ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com, url:grpcs://192.168.1.3:7051, connected:false, connectAttempted:true
[ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.example.com url:grpcs://192.168.1.3:7051 timeout:3000
[ServiceEndpoint]: Error: Failed to connect before the deadline on Discoverer- name: peer0.org1.example.com, url:grpcs://192.168.1.3:7051, connected:false, connectAttempted:true
[ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.example.com url:grpcs://192.168.1.3:7051 timeout:3000
[DiscoveryService]: send[mychannel] - no discovery results
192.168.1.3 is the ip of the remote peer the api is connected 2.
The api and HF peer are running in a separate VM and they are always connected. And using HF 2.2
Any leads?
I have inspected the peer and it seems to be working nomal
Related
I am trying to run the fabcar-UI code provided here using 2 org (1 peer each) network using docker swarm network on AWS. 2 Orderer services are running on peer0.org1. I am able to run peer chaincode invoke command successfully from the terminal. After integrating the fabcar-ui I am running into below error when registering the admin. Can someone please guide me?
ubuntu#ip-172-31-11-233:~/hlf-docker-swarm/fabcar-ui$ node app
fabcar-ui listening on port 3000
Wallet path: /home/ubuntu/hlf-docker-swarm/fabcar-ui/wallet
2022-03-12T05:49:09.886Z - error: [FabricCAClientService.js]: Failed to enroll admin, error:%o message=Calling enroll endpoint failed with error [Error: getaddrinfo ENOTFOUND ca.org1.example.com ca.org1.example.com:7054], stack=Error: Calling enroll endpoint failed with error [Error: getaddrinfo ENOTFOUND ca.org1.example.com ca.org1.example.com:7054]
at ClientRequest.request.on (/home/ubuntu/hlf-docker-swarm/fabcar-ui/node_modules/fabric-ca-client/lib/FabricCAClient.js:327:19)
at ClientRequest.emit (events.js:198:13)
at TLSSocket.socketErrorListener (_http_client.js:401:9)
at TLSSocket.emit (events.js:198:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at process._tickCallback (internal/process/next_tick.js:63:19), errno=ENOTFOUND, code=ENOTFOUND, syscall=getaddrinfo, hostname=ca.org1.example.com, host=ca.org1.example.com, port=7054
Failed to enroll admin user "admin": Error: Calling enroll endpoint failed with error [Error: getaddrinfo ENOTFOUND ca.org1.example.com ca.org1.example.com:7054]
Failed to enroll admin user "admin": Error: Calling enroll endpoint failed with error [Error: getaddrinfo ENOTFOUND ca.org1.example.com ca.org1.example.com:7054]
Since this is related to fabric CA, I am providing the docker-compose file for CA Org1.
services:
ca_org1:
deploy:
placement:
constraints:
- node.labels.name == manager
image: hyperledger/fabric-ca:latest
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca-org1
- FABRIC_CA_SERVER_TLS_ENABLED=true
- FABRIC_CA_SERVER_PORT=7054
ports:
- "7054:7054"
command: sh -c 'fabric-ca-server start -b admin:adminpw -d'
volumes:
- /home/ubuntu/hlf-docker-swarm/test-network/organizations/fabric-ca/org1:/etc/hyperledger/fabric-ca-server
container_name: ca_org1
networks:
test:
aliases:
- ca.org1.example.com
I have a problem with installing Hyperledger Fabric.
Error: failed to create deliver client for orderer: orderer client failed to connect to localhost:7050: failed to create new connection: connection error: desc = "transport: error while dialing: dial tcp 127.0.0.1:7050: connect: connection refused"
i also faced the similar issue and i resolved it by applying below changes in docker-compose yaml files
changed networks to this
networks:
fabric_test:
Replaced
1. image: hyperledger/fabric-orderer:latest
2. image: hyperledger/fabric-peer:latest
3. image: hyperledger/fabric-tools:latest
TO
image: hyperledger/fabric-orderer:2.2
image: hyperledger/fabric-peer:2.2
image: hyperledger/fabric-tools:2.2
Run ./network.sh down and then ./network.sh up createChannel -ca -s couchdb
I have a hyperledger-fabric project with 3 separate hosts and each host includes 1 Organization. Each Organization has 2 Peers, 2 Orderers, 1 CA Service(I don't have a separate CA for orderers and all the nodes in each organization get their certificates from the same CA service).
All the channels and chaincodes and everything else work correctly. I use an API service to communicate with the network, and I can add users with API correctly to the network. But the problem is:
When I'm trying to invoke the chaincode with API, e.g: adding an asset or querying the ledger (submitting and evaluating transactions), the asset will be added after a while(about 20 seconds), but I get the following error:
E1116 07:17:03.978929679 1 ssl_transport_security.cc:1245] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
2021-11-16T07:17:06.893Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Committer- name: orderer0.org2.example.com:9050, url:grpcs://orderer0.org2.example.com:9050
2021-11-16T07:17:06.894Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server orderer0.org2.example.com:9050 url:grpcs://orderer0.org2.example.com:9050 timeout:3000
2021-11-16T07:17:06.894Z - error: [DiscoveryService]: _buildOrderer[mychannel] - Unable to connect to the discovered orderer orderer0.org2.example.com:9050 due to Error: Failed to connect before the deadline on Committer- name: orderer0.org2.example.com:9050, url:grpcs://orderer0.org2.example.com:9050
E1116 07:17:06.909267625 1 ssl_transport_security.cc:1245] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
2021-11-16T07:17:09.897Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Committer- name: orderer1.org2.example.com:10050, url:grpcs://orderer1.org2.example.com:10050
2021-11-16T07:17:09.898Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server orderer1.org2.example.com:10050 url:grpcs://orderer1.org2.example.com:10050 timeout:3000
2021-11-16T07:17:09.898Z - error: [DiscoveryService]: _buildOrderer[mychannel] - Unable to connect to the discovered orderer orderer1.org2.example.com:10050 due to Error: Failed to connect before the deadline on Committer- name: orderer1.org2.example.com:10050, url:grpcs://orderer1.org2.example.com:10050
E1116 07:17:09.928123699 1 ssl_transport_security.cc:1245] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
2021-11-16T07:17:12.901Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Committer- name: orderer0.org3.example.com:11050, url:grpcs://orderer0.org3.example.com:11050
2021-11-16T07:17:12.902Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server orderer0.org3.example.com:11050 url:grpcs://orderer0.org3.example.com:11050 timeout:3000
2021-11-16T07:17:12.902Z - error: [DiscoveryService]: _buildOrderer[mychannel] - Unable to connect to the discovered orderer orderer0.org3.example.com:11050 due to Error: Failed to connect before the deadline on Committer- name: orderer0.org3.example.com:11050, url:grpcs://orderer0.org3.example.com:11050
E1116 07:17:12.952061671 1 ssl_transport_security.cc:1245] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
2021-11-16T07:17:15.948Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Committer- name: orderer0.org3.example.com:11050, url:grpcs://orderer0.org3.example.com:11050
2021-11-16T07:17:15.949Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server orderer0.org3.example.com:11050 url:grpcs://orderer0.org3.example.com:11050 timeout:3000
2021-11-16T07:17:15.949Z - error: [DiscoveryService]: _buildOrderer[mychannel] - Unable to connect to the discovered orderer orderer0.org3.example.com:11050 due to Error: Failed to connect before the deadline on Committer- name: orderer0.org3.example.com:11050, url:grpcs://orderer0.org3.example.com:11050
E1116 07:17:16.120702832 1 ssl_transport_security.cc:1245] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
Thanks in advance for your help
I am trying to get a multimachine network of Hyperledger Fabric running. I encountered some errors. I was able to reproduce the same errors on a single machine in the Fabcar example of Fabric v2.1 by changing one line in fabric-samples/fabcar/javascript/query.js. I changed the line
await gateway.connect(ccp, { wallet, identity: 'appUser', discovery: { enabled: true, asLocalhost: true } });
to the line
await gateway.connect(ccp, { wallet, identity: 'appUser', discovery: { enabled: true, asLocalhost: false } });
So I am setting discovery.asLocalhost to false instead of true. When I run node query.js in the fabric-samples/fabcar/javascript/ directory. I get the following errors.
Wallet path: /home/userName/my/code/fabric-samples/fabcar/javascript/wallet
2020-10-23T06:09:56.505Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Committer- name: orderer.example.com:7050, url:grpcs://orderer.example.com:7050
2020-10-23T06:09:56.507Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server orderer.example.com:7050 url:grpcs://orderer.example.com:7050 timeout:3000
2020-10-23T06:09:56.508Z - error: [DiscoveryService]: _buildOrderer[mychannel] - Unable to connect to the discovered orderer orderer.example.com:7050 due to Error: Failed to connect before the deadline on Committer- name: orderer.example.com:7050, url:grpcs://orderer.example.com:7050
2020-10-23T06:09:59.522Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com:7051, url:grpcs://peer0.org1.example.com:7051
2020-10-23T06:09:59.523Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.example.com:7051 url:grpcs://peer0.org1.example.com:7051 timeout:3000
2020-10-23T06:09:59.523Z - error: [DiscoveryService]: _buildPeer[mychannel] - Unable to connect to the discovered peer peer0.org1.example.com:7051 due to Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com:7051, url:grpcs://peer0.org1.example.com:7051
2020-10-23T06:10:02.528Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.org2.example.com:9051, url:grpcs://peer0.org2.example.com:9051
2020-10-23T06:10:02.528Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org2.example.com:9051 url:grpcs://peer0.org2.example.com:9051 timeout:3000
2020-10-23T06:10:02.529Z - error: [DiscoveryService]: _buildPeer[mychannel] - Unable to connect to the discovered peer peer0.org2.example.com:9051 due to Error: Failed to connect before the deadline on Endorser- name: peer0.org2.example.com:9051, url:grpcs://peer0.org2.example.com:9051
2020-10-23T06:10:02.564Z - error: [SingleQueryHandler]: evaluate: message=Query failed. Errors: [], stack=FabricError: Query failed. Errors: []
at SingleQueryHandler.evaluate (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/node_modules/fabric-network/lib/impl/query/singlequeryhandler.js:45:23)
at Transaction.evaluate (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/node_modules/fabric-network/lib/transaction.js:287:49)
at Contract.evaluateTransaction (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/node_modules/fabric-network/lib/contract.js:115:45)
at main (/home/sarva/my/code/viacomrepos/temp-fabric-samples/fabric-samples/fabcar/javascript/query.js:46:39)
at processTicksAndRejections (internal/process/task_queues.js:85:5), name=FabricError
Failed to evaluate transaction: FabricError: Query failed. Errors: []
I followed the instructions in the Fabcar tutorial. Here are the steps to reproduce the error.
cd fabric-samples/fabcar
./startFabric.sh javascript
cd javascript
npm install
node enrollAdmin.js
node registerUser.js
[Change the line in query.js]
node query.js
If I can figure out how/where to specify the grpcs URLs, I think I can get my multimachine network to work. Any help is appreciated.
More detailed error logs can be obtained by setting export GRPC_TRACE=all and export GRPC_VERBOSITY=DEBUG as suggested by this answer.
In the single machine case your client application is running on your local machine but the nodes (peers and orderers) are running in Docker containers on your local machine, and have hostnames like peer0.org1.example.com.
Within the Docker network the nodes can talk to each other using their hostnames, e.g. peer0.org1.example.com. Your client (outside the Docker network) can't talk to peer0.org1.example.com because that DNS name does not exist. Instead it needs to connect to localhost on ports mapped (by Docker) to/from specific hosts/ports within the Docker network.
Your client application gets node endpoints in two ways:
Entries in its local connection profile.
Network topology returned by service discovery.
You can set appropriate (localhost) endpoint URLs in your connection profile. However, the endpoint URLs returned by discovery will be the ones exposed within the Docker network, e.g. peer0.org1.example.com. To facilitate this scenario, the SDK provides the discovery.asLocalhost setting which, when enabled, maps all endpoint addresses returned by discovery to the same port at localhost. An alternative approach is to add entries to your local hosts file that map the node names to localhost.
You should not have the discovery.asLocalhost setting enabled if your nodes are accessible on the real network using their configured hostnames. These hostnames must be resolvable in DNS (so for a real deployment you can't use non-resolvable addresses like example.com), and must match your Fabric network configuration (and server certificate details if using TLS).
For setting up the multiMachine network. The changes you have to make is on the config.json and connection.yaml ( basically connection profiles).
await gateway.connect(ccp, { wallet, identity: 'appUser', discovery: { enabled: true, asLocalhost: false } });
The localhost Property should be set as "false" as you are going to setup a multi-host Blockchain Network. You can create an overlay network to connect to it.
---
name: mychannel.firstnetwork.connectionprofile
x-type: "hlfv1"
description: "connection profile for 2.1 network"
version: "1.0"
channels:
mychannel:
orderers:
- orderer.example.com
peers:
peer0.org1.example.com:
endorsingPeer: true
chaincodeQuery: true
ledgerQuery: true
eventSource: true
peer0.org2.example.com:
endorsingPeer: true
chaincodeQuery: true
ledgerQuery: true
eventSource: true
organizations:
Org1:
mspid: Org1MSP
peers:
- peer0.org1.example.com
certificateAuthorities:
- ca.org1.example.com
adminPrivateKey:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/users/Admin#org1.example.com/msp/keystore/8717c2e82c3c4caff76fca964bb70_sk
signedCert:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/users/Admin#org1.example.com/msp/signcerts/cert.pem
Org2:
mspid: Org2MSP
peers:
- peer0.org2.example.com
certificateAuthorities:
- ca.org2.example.com
adminPrivateKey:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/users/Admin#org2.example.com/msp/keystore/9592495c719c2e87fc4a8_sk
signedCert:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/users/Admin#org2.example.com/msp/signcerts/cert.pem
orderers:
orderer.example.com:
url: grpcs://orderer.example.com:7050
grpcOptions:
ssl-target-name-override: orderer.example.com
tlsCACerts:
path: ./caro-blockchain-commonfiles/config/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
peers:
peer0.org1.example.com:
url: grpcs://peer0.org1.example.com:7051
grpcOptions:
ssl-target-name-override: peer0.org1.example.com
tlsCACerts:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/msp/cacerts/localhost-7054-ca-org1.pem
peer0.org2.example.com:
url: grpcs://peer0.org2.example.com:9051
grpcOptions:
ssl-target-name-override: peer0.org2.example.com
tlsCACerts:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/msp/cacerts/localhost-8054-ca-org2.pem
certificateAuthorities:
ca.org1.example.com:
url: https://ca.org1.example.com:7054
httpOptions:
verify: false
tlsCACerts:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org1.example.com/ca/ca.org1.example.com-cert.pem
registrar:
- enrollId: admin
enrollSecret: adminpw
caName: ca-org1
ca.org2.example.com:
url: https://ca.org2.example.com:8054
httpOptions:
verify: false
tlsCACerts:
path: ./caro-blockchain-commonfiles/config/crypto-config/peerOrganizations/org2.example.com/ca/ca.org2.example.com-cert.pem
registrar:
- enrollId: admin
enrollSecret: adminpw
caName: ca-org2
The URLs property here like grpcs://peer0.org1.example.com:7051 should be changed from hostname to the IP of the other machine to VM ( grpcs://XX.XX.XX.XX:7051)where peer container is running.
Two months ago, I've written an application using Hyperledger Fabric. Now, I try to run it again, but I get error and I can't resolve it. So I go the the Hyperledger Fabric doc again(https://hyperledger-fabric.readthedocs.io/en/latest/install.html), and reinstall everything(including cmd tools, docker images and even my MacOS).
I've tried two versions(v2.0.1 and v2.1.0), and my node version is v12. What I do is:
download the fabric-samples repo using cmd (curl -sSL https://xxx/2ysbOFE | bash -s -- 2.1.0 1.4.6 0.4.18)
and go the fabcar/ directory and run ./startFabric.sh
and go the fabrcar/javascript/ directory and run npm install && node enrollAdmin.js && node registerUser.js && node invoke.js
However, I meet this error again:
2020-05-07T06:39:28.354Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com, url:grpcs://localhost:7051
2020-05-07T06:39:28.356Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.example.com url:grpcs://localhost:7051 timeout:3000
2020-05-07T06:39:28.360Z - error: [NetworkConfig]: buildPeer - Unable to connect to the endorser peer0.org1.example.com due to Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com, url:grpcs://localhost:7051
at checkState (/Users/xiebei/code/fabric-samples/fabcar/javascript/node_modules/grpc/src/client.js:883:16) {
connectFailed: true
}
2020-05-07T06:39:31.363Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Discoverer- name: peer0.org1.example.com, url:grpcs://localhost:7051
2020-05-07T06:39:31.363Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.example.com url:grpcs://localhost:7051 timeout:3000
Failed to evaluate transaction: Error: Failed to connect before the deadline on Discoverer- name: peer0.org1.example.com, url:grpcs://localhost:7051
I use docker ps:
enter image description here
Seems everything goes ok. And the orderer node docker logs is :
2020-05-07 06:12:07.554 UTC [orderer.consensus.etcdraft] propose -> INFO 059 Created block [5], there are 0 blocks in flight channel=mychannel node=1
2020-05-07 06:12:07.558 UTC [orderer.consensus.etcdraft] writeBlock -> INFO 05a Writing block [5] (Raft index: 7) to ledger channel=mychannel node=1
2020-05-07 06:12:07.788 UTC [orderer.common.broadcast] Handle -> WARN 05b Error reading from 172.18.0.1:36216: rpc error: code = Canceled desc = context canceled
2020-05-07 06:12:07.789 UTC [comm.grpc.server] 1 -> INFO 05c streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Broadcast grpc.peer_address=172.18.0.1:36216 error="rpc error: code = Canceled desc = context canceled" grpc.code=Canceled grpc.call_duration=2.257323471s
2020-05-07 06:12:14.117 UTC [orderer.common.broadcast] Handle -> WARN 05d Error reading from 172.18.0.1:36256: rpc error: code = Canceled desc = context canceled
2020-05-07 06:12:14.117 UTC [comm.grpc.server] 1 -> INFO 05e streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Broadcast grpc.peer_address=172.18.0.1:36256 error="rpc error: code = Canceled desc = context canceled" grpc.code=Canceled grpc.call_duration=61.170986ms
2020-05-07 06:12:16.114 UTC [orderer.consensus.etcdraft] propose -> INFO 05f Created block [6], there are 0 blocks in flight channel=mychannel node=1
2020-05-07 06:12:16.117 UTC [orderer.consensus.etcdraft] writeBlock -> INFO 060 Writing block [6] (Raft index: 8) to ledger channel=mychannel node=1
It seems like that the orderer can't connect to something(maybe the peer node). So I think there's some possible problems:
Maybe the peer node's network is failed, we can't connect to it.
Maybe the NodeJS Fabric SDK is buggy, it's GRPC client can't works well.
Maybe my computer goes wrong.
Could you please help me? Thanks!!!