Hyperledger Fabric CA: http: TLS handshake error from 127.0.0.1:53356: tls: oversized record received with length 21536 - hyperledger-fabric

we have a very simple setup where we have spin up an instance of fabric-ca-server by running
/bin/bash -c "fabric-ca-server start -b admin:adminpw"
in a docker container as explained e.g., here. but when we try to enroll a user by running
root#a7094a09a3b7:/etc/hyperledger/fabric-ca-client# fabric-ca-client enroll -u http://user:userpw#localhost:7054
we get below error:
2018/12/07 21:06:17 [INFO] generating key: &{A:ecdsa S:256}
2018/12/07 21:06:17 [INFO] encoded CSR
Error: POST failure of request: POST http://localhost:7054/enroll
{"hosts":["a7094a09a3b7"],"certificate_request":"-----BEGIN CERTIFICATE REQUEST-----\nMIIBQzCB6wIBADBfMQswCQYDVQQGEwJVUzEXMBUGA1UECBMOTm9ydGggQ2Fyb2xp\nbmExFDASBgNVBAoTC0h5cGVybGVkZ2VyMQ8wDQYDVQQLEwZGYWJyaWMxEDAOBgNV\nBAMTB3NqYWluNjgwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAASiWBUZ5vG5i9X9\nAV8fdfuxFAFsWHeg4JRpNFs/m7knB3QaDWDbbk/G53gEbgyv2scg61cYMNdLDaYK\nGi9ywX0noCowKAYJKoZIhvcNAQkOMRswGTAXBgNVHREEEDAOggxhNzA5NGEwOWEz\nYjcwCgYIKoZIzj0EAwIDRwAwRAIgTY0gn18UIsKfOLgS33F41ORAlKNz2tZS6ip5\nXS03XNICIAE3C9nhv2d4ZrCcO+7rycWLoR6wEU6cnAqZsSf1Txz4\n-----END CERTIFICATE REQUEST-----\n","profile":"","crl_override":"","label":"","NotBefore":"0001-01-01T00:00:00Z","NotAfter":"0001-01-01T00:00:00Z","CAName":""}: Post http://localhost:7054/enroll: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02\x16"
and the fabric-ca-server output shows:
http: TLS handshake error from 127.0.0.1:53356: tls: oversized record received with length 21536
wondering what are we doing wrong here and how can we fix this?

Looks like you started the fabric-ca-server with TLS enabled and you are not using TLS with the fabric-ca-client. You need to specify https in your URL:
fabric-ca-client enroll -u https://user:userpw#localhost:7054
Alternatively, disable TLS in the Fabric CA config.

Related

Debugging TLS handshake failure

I'm trying to access my peer through the fabric-network nodejs sdk.
However, I encounter an error during the gateway.connect in the sdk and the logs I find in the peer container is not helpful.
All I have, even with the grpc=debug logging mode is :
peer0.catie-test | 2020-09-21 13:27:07.731 UTC [core.comm] ServerHandshake -> ERRO 087 TLS handshake failed with error remote error: tls: handshake failure server=PeerServer remoteaddress=172.17.0.1:49918
peer0.catie-test | 2020-09-21 13:27:07.731 UTC [grpc] handleRawConn -> DEBU 088 grpc: Server.Serve failed to complete security handshake from "172.17.0.1:49918": remote error: tls: handshake failure
Any way to have more helpful logs ? I would like to know, for example, which key are used for the TLS handshake check.
Edit with more infos: Configuration files and TLS verification
My peer is configured with TLS with the env variables :
CORE_PEER_TLS_ENABLED=true
CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/crypto/peer/tls-msp/keystore/key.pem
CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/crypto/peer/tls-msp/signcerts/cert.pem
CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/crypto/peer/tls-msp/tlscacerts/tlsca.catie-test-cert.pem
I have the correct tlscacert of my peer on the client side, because the output from the peer and from client side are the same:
cat /etc/hyperledger/crypto/peer/tls-msp/tlscacerts/tlsca.catie-test-cert.pem # From the peer, output ZTd/o8LLw== at the end
cat /tmp/fabric-start-catie-test/building/artifacts/peer0.catie-test-crypto/tls-msp/tlscacerts/tlsca.catie-test-cert.pem # From the client, output ZTd/o8LLw== at the end
Path to the peer tlscacert is filled in the client side connection-profile.json :
"peers": {
"peer0.catie-test": {
"tlsCACerts": {
"path": "/tmp/fabric-start-catie-test/building/artifacts/peer0.catie-test-crypto/tls-msp/tlscacerts/tlsca.catie-test-cert.pem"
},
"grpcOptions":{
"ssl-target-name-override": "172.17.0.7",
"grpc.keepalive_time_ms": 10000
},
"url": "grpcs://172.17.0.4:7051",
"eventUrl": "grpcs://172.17.0.4:7053"
}
}
And I also checked that the tlsCAcert is the one that generated my peer cert :
openssl verify -CAfile $CORE_PEER_TLS_ROOTCERT_FILE $CORE_PEER_TLS_CERT_FILE # Output : /etc/hyperledger/crypto/peer/tls-msp/signcerts/cert.pem: OK
Edit 2 : Grpc option, peer name instead of IP and client logs
Also tried adding the grpcOptions to the peer section of the connection-profile.json (see the updated paragraph above) but it didn't change anything.
Also tried to add peer name to my /etc/hosts to reach my peer via its name instead of its IP. It makes a warning disappear but don't solve my problem and I prefer to work with IPs in my scripts.
Here are the logs of the nodejs sdk client in case it helps to diagnose the problem, but it only says that the Endorser must be connected and I think it is, because it reaches my peer as I have this TLS error in my peer's logs.
(node:59350) [DEP0123] DeprecationWarning: Setting the TLS ServerName to an IP address is not permitted by RFC 6066. This will be ignored in a future version.
2020-09-23T06:42:20.704Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Endorser- name: peer0.catie-test, url:grpcs://172.17.0.7:7051, connected:false, connectAttempted:true
2020-09-23T06:42:20.705Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.catie-test url:grpcs://172.17.0.7:7051 timeout:3000
2020-09-23T06:42:20.708Z - error: [NetworkConfig]: buildPeer - Unable to connect to the endorser peer0.catie-test due to Error: Failed to connect before the deadline on Endorser- name: peer0.catie-test, url:grpcs://172.17.0.7:7051, connected:false, connectAttempted:true
at checkState (/home/rqueraud/CATIE/Myrmica/myrmica-start/node_modules/#grpc/grpc-js/build/src/client.js:69:26)
at Timeout._onTimeout (/home/rqueraud/CATIE/Myrmica/myrmica-start/node_modules/#grpc/grpc-js/build/src/channel.js:292:17)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7) {
connectFailed: true
}
(node:59350) UnhandledPromiseRejectionWarning: Error: Endorser must be connected
at Channel.addEndorser (/home/rqueraud/CATIE/Myrmica/myrmica-start/node_modules/fabric-common/lib/Channel.js:259:10)
at buildChannel (/home/rqueraud/CATIE/Myrmica/myrmica-start/node_modules/fabric-network/lib/impl/ccp/networkconfig.js:50:21)
at Object.loadFromConfig (/home/rqueraud/CATIE/Myrmica/myrmica-start/node_modules/fabric-network/lib/impl/ccp/networkconfig.js:34:19)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async Gateway.connect (/home/rqueraud/CATIE/Myrmica/myrmica-start/node_modules/fabric-network/lib/gateway.js:279:13)
at async queryChaincode (/home/rqueraud/CATIE/Myrmica/myrmica-start/test/chaincode-sdk/index.js:41:5)
at async /home/rqueraud/CATIE/Myrmica/myrmica-start/test/chaincode-sdk/index.js:57:5
Edit 3 : Docker IPs ? Trying with EC2 instances.
As #Urko mentionned, my nodes are in fact docker container running docker-in-docker (dind) images. Inside these containers are some other containers running the hyperledger peers, cli, ... images.
I access them from the host which is also where I run the fabric sdk nodejs client. I cannot access them via their container name, I think it is only possible in a docker-compose configuration, isn't it ? I already tried (see Edit 2 above) to add their name to my /etc/hosts to reach them via a name instead of an IP but it didn't change anything.
However, as my network startup is scripted, I deployed it using docker-machine in AWS instead of the dind docker containers this time, so these are real instances reachable on the internet. But I still encounter the same errors, here is the log from the peer where you can see this is coming from a public IP :
2020-09-24 08:32:57.653 UTC [core.comm] ServerHandshake -> ERRO 0d7 TLS handshake failed with error remote error: tls: handshake failure server=PeerServer remoteaddress=31.36.26.4:35462
It seems that the connection with your Peer have been defined to be secured by the TLS protocol. So, you may configure you Peer configuration to know wich certificates are you using at the TLS.
As when you connect to any server using this protocol, the communication among the parties is encripted using the certificate of the server (in this case, the Peer will be the server). So, you need to configure your client to trust on the server by the Root CA that haven been used to issue the Peers TLS certificates.
The client is where you use the SDK, so, you should configure it to trust on the Peer TLS certificate. When you configure the connection to the Blockchain nodes (peers and orderers), you would define their direction, as well as their TLS certificate. This one is an example that you can find at the following link. There, you have to define the value of the tlsCACerts param:
orderers:
orderer.example.com:
url: grpcs://localhost:7050
grpcOptions:
ssl-target-name-override: orderer.example.com
grpc-max-send-message-length: 4194304
tlsCACerts:
path: test/fixtures/channel/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tlscacerts/example.com-cert.pem
peers:
peer0.org1.example.com:
url: grpcs://localhost:7051
grpcOptions:
ssl-target-name-override: peer0.org1.example.com
grpc.keepalive_time_ms: 600000
tlsCACerts:
path: test/fixtures/channel/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tlscacerts/org1.example.com-cert.pem
----- Edited ----
Also, you have to check the value of the ssl-target-name-override param. It should be the same to you nodes name, as you can see in the example file
----- Edited ----
Why are you using those IPs?! I understand that those IPs are internal from the Docker network, so you should not use them. Could you try using your containers names instead of the docker networks IPs?
----- Edited ----
Could you verify your ca-server configuration file and check that the tls is set to true?
You try a gRPC call in peer Server where peer server is secured with it's TLS system. So if you fail to provide the valid TLS certificate, server tls handshake will be failed and you will not get success to establish the connection.
Please check that your network config file is properly develop, also check that you are using the same TLS certificate that is used to run the peer server and your TLS certificate path is correct.

TLS handshake failed with error remote error: tls: bad certificate server=Orderer

I am trying to set up a hyperledger fabric on a VM manually. I have generated all the artifacts and configured the orderer.yaml and core.yaml. I have orderer running on port 127.0.0.1:7050. When I try to create channel using the peer cli channel create command I am getting a context deadline exceeded message on peer terminal.
./bin/peer channel create -o 127.0.0.1:7050 -c $CHANNEL_NAME -f ./channel-artifacts/channel.tx --tls --cafile /home/fabric-release/mynetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
Error: failed to create deliver client: orderer client failed to connect to 127.0.0.1:7050: failed to create new connection: context deadline exceeded
On the orderer terminal I am getting the following error:
2019-04-23 09:22:03.707 EDT [core.comm] ServerHandshake -> ERRO 01b TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=127.0.0.1:38618
2019-04-23 09:22:04.699 EDT [core.comm] ServerHandshake -> ERRO 01c TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=127.0.0.1:38620
2019-04-23 09:22:06.187 EDT [core.comm] ServerHandshake -> ERRO 01d TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=127.0.0.1:38622
I have gone through the configurations a few time, I am not sure if I am missing something. Following is my orderer.yaml
General:
LedgerType: file
ListenAddress: 127.0.0.1
ListenPort: 7050
TLS:
Enabled: true
PrivateKey: /home/fabric-release/mynetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/server.key
Certificate: /home/fabric-release/mynetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/server.crt
RootCAs:
- /home/fabric-release/mynetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt
ClientAuthRequired: true
Keepalive:
ServerMinInterval: 60s
ServerInterval: 7200s
ServerTimeout: 20s
GenesisMethod: file
GenesisProfile: OneOrgOrdererGenesis
GenesisFile: channel-artifacts/genesis.block
LocalMSPDIR: /home/fabric-release/mynetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp
LocalMSPID: OrdererMSP
Authentication:
TimeWindow: 15m
FileLedger:
Location: /var/hyperledger/production/orderer
Prefix: hyperledger-fabric-ordererledger
The issue is that the TLS server certificate used by the orderer does not have a SAN matching "127.0.0.1". You can add "localhost" and/or "127.0.0.1" to you TLS certificates by using a custom crypto-config.yaml when generating your artifacts with cryptogen:
# ---------------------------------------------------------------------------
# "OrdererOrgs" - Definition of organizations managing orderer nodes
# ---------------------------------------------------------------------------
OrdererOrgs:
# ---------------------------------------------------------------------------
# Orderer
# ---------------------------------------------------------------------------
- Name: Orderer
Domain: example.com
EnableNodeOUs: false
# ---------------------------------------------------------------------------
# "Specs" - See PeerOrgs below for complete description
# ---------------------------------------------------------------------------
Specs:
- Hostname: orderer
SANS:
- "localhost"
- "127.0.0.1"
# ---------------------------------------------------------------------------
# "PeerOrgs" - Definition of organizations managing peer nodes
# ---------------------------------------------------------------------------
PeerOrgs:
# ---------------------------------------------------------------------------
# Org1
# ---------------------------------------------------------------------------
- Name: org1
Domain: org1.example.com
EnableNodeOUs: true
Template:
Count: 2
SANS:
- "localhost"
- "127.0.0.1"
Users:
Count: 1
- Name: org2
Domain: org2.example.com
EnableNodeOUs: false
Template:
Count: 2
SANS:
- "localhost"
- "127.0.0.1"
Users:
Count: 1
I also faced the same problem and in my case, the issue was that I made some changes to the local directory files and apparently those changes were not successfully reflected while mounting those files back into the docker containers. What fixed the problem for me was
docker volume rm $(docker volume ls)
I restarted the network again and didn't see any more certificate errors. Worth a try.
when the problem of TLS handshake failed occurs between the orderer and orderer, it is most likely that there is an error in the configuration parameters when generating the TLS file.
if you are registered with TLS via fabric-ca, then you need to check whether the CSR properties in the TLS files of the two orderer are the same. You can use the following command "openssl x509 -in certificate.crt -text -noout".
The following you need to check whether the --cer.names, -m and other parameters of the orderer enroll are duplicate or incorrect.
In cases where the contents of the TLS file are consistent and the HostName specified, it is rare for the handshake to fail

Error while submitting transactions in Hyperledger Fabric

I am running Hyperledger Fabric with 4 peers of 1 organization, 1 orderer and 1 CA. All 4 peers are on different VMs, orderer and CA are running on different VMs. Chaincode is up and running on all the VMs. I want to setup client on a different VM, which can send transaction requests to the network. Using this code, I have changed the address of VM to my peer0.
I run the following 2 files first:
node enrollAdmin.js
node registerUser.js
I am getting the following error on running the last command:
Store path:/root/gopath/src/github.com/hyperledger/fabric-samples/fabcar/hfc-key-store
Successfully loaded admin from persistence
Failed to register: Error: fabric-ca request register failed with errors [[{"code":20,"message":"Authentication failure"}]]
I checked the logs of CA container on the . Container log is as follows:
2019/04/16 17:34:55 [DEBUG] Received request for /api/v1/register
2019/04/16 17:34:55 [DEBUG] Caller is using a x509 certificate
2019/04/16 17:34:55 [DEBUG] Failed to verify token based on new authentication header requirements: %!s(<nil>)
2019/04/16 17:34:55 [INFO] 192.168.1.22:44826 POST /api/v1/register 401 26 "Untrusted certificate: Failed to verify certificate: x509:
certificate signed by unknown authority (possibly because of
"x509: ECDSA verification failure" while trying to verify candidate authority certificate "ca.org1.example.com")"
I have copied the same generated crypto material on all the VMs, including the client. How to resolve this error?
UPDATE: When I place the client code on one of the VMs running peer containers, it works fine. Transactions are executed successfully.

TLS error enrolling with Fabric CA Client

I prepared multi-organization with Fabric CA environment by following tutorial. This tutorial itself was worked with no problem.
Deploying a Hyperledger Composer blockchain business network to Hyperledger Fabric (multiple
organizations)
https://hyperledger.github.io/composer/unstable/tutorials/deploy-to-fabric-multi-org.html
In this environment, I'm trying Fabric CA client commands on both ca_peerOrg1 and host environment. I copied tls crt, public key and private key from crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/ and tried following command, but I got error "x509: certificate is valid for ca.org1.example.com, not localhost".
linux1#fabric:~$ fabric-ca-client enroll --caname ca-org1 --tls.certfiles ca.crt --tls.client.certfile Admin#org1.example.com-cert.pem --tls.client.keyfile 0dbfd9d156b90331e6bc4a0529e07cb123f7de586935019499623062f182a8fa_sk -u https://admin:adminpw#localhost:7054
2017/12/19 16:34:35 [INFO] User provided config file: /home/linux1/fabric-ca/clients/admin/fabric-ca-client-config.yaml
2017/12/19 16:34:35 [INFO] generating key: &{A:ecdsa S:256}
2017/12/19 16:34:35 [INFO] encoded CSR
2017/12/19 16:34:35 [INFO] TLS Enabled
Error: POST failure [Post https://localhost:7054/enroll: x509: certificate is valid for ca.org1.example.com, not localhost]; not sending
POST https://localhost:7054/enroll
Authorization: Basic YWRtaW46YWRtaW5wdw==
{"hosts":["fabric"],"certificate_request":"-----BEGIN CERTIFICATE REQUEST-----\nMIIBOzCB4wIBADBdMQswCQYDVQQGEwJVUzEXMBUGA1UECBMOTm9ydGggQ2Fyb2xp\nbmExFDASBgNVBAoTC0h5cGVybGVkZ2VyMQ8wDQYDVQQLEwZGYWJyaWMxDjAMBgNV\nBAMTBWFkbWluMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEI8Y4aR/YxeyMXIQ3\nHGydrTLaoI+6tZ57ho1uv6gNonbFfftE7lrdsIbL8d8QLimztDjtqkg6rk612nso\nD66M9aAkMCIGCSqGSIb3DQEJDjEVMBMwEQYDVR0RBAowCIIGZmFicmljMAoGCCqG\nSM49BAMCA0cAMEQCIH9u2w9xO43Ba5rJ1SCKgtEFfokj1DKhg2cHjnbu8ucEAiBn\nwZKjjSbQage+ElofYbgagWpuCFz0t57Zwycz22M00w==\n-----END CERTIFICATE REQUEST-----\n","profile":"","crl_override":"","label":"","CAName":"ca-org1"}
linux1#fabric:~$
How can I avoid this error ? When I'm looking at the tutorial, I realized that composer connection profile has configuration like "hostnameOverride": "ca.org1.example.com". I guess I need to do same kind for fabric-ca-client.

Hyperledger Fabric Client Registration failed

I am new to Hyperledger Fabric and i am trying to register the fabric client with the server locally. here is what i have done so far.
go get -u github.com/hyperledger/fabric-ca/cmd/...
fabric-ca-server start -b admin:adminpw
after that i got the logs
2017/12/11 13:26:09 [INFO] Home directory for default CA:
/home/akash/fabric-demo 2017/12/11 13:26:09 [INFO] Listening on
http://0.0.0.0:7054
then i tried to register the fabric client with server by using the following commands
export FABRIC_CA_CLIENT_HOME=$HOME/fabric-ca/clients/admin
fabric-ca-client enroll -u http://admin:adminpw#localhost:7054
and I got the following logs
2017/12/11 13:28:48 [INFO] Stored client certificate at
/home/akash/fabric-ca/clients/admin/msp/signcerts/cert.pem 2017/12/11
13:28:48 [INFO] Stored CA root certificate at
/home/akash/fabric-ca/clients/admin/msp/cacerts/localhost-7054.pem
but when i tried to register the client then i got the following error
fabric-ca-client register --id.name admin --id.affiliation org1.department1 --id.attrs 'hf.Revoker=true,admin=true:ecert'
2017/12/11 13:37:05 [INFO] Configuration file location:
/home/akash/fabric-ca/clients/admin/fabric-ca-client-config.yaml
Error: Error response from server was: Authorization failure
on the fabric-server side i got the following logs
2017/12/11 13:37:05 [ERROR] No certificates found for provided serial and aki
Can anyone tell what i am doing wrong here.
The sequence to obtain new user certificates is to 1) register and then 2) enroll.
For the bootstrap admin (enrollment id:admin) this is already registered when the CA server is started (thus providing the secret when starting the instance). As such, you are able to run the enroll command for admin.
Not too sure what you are trying to do here, but if you are:
1) trying to register a new user. You have to use a new unique enrollment id (not admin)
2) updating the affiliation of the admin enrollment id, refer to the fabric-ca-client command to update an identity (https://hyperledger-fabric-ca.readthedocs.io/en/latest/users-guide.html)

Resources