we have done a migration of a Fabric network, from one tenant to another Azure Cloud. The nodes are on three IaaS and we have done a snpashot of them, the network did not stop . So, as a result when we use this snpashots on the new tenant, it looks like the peers have more blocks than the orderers. Orderers have RAFT consensus without problems and update their blocks. But they can't get the latest blocks from the peers.
I can run chaincode, and read values, but cannot insert new transanctions.
There is this error in the peers :
"2023-01-17 13:20:56.933 UTC [peer.blocksprovider] func1 -> WARN 269 Encountered an error reading from deliver stream: EOF channel=test orderer-address=orderer5.example.com:7050
2023-01-17 13:20:56.933 UTC [peer.blocksprovider] DeliverBlocks -> WARN 26a Got error while attempting to receive blocks: received bad status NOT_FOUND from orderer channel=test"
.
and on the orderer: " ERRO 05b [channel: test] Error reading from channel, cause was: NOT_FOUND".
It seems that the peer to request blocks the orderer does not know the channel, but when it starts it detects it and inserts the blocks it had pending from other orderers.
Can the orderers read the missing blocks from the peers, or the only way to start a network from snpashots from different IaaS is to stop the network first?
Related
Here is the log of Hyperledger Fabric test network. There are 2 peer node(peer0.org1.example.com & peer0.org2.example.com) and 2 orderer node(orderer.example.com & orderer2.example.com).
My question is what 「writeBlock」,「StoreBlock」，「Validate」「commit」stage actually does? I know that at 「propose」stage, a chosen orderer(in this case, orderer.example.com) will create a block and then send the created block to other orderer such as orderer2.example.com. However, I don't understand what orderers does at「writeBlock」stage. Do they sign the created block at 「writeBlock」stage? Next, peer nodes received the block from who? orderer.example.com or orderer2.example.com? In my guess, I think it's from orderer.example.com but I'm not sure.
I don't understand what orderers does at「writeBlock」stage.
I'm not sure about those stages (writeBlock looks self-explanatory). I don't know about "StoreBlock", it isn't mentioned in the official docs. I'm wondering which Fabric version you are using.
Next, peer nodes received the block from who?
In a RAFT network you have a leader/follower model. You can think of a leader peer as an entry point/doorman for all communication coming from the ordering nodes. An ordering node sends blocks to the leading peer(s) of each organization; the leading peer is then responsible for distributing those blocks to other peers belonging to the same organization. Distribution is done using gossip protocol.
Hyperledger Fabric is a complex system, with many components and a lot of moving parts.
The log that you are referring to is the log of the orderer node - orderer.example.com. It's a part of the Fabric test network. You can see what's happening in detail by looking at the logs of other Fabric components, such as peer node logs.
The Fabric network has a set of orderer nodes, which are responsible for ordering the transactions and creating blocks in the blockchain.
The orderer nodes are configured to use a consensus algorithm (such as Solo or Kafka) to achieve a consensus about the order of the transactions.
A Fabric network has a set of peer nodes, which are responsible for executing the chaincode (smart contracts) and catching and validating the transactions.
The Fabric test network consists of one orderer node, which is configured to use Solo consensus algorithm. This orderer node is orderer.example.com.
The Fabric test network consists of two peer nodes, which are peer0.org1.example.com and peer0.org2.example.com.
The log that you are referring to is the log of the orderer node - ord
I am testing the Kafka mode in Fabric Network, after I create a channel
named 'mychannel', I joined two ORG in it. After these actions, I tried to use 'down' and 'up' command to refresh the kafka、zookeeper and the orderer's containers.In this way, I want to test if a peer can persistent 'channels' in different fabric network.
When tailing the logs of Orderer, I found the problem :
[common.deliver] deliverBlocks -> DEBU dc9 Rejecting deliver for 192.168.11.61:60156 because channel mychannel not found
and I used the kafka shell tool to check topic list, and find the 'mychannel' topic disappeared.
After doing above, I create a new channel 'mychannel' which used the same 'channel.tx' . And I found the error in log:
UTC [common.deliver] deliverBlocks -> ERRO b1b [channel: mychannel] Error reading from channel, cause was: NOT_FOUND
I used command:
peer channel getinfo -c mychannel
in one org's peer , and get info, the block height is 16:
Blockchain info: {"height":16,"currentBlockHash":"gHOfUnVRT0pGMRssz8fUXWH4jdH/1hcPUPLBqau7L9c=","previousBlockHash":"yvKUrJDg3+60Sbc0HHKs+N5vVkW2WBJWhy9TLFGmMug="}
I guess the orderer genesis block's height is 0, and can't match the current block height.
How can I fix this problem? Can I use channel update method to update the channel config?
kafka mode: 4 kafka brokers, 3 zookeepers
1 orderer
2 orgs
restart orderer and kafka cluster(cmd: 'docker-compose down & up')
It appears that you were not using externally mounted volumes with your Kafka, Zookeeper and Orderer containers. When you run docker-compose down it actually destroys the containers. If you want to start/stop the containers, you need to use docker-compose stop and docker-compose start.
If you want to preserve data in the event that the containers are destroyed (or even to upgrade them), then you need to attach external volumes to your containers.
I'm trying to setup hyperledger fabric 1.2 on different hosts.
But when creating channel via: ./peer.sh channel create -o orderer0.trade.com:7050 -c tradechannel -f ../tradechannel.tx --cafile tlsca.trade.com-cert.pem -t 150s.
I got this error in CLI: got unexpected status: SERVICE_UNAVAILABLE -- will not enqueue, consenter for this channel hasn't started yet
and here is log from orderer:
[channel: tradechannel] Rejecting broadcast of message from 192.168.167.149:60655 with SERVICE_UNAVAILABLE: rejected by Consenter: will not enqueue, consenter for this channel hasn't started yet.
Closing Broadcast stream
transport: http2Server.HandleStreams failed to read frame: read tcp 192.168.171.33:7050->192.168.167.149:60655: read: connection reset by peer
It seems I have problem with grpc but totally have no idea about this.
the CLI is in Macbook and Orderer runs in RedHat.
I also faced the same issue when I was spawning the hyperledger fabric on kubernetes using kafka as ordering service.
The error is coming because kafka need some to sync with zookeepers. So you can wait for some time after the blockchain network creation. Then create the channel. In my case I wait for 10 minutes and I am able to create a channel.
But when I am spawning the fabric network using docker and kafka as ordering service this error never occurs and in this case kafka synchronise with zookeepers very fast, I don't why no error "rejected by Consenter: will not enqueueenter image description here".
While adding a new organization to the hyper ledger fabric network, the peer logs shows the below error.
[gossip/service] updateAnchors -> ERRO 327 Tried joining channel
mychannel but our org( org4), isn't among the orgs of the channel:
[org1 org2 org3] , aborting.
And a warning like below.
[gossip/gossip] Gossip -> WARN 3d9 Failed obtaining gossipChannel of
[99 104 45 99 111 109 109 111 110] aborting.
Hyper ledger Fabric version : v1.1.0
Dynamic leader election properties added into peer-base.
I was following the instructions here.
Because the peer is being joined with the genesis block (which knows about the original members of the channel, but not the new members), these log statements are emitted.
Usually, these statements will be benign; so long as the peer is able to retrieve and commit blocks, you may ignore these messages. Eventually, the peer will receive the config block which adds its org, and gossip networking will begin working properly again.
This is a known issue, and in the future, we hope to allow peers to be joined to a channel using the latest configuration block, and not only the genesis block.
I built Fabric network using Kafka-based orderer service.
(2 peer, 1 orderer and 4 kafka)
When I invoke a chaincode, below message appears in peer's log.
[gossip/state] queueNewMessage -> WARN 7a24 Failed adding payload: Ledger height is at 5685, cannot enqueue block with sequence of 938
What is this message?
Can I ignore this warning?
This is probably because the second peer's height is much less than the first peer's height, and it (the second peer) pulls from the orderer blocks that the first peer doesn't need, and forwards to it as a token of good will.
The peer logs that he got a block he doesn't need.