I'm currently working on a POC using hyperledger fabric + composer. I am somewhat confused when it gets to the consensus mechanism between orderers. From what i understand reading the documentation, hyperledger currently only supports SOLO and KAFKA. My understanding of SOLO is that the network only exists out of a single ordering node and no network consensus is reached between orderers (only 1 exists).
But this doesn't make sense to me; my gut tells me it must be possible to add multiple orderers without the use of KAFA / Zookeeper and that hyperledger has another mechanism to reach consensus between these two. The documentation regarding this is somewhat spotty and all over the place if i try to google it so i'm hoping someone can shed some light on the matter.
If no consensus mechanism exists between orderes besides a centralised one then what is the point of a distributed ledger platform?
The primary usage of SOLO ordering is for development mode, where you would like to test functionality without a need to span complex distributed consensus solutions. Moreover, note that consensus is a pluggable mechanism and could be replaced with anything you'd like, for example there is a recent proposal to use SmartBFT as an additional ordering service. Very similar to this you can add consensus mechanism of your own.
Currently if you need to distributed and highly available solutions for consensus there is a KAFKA based ordering service.
The Raft-based consensus protocol was added in Hyperledger Fabric v1.4.1, which simplifies deployment and adds decentralization to OSNs. It removes the additional dependency of Kafka/ZooKeeper needed to run a fault-tolerant network.
Related
What are the advantages of hyperledger-fabric compared to a private Ethereum network? According to what I've read, Ethereum seems to be more like a general-purpose blockchain network whereas hyperledger-fabric can be more business-oriented but so far the features offering that advantage are not very clear to me.
Fabric is a Hyperledger project. It is an enterprise-grade distributed ledger that allows the development blockchain solutions
with a modular architecture. It has a permissioned architecture,
support modularization, pluggable consensus, and supports smart
contracts(aka ‘Chaincode’ in case of Hyperledger Fabric).
Various functions in the fabric are pluggable, and it also allows the use of any language to develop smart contracts. This functionality
is possible because it is based on container technology (Docker),
which can host any language.
As of now smart-contract in the blockchain run on an order-execute environment where the consensus protocol validate and order
transactions and broadcasts it to all the peer nodes in the network,
each peer node then sequentially executes the transaction. Most
blockchains today run on this order-execute blockchain environment.
The issue in this order-execute environment is that the smart contract
running here needs to be deterministic otherwise a consensus might
never be reached. To solve this issue, smart contracts have been
written in non-standard languages like Solidity with which the
non-deterministic operations are eliminated. However, this hinders
blockchain large scale adoption as smart contracts need to be written
in an altogether new language and it may lead to programming errors.
Moreover, as all transactions are only executed sequentially,
performance and scale are limited.
As Fabric eliminates the non-determinism issue, it has become the
first blockchain to enable the use of standard programming languages
to write smart contracts. In the 1.1.0 release, the smart contracts
will be written in either Node.js or Go; in the subsequent releases it
is envisioned that the platform will support other popular programming
languages like Java.
As widely known, the public blockchain network runs on Proof-of-Work concept and the transactions are executed on every node. Every
transaction and code that is implemented is visible to every node in
the network. This means neither the smart contract stays confidential
nor the transaction data which is processed. This means we have
exchanged the privacy of the contract and data for the public
consensus concept of PoW.
This lack of privacy is problematic for businesses. For example, in a supply chain business, some partners might be offered additional
discounts as a means of solidifying business. If everyone in the
network is able to see the contract and transaction, it won’t be
possible to do business – everyone will ask for the discounted rates!
Hyperledger Fabric as a permissioned blockchain offers privacy and confidentiality through its channel architecture. With Fabric, one can
establish a “channel” between the subset of participants who should be
granted the right to access a set of transactions. Like a network
overlay, only the nodes that participate in a channel have the access
to the chaincode (smart contract) and transaction data. This helps to
preserve confidentiality and privacy and builds trust in the network.
To enhance the privacy and confidentiality aspect, Fabric is offering
support to keep the data private and is developing zero knowledge
proofs (ZKP).
I have a problem with understanding why Hyperledger Fabric (HLF) uses blockchain structure.
I know that in Bitcoin blockchain structure ensures big security due to PoW algorithm and longest chain rule, but what are advantages of using a blockchain structure in HLF?
It seems to me that in Hyperledger Fabric, instead of the blockchain structure, there could be one transaction history log and network could work in the same way - I bet I'm wrong, but I haven't been able to find an explanation yet.
I would be grateful for the clarification of this issue.
I think a lot of questions you have in your mind comes from the overlapped definitions of DLT and blockchain.
DLT:
A DLT is simply a decentralized database that is managed by various participants. There is no central authority that acts as an arbitrator or monitor. As a distributed log of records, there is greater transparency – making fraud and manipulation more difficult – and it is more complicated to hack the system.
All of this could well be familiar because it’s written about the features of blockchain as well.
Blockchain:
Blockchain is nothing else but a DLT with a specific set of features. It is also a shared database – a log of records – but in this case shared by means of blocks that, as the name indicates, form a chain. The blocks are closed by a type of cryptographic signature called a ‘hash’; the next block begins with that same ‘hash’, a kind of wax seal. That is how it is verified that the encrypted information has not been manipulated and that it can’t be manipulated.
DLT platforms that are not blockchain provide immutability too, but it's just the way Hyperledger Fabric provides this characteristic which makes it a blockchain framework.
Every blockchain framework, be it the Ethereum, Bitcoin, etc all store the transaction information in blocks, where each block is linked to its predecessor by a hash and provides immutability.
Corda is very much similar to Hyperledger Fabric, but it is said to be both a blockchain and not a blockchain. Architecturally, it's very much similar to Hyperleder Fabric, but with only a key difference which makes Hyperledger Fabric a blockchain framework, and Corda a DLT.
I'll try to answer your question by emphasizing on the point that why Corda is not a blockchain.
Why is Corda a blockchain, and not a blockchain?
A Transaction in Corda is cryptographically linked (chained) to the transactions it depends on. Just like Bitcoin, but the range of concepts that can be expressed is far wider.
Transactions in Corda are processed by having each participant in the transaction execute the same code deterministically to verify the proposed updates to the ledger. Just like Ethereum, but the languages you can use are high-level and productive, like Java, rather than obscure ones like Solidity.
Transactions in Corda are shared only with those who have a need to know. Just like channels in Fabric but designed in from day one and fully integrated into the programming model.
Transactions in Corda are confirmed through a process of consensus forming using one of a range of algorithms, including Byzantine Fault Tolerant algorithms. Just like any other blockchain, but with the unique features that a Corda network can support multiple different consensus pools using different algorithms.
So, for all intents and purposes, Corda is a blockchain. And yet… there’s also an utterly critical difference.
Unlike the platforms mentioned above, Corda does not periodically batch up transactions needing confirmation — into a block — and confirm them in one go. Instead, Corda confirms each transaction in real-time. No need to wait for a bunch of other transactions to come along. No need to wait for a “block interval”. Each transaction is confirmed as we go.
Now coming onto your question why Hyperledger Fabric (HLF) uses blockchain structure? The answer is simply because they chose to.
References:
https://www.bbva.com/en/difference-dlt-blockchain/
https://cointelegraph.com/news/what-is-the-difference-between-blockchain-and-dlt
https://www.corda.net/blog/corda-top-ten-facts-7-both-a-blockchain-and-not-a-blockchain/
To keep the record immutable, Hyperledger Fabric uses blockchain structure. So by using Hyperledger Fabric, you can get an immutable record of all the transactions which is tough to temper with fraudulent activities.
Suppose you buy an valuable asset and you are the current owner of that asset. Now it is very hard for others to temper that records or change your ownership without your permission as Hyperledger Fabric uses blockchain structure to keep the record immutable.
I don't know if this question make sense, I know Raft is consensus algorithm and use etcd to distributed the data, and i know etcd in Raft Ordering Service have a similar job with zookeeper in Kafka Ordering Service, but what I don't understand is, what kind of consensus used in Kafka ordering service?
Right now ordering service can use Raft or Kafka (deprecated), but Raft is a consensus algorithm yet Kafka is not. Or actually both of them just part of the consensus ordering phase? then does that mean now Fabric uses consensus algorithm to be part of consensus??? then what kind of consensus used in Fabric? I've read somewhere Fabric is not PBFT yet.
Let's talk about it as ordering and consensus and bring in Kafka and Raft.
In a distributed system, where messages are going to multiple nodes, the said nodes need a way to know which message came first, which was second, etc. Think of it as transactions on your bank account. If you have $20 in your account and someone pays you $30 so your account goes to $50, and you pay me $50 and your account goes to $0, its a valid sequence. But if your bank messes the order and you start with $20 and the transfer to me for $50 comes next, that check is going to bounce.
So that sequence (also known as order) is important, and in Fabric this is done by The Order Node.
For redundancy, to mitigate malicious intent, for decentralization and other reasons, you may not want just one node providing order. But, if you have n ordering nodes, how do you make sure they come up with one order of messages and not n variations of that order? You get a consensus among those nodes on the order of those messages.
As one of the responders posted - you can achieve that consensus with RAFT or Kafka. Both are Crash Fault Tolerant (CFT) consensus algorithms, which means theoretically as long as majority of the ordering nodes are good, (2 out of 3, or 3 out of 5, etc) you are in good shape.
You are correct and RAFT does use etcd, but I think that's an implementation detail and not tied to the consensus conceptually. Etcd is an open source key-value store used to hold and manage information that distributed systems need to keep running. Its used by RAFT in Fabric, but it's also used by other projects like I think kubernetes uses it to manage all the configuration and metadata, etc
I am not aware of a Byzantine Fault tolerant library (where 2/3rd or fewer ordering nodes can be faulty I think and the system would still function) being available for Hyperledger Fabric yet, although there have been and continue to be discussions on it and the Fabric documentation states that RAFT CFT is a stepping stone to a BFT consensus library for Fabric in the future.
I would also reiterate reviewing the link to The Ordering Service Docs that was posted by another poster as good material to review for more information.
I also really like this introduction to RAFT video, it's not related to Fabric, but does an excellent job of explaining RAFT in general, if you are interested.
In its entirety, a consensus in the blockchain is a mechanism that ensures all copies of a distributed ledger are the same.
Hyperledger Fabric achieves consensus by relying on a backend service (known as the ordering service) that intermediates the messages between senders and receivers. This backend service will ensure that all receivers will see messages in the same order – it follows that if all receivers see messages in the same order(prior to version 1.4, used Kafka, and later RAFT), they will perform the same actions/commits, etc. and the consensus is achieved.
Hyperledger Fabric uses Crash Fault Tolerance(CFT) to achieve consensus for single as well as multiple org systems. Crash Fault Tolerant model guaranties to withstand system failures, such as crashes, network partitioning. Having N nodes in your consensus system CFT capable to withstand up to N/2 such crashes.
For more information, you can read this article which does a good job on explaining consensus in Hyperledger Fabric.
I am not an expert on the subject , but I will try to respond to your questions.
Apache ZooKeeper (used in Kafka) , does not use a consensus algorithm , it is a centralized service that save configuration and expose endpoints (https://zookeeper.apache.org/) , so Zookeeper is used as a central communication point and it use Zab to propagate state update. If you want more info , go here : https://kafka.apache.org/intro
Now Fabric use etcd to maintain the state of the world state , etcd use Raft wich is Leader/Follower type consensus algorithm.
So Raft is the consensus used in HyperLedger Fabric as 2.x , but as it is a Leader/Follower type algorithm , it is not Byzantine Fault Tolerant (at is core , modification can be made to make it PBFT).
I recommend you read the Hyperledger documentation which is very complete , and probably explain better than me: https://hyperledger-fabric.readthedocs.io/en/release-2.2/orderer/ordering_service.html
Also , the RAFT documentation if you want to understand how the algorithm work : https://raft.github.io/
I am little confused bw fabric and sawtooth for supply chain application development from documentation it appears that sawtooth is best for supply chain but all the validator node keeps the copy of the distributed ledger there is no concept of channels and private data.
Sawtooth has an open source solution for supply chain.
Fabric Channels may solve your use case. It doesn't scale if you need several channels. You can also have multiple blockchains with Sawtooth, which is essentially what a channel is anyway.
The blockchain concept is that all data is transaparent and viewable and auditable by everyone. That doesn't always work in some use cases and is an active research area. Encrypting payload data and storing some data off-chain are some solutions.
Both, Hyperledger Fabric and BichainDB offer the possibility to have a private, permissioned blockchain database. With their concepts they try to address the main disadvantages of public blockchains like lack of privacy and lack of performance (low throughput etc).
What are the main differences between the two technologies?
If you try out example application of both frameworks, you will quickly notice that BigchainDB is easier to start with. Hyperledger Fabric involves a lot of more knowledge to master it.
Fabric knows different kinds of nodes (peers, peers additionally being endorser nodes, orderer) and thereby allows a very flexible setup, depending on the consortium design and organisations themselves. BigchainDB has one kind of node that can be deployed. Also every involved organisation gets one node of course.
Fabric has richer capabilities to model assets and all kinds of transactions. One transaction kind can always be implemented by a custom processor function doing whatever is needed to query or modify the state of the ledger. BigchainDB only knows CREATE and TRANSFER transactions on every defined asset. You can create something (that may be also divisable, e.g. amounts of tokens) and can transfer them completely or partly.
Both seem to have pretty low level APIs. Fabric has more APIs and config models that need to be mastered. But Fabric is being complemented by frameworks such as Composer (with all its nice libraries involved, like playground and rest-server) that really improve the programming model. As far as I know there is nothing like that for BigchainDB, also because it is pretty simple from the start.
Consensus: BigchainDB uses Tendermint which is Byzantine Fault Tolerant. Fabric does PBFT which is Practically Byzantine Fault Tolerance, based on the idea of Miguel Castro.
In general I would say that Fabric is intended to be used for complex business use cases. BigchainDB is simpler and nice for assets that can be divided (financal stuff like coins/tokens maybe).