We are currently trying to deploy a POC Fabric network with 5 orgs participating across two channels, with private data on each channel, using Raft. This mirrors the HLF sample network for 4 orgs outlined in the docs.
It has been suggested on RocketChat and elsewhere that we may have a system resource issue however there is no, as in ZERO, info on minimum system requirements for HLF installs -- so we are asking for input based on the scenario / issues below. We are running on current macOS. Docker is dedicated to only HLF test cases so there are no other containers running -- ever.
We have run through all the tutorials successfully with a couple of exceptions:
-- When we spin up BYFN per HLF tutorial steps, while console (terminal) output indicates success for START and END, we see via Kitematic that two entities -- sometimes orderers, sometimes peers, sometimes one of each -- are stopped. These containers exhibit no errors. Containers for other running entities may have errors listed, but eventually recover.
-- Commercial Paper seems to run as per the docs. We will be testing again FabCar today.
So my ask: what are the minimum and suggested system configurations for a viable HLF POC test case, memory, available disk, etc? We are more than confident in all prerequisite software installed.
One final note -- we do not believe this is a sys sizing issue but wanted to confirm as HLF's dev team has confirmed to us that there were in-fact issues with 1.4.x and 2.0 builds mixed on github which appears may have been resolved.
We want to take system config off the table so we can move forward with actual HLF implementation issues.
Thanks for any suggestions in advance -- reviewing AWS, et al guides do not apply as we are running locally in a private network.
Related
As of now I have used fabric-samples repo and used network.sh to start network . They already have connection-org.yaml file which has necessary information.
When I need to use fabric for my app , I know I need to start fabric network right ? Then I need to also create channel and user into it . How do I do it ? Should I just copy and paste that network.sh from fabric-samples ? What about connection-org.yaml? I think all of them is hardcoded right ? What should I do about it ?
Every tutorial has prebuilt these things never explaining what they are. Any help would be heavily appreciated
As you have mentioned that you have used Fabric Repo, I am expecting you to to be familiar with the Hyperledger Fabric Blockchain Framework.
Following factors related to the network should be decided first.
Channel name.
How many and Which are the Organizations are participating in the Consortium,
How many peers per Organization?
Ordering Network would RAFT based, But how many orderer nodes ?
Whether state database would use Couch DB or LevelDB
How the MSP Crypto would be generated ( Is Fabric CA going to be used[ If yes, then own root certificate/rootCA ?] or Cryptogen Tool)
Once the above has been laid out, then the next step is to start coding the network script.
The images should be already loaded into the local docker repository, and the Fabric binaries should be available in a location accessible to the script. If the docker images are not loaded, then the machine should have connectivity to internet and then to docker-hub.
It would be good to start with a docker based network setup.
The network and persistent data stores ( docker network, ports and volumes) should be planned.
Once that is sorted out, the coding of the docker compose files could start. Following are the points to be noted during this step.
Create a single compose with all the organizations Or create individual compose files for each organizations. Take a look at the docker compose yaml files present along with the network.sh to get an idea.
Decide on the docker subnet ( network reference)
Provide the same network reference against each service / each
individual compose files.
Provide the env variables for the below items.
Map the MSP folders.
Decide on the SSL as applicable
Provide CouchDB ports(if applicable), Peer Ports, Gossip ports,
Orderer ports etc.
If planning to use cryptogen, then create the config files as per the Org structure. If its CA, then write commands as per the org structure.
Now again refer to the network.sh script and try to figure out how the crypto is generated ( as applicable to your choice). Also refer to the cleanup part from network.sh to understand how it is being done, what is being removed, and what is being retained.
Every time the script bombs, make sure that you cleanup and then start. ie, all the docker containers and volumes to be removed. You could retain your MSP cryptos if you want to.
Locate the command to create the channel, and adding peers to the channel.
The content from env.sh is a good example on how to set the environment variables needed within your script.
Once all the members have joined the channel, the setup the anchor peers per organizations.
Write a version of the script after referring to the example.
By the end of proper execution of the steps above, the script should be able to get a Hyperledger Fabric network up and running.
Hyperledger explorer showing 1 node for 7 peers configured under 1 organization
A single organization is created called ORG1MSP. 7 peers are configured under same organization starting from peer0.org1.example.com through peer6.org1.example.com. All transactions running fine.
To visualize the fabric network I setup an Hyperledger Explorer which shows up only 1 node as against expected 7 nodes. I am expected to setup 7 nodes to be setup under single organization. What am i missing on?
Bassel is right here.
Shashank, I think you have done a fantastic job so far.
want to give a little perspective here. The idea of decentralized systems is to have multiple nodes for its known benefits. I would motivate you to do that with your app too. Deploy it to multiple nodes.
Then visualize it on explorer. Get a sense of pride. At this stage u earned it my friend.
Kill one node ( mock the DDoS scenario) and see if your network is still stable.
Run transactions and Monitor the speed as compared to single node ( single VM as mentioned above in the comments)
You just graduated to develop de-cen systems of which blockchain is a part ( if fabric is your first attempt at de-cen systems, I would assume in this case).
Good luck 👍
I have deployed 2 peers now on 2 separate virtual machines using docker swarm networking.
On first virtual machine, we have ca, orderer, peer0 and couchdb0 running. On second virtual machine we have peer1 and couchdb1 running.
The business network is installed on the fabric using hyperledger composer. Still the blockchain explorer is showing nodes as 1 against expectation of 2 nodes.
Blockchain explorer showing just 1 node
I have been following this tutorial to connect the consortium i have created using the Kaleido UI to truffle: link . When i finally do : ./truffle_migrate.sh
it gets stuck in there, here is the output:
$ ./truffle_migrate.sh
+ truffle migrate --network supnode --reset Using network 'supnode'.
Running migration: 1_initial_migration.js Saving artifacts... Running
migration: 2_deploy_simplestorage.js Deploying SimpleStorage...
... 0xd6d9cfe1ab5b01abb759fb8280920d8f7ba0cef73340af22e47a9c7e40120c14
I don't understand where is the problem, i'm sure i have followed the tutorial carefully and i ve created the same scenario.. If anyone have any idea i would appreciate it. Thanks.
So I went through step by step in an attempt to recreate your scenario - 3 nodes running Quorum + Raft and a private transaction between nodes 1 & 3.
On the initial migration attempt I came across the same hung state as you observed. Inspection of the block explorer reveals that both contracts were actually deployed, however neither were invoked (i.e. no state was set for simple storage via the migration file).
I then changed the truffle_migrate.sh file to target the original privateFor node (3) and used the original targeted node (1) as the new privateFor recipient. This worked immediately. The question is why :)
Truffle is finicky sometimes, especially when using RAFT with private transactions.
I would suggest:
Check your block explorer to ensure that the connection to the network was successful and the contracts were deployed.
Kill the running migrate process and just kick it off again.
If that doesn't work, try the flip flop process I described.
I'm curious if there is a correlation with targeting the RAFT leader or if perhaps truffle just needs a few extra nudges sometimes. We will investigate.
FYI I tried originally with a public transaction and used a truffle.js file with only a single node. This also worked immediately. So my supposition is there is some nuanced approach for private transactions and RAFT.
In the meantime this should give you a potential workaround; you'll just have the original migration and simple storage contracts as orphans in your environment.
I have successfully installed a few hyperledger demos, including the marbles one (https://github.com/IBM-Blockchain/marbles)
A few questions,
How can I move some of the marbles demo nodes to another host/s and still get this demo to work?
I have read the following two posts on the same topic already (where
docker-swarm has been used for intra-host communication)
How can I set up hyperledger fabric with multiple hosts using Docker?
hyperledger-fabric-with-multiple-hosts-using-docker &
How can I make a communication between several docker containers on my local network
communication-between-several-docker-containers-on-my-local-net
I still couldn't decipher installing additional nodes and running them
on different hosts.
As running blockchain nodes on multiple hosts seems to be a common task,
how is it being done now? I saw references to Cello and an ansible
script, though they look not so mature and sure shot solutions.
Could I install the fabric nodes manually by pulling the hyperledger/fabric peer images from the docker hub? How do I then install & run the marbles demo on this pulled images?
Thanks
How can I move some of the marbles demo nodes to another host/s and still get this demo to work?
What do you want to do? I don't understand why you want to move a node. Has it got any sense? If you move some nodes, you are removing them from your Blockchain. If they are part of the Ordering Service or they endorsement is required for the endorsement policy, your demo will not continue running.
The intra-host communication and the communication among multiple docker containers are different things from what you are asking.
Could I install the fabric nodes manually by pulling the
hyperledger/fabric peer images from the docker hub? How do I then
install & run the marbles demo on this pulled images?
You can install you nodes manually via the docker-compose. You should define what you want to start up and then execute it. Of course, you should have in your machine the corresponding docker images. Then, you should deploy the marbles Smart Contract in your peers. You have more info about it here.
I have been working on hyperledger-fabric node sdk v 1.0 and successfully created prototype based on dockers . However now I wanted to implement this architecture on real systems. I haven't found any documentation which helps in setting up environment in real systems. All I found is to set up different peers and organization using dockers and then invoke transactions etc.Can we connect different computer machines using dockers and then spin up the network on all these different machine to create private blockchain?
Yes, you can do it. For that, first of all you should define your network configuration. Then, you would create the artifacts that are required for the network: the keys, the channel artifact, the genesis block... You would follow the steps that are defined on the Fabric documentation to Build your first network. Also, you should share the public keys and the genesis block.
Then, in each machine you would install the docker images, like is explained in the Fabric documentation. After that, you would define the containers that you are going to set up in each machine. You do that on the docker configuration files (docker-compose.yaml, docker-base.yaml...). There, be aware of defining well the docker network configuration. You have more info about it in the answer of this question.
At the end you would switch on each container executin the docker-compose.
I don't know if I've given you enough information. If not, ask again please.
Docker is good for production systems. Docker swarm can be used for connecting multiple machines.