Please advise if it is possible to somehow remove old blocks from hyperledger fabric ?
I understand that it must be immutable, but what if we do not want to store years old data?
There is no "Archive" feature in Fabric at the moment, but there is an "Epic" in the jira system for an Archive feature. It has been around for quite a while but it now looks like a high priority.
Here are the details.
No, you can not remove transaction from the chain itself. Since otherwise you would destroy the whole thing (merkle tree properties).
But only 'recent' data will be stored in the StateDBs. This is data which is labeled as not deleted. Deleted data will be moved out of this DB. So you could actually think of this as a cache where you can 'quickly' access up to date data. (maybe read docs about StateDB and Ledger etc in Fabric to get more insights on this)
Related
Since Hyperledger Fabric has a workaround to avoid double spending by checking the read set versioning with the world state versioning.
Assuming there is a client that issues transaction to update the same asset then the coming transactions by my knowledge will be invalidated by the peers.
To my understanding this means in one block there should only be one transaction for one asset update and note more.
Any help, could you please confirm my assumptions.
Highly appreciated.
Thanks.
Hyperledger Fabric uses Multi-Version Concurrency Control (MVCC) is used to address double-spending problems.
Fabric’s underlying databases use MVCC to make sure no double-spending or inconsistency in data will occur. In understandable terms, this means it’s using versioned documents or records (Since it’s a key-value store, we call each row a document or record). When updating the same state, a new version of an existing document will be made to overwrite the old one. Any transaction executed between this time will not be cleared and an MVCC error will be triggered. The advantage of using versioning overlocking like other databases is that even under high load, the DB can run at full speed asynchronously. This is why in modern databases MVCC is more widely used.
We are currently researching on Hyperledger Fabric and from the document we know that a private data collection can be set up among some subset of organizations. There would be a private state DB (aka. side DB) on each of these organizations and per my understanding, the side DB is just like a normal state DB which normally adopts CouchDB.
One of our main requirements is that we have to distribute files (e.g. PDFs) among some subset of the peers. Each file has to be disseminated and stored at the related peers, so a centralized storage like AWS S3 or other cloud storage / server storage is not acceptable. As the file maybe large, the physical copies must be stored and disseminate off-chain. The transaction block may only store the hash of these documents.
My idea is that we may make use of the private data collection and the side DB. The physical files can be stored in the side DB (maybe in the form of base64string?) and can be distributed via Gossip Protocol (a P2P protocol) which is a feature in Hyperledger Fabric. The hash of the document along with other transaction details can be stored in a block as usual. As they are all native features by Hyperledger Fabric, I expect the transfer of the files via Gossip Protocol and the creation of the corresponding block will be in-sync.
My question is:
Is this way feasible to achieve the requirement? (Distribution of the files to different peers while creating a new block) I kinda feel like it is hacky.
Is this a good way / practice to achieve what we want? I have been doing research but I cannot find any implementation similar to this.
Most of the tutorial I found online pre-assumes that the files can be stored in a single centralized storage like cloud or some sort of servers, while our requirement demands a distribution of the files as well. Is my idea described above acceptable and feasible? We are very new to Blockchain and any advice is appreciated!
Is this way feasible to achieve the requirement? (Distribution of the files to different peers while creating a new block) I kinda feel like it is hacky.
So the workflow of private data distribution is that the orderer bundles the private data transaction containing only a hash to verify the data to a new block. So you dont have to do a workaround for this since private data provides this per default. The data itself gets distributed between authorized peers via gossip data dissemination protocol.
Is this a good way / practice to achieve what we want? I have been doing research but I cannot find any implementation similar to this.
Yes and no. Sry to say so. But this depends on your file sizes and amount. Fabric is capable of providing rly high throughput. I would test things out and see if it meets my requirements.
The other approach would be to do a work around and use IPFS (a p2p file system). You can read more about that approach here here
And here is an article discussing storing 'larger files' on chain. Maybe this gives some constructive insights aswell. But keep in mind this is an older article.
Check out IBM Blockchain Document Store, it is the implementation of storing any document (pdf or otherwise) both on and off chain. It has been done.
And while the implementation isn't publicly available, there is vast documentation on it's usage, can probably disseminate some information from it
According to Hyperledger Fabric website, it states:
Assets can range from the tangible (real estate and hardware) to the
intangible (contracts and intellectual property). Hyperledger Fabric
provides the ability to modify assets using chaincode transactions.
If my use case for a HF Blockchain is to store service logs of vessels, then are the Assets the vessels in this case?
But what modifications are to be made to the asset when I am not transferring the assets or changing its states? I am only keeping an immutable record of the service performed to the vessels.
Could anyone explain this for my use case? Greatly appreciated.
It depends on your setup.
Your vessels could be assets yes, if you need to store them on the ledger. The logs are also assets and they could link back to vessels via an id which uniquely identifies the vessel.
You could also not store the vessels on the ledger at all. if they are defined and maintained in another system then you could only have the logs as assets, still linking via a VesselId.
If you don't want the logs modified at all then you could some ACL rules which stop any changes once the asset is created.
So yeah, you have multiple options available depending on what you want to do.
In very simple terms, assets are the objects you want to maintain on the ledger.
It completely depends on the way you model your Network.
In the use case which you stated, you can have Vessels as assets and Services on these vessels as Transactions. Additionally, you can have a "ServiceLog" Asset which is linked to a Vessel. This may or may not be required and depends on what information you want to log and how you want to process this information later.
If you just want to log the date and time of the service along with who conducted it, for the purpose of verification, then the blockchain does it for you (You can use the historian to track this information).
But if you want more details in your logs then it is a good idea to have a ServiceLog asset and link it to a Vessel.
The ASSET in this case is indeed the Vessel. The ServiceLog is no longer needed, as the LEDGER now acts as the ServiceLog, and will record all transactions.
The TRANSACTION in this case is SERVICE, and all its requisite information will be recorded in ordered, dated, format to the Ledger, in immutable form. In other words, transactions, first need to be setup, and then associated with asset types, after which the transaction "SERVICE" and the associated data elements can be recorded against the specific asset "VESSELid".
The title practically says everything.
In a near future I'm going to implement a real-time tracking system using possibly, a blockchain, and for certain reasons Hyperledger Fabric seems to be the chosen technology. After the information is recorded, it should be accessible in a map in a web application.
So the question is: If we save every one or two minutes the location of a truck to the blockchain using gps, it will hurt the general performance of the blockchain in a near future? (milions and milions of registers)
In the end I have to decide if I should save this information in the blockchain or, knowing that it would cause some serious issues, leave that information out of it and use an hybrid system with a classic database for that and a blockchain for other functionalities that won't cause performance issues.
Thanks.
There is no storage limitation on the Fabric Ledger other than the disk space. The current value of a key (say, the latest position of the truck)can be read via Fabric query which are saved in the world state for quick retrieval. There is also mechanism to look up the history of a key quickly via the historyDB that Fabric maintains.
Since the concept of blockchain is not support modification. All data that being write to Ledger will not support changing. I want to test on changing data value that store in Ledger. I try to find ways around on how to change data, but I couldn't an exact one. I know that Hyperledger Fabric v0.6, Data is stored in RockDB.
Really Appreciate if someone could help to figure out that part. Because I also want to know that it really support un-modification.
Also Auditor will involve in checking Data changing. I also still cannot get clear answer on What's Auditor?
How to configure Auditor in Fabric v0.6?
If you have access to a peer or hack into a peer, you can tamper with the data. The power of blockchain is not that the data on a single peer is unmodifiable, it's that modification can be detected since the hash chain and signatures would not be correct if the data was tampered. This peer would not be able to change other peers, or convince other peers of the accuracy of the modified data. The integrity of the overall blockchain would remain.