I have a question regarding the piece selecion stradegy. It is mentioned in a paper that there is a "Strick Priority" as below:
BitTorrent’s first policy for piece selection is that
once a single sub-piece has been requested, the remaining sub-pieces from that particular piece are requested before sub-pieces from any other piece. This
does a good job of getting complete pieces as quickly
as possible.
The stradegy above is easy to understand, but it does not mention from the peer's point of view.
So here is my question:
Is it true that all the blocks for a piece should be requested from the same peer?
If answer of question 1 is true, does the client will request the same block from diffrent peers at the same time in case some peer failed to respond?
Is it true that all the blocks for a piece should be requested from the same peer?
False. The idea is, by requesting different blocks from different peers, adding together the bandwidth from all the peers to complete the piece quickly.
does the client will request the same block from diffrent peers at the same time in case some peer failed to respond?
Normally a specific block is only requested from one peer. Only if that peer hasn't sent that block after a long time out, that block is requested from another peer.
A exception from this is the "End Game" mode.
When the download is almost complete and all remaining blocks has been requested:
Blocks may be requested in parallel from more than one peer to avoid needing to wait for the slowest peers to finish their blocks and make the download slow down to a crawl.
Related
I want to write a timed chaincode. I wish the chaincode won't response to any invoke after a certain period. In Ethereum, I can do this by counting blocks. For example, I can make a smart contract valid only before block 100000. But in hyperledger fabric, I don't know how to do it.
I had similar problem. Even though the block information is available via system chaincode I was not able to implement the querying of it from my chaincode elegantly. For my purpose, however, the following implementation was enough:
Introduce a counter state and initialize it with '0'.
Check this state in all your important chaincode functions. If it is lower than the certain limit, increase it by 1 and proceed with the function logic. Otherwise, throw error/print message/do nothing.
Based on the expected frequency of your transactions you can set more or less appropriate limit.
Maybe this soltion will work for you as well.
AFAIK part of block verification is checking that it contains valid previous block hash. But let's imagine, that for some reason (network glitch etc.) previous block never arrived. Wouldn't node discard all the following blocks?
I wonder how this problem is solved? Does node periodically checks whole chain, by requesting peers? How exactly is this implemented?
Thanks!
I'm assuming you're not talking about mining. So:
A node will ask for block number N before asking for block N+1. Yes, mostly block N+1 will be discarded if block number N doesn't exist, but that's irrelevant. Because it's not that a node gets blocks only by being the receiving end. It can contact all the other nodes around it, and ask for block number N. If all these nodes don't have block number N, well, then it doesn't exist!
Notice that on a broader scale every node asks its peers for node number N, and verifies that this block follows the rules of the protocol. That's part of the consensus.
If we're talking about mining, the situation is different. You can imagine two competing miners creating blocks. If a miner doesn't have block N, he'll keep trying to mine that block. If he receives block number N after having mined it, the miner just ignores it, and we have a temporary fork in the blockchain with two branches, and block N+1 will have two different versions. Now what determines which branch survives is basically the branch that is mined faster. An arbitrary consensus rule is that the longer chain is always the correct chain. When a longer chain exists, all miners agree that this chain is the correct one.
The reason why this works, is because the probability of mining a block is wide enough over 10 minutes in Bitcoin (or block time in general).
The following image can be seen as the probability of mining a block over time (in bitcoin).
This is why it's wrong to create blockchains with very short time. Because if the time is very short (like 10 seconds), there will be tons of forks before the blockchain stabilizes.
for loose endorsement policy, for same transaction, even if correct-behavior committers, different committer maybe have different validation result due to their different world state.
And how do commiter know who's validation result is correct(major) or wrong(minor) ?
Hint: at a moment, due to network delay, correct commiter may have different world state.
thanks for answer.
I have got it!
because each correct commiter receive block/transaction in exactly same order, and implement same endorsement policy, so that at same transaction's validation moment, all correct commiter indeed have exactly same world state, despite at not same time!
key point is :
(1) at same moment, different commiters may have different world state due to network delay, etc;
(2) but, different commiter do validation for same transaction at not same moment.
After enrolling, installing and instantiating the chaincode fabric/example/chaincode/go/chaincode_example02, I run the following steps.
peer chaincode instantiate --orderer orderer0:7050 --tls true --path example02 --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/orderer/localMspConfig/cacerts/ordererOrg0.pem --chainID mychannel --name example02cc --version 1.0 --ctor '{"Args":["init","A","1000","B","2000"]}'
peer chaincode query --chainID mychannel --name example02cc --ctor '{"Args":["query","A"]}'
peer chaincode query --chainID mychannel --name example02cc --ctor '{"Args":["query","B"]}'
So far, I confirm that A is equal to 1000 and B is equal to 2000. Afterwards, The result will be variable if I invoke the following step with different timings.
peer chaincode invoke --orderer orderer0:7050 --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/orderer/localMspConfig/cacerts/ordererOrg0.pem --chainID mychannel --name example02cc --ctor '{"Args":["invoke","A","B","1"]}'
Specifically, A will be equal to 998 and B will be equal to 2002 if I run the previous step twice with a 10 second pause. A will be equal to 990 and B will be equal to 2010 if I run the previous step ten times with 10 second pause between every step. However, without any pause, A will be equal to 999 and B will be equal to 2001 if I run the previous step twice. A will be equal to 999 and B will be equal to 2001 if I run the previous step ten times without pause between every step.
I have tested several times with different arguments. Furthermore, I have tested other chaincodes. It seems like that the chaincode only accept the first invoking request, and discards subsequent invoking requests. So, the questions are:
Is this a mechanism to prevent double-spending? or just a weakness?
How to solve this problem which limits the transaction rate.
I think that chaincode should support concurrent invocations. Can chaincode support concurrent invocations actually?
Can a single chaincode invoke multiple requests in a single block period?
The chaincode need to do endorsement and then commit it the ledger state before the next invoke.
When you call peer chaincode invoke ..., fabric response quickly after the endorsement has finished. But the commit will still take some time. If you run the second invoke directly after the first invoke, the second invoke will be endorse correctly, but no commit happened.
So, for your questions:
You can try to invoke chaincode by java-sdk or node-sdk. take the javasdk for example, the progress is first send transactionProposalRequest to chaincode, which is the endorsement process in fabric. And after the endorsement is finished, and your endorment policy is correctly passed. the transaction is send to fabirc by sdk-client, this API will return an CompletableFuture<BlockEvent.TransactionEvent>, which is similar to the Promise in js. when the transaction is done, the CompletableFuture.thenApply() is triggled. You may check the src/test/java/org.hyperledger.fabric.sdkintergration/End2endIT.java for example.
You may write a batch in your chaincode. This can support multiple invoke and query at once, it solves your question in some degree. The batch is designed as follow, new a m map[string]string and toDelete []string in your cc, when invoke put your key/val pairs in m, and when query get key/val from m first, if not found, query from stub, and when delete requirement, delete from m and put the key to toDelete , after all requirements are finished, commit all data in m and toDelete at once. I tried this mechanism in my project, it work good.
blockchain is designed not for high concurrent. It is designed for confidentiality, scalability and security.
see 2
From the point of view of fabric this is actually normal behavior - although may seem a bit counterintuitive first.
What you see happening follows logically from the documented transaction flow (+ knowing that there is some batching performed on the orderer with batch-timeouts). Let us assume that during simulation (endorsement), variable A is read and then marked for re-setting by the chaincode simulation. What value was read and what value you want the variable set to become part of the proposed transaction (+ whether endorsers accept the transaction + crypto-stuff). Then we go through submission to orderer + distribution to channel peers + channel peers checking (among others) whether the original "read value assumptions" of the proposal still hold. (See 5.: Validation on the referenced peers.) If the value of A has changed "since simulation", then the transaction will not be valid.
The impact of timing on this is the following. If there is enough "leeway" between two invokes, the following happens:
Call1 - Proposal created with original A value
Call1 proposal gets endorsements, sent to orderer, gets batched
After some time, peers get the ordered-endorsed Tx from the orderer, check it, find it fine, the value of A is modified (committed to the Ledger) as requested
Call2 - Proposal created with new A value read etc.
The crucial thing is that what happens if you create and submit the second proposal before the effects of the first would have been committed to the Ledger. (So without "waiting enough".)
Call2 simulation still sees the original A value, endorsements are computed based on that
This is bundled into the "read values" property of the transaction
By the time it reaches the channel peers for committing, the value of A has been already modified by Call1
Thus, this transaction will be invalid and have no effect on the ledger
Such effects are not exactly new, we did get bitten by similar stuff with fabric 0.6. Fortunately, there are a few "clean" ways out of this; I suggest to read up on "tokenization" in the Blockchain context. In the context of example02, you can make all "units" have a GUID and track essentially ownership vectors; or you can go full-UTXO, Bitcoin style. Option b) can be to write the Tx requests themselves into the Ledger (Tx1:+20, Tx2: -40, Tx3: +65, ...) and base the comuptation of "current acccount state" on Ledger-stored Tx logs, although this can get a bit messy.
And do note that massive (and painless) concurrency is eminently possible, as long as the "working sets" of Tx requests do not overlap much. For many domains, this is possible; e.g. for a cryptocurrency, it is not the same currency unit that is being passed around like hot potatoes, but a large set of currency units having many transactions, but levelled out across them.
On your specific questions:
see far above
see the two suggestions above
Depends on what you mean by concurrent invocation, but at the heart of the fabric 1.0 architecture is that a full ordering is set up on all endorsed requests and this ordering is enforced during validation and Ledger update. Massive amounts of peers requesting things concurrently - sure; "reentrant chaincode" - no-no-no.
Chaincodes don't "invoke requests". If you mean whether a single chaincode can have multiple transactions in a single block: sure, e.g. selling 100 different cars using a "sell_cars" chaincode.
is it possible to have multiple threads sending on the same socket? will there be interleaving of the streams or will the socket block on the first thread (assuming tcp)? the majority of opinions i've found seems to warn against doing this for obvious fears of interleaving, but i've also found a few comments that state the opposite. are interleaving fears a carryover from winsock1 and are they well-founded for winsock2? is there a way to setup a winsock2 socket that would allow for lack of local synchronization?
two of the contrary opinions below... who's right?
comment 1
"Winsock 2 implementations should be completely thread safe. Simultaneous reads / writes on different threads should succeed, or fail with WSAEINPROGRESS, depending on the setting of the overlapped flag when the socket is created. Anyway by default, overlapped sockets are created; so you don't have to worry about it. Make sure you don't use NT SP6, if ur on SP6a, you should be ok !"
source
comment 2
"The same DLL doesn't get accessed by multiple processes as of the introduction of Windows 95. Each process gets its own copy of the writable data segment for the DLL. The "all processes share" model was the old Win16 model, which is luckily quite dead and buried by now ;-)"
source
looking forward to your comments!
jim
~edit1~
to clarify what i mean by interleaving. thread 1 sends the msg "Hello" thread 2 sends the msg "world!". recipient receives: "Hwoel lorld!". this assumes both messages were NOT sent in a while loop. is this possible?
I'd really advice against doing this in any case. The send functions might send less than you tell it to for various very legit reasons, and if another thread might enter and try to also send something, you're just messing up your data.
Now, you can certainly write to a socket from several threads, but you've no longer any control over what gets on the wire unless you've proper locking at the application level.
consider sending some data:
WSASend(sock,buf,buflen,&sent,0,0,0:
the sent parameter will hold the no. of bytes actually sent - similar to the return value of the send()function. To send all the data in buf you will have to loop doing a WSASend until all all the data actually get sent.
If, say, the first WSASend sends all but the last 4 bytes, another thread might go and send something while you loop back and try to send the last 4 bytes.
With proper locking to ensure that can't happen, it should e no problem sending from several threads - I wouldn't do it anyway just for the pure hell it will be to debug when something does go wrong.
is it possible to have multiple threads sending on the same socket?
Yes - although, depending on implementation this can be more or less visible. First, I'll clarify where I am coming from:
C# / .Net 3.5
System.Net.Sockets.Socket
The overall visibility (i.e. required management) of threading and the headaches incurred will be directly dependent on how the socket is implemented (synchronously or asynchronously). If you go the synchronous route then you have a lot of work to manually manage connecting, sending, and receiving over multiple threads. I highly recommend that this implementation be avoided. The efforts to correctly and efficiently perform the synchronous methods in a threaded model simply are not worth the comparable efforts to implement the asynchronous methods.
I have implemented an asynchronous Tcp server in less time than it took for me to implement the threaded synchronous version. Async is much easier to debug - and if you are intent on Tcp (my favorite choice) then you really have few worries in lost messages, missing data, or whatever.
will there be interleaving of the streams or will the socket block on the first thread (assuming tcp)?
I had to research interleaved streams (from wiki) to ensure that I was accurate in my understanding of what you are asking. To further understand interleaving and mixed messages, refer to these links on wiki:
Real Time Messaging Protocol
Transmission Control Protocol
Specifically, the power of Tcp is best described in the following section:
Due to network congestion, traffic load balancing, or other unpredictable network behavior, IP packets can be
lost, duplicated, or delivered out of order. TCP detects these problems, requests retransmission of lost
packets, rearranges out-of-order packets, and even helps minimize network congestion to reduce the
occurrence of the other problems. Once the TCP receiver has finally reassembled a perfect copy of the data
originally transmitted, it passes that datagram to the application program. Thus, TCP abstracts the application's
communication from the underlying networking details.
What this means is that interleaved messages will be re-ordered into their respective messages as sent by the sender. It is expected that threading is or would be involved in developing a performance-driven Tcp client/server mechanism - whether through async or sync methods.
In order to keep a socket from blocking, you can set it's Blocking property to false.
I hope this gives you some good information to work with. Heck, I even learned a little bit...