I'm getting familiar with DHT and I mostly understand how it works. However, I don't quite understand what happens if you want to have separate DHTs with different entry types in each. Is this possible?
If I use a popular DHT library, does that mean I put and get entries using the same DHT as every user of said library? Or is DHT universal for everyone? How do you define an owner of a DHT, or how do you define a separate, contained DHT?
Yes, you can make separate DHTs. However you need to make the 'on the wire' protocols slightly diffrent so they can't speak to each other and get mixed up.
You can actually have unlimited numbers of DHTs using the same protocol as long as peers don't know each other.
This is the important part when you set up a network. You have to know a second peer to intitially create the network. The next peer would have to know one of the two initial nodes, the next one needs knowledge of one of the three above and so forth.
You are also able to be connected to multiple DHTs on the same host without the two interferring (at least in terms of data exchange, not in terms of local ressources). And you are also able to joins those two DHTs by telling one of them about the peers you are connected to in the other DHT. Though that might be not as easy as it sounds.
Related
Does someone know if it's possible to open multiple positions with only a single data feed? I am trying to do a second buy whilst in a position, which doesn't seem to be possible.
Nobody seems to adress this issue. Does anyone have any experience with Backtrader and have any input?
If you are just trying to buy more stock to add to your position, then yes, you should be able to do this and if you cannot recheck your strategy code in next.
If you are trying to track two separate positions of the same data...
One cannot have two separate positions in the same data feed. You may trade additional positions if you like but they will be combined in Backtrader. Even if you use two strategies you will still have one combined broker.
The reason for this is to simulate as near as possible real world conditions. If you have a brokerage account you most likely would have just one postion. (I know there are exceptions)
One solution would be to track your trading manually in a dictionary trades that result from different signals/sub-strategies. It's a bit more tedious to develop but very doable.
I am new to this tech but I am close to the concept of Channel. I understand that the channel are ledgers within a bigger ledger( i.e. small chain specific to some users only within the entire blockchain). Based on this I have few questions.
Lets say there are retail markets where there is scope for negotiations in rate and hence each vendor can sale same thing to different people with different prices. So assuming that there are 1000 people and 1000 vendors possibly there could be many channels. How these are managed?
I understand that the blockchains are linear data structure (unless two blocks are created simultaneously, there is a word for this but I forgot it), If I have to access previous data for certain user. How efficient it is going to be for such operations? i.e. take an example of bank credit card transactions. If I want to see all my transactions for last 5 months. How efficient it is going to be?
I am not claiming that I am completely correct in my understanding but these things are bugging me.It would be very nice if someone help me clear these?
Thanks :)
Update
I have gone through this link it also talks about something related to my questions in below comments. i.e. related to businesses (questions private blockchain).
I like more the definition of channels as different blockchains using a common network or common parts of a network. It allows privacy and different organisation structures.
Yes, it could have possibly so many channels as vendor-people pair. Of course, it depends on which privacy you want to achieve.
If you want to access previous datas for previous operations, you have the possibility to have a state database running alongside the peers, which are a state representation of the linear structure. You can use LevelDB or CouchDB. CouchDB allows you to use complex rich queries to access your data.
ledger can only exists in channel. And the channel is composed of peers. the peers out of the channel cannot get access to the ledger data.
I am not sure what you mean by "Operation". You can create it and close it(this will be supported in later version) When you need it ,create it and you can close after use.
If the business is independent, you have to create so many channels. you just say vendor can sale same thing to different people with different prices. . each of them will be used for different scenario.
if you want to get access to the previous data, of course you can. But not maybe as efficient as you the bank card now. But this is doable. Now we use sdk to access. maybe later with more development of the sdk, more graphic tool will be developed, enabling it used as easy as possible.
In my new weekend project I decided to write an bittorrent client from scratch, no ready to use libraries at all. After two days looking for documentation I'm already about to give up :smile:. I know there are the BEPs, but they are far from enough to understand all the specification. After reading a lot more I think the tracker and peer protocols seems to be old and easy to understand/implement (yes, I know, to write a good code with balance, peer selection, optimizations, this is not easy as I just said, but all I want to is to do the basics to learn, not to compete with tens of good clients out there.)
So, I have decided to start by the DHT which seems to the the more complex part and also the less documented. When you stop looking for bittorrent DHT or mainline DHT and start looking for kademlia DHT you have a lot more information but it not so obvious how to put it all together.
Here is what I understand so far (and there are gaps which I hope to fill in):
I start with my DHT tree empty
use find_nodes on my bootstrap node
add the received nodes to my own tree, so I can then select the ones closer to my own ID
start issuing find_nodes to the selected ones and add their responses to my tree
go back to 3 until I stop receiving unknown/new nodes
if I receive an announce_peer with an info_hash than I should save its information on a local DB (the info_hash and ip/port of the sender)
if a node uses get_peers with an info_hash I have in my DB then I send the information otherwise I should send a list of closer nodes I have in my own tree (closest to that info_hash)
when I use get_peers on other nodes I will receive peers or nodes, in the later case I think the nodes are closer to the info_hash and not to my own nodeId so, should I add these nodes to my tree or start a new tree based on them?
when I want to announce I am interested on an info_hash should I use announce_peer everywhere or just to the nodes with nodeId closer to the target info_hash? How much is closer enough?
At this point I have a lot of nodes which IDs are closer to my own ID, and informations about info_hash'es I am not really interested.
I am afraid that I have a giant stupid question: why I did that?
I mean: my selfish reason to do all this work is to locate peers to the info_hash I'm interested in. I understand that the information of one info_hash is likely to be saved on a node which ID is closer to that info_hash. So my chances to find its information is bigger if I create a tree of nodes closer to the info_hash and not closer to my own ID (at this point, if you know the subject, you already noticed how lost I am).
Should I create multiples trees? One for me (to be there to save the information of info_hashes closer to my nodeID people send me), and other tree closer to each one of my target info_hashes so I can retrieve their information?
Should I create a single tree closer to my node ID and hope for the best when querying this tree for the info_hashes I need?
Should I give up since I have completely misunderstood the idea behind DHT at all?
Well, any real documentation, flowcharts, any thing will be welcome!
So, I have decided to start by the DHT which seems to the the more complex part and also the less documented.
The original kademlia paper "Kademlia: A Peer-to-peer Information System Based on the XOR Metric" by Peter Maymounkov and David Mazieres is required reading. It is referenced fairly early in BEP-5
if I receive an announce_peer with an info_hash than I should save its information on a local DB (the info_hash and ip/port of the sender)
You only accept announces when they contain a token previously handed out via get_peers.
when I use get_peers on other nodes I will receive peers or nodes, in the later case I think the nodes are closer to the info_hash and not to my own nodeId so, should I add these nodes to my tree or start a new tree based on them?
You use a temporary tree - or a list ordered by contact-ID relative to the target ID - for iterative lookups since they are not balanced towards your node ID.
when I want to announce I am interested on an info_hash should I use announce_peer everywhere or just to the nodes with nodeId closer to the target info_hash? How much is closer enough?
You perform a get_peers lookup and when it is done you announce to the 𝑲 closest nodes set that returned a write token and verify the responses to make sure you actually get 𝑲. In case of bittorrent 𝑲 = 8.
my selfish reason to do all this work is to locate peers to the info_hash I'm interested in. I understand that the information of one info_hash is likely to be saved on a node which ID is closer to that info_hash. So my chances to find its information is bigger if I create a tree of nodes closer to the info_hash and not closer to my own ID (at this point, if you know the subject, you already noticed how lost I am).
When doing lookups you do not just visit nodes in your routing table, you also visit nodes included in the responses. This makes them iterative. The bias of each node's routing table towards their own ID ensures that the responses include neighbors closer and closer towards the target.
So the deal is that you are responsible for information close to your node ID and other nodes will provide information close to their node IDs that you are interested in. So your routing table layout serves others, their routing table layout serves you.
Note that all the information contained in this answer can be found in the BEP or Kademlia paper.
The Wikipedia page of BitTorrent says regarding Multitracker torrents, "One disadvantage to this is that it becomes possible to have multiple unconnected swarms for a single torrent where some users can connect to one specific tracker while being unable to connect to another. This can create a disjoint set which can impede the efficiency of a torrent to transfer the files it describes."
Can someone please give me an example of this?
Thanks.
The information on the wikipedia page is old and no longer relevant.
Split swarms was only a problem between the introduction of:
Multi-trackers extension:BEP12 around 2004
and the introduction of:
PeerEXchange(PEX):BEP11 and the DHT distributed tracker:BEP5 around 2005.
These three extensions working together creates a single unified swarm.
I'm looking for a generic charting solution, ideally not a hosted one that provides the following features:
Charting a tuple of values where the values are:
1) A service identifier (e.g. CPU usage)
2) A client identifier within that service (e.g. server IP)
3) A value
4) A timestamp with millisecond/second resolution.
Optional:
I'd like to also extend the concept of a client identifier further, taking the above example further, I'd like to store statistics for each core separately, so, another identifier would be Core 1/Core 2..
Now, to make sure I'm clearly stating my problem, I don't want a utility that collects these statistics. I'd like something that stores them, but, this is also not mandatory, I can always store them in MySQL, or such.
What I'm looking for is something that takes values such as these, and charts them nicely, in a multitude of ways (timelines, motion, and the usual ones [pie, bar..]). Essentially, a nice visualization package that allows me to make use of all this data. I'd be collecting data from multiple services, multiple applications, and the datapoints will be of varying resolution. Some of the data will include multiple layers of nesting, some none. (For example, CPU would go down to Server IP, CPU#, whereas memory would only be Server IP, but would include a different identifier, i.e free/used/cached as the "secondary' identifier. Something like average request latency might not have a secondary identifier at all, in the case of ping). What I'm trying to get across is that having multiple layers of identifiers would be great. To add one final example of where multiple identifiers would be great: adding an extra identifier on top of ip/cpu#, namely, process name. I think the advantages of that are obvious.
For some applications, we might collect data at a very narrow scope, focusing on every aspect, in other cases, it might be a more general statistic. When stuff goes wrong, both come in useful, the first to quickly say "something just went wrong", and the second to say "why?".
Further, it would be a nice thing if the charting application threw out "bad" values, that is, if for some reason our monitoring program started to throw values of 300% CPU used on a single core for 10 seconds, it'd be nice if the charts themselves didn't reflect it in the long run. Some sort of smoothing, maybe? This could obviously be done at the data-layer though, so its not a requirement at all.
Finally, comparing two points in time, or comparing two different client identifiers of the same service etc without too much effort would be great.
I'm not partial to any specific language, although I'd prefer something in (one of the following) PHP, Python, C/C++, C#, as these are languages I'm familiar with. It doesn't have to be open source, it doesn't have to be a library, I'm open to using whatever fits my purpose the best.
More of a P.S than a requirement: I'd like to have pretty charts that are easy for non-technical people to understand, and act upon too (and like looking at!).
I'm open to clarifying, and, in advance, thanks for your time!
I am pretty sure that protovis meets all your requirements. But it has a bit of a learning curve. You are meant to learn by examples, and there are plenty to work from. It makes some pretty nice graphs by default. Every value can be a function, so you can do things like get rid of your "Bad" values.