Background
I'm trying to add some active trackers for transmission daemon to speed it up as I have done this before when using aria2.
But all the resources I found is how to add trackers to a torrent.
Question
So I'm wondering which is tracker related to? A torrent file or the downloader? If is the torrent file, how does I add trackers in aria2? The only way I can image is that aria2 automatically adds trackers to the added torrent.
BTW, how to add default trackers in transmission daemon just like in aria2?
Trackers can be centralized servers which you can request list of peers from.
The torrent file and the download don't go through the tracking server, the tracking server simply tells you from whom you can ask for pieces of the file from.
If you ask for more trackers, you won't download faster, you'll just have a wider pool of peers to pick from. If you want the download to be faster, you'll have to increase the amount of peers you are downloading from. (Tracking servers usually return 80 at a time I think in any case)
There are decentralized means of doing this using DHT (distributed hash table)
Related
With IPFS being distributed p2p storage and sharing, isn't there then a chance that someone could store something illegal on your machine if you are an IPFS provider?
Is there some mechanism that IPFS systems use to prevent this? How would someone even know if illegal content is stored on their machine, especially if they are only storing a part of the file?
I want to run an IPFS node on my machine, but I am unsure if I have to worry about malicious actors using my IPFS node.
The law is very clear here, you are not responsible for caching partial/complete files/metadata in almost all cases.
distributed p2p storage and sharing
No, it works more like in BitTorrent protocol and less like in TV series "Silicon Valley". You need to share a file and somebody will need to find a hash to download a file (the difference is that .torrent file was mostly preferred over magnet hash before around 2015 in BitTorrent, while in IPFS hashes are the only way, also the system is global, that means the complex hash function is used so that no collisions are possible (at least not in a billion years) and thus it can check for hashes over THE WHOLE network and thus do not store duplicate chunks of data that folder/file structure is reconstructed from).
The point here is that just like in BitTorrent you store no files you do not request, now just like in BitTorrent you can do IPFS BitSwap stuff to accelerate the swarms, that is what cloudflare-ipfs.com does and ipfs.infura.io and others (?). In BitTorrent such things also exist in particular to automatically attach to updated torrents that have the same hashes for file parts... That is very cool, but in IPFS it is done automatically. Also different servers exist that propagate .torrent file (a.k.a. magnet metadata) using just magnet hash. I believe even DHT crawlers play some role, like BTDigg or https://btdb.eu/, but not much of course, you can set up you own crawler (as Btdigg is open source) that will do precisely that: share metadata of torrents, that requires almost no resources... (You can even set up your own bootstrap supernode to create you OWN seperate DHT.) But is very cool to do as a lot of stuff can be found there. As I understand IPFS also does that by default, i.e. it stores some metadata to help data propagation. You can further read this:
https://discuss.ipfs.io/t/ipfs-propagation/4301
https://discuss.ipfs.io/t/how-fast-do-ipns-changes-propagate/311
https://docs.ipfs.io/concepts/bitswap/
There is also this: https://collab.ipfscluster.io/
I am studying some new emerging p2p protocols, then I found dat protocol.
In dat protocol, if I install dat and use it to share folders, I will become so-called peer and also store other peers's data, right?
I found no docs and FAQs saying that where and who are the so-called peers. Is it right that the PCs in which every users install dat will act as peers who share and store data for each others? and only dat client will do that? any other software?
In dat protocol, if I install dat and use it to share folders, I will become so-called peer and also store other peers's data, right?
It's important to know you control what data you download and share.
When you run the Dat CLI, you specify either a URL, or the path of a folder on your computer.
If you give an archive's URL, you will download the files in the archive, and then share those files on the p2p network until you close the CLI tool.
If you give a folder path, you will create a new archive, and Dat will give you a URL to share.
In practice, this is similar to how BitTorrent works. Each archive is a set of files, and so "swarming" an archive (that is, joining the network to exchange it) will only upload and download the files in that archive.
I found no docs and FAQs saying that where and who are the so-called peers. Is it right that the PCs in which every users install dat will act as peers who share and store data for each others? and only dat client will do that? any other software?
Peers are people who possess the URL, and who have told their Dat clients to swarm the URL. As of yet, there's no Dat client which tries to automatically download more than the URLs given explicitly by the user, but such a thing would be possible.
There are multiple clients for Dat already created:
The Official Dat CLI
The Official Dat Desktop App
Science Fair a tool for exchanging papers
Beaker Browser a browser which uses dat:// as a protocol
The dat-node library is written in javascript and is relatively easy to use for creating custom clients.
Hi I want to make an application that if given a torrent file (or hash) can give the number of peers without being active (i.e not responsible) in the process that allow the sharing of a file (for legal reason obviously). whether by being a "passive" (passive as define previously) tracker or a bittorrent client that counts "All time" peers (i.e. number of download for a torrent). Can it be done? I know some trackers keep track of download but I don't know if those who "seem not to" actually do as well. I look for something that can track the number of unique-ip transfers from when the torrent was added to the tracking system or something that count download (complete).
It's not possible to determine all peers just from a tracker. There can be multiple trackers for each torrent, and they may not store complete, fresh, or even truthful information. Additionally there's no obligation for peers to be honest with their trackers. There are also alternatives to centralized trackers, such as DHT and PEX. There's no guarantee that all peers are participating in the same DHT network. Peers might even establish disjoint PEX communities.
In short, you might make a best effort attempt at determining the total swarm participation for a particular torrent by checking trackers and querying DHT. But to be as thorough as the technology will allow, you'd actually have to participate in the swarm with all manner of transports and protocol extensions currently in use such as uTP and encryption, and scrape each peer for further peers and download states. Of course the BitTorrent community is familiar with such attempts to scrape data, and there a lot of security measures in place to prevent exploitation in this way. Examples include IP blocklists, and heuristics on peer behaviour.
I have been interested in using BitTorrent Sync to synchronize my files. My question is: Is my encrypted torrent data being shared through 3rd party computers or only the computers I register? For example, if I install Sync on 3 machines, will all of that sync'd data only ever stay on those 3 machines and no data - encrypted or not - ever travels to other computers with Sync installed.
While Sync uses BitTorrent technology, it creates swarm only from your machines. Particularly in your case it will create a swarm from your 3 machines, that nobody could join, and will use p2p to effectively send data between them.
PS. I am involved in Sync development.
This is from bittorrent website:
The data is transferred in pieces from each of the syncing devices,
and BitTorrent Sync chooses the optimal algorithm to make sure you
have a maximum download and upload speed during the process.
EDIT: there is also a statement in unofficial faq:
Will me devices still sync when switched off?
BitTorrent Sync is not a cloud storage solution like
SkyDrive/DropBox/GDrive, etc - it doesn't sync your files to a "cloud",
therefore for one device to sync with another, both devices need to be online
First statement does not tell anything about communication with devices of other users.
Last statement could hint, that there is no way to get to your data without
having at least one of your own syncing machines turned on.
I want to collect statistics from the spreading of a file in a new bittorrent swarm without actually downloading anything (or as little as possible). I need to know which peer has which pieces (to make file based statistics) knowing the number of seeders and leechers or percentages is not enough. Later when there are many peers I need to download the data to determine what it is. This part can be done with a regular torrent client.
I do not plan to implement the protocol myself so I looked at 2 implementations libtorrent and ktorrent's libbtcore. Neither is capable of collecting data while not downloading there are simply no connected peers when there is nothing to download. Libtorrent is simpler but ktorrent looks better commented.
I see 3 possibilities:
Use some application exactly for this. Are there any?
Modify a torrent implementation to do what I want. Is anyone familiar with them? Where to start?
Implement a small subset of the protocol. Just periodically ask the peers what they have. Is this feasible or would the program need to support almost the full protocol?
What do you recommend?
This is an old question, but perhaps this answer might be useful for others.
Use some application exactly for this. Are there any?
Not that I know of.
Modify a torrent implementation to do what I want. Is anyone familiar with them? Where to start?
I'm only familiar with the BitTornado core (that is used in e.g. ABC). It is written in Python, but it's an architectural mess.
However, you could just take any implementation and start stripping it from unnecessary functionality.
Implement a small subset of the protocol. Just periodically ask the peers what they have. Is this feasible or would the program need to support almost the full protocol?
Note that you cannot "ask" a peer what they have. The other peer informs you whenever it wants about the pieces it has (so it's push instead of pull). After the BitTorrent handshake, a peer may send a bitfield of pieces it has. Afterwards it may send HAVE messages informing you it has acquired a new piece. Also note that peers may lie about the pieces they have. Examples include superseeding peers and freeriding clients like BitThief.
If you want to implement a small subset of the protocol, you'd need at the bare minimum implement the BitTorrent handshake message and preferably the extended handshake message. The latter allows you to receive (and send) uTorrent PEX messages. PEX is useful to quickly discover other peers in the swarm.
For your statistics gathering purposes, you additionally need to support the bitfield and HAVE messages.