Peer exchange
Encyclopedia
Peer exchange or PEX is a communications protocol
that augments the BitTorrent file sharing protocol. It allows a group of users (or peers
) that are collaborating to share a given file to do so more swiftly and efficiently.
In the original design of the BitTorrent file sharing protocol, peers (users) in a file sharing group (known as a "swarm") relied upon a central computer server called a tracker
to find each other and to maintain the swarm. PEX greatly reduces the reliance of peers on a tracker by allowing each peer to directly update others in the swarm as to which peers are currently in the swarm. By reducing dependency on a centralized tracker, PEX increases the speed, efficiency, and robustness of the BitTorrent protocol.
file that describes the file(s) to be shared, as well as the URLs of one or more central computers called trackers
that maintain a list of peers currently sharing the file(s) described in the .torrent file. In the original BitTorrent design, peers then depended on this central tracker to find each other and maintain the swarm. Later development of distributed hash table
s (DHTs) meant that partial lists of peers could be held by other computers in the swarm and the load on the central tracker computer could be reduced. PEX allows peers in a swarm to exchange information about the swarm directly without asking (polling
) a tracker computer or a DHT. By doing so, PEX leverages the knowledge of peers that a user is connected to by asking them for the addresses of peers that they are connected to. This is faster and more efficient than relying solely on a tracker and reduces the processing load on the tracker. It also keep swarms together when the tracker is down.
Peer exchange cannot be used on its own to introduce a new peer to a swarm. To make initial contact with a swarm, each peer must either connect to a tracker using a ".torrent" file, or else use a router computer called a bootstrap node to find a distributed hash table (DHT) which describes a swarm's list of peers. For most BitTorrent users, DHT and PEX will start to work automatically after the user launches a BitTorrent client and opens a .torrent file. A notable exception is "private torrents" which are not freely available; these will disable DHT.
It was agreed between the Azureus and µTorrent developers that any clients which implement either of the mechanisms above try to obey the following limits when sending PEX messages:
Some clients may choose to enforce these limits and drop connections from clients that ignore them.
For "trackerless" torrents, it is not clear if PEX provides any value since the mainline DHT can distribute load as necessary. Each DHT node acting as a tracker may store only a subset of the peers, but these are maximal subsets constrained only by DHT node load rather than by a single peer's view. Private torrents disable the DHT, and for this case, PEX might be useful provided the peer obtains enough peers from the tracker.
Each of these clients implement some version of peer exchange:
Communications protocol
A communications protocol is a system of digital message formats and rules for exchanging those messages in or between computing systems and in telecommunications...
that augments the BitTorrent file sharing protocol. It allows a group of users (or peers
Peer-to-peer
Peer-to-peer computing or networking is a distributed application architecture that partitions tasks or workloads among peers. Peers are equally privileged, equipotent participants in the application...
) that are collaborating to share a given file to do so more swiftly and efficiently.
In the original design of the BitTorrent file sharing protocol, peers (users) in a file sharing group (known as a "swarm") relied upon a central computer server called a tracker
BitTorrent tracker
A BitTorrent tracker is a server that assists in the communication between peers using the BitTorrent protocol. It is also, in the absence of extensions to the original protocol, the only major critical point, as clients are required to communicate with the tracker to initiate downloads...
to find each other and to maintain the swarm. PEX greatly reduces the reliance of peers on a tracker by allowing each peer to directly update others in the swarm as to which peers are currently in the swarm. By reducing dependency on a centralized tracker, PEX increases the speed, efficiency, and robustness of the BitTorrent protocol.
Description
Users wishing to obtain a copy of a file typically first download a torrentTorrent file
A torrent file stores metadata used for BitTorrent. It is defined in the BitTorrent specification.Simply, a torrent is data about a target file, though it contains no information about the content of the file. The only data that the torrent holds is information about the location of different...
file that describes the file(s) to be shared, as well as the URLs of one or more central computers called trackers
BitTorrent tracker
A BitTorrent tracker is a server that assists in the communication between peers using the BitTorrent protocol. It is also, in the absence of extensions to the original protocol, the only major critical point, as clients are required to communicate with the tracker to initiate downloads...
that maintain a list of peers currently sharing the file(s) described in the .torrent file. In the original BitTorrent design, peers then depended on this central tracker to find each other and maintain the swarm. Later development of distributed hash table
Distributed hash table
A distributed hash table is a class of a decentralized distributed system that provides a lookup service similar to a hash table; pairs are stored in a DHT, and any participating node can efficiently retrieve the value associated with a given key...
s (DHTs) meant that partial lists of peers could be held by other computers in the swarm and the load on the central tracker computer could be reduced. PEX allows peers in a swarm to exchange information about the swarm directly without asking (polling
Polling (computer science)
Polling, or polled operation, in computer science, refers to actively sampling the status of an external device by a client program as a synchronous activity. Polling is most often used in terms of input/output , and is also referred to as polled or software driven .Polling is sometimes used...
) a tracker computer or a DHT. By doing so, PEX leverages the knowledge of peers that a user is connected to by asking them for the addresses of peers that they are connected to. This is faster and more efficient than relying solely on a tracker and reduces the processing load on the tracker. It also keep swarms together when the tracker is down.
Peer exchange cannot be used on its own to introduce a new peer to a swarm. To make initial contact with a swarm, each peer must either connect to a tracker using a ".torrent" file, or else use a router computer called a bootstrap node to find a distributed hash table (DHT) which describes a swarm's list of peers. For most BitTorrent users, DHT and PEX will start to work automatically after the user launches a BitTorrent client and opens a .torrent file. A notable exception is "private torrents" which are not freely available; these will disable DHT.
Peer Exchange Conventions
Extensions to BitTorrent such as PEX are typically implemented using one of two common extension protocols, AZMP or LTEP. Both types of peer exchange send messages containing a group of peers to be added to the swarm and a group of peers to be removed.It was agreed between the Azureus and µTorrent developers that any clients which implement either of the mechanisms above try to obey the following limits when sending PEX messages:
- There should be no more than 50 added peers and 50 removed peers sent in any given PEX message.
- A peer exchange message should not be sent more frequently than once a minute.
Some clients may choose to enforce these limits and drop connections from clients that ignore them.
DHT
To create a PEX protocol providing a uniformly-distributed peer selection, one could form a small DHT local to a torrent. For each desired new peer one would look up a (uniformly) random key, and use the node responsible for the key as a new peer. This is conceptually simple but would require quite some overhead.For "trackerless" torrents, it is not clear if PEX provides any value since the mainline DHT can distribute load as necessary. Each DHT node acting as a tracker may store only a subset of the peers, but these are maximal subsets constrained only by DHT node load rather than by a single peer's view. Private torrents disable the DHT, and for this case, PEX might be useful provided the peer obtains enough peers from the tracker.
Versions
There are three incompatible PEX implementation (making distinct "networks" in swarm)- Azureus – introduced in Azureus
- BitComet – proprietary – introduced in BitComet
- MainLine – introduced in µTorrent
Clients supporting peer exchange
Most BitTorrent clients use PEX to gather peers in addition to trackers and DHT. With the 3.0.5.0 release of Vuze, all major BitTorrent clients now have compatible peer exchange.Each of these clients implement some version of peer exchange:
- Ares GalaxyAres GalaxyAres Galaxy is an open source peer-to-peer file sharing application that uses its own decentralized supernode/leaf network. It was spun off from the gnutella network in 2002, and is currently hosted on SourceForge.net. Ares Galaxy has a simple, quick access interface with a built in audio/video...
- aria2, µTorrent PEX support
- VuzeVuzeVuze is a BitTorrent client used to transfer files via the BitTorrent protocol. Vuze is written in Java, and uses the Azureus Engine. In addition to downloading data linked to by .torrent files, Azureus allows users to view, publish and share original DVD and HD quality video content...
, formerly Azureus, and clients based on it (The Vuze PEX is only compatible with the Transmission client. PEX with other clients has been implemented into Vuze and into Azureus from 3.0.4.3 onwards) - BitCometBitCometBitComet is a cross-protocol BitTorrent, HTTP and FTP client written in C++ for Microsoft Windows and available in 52 different languages. Its first public release was version 0.28...
supports PEX using a proprietary protocol in its older versions. Starting with v.1.19 it also added support for the µTorrent/Mainline implementation of PEX, based on the Extension Protocol. - Bitflu
- BitTorrent
- FlashGetFlashGetFlashGet is a freeware download manager for Microsoft Windows...
- Free Download ManagerFree Download ManagerFree Download Manager is a free open source graphical download manager for the Microsoft Windows operating system.FDM was previously proprietary software, but with the release of version 2.5 it has been open source. Starting with version 3.0.850 , source code is no longer provided as a packaged...
- KTorrentKTorrentKTorrent is a BitTorrent client written in C++ for KDE using the Qt user interface toolkit. It is maintained in the KDE Extragear.- Features :*Upload and download speed capping / throttling & scheduling...
has implemented full µTorrent PEX support as of 2.1 RC1 - libtorrent and clients based on it (Deluge, qBittorrentQbittorrentqBittorrent is a free software cross-platform BitTorrent client GUI written with Qt4. The program uses libtorrent-rasterbar C++ library for the torrent back-end functionality. It is developed by Christophe Dumez, from the University of Technology of Belfort-Montbeliard in France...
, MooPolice) compatible with µTorrent - µTorrent
- OperaOpera (web browser)Opera is a web browser and Internet suite developed by Opera Software with over 200 million users worldwide. The browser handles common Internet-related tasks such as displaying web sites, sending and receiving e-mail messages, managing contacts, chatting on IRC, downloading files via BitTorrent,...
9.5, µTorrent PEX support - qBittorrentQbittorrentqBittorrent is a free software cross-platform BitTorrent client GUI written with Qt4. The program uses libtorrent-rasterbar C++ library for the torrent back-end functionality. It is developed by Christophe Dumez, from the University of Technology of Belfort-Montbeliard in France...
, µTorrent PEX support - rTorrentRTorrentrTorrent is a text-based ncurses BitTorrent client written in C++, based on the libTorrent libraries for Unix, whose author's goal is “a focus on high performance and good code”...
- TransmissionTransmission (BitTorrent)Transmission is a BitTorrent client which features a simple interface on top of a cross-platform back-end. Transmission is free software licensed under the terms of the GNU General Public License , with parts under the MIT License.-Features:...
(compatible with both the μTorrent and Vuze implementations) - XTorrent being based on Transmission source code, equally fully supports the Vuze and µTorrent implementations as of version 1.0 (v40)