Overlay network
An overlay network is a computer network which is built on top of another network. Nodes in the overlay can be thought of as being connected by virtual or logical links, each of which corresponds to a path, perhaps through many physical links, in the underlying network. For example, many peer-to-peer networks are overlay networks because they run on top of the Internet. Dial-up Internet is an overlay upon the telephone network.
Overlay networks can be constructed in order to permit routing messages to destinations not specified by an IP address. For example, distributed hash tables can be used to route messages to a node having specific logical address, whose IP address is not known in advance.
Overlay networks have also been proposed as a way to improve Internet routing, such as through quality of service guarantees to achieve higher-quality streaming media. Previous proposals such as IntServ, DiffServ, and IP Multicast have not seen wide acceptance largely because they require modification of all routers in the network. On the other hand, an overlay network can be incrementally deployed on end-hosts running the overlay protocol software, without cooperation from ISPs. The overlay has no control over how packets are routed in the underlying network between two overlay nodes, but it can control, for example, the sequence of overlay nodes a message traverses before reaching its destination.
Kademlia has (in my taste, that's why I decided to drop it) a bad hole which makes it's use in remote querying for packets pretty useless: When you have key->value mappings, which have an equal key, in even in semi-large networks it gets very unprobable that you get all mappings, the more hosts there are, the less probable it is, and the more mappings there are, the less probable it is too. I've tried to remedy this in a later implementation, at the cost of lookup speed, but have never managed to get all entries when the network had over 100 nodes. (the later implementation is based on my own server-framework and uses no threads, btw.) This made it unusable for me, as basically I basically had to change the lookup-algorithm to query all nodes (back to gnutella, then...), to get all answers. And that's what is important in the network I designed it for.
Tapestry offers fault-resilient mechanisms for both object location and point to point message delivery. For object location, Tapestry generates a small number of deterministic and independent GUIDs for each object. An object server publishes an object's availability on all of its GUIDs, and a client issues Tapestry locate requests for all of the object GUIDs in parallel. This greatly increases availability under fault conditions, while improving general performance and reducing performance variability. For point to point message delivery, Tapestry provides pre-calculated backup routes at each overlay hop. UDP soft-state beacons measure up-to-date link conditions. Tapestry uses such information and simple fault-avoidance protocols to route messages around failures, providing successful delivery with high probability if a path between the endpoints exists.
For example, Akamai Technologies manages an overlay network which provides reliable, efficient content delivery (a kind of multicast). Academic research includes End System Multicast and Overcast for multicast; RON (Resilient Overlay Network) for resilient routing; and OverQoS for quality of service guarantees, among others.
List of overlay network protocols:
* Distributed hash tables
* Many peer-to-peer protocols including Gnutella, Freenet and I2P
* Solipsis: a France Télécom system for massively shared virtual world
* Internet Relay Chat
See also:
OverLay Internet Solutions "Emotion Link" (FreeBit Co., Ltd. in Japan )