Ivip - a scalable routing and mobility architecture for the IPv4 and IPv6 Internets

Robin Whittle rw@firstpr.com.au 2011-11-03

To the main IP page

Multihoming service restoration with Ivip


Ivip - Internet Vastly improved Plumbing - is intended to help solve the Internet's routing scaling problem, for both IPv4 and IPv6.  It will also help improve IPv4 address utilization and be a good basis for the TTR Mobility (below) architecture.

I haven't done any work on this since mid-2010, due to the need to focus on paying work.

In the future I intend to return to Ivip and TTR Mobility.  I have not yet documented two new approaches for getting packets from ITRs to ETRs without conventional encapsulation-based tunneling, using existing packet formats (and so being compatible with all existing routers) and without making the packets any longer.  These approaches are related, but different.  They are applicable to an ITR-ETR arrangement as used in Ivip, so they are not a magic general form of tunneling.  The IPv6 approach is essentially the same as what Finnish
wannabe-polymath, libertarian, and all-round nerd Sampo Syreeni http://decoy.iki.fi/ suggested in a personal communication.  Inspired by his insights, I thought of a different approach which would be practical for IPv4.

None of the Ivip documents below mention these two approaches.  Future versions will do so.

Ivip could be used just for a non-mobile scalable routing solution.  More likely, one or more Ivip-like systems using TTR Mobility could be developed and commercially deployed today, for IPv4 and/or IPv6, for the primary purpose of providing mobility of one or more IPv4 addresses or one or more IPv6 /64 subnets, to mobile devices from hand-held cell-phones to ships or aircraft.

In October and November 2011, I am discussing ILNP and the LISP protocol on the main IETF mailing list.  I and others also argue that the LISP protocol should be ranamed.  See also the pages here on the meaning of "namespace" namespace/ and on why the LISP protocol is not
a Locator-Identifier Separation protocol: namespace/lisp-not-loc-id/

Please note that all references in these pages to "LISP" are to the IETF protocol known as "LISP" and not to the programming language. 

The IRTF Routing Research Group's work on Scalable Routing

Scalable routing has been discussed since early 2007 on the IRTF RRG mailing list and in RRG meetings.  It was also discussed on the IETF RAM list, and before that, in late 2006, at the RAWS workshop in Amsterdam.

Here's my quick intro to Scalable Routing, written in November 2011:

The main, or at least original, purpose of scalable routing is "scalable multihoming" of large numbers of end-user (AKA "edge") networks - to allow millions or billions of separate end-user networks to retain their own global unicast address space no matter which ISPs they use for connectivity, without all the world's 200k+ BGP routers having to add a new prefix for that end-user network into their Routing Information Bases and their Forwarding Information Bases.  It is expensive for each router to do so, and there are something like 200k+ BGP routers (no-one knows for sure).   In addition to the cost of these routers going up with the 380k+ and growing http://bgp.potaroo.net prefixes they handle today, this large number of prefixes makes the global BGP system less able to do is job - cope rapidly with failures and reconfigurations.

A second goal of scalable routing is to extend this concept of scalable multihoming for end-user networks not just to university, corporate networks etc. but to home and home-office networks which want this. 

A third goal is to extend this further for seamless mobility for potentially billions of hand-held devices, so they can maintain their communication sessions while changing their point of attachment to the global network.  To do this without changing application programs or host stacks will require making the one or more global unicast IP addresses which the device uses portable, without disruption, to any form of IP connectivity at all.

The LISP protocol, Ivip and IRON are intended to provide a "scalable" approach to multihoming in that it can, ideally, be expanded to millions or hundreds of millions of end-user networks, each with their own permanently assigned, ISP-portable, global unicast address space,
*without* burdening all the world's BGP routers with another prefix on top of the 380k+ and growing prefixes they already struggle with (at great expense to all ISPs and therefore all Internet users.)

In late 2010, the RRG work on scalable routing wound up.  The final report and recommendation is RFC-6115. Some RRG details:
Please see RRG-2010/ for files containing almost all the RRG messages to 2010-09-06 and complete set of RAM list messages from 2006 to 2008.

For people who are new to the IRTF Routing Research Group's discussion of the Internet's routing scaling problem, please see this RRG List message msg00623.html and the Ivip-glossary ID mentioned below for a description of some of the terminology and for an explanation of the above diagram, which depicts multi-homing service restoration.   Please see RRG-2009/ for links to the RRG and to other resources regarding scalable routing and addressing.  My attempt (2010-02-23) to describe the routing scaling problem is in the RRG mailing list archives: Scalable routing problem & architectural enhancements : msg06099.html .

In March, I wrote what I would like to see as an RRG Recommendation:  msg06219.html  This contains my attempt at a concise definition of the design goals, and an evaluation of the proposed architectures.  In late 2010, I stand by what I wrote here except that:
  1. IRON (RANGER) has developed a great deal since then, and I think is more promising. (However, I haven't read the most recent IRON I-Ds.)
  2. ILNP is even less suitable than I thought, since it seems to be impossible to make it work with existing IPv6 applications.  See msg07185.html .
The final report RFC-6115 contains summaries , critiques and responses to those critiques for 14 proposed architectures - with 500 word limits on each, and with only one critique chosen in some cases where more than one was written by RRG participants.  The Report does not contain a recommendation for a future architecture based on consensus of the RRG participants, though such a recommendation was a primary goal of the RRG.  No attempt was made to achieve this, and it would probably have been impossible to obtain even rough consensus.  So the co-chairs made their own recommendation, with ILNP being their long-term solution of choice.The RRG Design Goals I-D, which has not been updated or discussed to any significant degree (despite my efforts to do so) since version 01 in July 2007, was not worked on any further.  See msg07299.html .

My assessment of the co-chairs' Recommendation: msg06353.html .  Consensus is not required in IRTF Research Groups (msg06204).  The slides of Lixia's and Tony's presentation are here:  http://www.ietf.org/proceedings/10mar/slides/RRG-0.pptx and in PDF format: RRG-2010/RRG-2010-03-26.pdf .  The slide 19 material on Ivip is almost entirely inaccurate.  See msg06373 .  Here is a 64kbps MP3 of the 1 hour 13 minute meeting  RRG-2010/RRG-2010-03-26.mp3 , which consisted of the co-chairs' presentation and some questions and comments from the floor.

Ivip Internet Drafts and other important documents



The 04 version of 2010-03-08 covers the new DRTM mapping system.
  Includes a list of goals and non-goals.  Also, a quick description of Ivip's network elements:

ITR        QSC          QSR        QSA      ETR
DITR      (QSCs are
ITFH       Optional)

network-elements/ .

DRTM - Distributed Real Time Mapping for Ivip and (potentially) the LISP protocol

2010-03-26: please see the new description of the major parts of DRTM, with three detailed illustrations:

One of the illustrations is below.  This is the best way to understand the overview of DRTM - best to read it before reading the Internet Draft:


This is a new mapping system which I devised in Feb-March 2010, which is still real-time, but avoids the need for any server having the entire mapping database, and the need for ISPs and other networks with ITRs to run servers which gets real-time mapping feeds.  All the ITRs and query servers needed by ISPs etc. are fully caching devices. 

Significant adoption of scalable-routing-friendly SPI address space can be achieved with little or no ISP investment or involvement.  The initiative is taken by, and the investments are made by, organizations which need not be ISPs.  This requires only DITRs, QSAs and ETRs.  Compared to the original Ivip mapping distribution system, DRTM will be simpler for ISPs etc. to implement - and will be more robust. 

Glossary of Ivip and scalable routing terms 


2010-03-06 - intended to help people come up to speed in this field.

Instead of encapsulation, Modified Header Forwarding for IPv4 


Revised 2010-01-13 with a minor update in 2010-07-08.  With a modest upgrade to all DFZ router FIBs, and by taking liberties with the IPv4 header, we can avoid encapsulation, its overhead and its Path MTU Discovery problems.  This is for the long-term future - it is not an essential part of Ivip for initial introduction. 

(An update is that perhaps it would be better to do the same thing, with a new protocol, with the same header length, and so use the Evil bit too, which would enable it to carry 31 bits of ETR address.  However, in the future I intend to replace this technique with the IPv4 version of the new techniques referred to above, as inspired by Sampo Syreeni's suggestion.)

TTR Mobility


By Steve Russert and myself, August 2008, with minor revisions in January 2010 and October 2011. More on this below: #mobile  TTR Mobility does not require a mapping change whenever the MN gains a new address or uses a new access network, which may be a very frequent occurrence.  So the correspondent hosts do not need to get any new mapping or do anything different when this happens.

My suggestion for a Recommendation

msg06219.html (2010-03-09) Includes a list of goals, analysis of all the proposals and arguments against Core Edge Elimination (CEE, AKA Locator / Identity Separation) architectures.  (For a longer version of goals and non-goals for Ivip, see draft-whittle-ivip-arch.)

This does not cover the current version of Fred Templin's IRON architecture, which he developed extensively developing the middle of 2010 and to early September, when it was being readied to become an RFC.

"Overloading" of Loc & ID functions is good for hosts and should be maintained

msg07017.html (2010-06-22.)  A critique of all Core Edge Elimination (CEE, AKA Locator / Identity Separation) architectures including GSE, HIP and ILNP.  (The LISP protocol is not such an architecture - it is a Core-Edge Separation architecture, as is Ivip and Fred Templin's IRON.)

On 2010-07-08, Dae Young Kim supported this viewpoint (msg07074.html), though I understand his ideal arrangement involves the IP address pointing to a node (host) rather than to one of its interfaces, as it does today.

The RRG's capacity to handle detail is not up to the task of designing architectural enhancements for v4/v6 Internets

msg07043.html (2010-06-26) 

Since I wrote this, which includes a lament on the lack of support for, or critiques of, Ivip, TTR Mobility and some other things I wrote, I have received some encouraging support.  I understand that Fred Templin's IRON architecture now uses some elements of TTR Mobility. 

Also, see Dae Young Kim's feedback mentioned above, and Tom Petch's support (msg06823 & msg07049)  for DRTM and my attempt to document the CEE/CES distinction - by stating he would like to see these written up as RFCs.  Unfortunately, I don't have time to do this in the foreseeable future, but perhaps someone will write up the CES/CEE material in an RFC.

Constraints due to the need for widespread voluntary adoption

We can't force the adoption of a new architecture or architectural enhancement.  So a number of constraints arise, on any proposed enhancement, due to the need for very widespread voluntary adoption, if the enhancement is to be widely enough used to substantially solve the routing scaling problem.


Distinction between Core Edge Separation (CES) and Core Edge Elimination (CEE) architectures

An attempt at a concise description of the distinctions between CES and CEE architectures is in section 17.3.3 of my proposed Recommendation text: msg06219.html .  A more detailed version is: msg06250.html and msg05865.html .

LISP, Ivip, IRON (RANGER) and TIDR are CES architectures.  (TIDR doesn't offer substantial scaling benefits.)

GLI-Split, ILNP, Name-Based Sockets and RANGI are CEE architectures.

I believe the CES architectures LISP, Ivip and IRON are capable, in broad principle, of solving the routing scaling problem, for both IPv4 and IPv6 - and doing so without adding additional burdens on hosts, and without the requirement to change host stack or application software.  The benefits to adopters and to scalable routing in general are in proportion to the adoption of these architectures.

I believe that the CEE architectures could only solve the routing scaling problem for IPv6 - none of them are practical for IPv4.  To do so, they would only provide significant benefits to adopters or to scalable routing once they were adopted by almost all hosts and networks.  Furthermore, to do so, they would require major changes to both the stack and application software of hosts.  (ILNP is claimed to support IPv6 applications via a modified stack, but I can't see a way of making this work - and Tony Li and Ran Atkinson have not shown how it can be done.  (msg07185.html)  All CEE architectures place extra burdens on hosts, in the form of extra complexity of software, extra state and extra packets and/or longer packets.  This also involves delays in establishing initial communications compared to the current arrangement in which the functions of "Locator" and "Identifier" are both performed by one thing: the IP address.  See msg07017.html .

List of attempts at creating a taxonomy of RRG architectures

Here is a list I made on 2010-09-07: msg07307.html .

Ivip's DRTM Distributed Real-Time Mapping system Fig. 3 of 4

Selected developments not mentioned above

I also made a page to record the most recent version of my arguments against hosts being expected to take on any more Routing and Addressing responsibilities, other than those really needed for mobile hosts:  RRG-2009/host-responsibilities/ .  See also a later account of this subject: msg07017.html (2010-06-22.)

In early December 2009 I discussed on the LISP WG list the inability of LISP-ALT to scale well - to provide short paths, with large numbers of EIDs, whilst also being structured for robustness against link and router failure.  The initial message is ALT structure, robustness and the long-path problem: www.ietf.org/mail-archive/web/lisp/current/msg01801.html .  This is a continuation of a critique I first made in late 2007, soon after ALT was announced.  Still no-one has proposed how these problems can be solved.  The attraction of ALT is now said, by some people, to be its ease of implementation (or perhaps design), since it is constructed using existing router functionality.  That's fine, but why bother making an ALT test network if ALT can't scale to billions of EIDs or whatever?  If ALT can't do this, LISP's future must be with some other mapping system - which should be developed and prototyped ASAP.  LISP has no other mapping system under development.  NERD was retired on 2009-11-17 (rrg/current/msg05274.html) and there has been CONS development since ALT was launched in late 2007.

The debate about whether LISP involves separate namespaces for EIDs and RLOCs has fired up again, on the LISP list:  namespace/  Also, I wrote a critique of the LISP Mobile I-D (draft-meyer-lisp-mn-00): http://www.ietf.org/mail-archive/web/lisp/current/msg00749.html.   See the #mobile section below for my suggestion that LISP use the TTR mobility approach rather than the "MN is its own ETR" approach of the recent Mobile LISP Internet draft.

A new page RRG-2009/constraints/ has a draft list of constraints on any scalable routing solution which result from our reliance on widespread voluntary adoption.  Please comment on this in the RRG list.

Added a page namespace/ as part of trying to determine the meaning of the noun "namespace".  This is part of a debate on the LISP list about whether or not statements about LISP involving separate namespaces for EIDs and RLOCs are accurate or not.  This is probably more than anyone ever wanted to know about the meaning of the term, but since the debate has involved the suggestion that "namespace" may have "a lot of other definitions floating around" I tried to find them - without success.

This paper:

Towards a Future Internet Architecture: Arguments for Separating Edges from Transit Core
Dan Jen, Lixia Zhang, Lan Wang, Beichuan Zhang

the authors argue that "core-edge separation" approaches  is a better way of achieving scalable routing and addressing than  "core-edge elimination" schemes.  Core-edge separation schemes are "Strategy A" in Bill Herrin's taxonomy (below) and in the 3.3 taxonomy of the draft RRG recommendations.  Core-edge separation proposals include LISP, APT, Ivip, TRRP and Six/One Router.

Added a page rrgarch/ which attempts to match actual scalable routing proposals to the strategies in Bill Herrin's Summary of Routing Architectures: http://bill.herrin.us/network/rrgarchitectures.html


Forwarding with modified headers: I have devised two systems for getting packets from ITRs to ETRs without encapsulation or address translation.  Both systems require a modest upgrade to most or all core (DFZ) routers and for some or many internal routers - but would make for a much simpler, more efficient system, with no PMTUD problems and simpler ITRs and ETRs. 

ETR Address Forwarding (EAF) - for IPv4

tools.ietf.org/html/draft-whittle-ivip-etr-addr-forw.  It puts the 30 most significant bits of the ETR address in the header bits for:
More Fragments bit.
Fragment Offset.
and marks the packet as being in this EAF format by setting bit 48, the "Evil Bit" tools.ietf.org/html/rfc3514.

Prefix Label Forwarding (PLF) - for IPv6

I don't yet have an Internet draft for this, but the proposal is described in detail here:  PLF-for-IPv6/  .

The current plan is to use a 19 bit value in the proposed new Forwarding Label to get the packet from the ITR to one of 524,287 provider networks to which the new kind of end-user networks connect.  Each such recipient provider network can have effectively unlimited numbers of ETRs.

There is a another page exploring extending this system to be used by edge networks, with each such network, including the just-mentioned recipient provider networks, able to use 2^19 values of the proposed Forwarding Label to get the packet from its border router to any one of 524,287 ETRs, again without any encapsulation, PMTUD problems etc. PLF-for-IPv6/lfe/

Also, a little research page on which upper layer protocols look at bits in the IPv4 or IPv6 headers, in terms of their checksums - and which fields the IPsec Authentication Header looks at:  checksums/ .

Comparing LISP-NERD/ALT, APT, Ivip and TRRP (Does not cover Ivip's  Modified Header Forwarding arrangements.)
See also the rrgarch/ page which attempts to match actual proposals to Bill Herrin's summary of strategies.

Ivip's approach to mobility

TTR Mobility Extensions for Core-Edge Separation Solutions to the Internet's Routing Scaling Problem
Robin Whittle, First Principles, Rosanna, Vic, Australia
Steven Russert (formerly of) Boeing Phantom Works, Seattle, WA, USA
2008-08-25 http://www.firstpr.com.au/ip/ivip/#mobile 
(Minor revisions 2010-01-12 and 2011-10-30.)

Some illustrations (using DITR - Default ITRs in the DFZ - previously known as OITRDs):

TTR mobility extensions for Ivip or other core-edge separation solutions to the Internet's routing scaling problem


TTR mobility extensions for Ivip or other core-edge separation solutions to the Internet's routing scaling problem

Path MTU Discovery and fragmentation problems

On 2008-04-21 I devised a new approach to the PMTUD (Path Maximum Transmission Unit Discovery) problems which all map-encap schemes need to deal with.  Please see:

and some 2010 research by Fred Templin and others: msg05910.html .

Notable RRG list discussions

To-do, but see LISP critiques at  lisp-links/ and the Ivip business models message: psg.com/lists/rrg/2008/msg01158.html .