Comparing LISP-NERD/ALT, APT, Ivip and TRRP

To the main Ivip page

Robin Whittle rw@firstpr.com.au 2008-04-23
(Note 2009-01-03: This covers Ivip's encapsulation modes and does not consider the IPv4 and IPv6 forwarding alternatives.  Please also see the list of proposals which match Bill Herrin's Summary of strategies for solving the routing scaling problem: ../rrgarch/ .)

Introduction

On 2008-04-02 I wrote a Routing Research Group message "Taxonomy: 25 questions" psg.com/lists/rrg/2008/msg01091.html comparing LISP-NERD, LISP-ALT, APT, Ivip and TRRP.  On 2008-04-22 I made this page as an updated version of that message, so far including updates resulting from Bill Herrin's message concerning TRRP psg.com/lists/rrg/2008/msg01094.html hereafter linked to as BH01 . Please see psg.com/lists/rrg/2008/msg01195.html (2008-04-21) where I respond to Bill's input, debate some parts of it and list other changes I made from the original message in order to arrive at what lies below.

At this page, from October 2007 to April 2008 there was a comparison of LISP NERD/CONS, APT and Ivip.  That older material is in archive-1.html and archive-2.html in this directory, but they date from late 2007, don't mention ALT or TRRP, and are probably add little to what follows.

Please discuss this comparison on the RRG list, or by private email.  I will try to keep it up-to-date with correction, new questions etc. and by linking to discussions about it.



Index to questions

All proposals at the RRG wiki:

Q01  Does the scheme require host changes or only work with IPv6?
Q02  No loss of communications ability for any hosts as the scheme is adopted?

LISP-NERD, LISP-ALT, APT, Ivip and TRRP:

Q03  How is mapping delivered to the ITRs?
Q04  How fast?
Q05  End-users pay for each mapping change?
Q06  End-users pay for traffic on their micronet (EID prefix)?
Q07  What information is included in the mapping?
Q08  What functionality is required in ITRs and ETRs?
Q09  Monolithic or modular?
Q10  How does the scheme tackle PMTUD and fragmentation problems?
Q11  Does the mapping and ITR functionality explicitly support load sharing between ETRs?
Q12  Portability of address space in terms of ISPs used for Internet access?
Q13  Support for existing or new forms of mobility?
Q14  Granularity of address management?
Q15  How would the scheme help alleviate IPv4 address depletion?
Q16  Business case for supporting communication with hosts in non-upgraded networks?
Q17  Where are the ITRs located?
Q18  Where are the ETRs located?
Q19  How involved is each ISP's internal routing system? 
Q20  Must every packet to mapped space go through an ITR and ETR?
Q21  What is the outer source address of the tunneled packets?
Q22  Can the sending host traceroute the routers in the ITR--> ETR tunnel?
Q23  Can the ETR support any source address filtering the ISP does on incoming packets?
Q24  Delay in initial packets when caching ITR has no mapping? 
Q25  Alternative system for delivering these initial packets to the ETR?

Q26  <<< Please suggest more questions. 



The five proposals

The best guide to LISP-NERD and LISP-ALT is my own page: ../lisp-links - which also lists major critiques of LISP.

APT has no web page, but please see the APT section of the RRG wiki: www3.tools.ietf.org/group/irtf/trac/wiki/RoutingResearchGroup .

Ivip is described at the parent page ../ .

Bill Herrin's TRRP proposal is described at: bill.herrin.us/network/trrp.html . At present it is not described in any Internet Drafts.


Questions and answers

First, are 2 questions which weed out the impractical proposals and leave us with five potentially practical map-encap proposals.

My criteria for "practicality" includes that the scheme would likely to be widely adopted, without significant disruption and without significant inducements.  Please see item 10 in:

Re: [RRG] What does incremental deployment mean - 2 questions
2008-03-22 psg.com/lists/rrg/2008/msg00957.html

I think criteria such as this must be met by any scheme which would have a hope of being deployed widely enough to significantly reduce the routing scaling problem.  The discussion about "incremental deployability" indicates that many people understand this term to be far less restrictive than my item 10, yet the RRG Design Goals tools.ietf.org/html/draft-irtf-rrg-design-goals-01 requires that the chosen solution be "incrementally deployable".  I think we need some stronger deployability criteria than the common understanding of this term implies.

I see many problems with Six-One Router, including that it requires the routers to do deep packet inspection and modification of traffic packets, to understand all application protocols and rewrite any IP addresses carried in those packets.  This is completely impossible for several different reasons, such as the technical impossibility of doing this with encrypted packets, and of the routers ever being programmed to understand all application protocols.

I considered IPvLX and ISATAP because they were on the RRG list, but I don't think these are really proposals which help with the RRG's task of solving the routing scaling problem.  See the discussions from psg.com/lists/rrg/2008/msg01109.html .

Then, 23 questions which I intended to lead to a taxonomy within which these five proposals sit.  Lixia Zhang (and no-doubt others) had a different view of what constitutes a taxonomy, as we discussed in msg01098.html and msg01099.html .    

I think the answers to these help us to understand the problems and potential solutions.

Q01

Does the scheme require host changes in order for hosts to
participate in it?  This includes adopting dual stack IPv4/v6 and
applications which in fact use IPv6. (A No for this answer means the
scheme could work, in different versions, with both IPv4 and IPv6.)


  APT                              No.

  AIRA                             ?

  CRIO                             No?

  LISP-ALT/NERD                    No.

  IPvLX                            Yes. "long-term IPv6/IPv4
                                   coexistence, with IPv6 addresses
                                   serving as identifiers."

  Anything proposed by
  Heiner Hummel.                   Yes, as much as I understand
                                   his proposals, which seem to
                                   involve radical changes to
                                   routing and addressing.

  ISATAP                           Yes - IPv6.

  Ivip                             No.

  Six/One Router                   Yes - IPv6.

  Six/One                          Yes - IPv6 with host changes to
                                   implement Six/One.

  TAMARA                           Yes - IPv6.

  TRRP                             No.

I think those schemes above for which the answer is Yes should be
mentioned in the Taxonomy in this respect, but not considered any
further - unless perhaps in a completely separate section reserved
for long-term plans not restricted by practical considerations of
being deployable in currently foreseeable circumstances.

Trying to solve the IPv4 routing scaling problem by migrating users
to IPv6 addresses only (no IPv4 address at all) is nowhere near a
practical solution.  There is no significant adoption of IPv6 at
present amongst general Internet users and no such user could
survive for a moment without their IPv4 address, because without it
they cannot communicate with the vast majority of other hosts.

Furthermore, there
is no routing scalability problem in IPv6 now, so
IPv6 routing
 scalability, lack of multihoming (now /48 PI space is
available)
etc. is not a problem restricting the adoption of IPv6. 
So a
proposal which proposes to solve only the IPv6 scaling problem
is of
no importance in the foreseeable future - only in the long
term if
and when IPv6 is ever really widely used.

Q02

For the proposals for which the Q01 answer was No, do they aim to
provide a mechanism by which there is no loss of connectivity for
any hosts, as the system is deployed, including for communications
to and from networks with no upgrades?

  APT             Yes.  However, there are limitations on the
                  support for EIDs, which might be resolved if APT
                  used allowed a somewhat different type of link
                  between APT networks, so there would naturally be
                  only one global APT system, rather than multiple
                  APT islands.  See my message "APT: no need for
                  islands?" 2008-03-12:
                  psg.com/lists/rrg/2008/msg00734.html

  AIRA            No - there are two separate namespaces for
                  Identifiers and Locators.  That means that hosts
                  using the new Identifiers for their own addresses
                  won't be able to communicate with hosts which are
                  not upgraded.

  CRIO            This is a complex proposal I have very little
                  understanding of, which has not been discussed
                  much recently in the RRG, and for which no RRG 8
                  page summary and analysis has been prepared.
                  AFAIK, it involves radical address changes
                  and so I think the answer is No.

  LISP-ALT/NERD   Yes - "Proxy Tunnel Routers".


  Ivip            Yes - OITRDs (Open ITRs in the DFZ, formerly
                  "anycast ITRs in the core":
                  psg.com/lists/rrg/2008/msg00937.html

                           
  TRRP            Yes, in a stepwise change of the longest prefixes
                  which are advertised in BGP, starting with /24,
                  and then going to /23, /22 etc.  See BH01 for a
                  proper description.




This leaves five proposals which have a chance of meeting my message 957 (10) criteria (ignoring LISP-CONS, which I think is on the back-burner):

  APT
  Ivip
  LISP-NERD
  LISP-ALT
  TRRP

Since all other proposals have no hope of being practical, I think rest of the Taxonomy should only consider these five proposals - and any new proposals or variants which pass muster in the above two questions.

Here are some Taxonomy questions which I think are illuminating:

Q03

How is mapping delivered to ITRs?

  APT        Hybrid push-pull.

             Push to Default Mappers.
             Pull to caching ITRs from Default Mappers.

  Ivip       Hybrid push-pull.

             Push to full database ITRs (ITRDs) and to full
             database Query Servers (QSDs).

             Pull to caching ITRs (ITRCs), ITR functions in
             sending hosts (ITFHs - not behind NAT) and
             caching Query Servers (QSCs) from QSDs and QSCs.

  LISP-NERD  Pure push.  (Not expected to be scalable to
                          hundreds of millions or billions of
                          EIDs.)

  LISP-ALT   Pure pull.  However, in the Friday 14th March 2008
             RRG meeting (for which I have some sound recorded,
             but not all - and for which there is no archive or
             sufficiently detailed minutes, Dino Farinacci spoke
             of ETRs sending some kind of map solicitation
             request to ITRs which are currently sending it
             packets, so that those ITRs will request fresh
             mapping information.  This is arguably a form of
             push, although directed only where needed.  It could
             overcome some limitations in ALT, but I can imagine
             it has its own limitations.  To-do: find out more
             and write a critique on the RRG list.
 

  TRRP       Pure pull.  However, there are two methods by
             which the ITR could be given new mapping information.
             See the TRRP section of Q04 for more details.             

Q04

How fast can an end-user's mapping change be sent to all ITRs?

(See my message:

    Mapping model discussion - push, pull, hybrids
and notify (2008-03-11)
    psg.com/lists/rrg/2008/msg00707.html


and
    Notify = "proactive" updates; flexible placement of full db
   
ITRs & query servers  (2008-03-13)
    psg.com/lists/rrg/2008/msg00707.html

       

  APT        Slowly.  Depends on slow flooding protocol and the
             ability of Default Mappers to Notify ITRs of
             changed mapping.  No times are mentioned, but I
             recall the earlier version, with BGP flooding,
             anticipated the mapping not changing for days or
             months.

  Ivip       Fast - ideally 5 seconds.  Fast push to ITRDs and
             QSDs.  Immediate Notify from QSDs (via any intermediate
             QSCs) to all ITRs which might be sending packets
             for the micronet with changed mapping (according
             to the caching time of map replies given out by
             the QSD): ITRCs and ITFHs.

  LISP-NERD  Slowly.  Depends on frequency by which central
             servers files are updated with user mapping
             changes and then on how frequently each ITR
             downloads all the most recent files, or update
             files.

  LISP-ALT   In practical terms, slowly, such as minutes or
             tens of minutes.  Depends primarily the caching
             time of map replies - and shorter times means
             more queries, responses etc.  There is no Notify
             (ITR cache invalidation) mechanism.

  TRRP       I think:
               
                As for LISP-ALT, since both use a global query
                
server network.  TRRP doesn't have a workable
                Notify system.  It has a limited cache
                invalidation
system where an unreachable ETR
                leads to an ICMP
message to an ITR, which can
                then issue a new
mapping request.

                
bill.herrin.us/network/trrp-preempt.html

                See my critique (2008-02-27):
                psg.com/lists/rrg/2008/msg00532.html

                I haven't yet fully understood the following:

             Bill wrote
BH01:
               
                Notification is an optional component.  The
                system is designed to work acceptably without
                it but work faster with it.

                Without notifications, the maximum effective
                change rate per map is about once every 10
                seconds.  That interval has enough overhead
                that folks who are just doing multihoming
                service restoration will normally back it off
                further.  For every communication where
                notification is supported at both the ITR and
                map server, TRRP can in the average case
                switch to the new map within a fraction of a
                second after the map is published. 

Q05

Would end-users pay for mapping changes?

  APT        I don't think so, but ISPs would be annoyed at
             constant flurries of mapping changes tying up
             their flooding system.

  Ivip       Yes - a few cents, I guess.  
             See my later message, 2008-04-18:
             Ivip business models: fast push & OITRDs
             psg.com/lists/rrg/2008/msg01158.html

  LISP-NERD  I guess not, but again, if whoever runs the
             central server gets sick of some end-users
             constantly changing their mapping, some fees
             would be charged.

  LISP-ALT   No.  The end user runs their own authoritative
             query servers in a pure pull global network.
             They can change mapping as often as they like
             without burdening anyone.  But what about the
             burden of mapping replies with short caching
             times?

  TRRP       As for LISP-ALT.

Q06

Does the end-user have to pay for traffic directed to their micronet
/ EID prefix?

  APT        ?

  Ivip       Yes - the RUAS runs the OITRD ITRs in the DFZ, and can
             be expected to charge end-users according to what
             traffic these ITRs handle.  However, this is probably
             just a server sitting at an Internet exchange, so the
             cost would be a lot less per gigabyte than for
             connectivity via an ISP.

  LISP-NERD  } Not with current proposals - but these proposals
  LISP-ALT   } have no business plan for Proxy Tunnel Routers

  TRRP       Bill wrote BH01:
             
                I expect the answer is yes, but it's not formal part
                of the architecture.  The most accurate answer is
                that the end-user can do whatever he wants to do.
                If he gets acceptable connectivity without paying
                someone to reroute traffic his direction from a BGP
                supernet route then the answer is no.  If he doesn't,
                then something like:
                   
                   bill.herrin.us/network/trrp-aapip.html

                [Waypoint Routers - a faster method of delivering
                 initial packets - RW] or

                   lists.arin.net/pipermail/ppml/2008-March/010475.html

                comes in to play and he'll  pay the aggregator.

Q07

What information is contained in each micronet's mapping (each EID
prefix's mapping)?

  Ivip       The ETR address.

  APT        } One or more ETR addresses, with weights and
  LISP-NERD  } priorities to enable each ITR to implement
  LISP-ALT   } multihoming service restoration and/or
  TRRP       } inbound load sharing (explicit TE).


Q08

Therefore, what functionality is required in the ITR and ETR?

  Ivip       No reachability functionality, except as a by-product
             of Ivip's inbuilt PMTUD and fragmentation management
             system (yet to be finalised, but see my PMTUD-frag
             page).

  APT        } ITRs must individually determine reachability
  LISP-NERD  } of each ETR and whether the ETR can reach the
  LISP-ALT   } destination.  Also, I think in LISP, the ETR
  TRRP       } may, or must, somehow tell ITRs about reachability
               of other ETRs which are in the mapping.  (But
               how does the ETR know exactly what mapping the
               ITR has??)

             
Bill wrote BH01:

                For TRRP, the ETR need only be a dumb GRE endpoint
                with an acl specifying the destination addresses
                which are allowed to use it.  It may do more than
                that and you can get better results if it does, but
                dumb GRE endpoint is the base requirement.  The ITR
                is responsible for finding currently valid ETRs in
                a timely manner (primarily by looking them up from
                the map server) and engaging in much of the complex
                activity.

Q09

Therefore, is the system a monolithic approach, permanently
integrating reachability and multihoming service restoration
decision-making functions in the map-encap scheme, or enabling and
requiring end-users to supply their own reachability and
decision-making systems?

  Ivip       Modular = non-monolithic.

             End-users must do their own reachability testing
             and make their own decisions on how to restore
             service.  (They would typically do this by hiring
             some company to do it however they want it done.)


  APT        } Monolithic system.  Since all these do not
  LISP-NERD  } get end-user mapping changes to all ITRs
  LISP-ALT   } quickly, the map-encap system must do all
  TRRP       } reachability testing and make all decisions.
   |           ITRs and ETRs must do this, using relatively simple
   |           mechanisms and mapping data which are hardwired into
   |           the entire map-encap system.  This will not
   |           satisfy all needs of end-users, since it is
   |           impossible to anticipate all the ways end-users might
   |           want to test reachability and since it is impossible
   |           to define exceedingly complex or extensible
   |           arrangements into the map-encap scheme itself.
   |           Also, ITRs acting alone are a poorly scalable and
   |           inefficient mechanism for testing reachability and
   |           for making decisions.
    \
     \_______  
Bill wrote BH01:

                 This is not an accurate description for TRRP.  TRRP
                 requires end-users to supply their own reachability
                 and decision-making systems.  The ITRs have some
                 flexibility based on local knowledge but they're not
                 expected to be able to be able to make complete
                 routing decisions on their own.
 
                 The workings of the decision-making systems are
                 intentionally not a part of TRRP's specification.  I
                 foresee them starting as simple pingers and advancing
                 to something akamai-like.

              My response:  OK - it seems I have misunderstood TRRP
              in this important respect.  I will try to understand it
              this way in the future, but will have to discuss this
              more with you.

              (I proposed that Bill and I discuss this on the list.)


Q10

Does the proposal tackle the PMTU and Fragmentation problem?

  APT        No.

  Ivip       Yes.  At the time of writing the 25 question
             message, this was not finalised.  Now there is a
             much more complete solution:


                www.firstpr.com.au/ip/ivip/pmtud-frag/


  LISP-NERD } Yes, but this is subject to critiques which have
  LISP-ALT  } not yet been answered:

                psg.com/lists/rrg/2008/msg00678.html
                psg.com/lists/rrg/2008/msg00687.html
                psg.com/lists/rrg/2008/msg00692.html
                psg.com/lists/rrg/2008/msg00694.html etc.
 
             

  TRRP       Bill BH01 refers to the Fragmentation section of:
             
                bill.herrin.us/network/trrp.html

             but this looks inadequate to me.

             For instance, there is no suggestion of how the ITR
             can discover the Path MTU to each ETR.

             Also, the suggestion that the ITR alter traffic
             packets to adjust TCP MSS in any SYN packets sounds
             really undesirable and impractical to me.  
 

Q11

Does the system involve explicit TE?

  Ivip       No.  Load sharing must be done by splitting the traffic
             over multiple micronets (each of which can be a single
             IP address) and then mapping the micronets to different
             ETRs, so the traffic is split over multiple links to
             the provider networks with these ETRs.

             This can be changed (for a small fee per update) in
             real-time, which may make it more useful - at least
             in some settings - than the slowly controlled, explicit
             TE functions of the other schemes.


  APT        } Yes - they all use a similar method of specifying
  LISP-NERD  } the share of traffic to be sent to two or more
  LISP-ALT   } ETRs, by each ITR operating independently - except
  TRRP       } that in the case of APT, the Default Mapper makes the
               splitting decision and tells each querying ITR a
               single ETR address to use.

Q12

Does the system provide complete portability of the end-users' mapped
address space?  This is in terms of which one or more ISPs they choose
for connecting their network to the Net.  
 

  Ivip       } Yes - as long as the end-user network uses an ISP
  APT        } which has an ETR and which will accept outgoing
  LISP-NERD  } packet with the source address being within the
  LISP-ALT   } end-user's micronet (EID prefix).
  TRRP       } For ISPs which don't do this, Ivip's mobility
               system, which requires the end user to be a
               customer of a TTR network, will provide forwarding
               of the end-user's outgoing packets.

A related question which I may add later is how independent the end-user
can be of the organisation through which they get their address space.
For instance, if they wanted to keep their address space, but pay someone
else for it, or for handling their mapping changes, or for handling the
traffic addressed to this address space in OITRDs (Ivip) or PTBs (LISP),
what restrictions do they face.  For Ivip, please see:

   Ivip business models: fast push & OITRDs (2008-04-18)
   psg.com/lists/rrg/2008/msg01158.html


Q13

Does the system aspire to support mobility, and if so, how?

  Ivip       Yes, without any extra complexity for the basic
             map-encap requirements for solving the routing
             scalability problem. See:

                www.firstpr.com.au/ip/ivip/#mobile

             which has new diagrams.  This is a new approach to
             mobility.  See message 994 for using these "mobility"
             techniques as a lightweight approach to multihoming
             a fast, generally reliable, but expensive fibre link
             for a moderate sized commercial end-user network.

  APT        } No specific mechanisms for supporting new
  LISP-NERD  } approaches to mobility.  Should work fine
  LISP-ALT   } with existing mobile-IP, although any interactions
  TRRP       } have not yet been explored in detail.

             All these map-encap systems can support the TTR
             approach to mobility:

                www.firstpr.com.au/ip/ivip/#mobile

             but I think only Ivip promises to provide fast changes
             to the mapping for all ITRs which need it, which would
             make the mobility system much more attractive.

             Bill discussed TRRP's support for mobility:

                psg.com/lists/rrg/2008/msg00766.html

             but I don't yet know how fast it could get fresh
             mapping to all ITRs, since I don't believe either
             of the TRRP approaches to Notification are really
             usable.  


Q14

What is the granularity of address management?

  Ivip       IPv4 addresses or IPv6 /64s.  Within these
             increments, each micronet has an arbitrary starting
             point and length (except that the maximum length
             of an IPv4 micronet is 2^24).  This is not yet in
             the Ivip IDs, but is described in message:

                
psg.com/lists/rrg/2008/msg00958.html

  APT        } Not yet clear, but I think they can all support
  LISP-NERD  } EID prefixes starting at any IPv4 address or
  LISP-ALT   } IPv6 /64, in CIDR prefix format and therefore
             } powers of two lengths on binary boundaries.

  TRRP      
Bill wrote BH01:
             
                 The primary granularity is one IP address.  
                 Administrative grouping can be arbitrary; its an
                 implementation detail in the map server.

                 Efficiencies are gained at power-of-two
                 boundaries, 4-bit boundaries and 8-bit boundaries.
     
             I think this relates to how many DNS servers the
             ITR needs to query before one of them responds with
             authoritative mapping.


Q15

What role might the scheme play in the IPv4 address depletion
problem, other than the long-term possibility of helping make IPv6
more attractive in various ways, including by providing a solution
to the multihoming, routing scaling problem etc. in IPv6?

  Ivip       Intended to help via lower-cost, more flexible,
             management of finer slices of address space, including
             individual IPv4 addresses.  This will enable more
             end-user networks to have MH, portability etc.
             and will enable IPv4 space to be utilised significantly
             more efficiently.  This capability does not involve any
             additional complexity over what is required for solving
             the routing scalability problem.

  APT        } I think they would all be nearly as good as Ivip
  LISP-NERD  } for helping improve IPv4 address utilization.
  LISP-ALT   } Ivip's finer control over micronet length makes it
  TRRP       } somewhat better in this regard.

               I think none of them have better IPv4 address
               utilisation as an explicitly goal.

               Nonetheless, any proposal which solves the routing
               scalability problem for IPv4 must surely be
               improve the utilization of IPv4 address space to
               some degree.  Since they could all do this, with
               potentially smaller chunks than BGP can do (256
               addresses at present) I think they are all capable
               of making a big difference to IPv4 utilization and
               therefore to make a significant contribution to the
               IPv4 address depletion problem.

(This won't solve the problem - just extend it to give more time for
IPv6 or some other alternative to become attractive and widely used.
Alternatively, it will enable more of us to bury ourselves more
deeply entangled in IPv4 and so make the prospect of something else
even more unthinkable!)

Q16

What is the business case for the proposal's mechanisms for
supporting communication between hosts in upgraded and non-upgraded
networks?

  APT        No explicit business case, I think, other than general
             benefits due to APT.  I am not quite sure how it
             works, but I think networks in APT islands have all
             their border routers advertising prefixes to the BGP
             system which cover APT-mapped EID prefixes of end-user
             networks which use ISP networks inside the island.
             So what traffic flows where, and why should one ISP
             carry and advertise to non-APT networks for traffic
             which will be sent to another ISP for that second ISP's
             customer?

  Ivip       For the business case for RUASes deploying OITRDs:

                "Open ITRs in the DFZ" (OITRDs), formerly "Anycast
                ITRs in the core"  2008-03-21
                psg.com/lists/rrg/2008/msg00937.html  

                Ivip business models: fast push & OITRDs  2008-04-18
                psg.com/lists/rrg/2008/msg01158.html
  
     
  LISP-NERD  "Proxy Tunnel Routers" - there has been no discussion
             of the business case for these in NERD specifically.

  LISP-ALT   Recent long discussions in the RRG seem to indicate
             the LISP designers have no business case in mind for
             "Proxy Tunnel Routers":

               psg.com/lists/rrg/2008/msg00963.html
               psg.com/lists/rrg/2008/msg00965.html

             As far as I know, the alternative - LISP-NAT -
             won't work.  See my critique of LISP-NAT on
             2007-12-01:

               psg.com/lists/rrg/2007/msg00674.html

  TRRP       I don't recall the details right now, but
             bill.herrin.us/network/trrp-implementation.html
             mentions various carrots and sticks.


Q17

Where are ITRs placed?

  APT        Default Mappers are in ISP networks, perhaps at
             border routers if I remember correctly.  All other
             ITRs are caching ITRs and I think they can be located
             at Provider Edge routers, or is it Customer Edge, or
             both, or in other places too?

  Ivip       OITRDs in the DFZ.  ITRs can be pretty much anywhere
             in provider and end-user networks, as long as they
             have an ordinary ("RLOC" = public, stable, conventional
             BGP managed) address.  Also, ITFHs in sending hosts
             with such addresses.

  LISP-NERD  } PTRs in the DFZ, and ITRs where?
  LISP-ALT   }

  TRRP       I wasn't sure, but Bill wrote BH01:

                 "Anywhere." The ITR can be on any originating host
                 (or nat gateway) with a public IP address, way
                 deep in the core, around the other side being fed
                 by a supernet route or anywhere in between.  It's
                 more cost effective to place them close enough to
                 the edge that they only deal with sub-gigabit
                 traffic flows, but not mandatory.  

             TRRP, like Ivip, enables the sending host to be its
             own ITR, as long as it is on a public IP address.

             I think this includes addresses which are mapped by
             TRRP or Ivip.

Q18

Where are ETRs placed?

  APT        The same devices as ITRs? (Not counting Default
             Mappers, which are ITRs too.)

  Ivip       ETRs can be pretty much anywhere in provider and
             end-user networks, as long as they have an ordinary
             ("RLOC" = public, stable, conventional BGP managed)
             address.

  LISP-NERD  } ETRs are generally in the same devices as ITRs
  LISP-ALT   } except PTRs?

  TRRP       I wasn't sure, but Bill wrote BH01:

                 The ETR can be anywhere that can natively reach
                 the tunneled destinations via a direct connection
                 or an IGP.

             See Q19 and Q20 for more on whether all packets must
             go through an ITR and the ETR.


Q19

Does the internal routing system of ISP networks need to know about
each end-user network's micronet (EID prefix)?  In other words, can
an ETR in an ISP network put out raw, decapsulated, traffic packets
it receives from an ITR and expect the internal routing system to
take these packets to the proper destination?  See Q18 and Q20 too.

  APT        ?

  Ivip       No.  ETRs need some kind of explicit link to get their
             decapsulated packets to the destination.  This could
             be a physical link, or a tunnel of some kind.
             (TTRs are special ETRs for mobility which do this via a
             2-way tunnel established by the Mobile Node.)

  LISP-NERD  } ? I guess so.
  LISP-ALT   }

  TRRP       I wasn't sure, but Bill wrote BH01:

                 In part, it depends on the ETR placement.  The ISP
                 only needs to know a route if the ETR is at the ISP.
                 If the ETR is deployed at the customer premise then
                 the ISP doesn't need an IGP route to it.

                 The ISP will need to know not to drop packets with
                 the micronet's source addresses.


Q20

Does every packet addressed to a micronet (EID prefix) have to go
through an ITR and an ETR?  If the answer is No to this question,
then it means that the proposal expects packets to be delivered to
the correct ETR not just by the map-encap scheme, but also by the
local routing system of any ISP network which is configured to have
the end-user's prefix in its local routing system.  Where two ISP
networks have ETRs for the one end-user network, and they rely on
the internal routing system to deliver raw decapsulated packets from
the ETR to the proper point which links to the end-user network,
then each such ISP must have the micronet / EID of the end-user's
network in its internal routing system.  For consistent operation,
this means the internal routing system needs to perform the same
functions as an ITR, including deciding whether this ETR is
reachable and has a connection to the end-user network (and if not
so, then tunneling the packet to another ISP's network and its ETR,
which requires an ITR function).

  APT        ?

  Ivip       Yes.

  LISP-NERD  } ?
  LISP-ALT   }

  TRRP       Bill wrote BH01:

                 As with Q19.

             I hope to discuss this further.


Q21

What is the source address of the outer header of the packets in the
ITR-ETR tunnel?

  Ivip       Sending host's address. However, the IPTM - RPD2 PMTUD
             probing system sends some packets with the outer source
             address set to the ITR's address.


  APT        } ITR's address.
  LISP-NERD  }
  LISP-ALT   }
  TRRP       }


Q22

Following the above answers, can the sending host traceroute through
the tunnel?

  Ivip       Yes, with a somewhat modified traceroute program which
             would recognise the outer header in the initial bytes
             of the packet which are returned in the ICMP message. 
             This program would be able to detect every router in
             the tunnel and would be able to identify the routers
             which act as
the ITR and ETR.

  APT        } No, unless the ITR did heroics to convert the ICMP
  LISP-NERD  } messages it receives into ICMP messages with the
  LISP-ALT   } correct data for a standard or modified traceroute
  TRRP       } program on the sending host to recognise.
    |
     \____   
Bill wrote BH01:

                
Note #1: as the originating host may function as a
                TRRP ITR, a modified traceroute would also work for
                TRRP.  TRRP's traceroute issues are noted here:
                   
                    bill.herrin.us/network/trrp-breaks.html

                Note #2: ICMP unreachables as commonly implemented
                include enough of the original packet that the ITR
                can translate it into valid unreachable for the
                source with no particular heroics.  Not every case
                to be sure and the standards don't require it, but
                when I looked at the packets they usually had enough
                data.

             If this is true to a large extent, then maybe the same
             is true for Packet Too Big messages from these same
             routers.  If so, then this would be contrary to my
             arguments about it being impractical to have ITRs
             respond to PTBs from the tunnel in a way which
             reliably created a PTB the Sending Host would
             recognise:

                 www.firstpr.com.au/ip/ivip/pmtud-frag/
                 www.ietf.org/mail-archive/web/ram/current/msg01766.html  
                 www.ietf.org/mail-archive/web/ram/current/msg01769.html


Q23

Following the answer about outer source address, how can an ETR
perform the same filtering on packets it decapsulates as an ISP
border router might perform on packets coming into the ISP network -
specifically, rejecting any packets from outside the ISP network
which have source addresses matching any internal prefix of that
network?  See the note on filtering #filtering for more on this.

  Ivip       The ETR simply rejects any inner packet which has a
             different source address than the source address
             of the outer header.  See "ETRs checking src & dest
             addresses", RAM list, 2007-07-12:

            
www.ietf.org/mail-archive/web/ram/current/msg01691.html


  APT        } The ETR would need to do a complete, expensive,
  LISP-NERD  } filter operation on the inner source address
  LISP-ALT   } against whatever list of prefixes the ISP's
  TRRP       } border routers use - which could be huge in a
    |          large ISP network.
    | 
     \_____  
Bill wrote BH01:

                With TRRP, the ETR need only filter for the
                [destination end-user network - RW] addresses
                which it serves.

                This could be an ISP's entire border list but it
                could also be a customer's single micronet or
                anything in between depending on where you place
                the ETR. 

             See further text from Bill and discussion in the
             note below on
#filtering .


Q24

To what extent would packets be delayed when the ITR has no mapping
for the micronet (EID prefix) it is part of?  (Not counting
occasional failures due to lost packets in local networks.)

  APT        Very short - a few 10s of milliseconds I guess, since
             all caching ITRs send their packets to a local Default
             Mapper, with both tunnels the packet to the correct
             ETR without delay, and sends back an ETR address for
             the ITR to use.  (Does the map reply include a
             specification for the EID prefix, so the ITR doesn't
             have to ask again when it gets a packet for a nearby
             address which is part of the same EID?  If so, then
             this is good in many ways.  But if so, then how well
             will load sharing work, since the ITR would send
             all its packets to the ITR nominated by the Default
             Mapper?)

  Ivip       No delay if the ITR is an ITRD.  Very little delay
             if it is an ITRC.  This depends on how close the
             nearest QSD is, though a QSC may have the mapping
             and respond without having to ask an upstream QSD.
             A few 10s of milliseconds I guess.  However, caching
             ITRCs in the same rack as QSDs, connected by an
             Ethernet cable or two, will get their mapping in a few
             milliseconds.

Apart from potential trouble with NTP packets (which could be fixed
by the NTP client sending an initial packet to get the ITR up to
speed, and then a timing-related packet a few seconds later), I
think both APT and Ivip would involve initial packet delays which
are firstly "no problem" for end-users and secondly so low that it
would be unlikely for them to be credibly trumped up into something
which appears significant to end-users, by some anti-APT or
anti-Ivip marketing campaign.

  LISP-NERD  No delay.

  LISP-ALT   Significant delays due to the global nature of the
             query system - fractional second at worst.  PLUS
             a significant risk of a long delay due to the
             possibility of the request or reply packets being
             dropped.  PLUS an often much longer delay in the ALT
             network than would be indicated by geographical
             distance due to ALT's insistence on strong
             aggregation.  See: "ALT's strong aggregation often
             leads to *very* long paths" 2008-01-28:
   
                 
psg.com/lists/rrg/2008/msg00229.html

             and the
long discussion which followed. 

             The closest thing
to a defense against this critique
             was Scott's
message:

                 Re: Why delaying initial packets matters 
2008-03-04:
                 psg.com/lists/rrg/2008/msg00584.html


             and my response to this towards the end of:
 
                 
psg.com/lists/rrg/2008/msg00791.html

  TRRP       Significant delays due to the global nature of the
             query system - fractional second at worst.  PLUS
             a significant risk of a long delay due to the
             possibility of the request or reply packets being
             dropped.  PLUS often longer delays due to the need
             to perform recursive DNS queries.  See:

                Delays
 inherent in TRRP's DNS-like lookup?" 2008-02-28:
                
psg.com/lists/rrg/2008/msg00450.html

             and
the discussion which followed.

Q25

Does the proposal have an alternative system for delivering packets
to ETRs for which the ITR currently has no mapping?

  APT        No - the caching ITR or the Default Mapper always
             has the mapping.

  Ivip       No - the packet is always handled by a full database
             ITR or a caching ITR which has a reliable, close
             full database Query Server.

  LISP-NERD  No - all ITRs are full database.

  LISP-ALT   Yes.  The ALT network delivers the packet to the.
             correct ETR, where it also functions as a mapping
             request.  But the ALT network is global and likely
             to involve very long paths, and therefore delays.
             See my critique mentioned in Q24.




  TRRP       Yes - Waypoint Routers.  But see my critique:

                 
psg.com/lists/rrg/2008/msg00475.html



Note on filtering of source addresses

Note in addition to Q23, with some material from Bill Herrin.

Even if the end-user runs their own ETR, the ISP - or ISPs - who

provide connectivity are likely to want to ensure there is some
source address filtering on the decapsulated packets.

Without the map-encap scheme, the ISP can filter at its border
routers and so prevent any attacker (implicitly, attackers are
outside the ISP and end-user network) from sending packets into
either network with the source address spoofed to be an address
inside the ISP network.

Since it is improbable that any ETR will have the grunt to do source
address filtering on decapsulated packets according to a long list
of prefixes in the ISP's network (except in the case of Ivip, which
does it automatically, with minimal effort and without having to
know the list of prefixes to filter, by dropping inner packets with
source addresses which do not match the outer source address), the
other map-encap proposals
would lead to the end-user network being
wide open to packets with
spoofed source addresses.

An attacker could send a packet from anywhere to the ETR and the
decapsulated packet could have the source address of any host in the
ISP's network.  This is likely to be a security concern, if only
because an attacker could prompt the destination host to send
packets to the host at that spoofed address.

Bill wrote BH01:

    I think this concern fits in a different location than you think.

    Addresses used within an end-user's network are out-of-scope for
    the ISP and always have been.  A user can generate bad packets
    just as easily now as they can after deploying an ETR.
 
    The ISP needs to assure that the source addresses on packets
    reaching them from the user may legitimately do so.  That's not
    so problematic at the ETR but it becomes a problem in two other
    locations:

       1. The ITR.

          The tunnel packet's source may be valid but the
          encapsulated packet's source may not be.  I gather this
          doesn't apply to Ivip but it certainly applies to all the
          rest.

      2. The ISP border with his upstream.

         Filtering for the ISP's 10 prefixes that change once a year
         is not so problematic.  Filtering for the hundreds of
         downstream micronets that change weekly is.  This problem
         will appear with any solution that attempts to restore
         provider-independent addresses to the masses, not just
         map-encap.

OK - I will think about this more in the future.
.