Extending the IPv6 Label Forwarding idea to Edge Networks

To the parent page (Prefix Label Forwarding (PLF) - Modified Header Forwarding for IPv6)

I intend to write this up as an ID, but for now, this page and its parent is the best account of this work in progress.

Robin Whittle rw@firstpr.com.au 2008-08-06

Short version

I contemplate how the proposed 20 bit Forwarding Label in the IPv6 header (currently the unused Flow Label) could be used with one half of the range reserved for forwarding packets in the core (primarily or solely for the Ivip6 scalable routing solution) with the other half of the range to be used in a similar way, with greater flexibility, inside any edge network which choses to do so.  Each such edge network would have its own way of using this half of the range, so it would be purely for forwarding packets within one edge network (unless several coordinated their routing systems and their use of this part of the Forwarding Label Range.

The most obvious use I can think of for this Label Forwarding in Edge networks is to forward Ivip6 traffic packets from the border router which receives them from ITRs via the core, to the correct ETR, wherever that is located inside this recipient provider network.  Being able to use Label Forwarding like this inside the recipient provider network could be a lot more efficient, and have greater flexibility, no PMTUD problems etc, compared to alternative methods of bridging the gap from border router, via the ETR, to the end-user network.

Note, the bit numbers I give here are ordinary binary bit numbers, where 0 is the least significant.  I need to update it to also show the IETF bit numbers, where 0 is the most significant.


>> Introduction
>> This is not re-inventing MPLS
>> How the Forwarding Label could be used in an Edge network
>> A word about namespaces
>> Core routers drop packets with FL values of 8 0000 and above
>> List of new functions for core routers
   > FIB functions
   > RIB Functions
>> What do we want this to do in Edge Networks?
   > Preliminaries
   > More flexible assignment of prefixes to the Forwarding Label values
   > Carrying Ivip6 packets inside the provider network from the FLPER to the end-user network, via the ETR
   > Conclusion for now
>> What about not splitting the 2^20 values into Core and Edge?


The Ivip6 page discussed how valuable it would be, for a scalable routing solution, to implement Label Forwarding in the Core  (LFC), by adding some new functions to core routers and by using the current Flow Label for a new purpose - a 20 bit Forwarding Label.

This page considers in detail how that could be done, with the bottom half of the range reserved for use in the core, and the top half for Label Forwarding in Edge networks (LFE).

These are relatively simple and potentially very useful functions.  They are rather different from any other existing facility, and we need to think carefully what they might be useful for.

This page is a speculative discussion based on several assumptions:
  1. That, as described in the Ivip6 page, it is decided to change the function of the IPv6 Flow Label into a 20 bit Forwarding Label, for the primary or sole purpose of using it for an Ivip6-like core-edge separation scalable routing and addressing solution.
  2. That it is decided that all 2^20 combinations will never be needed for this Forwarding Label in the Core approach, and that it would be acceptable to reserve half of these for the Core alone.  (Maybe it would be decided that even less would ever be needed, but the following assumes that 2^19 is a good number for the core.)
  3. That existing core routers (BGP border routers and transit routers) are upgraded, where possible to perform the required new functions - or they are replaced by new models which perform these functions. (It is tricky having non-upgraded core routers, but is acceptable as long as no ITRs or ETRs depend on them.)
  4. That it is desired to enable each "edge network" to do a similar thing with its internal routers.  An edge network would be defined as a particular provider network, a particular end-user network with conventional space or a particular SEN end-user network with the new, scalable, SPI address space.
  5. That most of what is needed would already be implemented in larger routers in general because these classes of routers could be used in the core.  Therefore little extra functionality (none in the FIB) beyond that for core routers would need to be added to routers in general to enable them to do similar work inside each edge network.
  6. That it would be best to define an edge network mode of using half the 2^20 possible values.
In the discussion which follows, I will assume that the whole 2^20 space of possible values is divided in this manner:

0 0000            This indicates the Forwarding Label is empty.  The
                  packet should be forwarded as per usual: according
                  to its destination address.
0 0000 to 7 FFFF  524287 values available for the Core.  All, or most
                  of them would correspond to the same numbered
                  Core Egress Prefix (CEP).  CEP prefixes are all the
                  same size and are all aligned in a contiguous block
                  as exemplified in the Ivip6 page.  However, since
                  there are 2^19 of them, it wouldn't be E00://12 as
                  mentioned on that page, but:
                      E000::/13 = 524288 /32 prefixes

8 0000            Part of the Edge Network range, but also to be regarded
                  as empty: forward the packet as usual.

8 0001 to F FFFF  524287 values available for each Edge Network.  

I will also assume that any edge network which decides to use this will upgrade all its internal routers to the required functionality.  This gets around some problems we have in the core interworking with non-upgraded routers.  

Please let me know your thoughts.

Note: I have not considered multicast at all yet.  

This is not re-inventing MPLS

Please remember that the proposed conversion of the IPv6 Flow Label to a Forwarding Label is not an attempt to re-invent MPLS.

(That said, it would be possible to implement, in private networks, an MPLS-like usage of some or all of the proposed 2^19 values 8 000  to F FFFF could be rewritten by each router the packet passes through.  I am not suggesting this.)

MPLS requires setting up a path.  This takes time, ties up resources on the routers, involves other protocols etc.  The proposed IPv6 Label Forwarding approach involves no path or setup.  The proposed IPv6 Label Forwarding simply takes a packet to a router which advertises a prefix, while that prefix is identified by the value of the Forwarding Label, which can be a prefix completely unrelated to the destination address of the packet.  

MPLS can handle any kind of packet.  This technique is purely applicable to IPv6 - although of course and IPv6 header could be used to encapsulate any other kind of packet.

MPLS typically involves QoS.  This proposal has no QoS arrangements.  

MPLS involves an additional header. This proposal has no additional header.

MPLS involves significant state to be set up in each router, and involves writing the label to another value before forwarding the packet.  This proposal involves no rewriting of the packet by the routers it is forwarded by.

MPLS is generally only applicable within a particular network, and is not used in the core of the Internet - the interdomain routing system between BGP routers.  The Edge part of this proposal is like MPLS, its scope is a given edge network which adopts it and gives its own meanings to the 2^19 potential values of the Forwarding Header.  However the core part is a global system for the core of the Net, handling packets sent from one AS to another where each AS has no business relationship whatsoever with the other and where the packet is forwarded by core routers of any number of similarly unrelated ASes. MPLS can't be used like this.

How the Forwarding Label could be used in an Edge network

This proposal is applicable to the core, as a particular "zone" in which all the bottom half (2^19 - 1) valid Forwarding Label values must have the same meaning and function.  

This proposal is also applicable to any other zone for the top half (2^19 - 1) valid Forwarding Label values, and each such zone is most likely to be a separate edge network.

In principle it would be possible for several edge networks to form a larger zone, by agreeing that each network only use a separate subset of the 524,287 available values, and by coordinating their routing systems so that a given label value would have the same effect (forwarding the packet according to one prefix) in all the three networks.  However, here I focus on the scenario in which each edge network uses the full 524,287 values in its own way, without regard to how the same range of values are used in any other edge network.

A word about namespaces

The 2^20 range of Forwarding Label values is split into two.

The bottom half is for use in the core.  The free 19 bits in this range constitute a single namespace 0 to 524,287.  While, in principle, any edge network can do whatever it likes with these values for the Forwarding Label, as long as this usage never affects packets in the core, the following discussion assumes that the bottom half is reserved only for core use, where these values constitute a single global namespace which is always applied to these 2^19 values.

The top half of the range is for use in edge networks.  Every edge network has its own namespace for this range of numbers.  That is to say that a particular value such as 9 603B has a particular meaning and function everywhere in edge network X, and a completely different and independent function everywhere in edge network Y.  Each network has its own namespace in which these values have their own local meanings.

The core is a single namespace.  If the Forwarding Label is set to 1 6703 in packets in core routers operated by network X and Y it has the same meaning and function in both packets.

Every edge network has different public and non-public (Unique Local Unicast) prefixes and will be free to assign particular values in the 8 0001 to F FFFF range to whatever prefix they like.

All the internal routers (or the internal facing functionality of border routers) in a given edge network will be coordinated so that the use of the top half of the range can be defined according to the wishes of that network's administrators, without any reference to how it is used by other networks.

Consequently, the 8 0001 to F FFFF range has its own unique namespace for every edge network which uses this proposed facility.

Naturally, wherever a network connects to another network, the routers need to be careful not to propagate a value in the 8 0001 to F FFFF range across the boundary to the other edge network.

Core routers drop packets with FL values of 8 0000 and above

One option is for core routers to drop a packet which has any value of 8 0000 and above.

Already, a core router is expected to drop any packet with a Forwarding Label value between 0 0001 and 7 FFFF for which the value points to a prefix for which the router has no best path (no valid FEC value).

But what of the idea that packets with values in the range 8 0000 to F FFFF should be handled by core routers as normal packets?  They would be transported across the core without any changes, to a border router which advertises the most specific prefix which encloses the destination address.

What good would the non-zero Forwarding Label value have after that journey?  It would generally or almost always be arriving in a network which is different enough from the source network not to share the same method of assigning meaning to the values 8 0001 to F FFFF.

For the following discussion, I will assume that border routers should never send packets to their core BGP neighbours with a Forwarding Label value of 8 0000 or above, and that any core router (or core part of a border router) which receives such a packet should drop it, probably without generating any messages.

Folks can use Traceroute to debug such a situation.  To Do: test if current Traceroute programs report the value of the Flow Label from the returned packet header.

List of new functions for core routers

In the Ivip6 page I described the additional functionality, but in various places.  Here is an exact list - however here I am assuming a 2^19 range for the core, and that the top half of the full 2^20 range is reserved for edge networks.

Naturally, these new functions would be configurable in terms of being on and off.  They may be made more flexible than described here - this is the minimum new functionality to be added.  For simplicity I am assuming the terminology and the values used in the example on the Ivip6 page.  Probably, these things would be adjustable in each router.

So for the core, the 2^19 values correspond to the prefixes:

Name of      Forwarding     Prefix           Notes
prefix       Label value

     CEP-0   0 0000        (E000:0000::/32)  Forward packet as usual:
                                             according to the prefix of
                                             the destination address.

     CEP-1   0 0001         E000:0001::/32    }          
     CEP-2   0 0002         E000:0002::/32    } Forward as for this prefix.
CEP-524287   7 FFFF         E007:FFFF::/32    }    

FIB functions

Forwarding Label recognition: dropping the packet

If the value is 0 0000, forward the packet as usual: according to its destination address.

If the value is equal to or above 8 0000, or to some lower value which the router is configured with, for instance according to the limited length of the FLFEC[] array (which means the total range of CEP prefixes advertised is below this limit), then drop the packet.

If the value points to an FLFEC[] entry which is zero, invalid etc. drop the packet.

Forwarding Label recognition: end of path across core

There needs to be a special FEC value, here called HOME, which indicates that the current router is the one which advertises the prefix which matches this Forwarding Label value.

If the Forwarding Label's value points to an FLFEC[] entry  == HOME, the packet has arrived at the router (perhaps one of several) which advertised the CEP prefix the Forwarding Label referred to.  So the router should zero the Forwarding Label, and then handle the packet with its ordinary FIB functions, according to its destination address.

Filtering out spoofed source addresses:

At this point, border routers might be configured to first check the source address against a list of internal prefixes, to drop the packet if its source address matches one which is inside the current network.

Exactly how the router handles the packet now is not important to the whole Label Forwarding in the Core system, because the system has achieved its goal of delivering the packet to a border router which advertised the specified CEP prefix.

Forwarding according to the Forwarding Label

If the packet did not match the above conditions, then the FEC value read from the FLFEC[] array is a valid FEC value, directing the router to forward the packet out of one of its interfaces, to one of its neighbours.  The router does this.

RIB Functions

The RIB has an additional function which periodically (every second, every 5 seconds or whatever) monitors the FEC values for the entire currently used range of  CEP prefixes.  There may be a configured limit on this, such as there being no more than 20,000 or some other number of contiguous prefixes assigned to providers who have end user networks - so the RIB function should not look to any further than a certain number of CEP prefixes.  Likewise, perhaps there is a defined limit on the size of the FLFEC[] array.

Assuming there are no such limits, the new RIB function reads these FEC values and copies them to the FLFEC[] array in the FIB.

An alternative arrangement is for the RIB code to monitor all changes to the FEC values, as the BGP implementation changes those FEC values whenever it decides on a new best path for a prefix.  If the change is for a prefix in the CEP range, then the new functionality writes the changed value to the appropriate element of the FLFEC[] array in the FIB.

What do we want this to do in Edge Networks?


Here is a rough discussion of what we might want to do with the Forwarding Label in edge networks.

There are all sorts of possibilities, but the focus is on basic usage, which is useful and robust enough to warrant somewhat extending the functions required for Label Forwarding in the Core so we can also do Label Forwarding in Edge networks.

It would be possible to reserve some values of the 8 0001 to F FFFF range for some special purposes we haven't thought of yet.  That should probably be done, but below I assume we use the whole range for forwarding along the lines of the way proposed for the core.

Am not considering here anything special with CoS or QoS.

There is only marginal value in using the Forwarding Label in a way which directly follows the value of the destination address.  Routers are already capable of forwarding the packet according to FEC of the prefix its destination address best matches.  However, the Forwarding Label approach will probably be easier for the FIB, especially if it has to chew through up to 48 bits of destination address to match a prefix.  If FIB speed is an issue, then this may be a valid use of the Forwarding Label: simply set it according to the destination address.  It will arrive at the same router, but use less FIB resources of all the routers en-route.

The most unusual quality of this Label Forwarding approach is:

It can forward a packet to a router which advertises some
prefix, which is unrelated to the destination address -
without any extra headers and without changing the rest
of the packet in any way.

There could be many uses of this facility - which performs a job normally done by encapsulation and tunneling.

Broadly speaking, it is a method by which some ITR-like function sets the Forwarding Header and the routing system carries the packet to some router inside the network which advertises some prefix to which this value of the Forwarding Header corresponds.

More flexible assignment of prefixes to the Forwarding Label values

The first change we would want to make to the LFC (Core only) arrangements is to provide each edge network with a way of assigning any of its internal prefixes to any of the values of the Forwarding Label: 8 0001 to F FFFF.

The LFE (for individual Edge networks) arrangements specify a fixed linkage between some publicly advertisable prefixes and the values of the Forwarding Label.

This won't do for Edge networks for several reasons:
  1. Each edge network has different prefixes it would want to use for the Forwarding Label system.
  2. Those prefixes surely won't all be contiguous, public, or the same size.
So we need to provide some robust system by which all routers in an edge network can be told which prefix's FEC value they should write into each of the 8 0001 to F FFFF elements of the FLFEC[] array.

This will involve the specification of some protocol and either one or more robust servers to provide this information or some way of flooding it to all internal routers.

Let's assume we need a bunch of servers and a way of configuring each internal router to periodically access at least one of these servers.

Then we need a standardised protocol for specifying the mapping of whatever prefixes the administrators like to whatever Flow Label values they like.

These only impact the RIB - route processor - part of the router.  So they are perfectly amenable to firmware updates, and would not be particularly complex or involved.

The resulting "Forwarding Label Value to Prefix Mapping List" (FLBPM) is then an elaboration of the extra RIB functionality for core routers.  Instead of copying from a contiguous list of FEC values for whatever /32 prefixes are advertised in the E000::/13 prefix, the FLBPM consists of a list of instructions, covering the 8 0001 to F FFFF entries of the FLFEC[] array.  The new functionality would step through the FLBPM, finding the FEC of each specified prefix and copying it into the appropriate FIB FLFEC[] array.

For instance, the FLBPM might start by specifying the particular prefixes to use for values 8 001, 8 002 and 8 003:

8 000         0
8 001         FC00:08B4:0100::/48
8 002         FC00:0002::/31
8 003         FD00:0001:0044::/46

The operators of the network can do whatever they like with this mapping list.  If they want to put global unicast prefixes in it too, that is fine.

Carrying Ivip6 packets inside the provider network from the FLPER to the end-user network, via the ETR

There could be all sorts of uses, but the most most obvious to me is as an internal extension to what we do with the Forwarding Label in the core.

The ITR has a task of getting a packet with destination address X to some router which connects to an end-user network, located somewhere in the world (or, if the network is a single host, to that host).

The address of the destination host within the destination end-user network is in the packet, but the destination end-user network is not directly accessible.  The mapping system in Ivip6 tells the ITR the 19 bit identifier for the CEP prefix which is advertised by the provider network where the ETR is.

The full mapping information for the micronet to which the packet is addressed consists of a 128 bit address - which we call the ETR address.

In Ivip6's current design, the ITR does not get all this - just the bits 96 to 114 of it: the 19 bits to write into the lower 19 bits (0 to 18) of the Forwarding Label, with bit 19 set to 0.

When the packet arrives at the recipient host network (the border router is the FLPER - Forwarding Label Path Exit Router), it needs to be forwarded to the destination network.  If the recipient provider network's internal routing system has a route for the end-user network, then all that needs to be done is to zero the Forwarding Label and let the FIB handle the packet normally.

However, there will be situations where it is not practical to handle every end-user network's address range in the recipient provider's internal routing system.  In these cases, the FLPER has a more difficult task.

This is where the Label Forwarding for Edge networks could be handy.

Let's say the recipient provider's network has 20,000 ETRs.  There may be 1,000,000 separate end-user networks with links to this provider networks.  Let's say each such connections goes through one of these ETRs, and each ETR is able to recognise from the destination address of the packet which end-user network it should be sent to, and therefore which link to send it out on - which tunnel to a DSLAM or whatever to forward it to.

There may be multiple border routers, each advertising one (ideally not more) CEP prefixes.  Each such border router could be acting as an FLPER: zeroing the Forwarding Label and having the task of getting the packet to the right ETR.

One approach is for the ETRs to all have unique addresses, and for the FLPER to look up the full 128 bit mapping for the packets it is handling.  Then the FLPER could encapsulate the packet and tunnel it to the ETR. For this to work, all those ETRs would need addresses within the CEP prefix, which should be fine.  The CEP prefix is a /48 in the current description of Ivip6, and so could be split up into longer prefixes, to handle 20,000 ETRs, at up to 20,000 different physical locations in the network. (These are very loose and probably extreme numbers, for the sake of discussion.) However, this encapsulation raises some PMTUD problems.

Another approach is for each ETR to advertise its own prefix within the CEP prefix.  For this discussion, let's assume these are all the same size prefix and are contiguously assigned from some defined prefix.  Then we can play the same game with some or all of the 2^19 values of the Forwarding Label inside each recipient provider network as we did in the Core.  

The network can have its own internal mapping lookup system.  There would be a query server, or some distribution of a complete mapping list, so that each FLPER could request which ETR (in effect) a packet should go to, and get back a number to identify that ETR, and the range of addresses (one or more micronets, for instance) which also have the same ETR.

So each FLPER router can easily find out, and cache, the Forwarding Label value in the 8 0000 to F FFFF range which corresponds to the prefix advertised by the ETR which handles.  Then it simply rewrites the Forwarding Label, and lets its upgraded FIB, and the upgraded FIBs of other internal routers, forward the packet to the ETR.

So the packet gets from the sending host to the ETR without any additional headers, purely use of the Forwarding Header, but with two values:
  1. From the ITR to the FLPER with a value 0 0001 to 7 FFFF written by the ITR after looking up the mapping of the packet's micronet in the global Ivip6 mapping system.  (But nice and fast, since the full database query servers are local.)
  2. From the FLPER to the ETR with a value 8 0000 to F FFFF written by the FLPER.  
Above, I discussed two ways the FLPER could use the destination address to determine the value to write to the Forwarding Label.

Here they are again in more detail:

A - Use the global mapping system and use the full ETR address

In this case, the global mapping system needs to be able to tell the FLPER the full 128 bit "ETR" address which the end-user network has mapped each micronet to.  This is clearly possible, but since the ITR only needs 20 bits of this, I was exploring the idea that only those 20 bits are fast pushed to all the full database Query Servers around the world, and that some other mechanism would be provided for the much lower demand for the full 128 bit ETR address, which would arise only from those FLPERs which needed this.

In this case, the addresses of the ETRs must be within the CEP prefix.  That CEP prefix can be split up arbitrarily inside this recipient provider network site, according to wherever the ETRs are.

The recipient provider network would have its own way of organizing the advertised prefixes on which these ETRs could be reached. It is conceivable that this would be standardised across all networks, for instance so that each /32 CEP prefix was assumed to be split into 2^19 smaller prefixes: /51s.  (There are a variety of other arrangements I won't explore now.)

Then it could be standard procedure in all provider networks for the FLPER to do a lookup of the full 128 bit ETR address, and to use a subset of those bits to set some or all of the 19 bits of the Forwarding Label so that the packet would be forwarded in the internal routing system towards - or all the way to - each ETR.

If an ETR goes down, it would be up to the end-user network's multihoming monitoring system to change the mapping to another ETR - either to another ETR in this recipient provider network, or if the end-user network is multihomed, perhaps to an ETR in another recipient provider network.

B - Maintain a local mapping system for ETR addresses/prefixes

In this case, there is no need to look up the full 128 bit ETR address to which the packet's destination micronet is mapped.  This is because the destination provider network maintains its own query servers which can tell each FLPER what value in the 8 0000  to F FFFF range to set the Forwarding Label to so that the packet will be Label Forwarded to the correct ETR.

This requires the recipient provider network to do a more work, since maintaining a local query server, keeping track of all these ETRs and customers etc. is a lot of work.

Here are two different approaches:

1 - Ignore the 128 bit ETR address which the end-user mapped the micronet to (This turns out to be a bad idea.)

The local mapping system could ignore the 128 bit value to which the end-user mapped each micronet.  This way, the local mapping system is free to use any ETR to connect to each end-user network - which provides some local flexibility.  It also means that the end-user doesn't have to worry about what ETR address they map the prefix to, as long as bits 96 to 114 of the address they use (which the ITR gets and copies to the Forwarding Label) causes the packet to arrive at the correct recipient provider network.  

However, this sounds like a bad idea.  The local mapping system would need some way of tracking the actual addresses used by all these end-user networks.  There's no particular reason why an end-user network must have a single nicely defined prefix.  If I have some micronet address space I am not using, and my friend has an end-user network somewhere, I should be able to define some of my space as a micronet or two and map it to his ETR, so he can use as he wishes for as long as I like.

How can all these end-user networks efficiently and securely communicate to this recipient provider network exactly which of their micronets they are mapping to this provider's ETRs right now?  The great thing about Ivip's real time mapping system is that each end-user network can switch their micronet to any ETR address in any ISP they like.

For example, a micronet is currently mapped to an ETR address at ISP-A.  ISP-A's local mapping system somehow is told about this, which means, in part, that any packets generated in ISP-A's network will be guided by this mapping system to that ETR.  But let's say the end-user network's link to this ETR fails and they change the mapping of their micronet(s) to another ETR in another ISP?  Or they change the mapping to another ETR at ISP-A or to another ETR in another ISP, for reasons of TE, or because they simply feel like it?

How can the end-user easily tell ISP-A's local mapping system about the change if its link to ISP-A has just failed and if this mapping system doesn't monitor the mapping changes concerning the micronets which are currently mapped to ISP-A's ETRs?

What should happen is that all packets, including those sent from this provider's networks, should go to an ITR and be forwarded to the new ETR in the other ISP's network.  That won't happen if the local mapping system can't be told to do this by the end-user, and if it doesn't constantly monitor the mapping updates for any which affect its current end-user networks' micronets.  

I could go on, but all this is a can of worms.

Every packet generated from outside the destination end-user network must go through an ITR and be tunneled (Ivip4) or Label Forwarded (Ivip6) to the ETR address to which the micronet (which matches the destination address) is currently mapped.

This is the plan with Ivip4 - but not of the other map-encap systems.

In Ivip6 we are implementing this path to the ETR in two stages - one from the ITR across the core to the FLPER, the other from the FLPER to the ETR - it is vital that the FLPER respond as quickly to mapping changes as the ITR does.

It is not good enough for the FLPER to look only at the destination address and think it knows which ETR the packet should be sent to, unless its notion of this is fully updated by the Ivip6 mapping system.

So for all recipient provider networks with more than one ETR, it follows that the FLPER must get the full 128 bit mapping information for the micronets of all the packets it is receiving from ITRs.  (The only exception would be some arrangement where the FLPER got just enough bits to write these into the Forwarding Label in order that the LFE system would forward the packet to the correct ETR, but this is rather inflexible.)

2 - Rely on the 128 bit ETR address from the Ivip6 mapping system

This is rather like option A above.  If the recipient provider network maintains a mapping system for the use of its FLPERs to figure out how to get the packet to the correct ETR, and if that mapping system depends on how each end-user network has mapped each micronet it uses, then maybe the whole thing should be simplified by having each FLPER look up the 128 bit ETR address itself.

There may be a good reason for doing this.

Suppose the recipient provider network had other uses in mind for some of the 2^19 values of the Forwarding Header it can use.  It could configure the FLPERs to use a subset of these values, such as the bottom 2^16 of them, based directly on bits 80 to 95 inclusive of the full 128 bit ETR address.  Then there can be 2^16 contiguous prefixes where ETRs are located, such as a small subset of the CEP prefix,or perhaps some regular set of Unique Local Unicast prefixes.  This way, the FLPERs could take care of their incoming packets in a coordinated fashion, without the local mapping system needing to respond every time a micronet's mapping changed.

This way, the rest of the 2^19 available values of the Forwarding Label could be used for other purposes.

(In Ivip4, for PMTUD reasons, the ITR needs to be able to send packets directly to the ETR and get responses from that address.  With Ivip6, there is no such requirement for ITR-ETR communication.  So exactly how the recipient provider network gets the packets to the destination network doesn't matter.  It might put its ETRs in Unique Local Unicast space and use some system of its own devising to make sure the traffic packets get to the right ETR.)

Conclusion for now

In larger recipient provider networks, if all routers were suitably upgraded, and if new protocols were used to coordinate how they write the FEC values for various internal prefixes into the 8 0001 to F FFFF entries in their FLFEC[] array, then the system could be used for a variety of purposes, not least to forward packets from FLPERs to ETRs, without any tunneling and so without any PMTU problems or additional packet length overhead.

What about not splitting the 2^20 values into Core and Edge?

It could be argued that core routers know they are core routers and internal routers inside a given organisation know they are internal routers - and that routers which do both know which part of the router the packet is in . . . so why not let all 2^20 values of the Forwarding Label be used for the Core, and allow each edge network to use the full 2^20 range as it sees fit within the confines of its network?

I have several reasons for maintaining that the split into two separate 2^19 spaces is the best approach:
  1. The idea of not splitting it, and of having packets with a Forwarding Header set for the internal system somehow get treated as if it is in the core . . . the idea gives me a terrible headache.
  2. I think 2^19 is plenty in the core and in any private network.  (Famous last words . . .)
  3. For now, in the early stages of developing this idea, I suggest we keep the two ranges separate. It is easier to think about it this way.  If we decide it is a good idea to change the function of routers and of the current Flow Label, then we will know a lot more about this and will consider carefully whether to split the range, and if so how.
In an edge network, people can do what they like - and if they keep it to themselves, why should anyone care.  If an edge network wants to use all 2^20 values of the Forwarding Label for its own internal purposes, or share some of that space with another network, that is just fine.

The idea is to set some standards which most people will follow and find not too restrictive, but which will discourage practices which are likely to cause trouble.