John Roese’s Blog CTO, Nortel

IP: Unifying Force or Mask of Complexity?

Location: Flying to Boston

As we began our focus at Nortel on addressing the reality of the hyperconnected world, it became obvious that in addition to all of the fantastic gains that will come from being more connected, wherever you are, and via whatever device or application you need, that there will also be a cost. That cost will mostly be seen in the added complexity of living and operating in a world where an enterprise might have a million nodes to manage and billions of sessions to support. It might also be a world where the Internet is measured in trillions of devices and sessions. Because we, as the IT industry, have never before operated at that scale, we need to consider every aspect of IT system complexity and begin addressing it now, in advance of the scale and complexity challenges that inevitably are in front of us.

One area that we looked into at Nortel was the complexity of transport technology. When you look at the network today, what you see if you are an applications person is a “cloud” that provides best-effort Internet Protocol access to your own networks and the Internet. That’s a pretty good accomplishment because without IP we would lack a ubiquitous layer of addressing and protocol definition, and global communications would be difficult, if not impossible.

We could argue that IP has been a huge simplification technology and we would probably be correct. The issue, however, is that IP is not, in and of itself, the transport network. Underneath that nice, uniform IP layer is a host of diverse and complex networking systems that lack commonality; vary widely in cost, complexity and speed; and introduce huge complexity into the system.

If you consider why most enterprises don’t even consider running their internal applications over the cellular networks of today, it is not because those networks are unable to transport IP. It is because those networks transport in very different ways than their LANs, resulting in less speed (144 kb/s versus 100 Mb/s), greater cost (cost per bit in the LAN is almost 0; cost in cellular is not), different intelligence (security, provisioning and QoS), and lack of control. As well, so not to pick just on cellular, if we look at the differences underneath IP in the LAN versus the WAN, in Wi-Fi versus wireline, and in the metro versus the LAN versus the WAN… what we find underneath the nice uniformity of IP is that the systems we rely on today to actually deliver the traffic are fragmented, diverse, in many ways incompatible, and very costly.

What should we do about this? My suggestion is that we begin to look at this lower layer and ask if it is possible to converge to some common lower-level transport technology across all next-generation networking technologies. If so, what should that look like? Is there already an emerging dominant technology model that should be the goal of everything from the broadband wireless world to LAN/WiLANs, to MAN and WAN technology?

Below is a chart that I have used to suggest that this convergence is not only possible but is already happening.

simplify_transport_infrastructure1.jpg

The top portion shows the diversity under the IP layer today in each area (incomplete I am sure). You see circuit-switched and packet-switched technologies, slow-speed and high-speed links, high-cost and low-cost systems, and a huge diversity that makes end-to-end transport for all that IP traffic pretty inconsistent.

The bottom portion of the chart suggests that in each of these domains a convergence to “Ethernet-like” transport is already happening and that convergence is creating a “clear pipe” for the IP traffic to flow. In the cellular world, 4G wireless technology, such as Mobile WiMAX, is based on a model that looks a lot like Wi-Fi and Ethernet LAN systems in that they are packet-based, have high capacity, are low cost and support an open edge so that a wide range of devices and applications can operate over these networks without added complexity.

In the MAN and WAN, there is a set of technologies around Metro Ethernet emerging, where the optical network provides an Ethernet MAN service using Ethernet directly over optical systems. Nortel has been advocating this model with our Provider Backbone Bridging technology, and carrier uptake, while early, has validated 60-80% savings over legacy models of operation. In the WAN, the shift to Ethernet has been slower but the shift to packet transport has already begun (away from circuit) and, inevitably, there will be expansion of Ethernet in the WAN given the emergence - and now validation - of this model in all other networks.

This chart is a subjective view and the timing could be over 10 years but, logically, if we simplify the underlay of IP systems we will drive consistent services and lower opex and capex. All this is needed to reign rein (thanks K. Ramesh) in the cost and complexity that hyperconnectivity will create.

One last point to make on this topic… I want to preempt the dialog where some may suggest that we simply deliver IP over a physical media and skip a common packet transport layer below it. In situations where the applications and devices are intelligent and able to be provisioned, this model (IP over optics, for example) may be correct. The hyperconnected world, however, is far more complex than that.

Many of the new consumers of network services (sensor networks, for example) cannot support the overhead of a full IP model and thrive in simplified self-configuring network systems like Ethernet. Additionally, many of the tasks that make networks secure and stable happen in advance of entering the IP layers. For example, the current accepted model to control access via the authentication of end systems is based on Extensible Authentication Protocol over Ethernet (EAPoE, or 802.1X) because the ability to exchange credentials and determine trust in advance of interacting with the Internet has been shown to be a simple and very robust way to control access for a wide range of end points and systems.

This debate over whether the answer is IP over a physical layer, or IP over a common packet transport, is a good one to have, but the subject for another blog entry. The subject of this post is more to dialog over the need to simplify the layers under IP as a way to create the “clear pipe” and to drive down the cost and complexity of the hyperconnected network.

Comments

  1. Switched Ethernet in the core, shared Ethernet at the edges or aggregation points. Possible move to Gpon/Gmpls in the core as the standards mature and the issues with managing a box that does not have the need for an electrical interface are solved.

    See how simple that is? :)

    I guess PBB and PBT address the issue, however there needs to be a resistance to complicated solutions such as what happened to MPLS. In my opinion the complexity happens to any technology as it is pushed closer to the edge. This is likely an essential complexity that cannot be easily overcome as discussed in you last post.

  2. ‘Many,’ great points. Having watched how MPLS morphed from a simple approach to accelerate routing to the complex set of technology today, it is critical that the work around simplified Ethernet transport not follow that same evolution. Some key principles are to: resist moving up the stack unnaturally (don’t try to do what the session layer is designed for, as an example); always ask if the solution to a new problem can be solved by existing technology capability used in slightly different ways (it’s always easier to make a brand new technology for every problem, but the result is complexity); and treat the frame format as unalterable except in extraordinary cases (Ethernet in IEEE802 has only modified the frame format a handful of times and the process to do so is so difficult that it encourages people to solve problems with existing mechanisms).

    Additionally, your comment on complexity at the edge is key. One advantage Ethernet transport has is that it can deal with the dynamic and individual elements of the edge well. My personal belief is that if you want an end-to-end technology, edge effectiveness is a better starting point than core because the edge is a much more complex place. If you can solve for the randomness and scale of the edge, dealing with aggregation and static cores should be easy.

    John

  3. John,

    Thank you. I think the complexity really began with forwarding plane QoS in the core and reached absurdity with pseudowire which morphed into at least four different stacks from three different standards bodies. Different implementations of IMA groups didn’t help either. All this while another less sexy part of MPLS, the control plane, languished. Bad combination of events. All is not lost though, you never know, someone may pull a rabbit out of their hat.

    The edge is where both the “action” and the complexity is. Country variants, vendor games in the standards bodies, service provider proprietary implementations in the name of competition all play a role in stirring the Byzantine protocol stacks.

    I think it will be difficult to build a ground up protocol from the edge unless you can get several vendors to support it. To do this you need to get it through the standards and unfortunately politics while still keeping it simple. Even Ethernet is not immune, remember when cisco tried to hijack the preamble? While I agree the edge is a good (maybe the best) starting point, I don’t think there is the time and will lately.

    As an aside, I have noticed that vendors are getting even more restrictive in their implementations tearing down sessions because of unrecognized parameters. The root concern? Stack overflow/underflow. So now stacks that should work together forward and backward compatibility and the notion of data not having effect without a process is no longer true. I wonder what the ultimate result will be, care to guess?

    Anyway, taking a step or two back, I think nortel could start to have a positive effect on the whole mess if they had a simple, clearly articulated and recognizable (almost a brand) architecture like the core-distribution-access model to hang their ideas on.

  4. Hi, John
    I agree with you over the need to standardize on a common infrastructure technology, on a layer low enough, and one that is clearly understood.
    Whether that technology is Ethernet, pure IP packet over optical, or some optical labelling different than any of those, it depends on the vendor, and the level of traction it has in various standards bodies and forums. NORTEL for instance, has a very good reputation with IEEE and ITU, therefore has a better chance to push for wired and wireless Ethernet standard initiatives (such as 802.x.y) that give first mover advantage. By the same token, CISCO and ALCATEL, with reputable presence in IETF, are pushing for IP/MPLS based solutions.

    The next question would have to do with the control model: how much control do we give to users, versus carriers. In an IP world, the users have a lot of control, as some of them may be more IP-literate than their ISPs. IP also presents a security problem for the ISPs, as a significant portion of the provider’s infrastructure is exposed to the user at the IP level (Ex: I can do a traceroute and find out where the provider has a problem in the same time with their tech support, in a reasonably flat IP network), and a lot of IP stack tools are available, for good or malicious use).
    Not the same can be said about MPLS or Ethernet, as it is more difficult for users to play around with lower level items such as MAC addresses (as you eluded with EAPoE, or 802.1X), and user-level tools are not as available. This is definitely a world where the ISPs will have most of control, as they have access to parameters and tools at a protocol level (Layer 0-2 - Ethernet, MPLS, optical) below their users.

    At the end of the day, the equipment is doing forwarding based on a hash table. Whether that table is based on Ethernet MACs, IP addresses or MPLS labels, and is using, static entries, IP/MPLS routing protocols, or flavours of Ethernet spanning tree to configure/reconfigure itself, becomes an implementation detail.
    This implementation detail is dictated by the vendor’s preference, and solutions’ perceived simplicity.

  5. Dragos,

    Good point.

    A lot does depend on the expertise/heritage as well as their legacy equipment. I agree also with your point that nortel has more influence in the IEEE/ITU.

    I think John’s point (please correct me if I am wrong) is that unless there is a single lower layer view (Layer 2)the different stacks need to interact and there will be throughput/complexity penalties for that interaction. reminds me of arguments I heard from the ATM Forum in the 1980’s :)

    I also agree with John that the edge will have a very large impact on what the transport network looks like. Today the applications such as American Idol are driving the core in a way I have never seen before.

  6. The word ‘reign’ is used where the word ‘rein’ was meant:
    “All this is needed to reign in the cost and complexity that hyperconnectivity will create.”

  7. Hi John,
    Good article. I believe, Layer 2 (and Layer 1) plays very important role in providing QoS. With IP becoming the de facto standard for all kinds of communication, Layer 2 technology should have capability to meet QoS requirements of applications sitting over IP. The problem seems to be less complex at LAN and MAN level, but getting QoS support for realtime applications (eg VoIP) remains big challenge.

Leave a Reply