Cloud Interconnection Models

By William B. Norton September 9, 2016

The Internet is evolving into two hemispheres: the Public Internet and the Private Internet. The public Internet is the shared best-effort packet delivery vehicle that we all know and love. The private Internet is attached to the side of the public Internet, and consists of dedicated private (not-shared) connections between entities. The private connection is architectural; you set it and forget it. The routing algorithms automatically flow traffic along the shortest path.

Why does this Public-Private Internet distinction matter?

Public Internet traffic is intermingled with others Internet traffic. This has some very important implications. First, the good news. Shared resources can lead to economies of scale and aggregation efficiencies. By aggregation efficiencies I mean that the peaks in one flow may be offset by the valleys in another flow. This means the combined flows do not require double the bandwidth. Internet Service Providers tell me that they can sometimes fit 2.5 times as much over an aggregated circuit than if each services required its own dedicated network path. This sharing of network resources has helped drive down the costs of Internet access.

However, this also means that Internet-based packet services experience a higher degree of performance variability.  Internet traffic ingress is not predictable. Where that traffic is going is not predictable. The scale of the traffic flow is not signaled. This random inflow of traffic means others traffic can negatively impact your Internet traffic. And you won’t necessarily know why the connectivity is no longer there. Massive spot events like an Oprah Winfrey webinar or a Victoria Secret fashion show result in seemingly random packet loss and congestion events for the rest of us.

Is the Internet Good Enough?

For some workflows, the occasional network failures and corresponding artifacts are annoying, but not problematic. However, for business-critical activities, the network issues could result in the organization failing to meet objectives. Examples of mission-critical network uses include collaboration with geographically distributed staff using cloud network folders, entering customer intel into a CRM site, report generation for a board meeting, company-wide or event video and voice conference calls, etc. If the traffic is important, it should now be exchanged directly.

In response to these customer demands, the major cloud companies (Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure (MAZ)) have created their own Internet-bypass solution. Let’s examine each in turn.

Amazon

In 2011 Amazon launched Direct Connect, an Internet-bypass solution. In my white paper “Cloud Interconnections” I conceptualize the AWS model visually as shown here:

AWS Diagram 1-01 Figure 1 – AWS Direct Connect interconnection model.

On the left side we have the cloud resources being externalized to the right side, the Customer Data Center. Amazon calls the network transport providers in the middle “Amazon Partner Networks.” Once established, the Direct Connect enables the enterprise to directly connect to its Virtual Private Clouds (VPCs) (shown as colored rounded rectangles) from their data center, bypassing the Internet. Each department’s colored VPCs for example could be routed directly back to the internal network for the group responsible for those resources.

We (Console) sell connectivity to AWS every day. It is the most well understood of the Internet-bypass solutions.

Azure

Microsoft unveiled their ExpressRoute interconnect model in 2014 in response to enterprise demands for an Internet-bypass solution, but as you will see, accomplishes it with a different approach.

AZURE diagram2-nospace

Figure 2 – Microsoft Azure ExpressRoute interconnection model.

One key difference is that Microsoft is externalizing three distinct categories of resources:

Private (for non-Internet accessible Azure resources),

Public (for Internet accessible Azure resources), and

Microsoft (for SaaS apps like Office 365, Skype for Business, etc.).

Each of these interconnections is accomplished with a distinct peering sessions. Interesting approach.

Another key difference is that Azure provides dual connections for redundancy. This means that one might have 6 peering sessions with Microsoft (two connections to each of Public, Private, and Microsoft SaaS).

Visually I present the ExpressRoute service like a six connector cable as shown below:

AZURE diagram1-nospace

Figure 3 – The ExpressRoute Circuit

The ExpressRoute customer may interconnect with any or all of the Microsoft resources. However, to take advantage of the Microsoft SLA, customers must connect redundantly.

Google Cloud Platform

Google Cloud Platform chose yet a different approach: you simply peer with Google and you get access to all Google services along with all of your cloud resources. There are no VLANs to multiplex.

GCP

Figure 4 – The Google Cloud Interconnect model.

The implementation may be a little strange. One model is for the enterprise to peer with the service provider, what Google calls “Google Cloud Interconnect (GCI) Service Providers.” The GCI Service Provider also peers with the Google, and transparently extends the announcements and traffic flow between the two.

Some network engineers call this approach “Peer and Leak.” Traditionally peering is non-transitive, but here it purposely makes peering transitive. This approach explicitly leaks routes and traffic across the BGP peering session, effectively delivering proxy peering between the enterprise and Google. Again, we see a different approach to interconnection.

Summary

In this blog we see a brief glimpse into the three different interconnection models used by the most popular cloud providers. We will explore this topic a bit more fully in the upcoming Webinar September 15th at 9AM. The full white paper is available as the “Cloud Interconnections” white paper.

About William B. Norton

William B. Norton currently serves as the Chief Scientist for Console, a leading global interconnection ecosystem. Previously he served as Managing Director for DrPeering International, Co-Founder and Chief Technical Liaison for Equinix, and served as the first chairman of NANOG. Read more