What Is BGP? How the Internet Decides Where Your Packets Go

FundamentalsAutonomous systems, the unit BGP works in

The textbook answer to "what is BGP" is a routing protocol, and it does not make sense until the shape of the internet does. The Border Gateway Protocol is built around one unit, and that unit is not the network you administer.

An autonomous system is one network under one administrative control with one routing policy: an internet provider, a large hosting company, a university, a bank with its own connectivity.

Each has a number, an ASN, assigned by a regional registry, and there are roughly 75,000 ASes in use. Routing within one of them is somebody's internal business. Routing between them is what BGP does.

Inside an autonomous system, routers use an internal routing protocol such as OSPF to find each other and share routing information. Those protocols optimize for the shortest path, and they assume every router involved is cooperating and trusts every other one.

Between autonomous systems, none of that holds. Nothing within one network is visible to the next. Two networks connecting to each other are not one organization, do not trust each other, and have commercial relationships that decide what they are willing to carry.

That is the problem the Border Gateway Protocol exists to solve, and it is why BGP looks so different from an internal routing protocol: BGP optimizes for policy, not for distance.

Not every AS number is handed out. RFC 6996 reserves two blocks for private use: 64512 to 65534 in the 16-bit registry, which is 1023 numbers, and 4200000000 to 4294967294 in the 32-bit one.

A private ASN from either block, the private BGP AS range, is what to use for an internal session or a lab. Private ASNs are the equivalent of private IP addresses in that they must not appear on the public internet.

A session that will not come up is a separate subject from the numbering, and the BGP states are where that gets read.

The exchangeWhat a BGP router actually says

BGP is unusual among routing protocols in running over TCP rather than directly on the network layer, which is what gives it reliable, ordered delivery of routing information without having to implement any of that itself.

Two BGP routers open a TCP connection on port 179, agree on parameters, and become BGP peers. Then they exchange routing information: each one tells the other which networks it can reach.

A BGP announcement carries a prefix, which is a block of addresses in CIDR form such as 203.0.113.0/24, and a list of attributes that qualify it. The most important of those is the AS path: the sequence of ASes the routing information has passed through, most recent first.

That path does two jobs. It gives a rough measure of distance, since a route crossing two networks is usually better than one crossing seven. And it prevents routing loops, because a BGP router that sees its own AS number already in the path discards the announcement instead of accepting it.

After the first exchange, which is a full copy of all the routing information a peer holds, BGP peers send only changes. A BGP session between two large networks might carry no data for minutes and then a burst of thousands of updates when something far away breaks. Keepalives every thirty seconds hold the BGP session up in between.

The full BGP table on the public internet now holds more than a million prefixes. That is why internet facing routers are sold with the memory to hold that much routing information, and why "does it take a full table" is a real purchasing question.

Path selectionHow a route is chosen

A BGP router usually hears about the same destination from several peers, and it picks one. The order BGP uses is long and vendor specific in the details, but the shape is always the same.

Local preference comes first. This is a number set by the network's own administrators saying which exit they prefer. It is internal policy, entirely local, and it beats every rule below it. A network that pays for one link and gets another cheaply expresses that here.

Then the shortest AS path. Fewer autonomous systems, fewer ASes traversed, wins. Note what this is not: it is not the fastest path, not the lowest latency, and not the highest bandwidth. Four networks with a fast path lose to three networks with a slow one.

Then a series of tie breakers within BGP itself, including the origin type, the multi exit discriminator or BGP MED, the BGP metric a neighbor can set to steer data between multiple links to itself, and eventually the oldest route or the lowest router address, so the outcome is deterministic rather than arbitrary.

The order in full, as most vendors implement it.

Steps 1 and 2 are yours. Step 4 is the one everybody quotes. Everything from 5 down only runs when the rows above it tied, which on a busy internet router happens more often than you would expect. The thing worth carrying away is the first rule.

BGP is a policy protocol wearing the clothes of a routing protocol. The path BGP routing finds is not the best one in any engineering sense. It is the path the networks in between were willing to carry, ordered by what they preferred.

External and internaleBGP and iBGP

The Border Gateway Protocol runs in two places and behaves differently in each. The names are external BGP and internal BGP, and every network of any size runs both. Most of eBGP vs iBGP comes down to where the two peers sit and what each one passes on.

eBGP, external BGP, runs between routers in different autonomous systems. This is the version people mean by BGP: two peers, usually directly connected, exchanging routing information across an organizational boundary. A BGP router receiving an external route adds its own AS to the path before passing it on.

iBGP, internal BGP, runs between routers inside a single autonomous system, carrying externally learned routes across the internal network. It exists because an internal routing protocol like OSPF cannot hold a million internet routes and was never meant to.

Internal BGP has one rule that trips people constantly: a router does not pass an internal BGP route on to another internal peer.

That prevents loops inside the AS, and it means every internal BGP router must peer with every other one, a full mesh that grows as the square of the routers. Route reflectors exist to break that, and are the standard answer in any network above a handful of routers.

The next hopThe next hop, and why iBGP needs next-hop-self

The most common way a working BGP configuration fails to produce working routing is the NEXT_HOP attribute, and the rule behind it is one sentence in RFC 4271.

What the attribute is. NEXT_HOP is a well-known mandatory attribute that defines the IP address of the router that should be used as the next hop to the destinations in the update. Every prefix carries one.

The rule that surprises people. RFC 4271 says that when a speaker sends an update to an internal peer, and the route was not locally originated, it should not modify the NEXT_HOP attribute unless it has been explicitly configured to announce its own address.

So a border router that learns a prefix from an external peer passes it to its internal peers with the external neighbor's address still in the attribute.

Why that breaks. The internal routers now have a route whose next hop is an address on the external link, which belongs to another network and is usually not in the interior routing protocol.

The path is invalid, the route is not installed, and BGP looks broken while every session is up. On the border router itself everything appears correct, which is what makes it a long evening.

The fix, and the two ways to do it. The explicit configuration the RFC alludes to is what vendors call next-hop-self: the border router rewrites the attribute to its own address, which the interior protocol already knows how to reach.

The alternative is to carry the external link's subnet in the interior protocol, usually as a passive interface, so the original next hop resolves. Most networks use next-hop-self because it keeps external addressing out of the interior routing table.

Where it does not apply. Between external peers on a shared subnet the sending router normally uses its own address anyway, so the problem belongs to iBGP. If a route reflector is in the path, apply the same reasoning at the router that first brings the prefix into the autonomous system, not at the reflector.

MultihopeBGP multihop, and what the TTL is guarding

External BGP expects its peer at the other end of the wire. On Cisco IOS the default is blunt: only directly connected neighbors are allowed, and the router checks a single hop eBGP session to confirm the peer really is on the same segment.

That default is quietly a security property as well, because a session that has to come from the next router along cannot be opened by something three networks away.

The common reason to break it is peering between loopback addresses, so the session does not depend on any one of several parallel links, or peering with a router that sits one or more routers beyond the wire.

neighbor ebgp-multihop is the switch. Cisco describes it as accepting and attempting BGP connections to external peers on networks that are not directly connected, with an optional TTL from 1 to 255. One safety rule is built in: the session is not established if the only route to the peer is the default route, which Cisco says prevents loops through oscillating routes.

For loopbacks on a directly connected peer there is a lighter option. neighbor disable-connected-check turns off only the connected check, for a single hop session that uses a loopback interface, and neighbor update-source is what lets the router use its loopback for the session at all.

The cost is the property the default had. A higher TTL means packets from further away are now accepted as possible session traffic, which is exactly what the one hop default refused. That follows from what the TTL was doing rather than from any vendor sentence, and it is the reason the next mechanism exists.

TTL security puts it back. RFC 5082, the Generalized TTL Security Mechanism or GTSM, turns the field around: every packet for a protected session is sent with a TTL of 255, the highest it can carry, so the receiver can tell how many routers a packet crossed and drop what came from too far away.

The RFC states the aim as protecting a router's control plane from CPU exhaustion attacks, and names BGP as the example.

Cisco implements it for eBGP as neighbor ttl-security hops: the router works out the lowest TTL it will accept from the configured hop count and silently discards anything below it, without sending an ICMP message.

One replaces the other. Cisco documents ttl-security and ebgp-multihop as mutually exclusive: with TTL security configured on a multihop session, multihop can be neither configured nor needed. TTL security is also eBGP only, and Cisco does not support it for iBGP peers.

What neither of them does. GTSM cannot stop an attacker who is as close to the router as the legitimate peer, and the RFC says outright that it is not a substitute for authentication. Cisco says the same of its feature: it does not protect the integrity of the data between peers or authenticate them.

Failure modesWhere it goes wrong

BGP security is the protocol's oldest problem. The Border Gateway Protocol was designed among operators who knew each other, and it accepts the routing information it is told. That assumption is the root of everything below.

BGP hijacking. An autonomous system announces a prefix it does not own. If the announcement is more specific than the real one, a /24 against a /16, it wins everywhere, because routers always prefer the most specific match.

Data for those addresses arrives at the wrong network. It has been done by accident and on purpose, and the famous cases redirected significant traffic for hours before anyone noticed.

BGP route leaks. A network announces routing information it learned from one peer to another peer it should not have.

Nothing is stolen, but a small network suddenly advertises itself as a path between two large ones, and traffic that should never have touched it arrives in volume. The link saturates and both directions suffer. This is the most common cause of a large regional outage.

Withdrawing the wrong prefixes. An operator misconfigures a BGP filter, their own addresses stop being announced, and the network disappears from the internet while everything within it keeps working perfectly.

Every large provider has done this at least once, and it is worse than an ordinary outage because the tools used to fix it are often inside the network that vanished.

Flapping. A BGP session that goes up and down repeatedly sends the change across the internet each time. Route dampening exists to suppress a prefix that keeps changing, and it is used carefully, because the cure can outlast the disease.

Depending on BGP for failover without testing it. Convergence after a change is not instant. A withdrawal has to propagate across the internet, and a table of that size takes time to reconverge. Failover measured in seconds is not what BGP gives you by default.

SecurityMaking it trustworthy

Three BGP security mechanisms are in real use, and none of them is universal.

Prefix filters are the oldest and still the most effective. A network accepts from each BGP peer only the prefixes that peer is registered to announce, built from the routing registries. It works, and it depends on registry data being current.

RPKI attaches a cryptographic record to a prefix saying which AS is allowed to originate it. A BGP router validating against RPKI rejects external routing information whose origin does not match.

Adoption passed half of announced address space and continues to climb, and it stops the simplest hijacks. It says nothing about the rest of the AS path, which is the limit of what this security control buys.

Maximum prefix limits are the blunt instrument that works. A BGP session configured to shut down if a peer suddenly announces ten times its usual number of routes contains a leak in seconds rather than hours. It is the cheapest security control here and the one most often missing.

When it appliesDo you need BGP

Most networks do not, and being clear about that saves money.

You need the Border Gateway Protocol when you have addresses of your own and more than one internet provider, and you want data to keep arriving when one of them fails. That is the actual use case: multihoming with provider independent addresses.

You do not need BGP for a single connection, however fast. You do not need it for two connections where a firewall picks between them, which is what most small office networks run. And you do not need BGP inside a small network, where an internal routing protocol is simpler and faster.

Getting there means a registry membership, an AS number, an address block, and a BGP router that can hold a full table of routing information, or accept a default route from each provider instead.

The last option is common and works well: take a default route from each provider, announce your own prefix to both, and you have failover without holding a million routes of internet routing information.

ComparisonBGP, OSPF and a static route, on what each one is actually for

The row that decides it is the last one. Everything else BGP does can be done another way within a smaller network. Announcing your own address block to two providers cannot.

FAQFrequently asked questions

Keep readingRelated concepts

Read next · Network security What Is a Firewall? A small office with two providers usually wants a firewall choosing between them, not BGP. Open this next14 min

• Addressing · 15 min What Is a Subnet? A BGP announcement carries a prefix, and a prefix is a subnet written in CIDR form.
• Protocols · 10 min TCP vs UDP BGP is the routing protocol that runs over TCP, which is why it gets reliable delivery for free.
• Infrastructure · 11 min MPLS Explained The label switching that carriers run underneath the routing this page describes.
• Addressing · 11 min What Is CIDR? The addressing scheme behind every prefix a BGP router advertises.
• Routing · 12 min OSPF Explained The interior protocol that decides paths inside each of the autonomous systems BGP connects.
• Design · 9 min Mesh Topology, and the Three Places It Actually Exists How the internet core does it.
• Routing · 10 min BGP States, and Why Only Three of the Six Ever Appear on Screen What the neighbor session does before any of this, and why it stalls.
• Switching · 12 min What Is VXLAN? The routing protocol that distributes which address sits behind which VTEP.
• Routing · 12 min The IS-IS Protocol, and Why It Runs the Networks You Never See What carries the routes between organizations.
• Routing · 10 min The RIP Protocol, and Why You Still Meet It Where routing goes once one organization is not enough.
• Routing · 12 min Static vs Dynamic Routing, and the Route That Never Fails The protocol at the threshold static routing cannot cross.
• Routing · 10 min OSPF LSA Types, Explained by Who Sends Them and How Far They Flood How routes learned from outside OSPF travel inside it, as Type 5 and Type 7 LSAs.
• Routing · 9 min BGP Community, the Tag That Signals Routing Policy How BGP carries routes, which communities then tag for policy.