Cisco SD-WAN separates a router into four jobs and gives each to a different component. That sounds like an architecture diagram exercise and it has a very practical consequence: when something breaks, the first question is which of the four is involved, and the answer usually eliminates three quarters of the possibilities. A site that has lost its management connection but is still forwarding traffic is a completely different problem from a site that has lost its control connection.
The separation also explains the property that surprises people most. The controllers are not in the data path. An edge router that cannot reach any controller keeps forwarding traffic on the policy and routes it already holds, indefinitely. Controllers being down is a serious condition because nothing new can be learned or changed — not because traffic stops.
This article covers what each component actually does, the sequence by which an edge router joins the overlay and why that order cannot be shortened, how the four planes map onto the components, how to design controller redundancy, and the onboarding failure catalogue — which is dominated by certificates and identity rather than by anything resembling a routing problem. It is written for the lab rather than for the written exam, and sits alongside the rest of the CCIE Enterprise Infrastructure lab certification track.
What Does Each of the Four Components Actually Do?
What is the division?
vBond authenticates devices and tells them where the other controllers are — it is an introducer, and the only component that must be reachable from the public internet. vManage holds the configuration and the monitoring, and is where every change is made. vSmart distributes routes and policy, acting as a reflector so that edges never peer with each other for control. The WAN Edge builds the encrypted tunnels and forwards the traffic.
A Deeper Dive into the Components
vBond, which everything starts with
An edge router knows one thing when it powers on: how to reach vBond. Everything else — the addresses of vManage and vSmart, its own configuration, the overlay routes — arrives as a consequence of that first conversation. This is why vBond must be reachable from wherever edges are, why it needs a public address or a one-to-one translation to one, and why a vBond problem stops onboarding completely while affecting nothing that is already running.
! What the edge is configured to look for
Edge# show sdwan control local-properties
organization-name SPOTO-LAB
certificate-status Installed
root-ca-crt-status Installed
system-ip 10.255.1.11
site-id 111
domain-id 1
vbond 203.0.113.10
!
! And what vBond itself sees
vBond# show orchestrator connections
vBond# show orchestrator valid-vedges
vManage, whose absence is survivable
vManage is the configuration and monitoring system, not a forwarding element and not a control-plane element. An edge that loses its vManage connection continues running the configuration it already has, continues its OMP session, and continues forwarding. What stops is the ability to change anything and the ability to see current statistics — serious, and not an outage.
vSmart, which is a reflector rather than a router
Edges do not exchange routes with each other. Each maintains a session with vSmart, advertises what it knows, and receives what everyone else advertised. vSmart applies policy in the middle of that exchange, which is what makes centralised policy possible: changing what a site can reach is a change on vSmart rather than a change on every edge. Traffic never traverses vSmart.
! The control sessions from an edge's point of view
Edge# show sdwan control connections
PEER PEER SITE DOMAIN PEER
TYPE PROTOCOL SYSTEM IP ID ID PRIVATE IP STATE
------------------------------------------------------------------------------
vsmart dtls 10.255.0.2 1 1 203.0.113.20 up
vmanage dtls 10.255.0.1 1 0 203.0.113.30 up
vbond dtls - 0 0 203.0.113.10 up
!
! The OMP session, which is what carries the routes
Edge# show sdwan omp peers
Edge# show sdwan omp summary
The WAN Edge and what a TLOC is
A TLOC is the identity of a tunnel endpoint, made of three things: the router's system address, a colour naming the transport, and the encapsulation. An edge with two circuits advertises two TLOCs. Tunnels are built between pairs of TLOCs rather than between routers, which is what allows a policy to say "use the broadband path for this application" — the policy is selecting a TLOC, not a device.
! One TLOC per transport, on a cEdge
sdwan
interface GigabitEthernet1
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service dhcp
allow-service dns
allow-service icmp
no allow-service sshd
exit
exit
interface GigabitEthernet2
tunnel-interface
encapsulation ipsec
color mpls restrict
! ^ restrict = only build tunnels to the same colour
exit
exit
!
Edge# show sdwan bfd sessions
Edge# show sdwan omp tlocs
Why the jobs were separated at all
A traditional router does all four jobs itself: it is configured directly, it computes its own routes, it authenticates its own peers, and it forwards. That works and it scales badly, because every one of those jobs then has to be performed identically on every device by whoever is holding the console. Separating them means the three that benefit from a global view are done once, centrally, and only forwarding stays distributed.
The clearest illustration is policy. Expressing "branches in this region reach each other directly and everything else goes via the data centre" on a traditional network means a consistent configuration on every branch router, maintained forever. Expressing it in this architecture means one policy on the controllers, and every edge receives a view of the network consistent with it. The saving is not typing; it is that the policy cannot be inconsistent between sites, because there is only one copy of it.
Colours, which decide who can reach whom
A colour names a transport and carries one behavioural rule: private colours expect to reach each other using private addressing, and public colours use the address seen after translation. Two sites on private colours across the public internet will not form a tunnel, because each is trying to reach an address the other cannot route to. This is the single most common cause of tunnels that refuse to come up between otherwise healthy sites.
| Colour type | Examples | Address used | Suits |
|---|---|---|---|
| Private | mpls, metro-ethernet, private1–private6 |
Pre-translation (private) | A private WAN where both ends see each other directly |
| Public | biz-internet, public-internet, lte |
Post-translation (public) | Anything crossing the internet or a NAT |
How Does an Edge Router Join the Overlay?
What is the sequence?
Address, then vBond, then the controllers vBond named, then routes, then tunnels. The order is a dependency chain rather than a convention: the edge cannot reach vManage until vBond has told it where vManage is, and it cannot build a tunnel to another site until vSmart has told it that site exists. Knowing the order means a failure at any point identifies which step did not complete, and everything after it is a consequence rather than a separate fault.
A Deeper Dive into Onboarding
What has to match before anything works
Three values must agree across every device in the overlay, and a mismatch in any of them produces a device that cannot authenticate. The organisation name, which is compared against the certificate. The root certificate authority, which must be the same one that signed every device. And the device's own identity, which must appear in the authorised list that vManage distributes. None of these produces a network-level symptom; all of them produce a control connection that will not establish.
! The three that must agree, seen on the edge
Edge# show sdwan control local-properties | include organization|certificate|root-ca
organization-name SPOTO-LAB
certificate-status Installed
certificate-validity Valid
root-ca-crt-status Installed
!
! The same value on the controllers
vManage# show running-config system organization-name
vBond# show running-config system organization-name
! ^ A single character of difference is a total failure.
!
! Is this device authorised at all?
vBond# show orchestrator valid-vedges | include 10.255.1.11
Reading the connection history
The current state tells you whether a connection is up; the history tells you why it is not. Every failed attempt is recorded with a reason code, and those codes are specific enough to identify the cause directly — a certificate that did not verify, a device not in the authorised list, a timeout reaching vBond. This is the single most useful command during onboarding and it is frequently skipped in favour of re-checking configuration that was already correct.
! Why did it fail, rather than that it failed
Edge# show sdwan control connections-history
!
! Reason codes worth recognising:
! VB_TMO - could not reach vBond at all
! DCONFAIL - the DTLS connection itself failed
! CRTVERFL - certificate verification failed
! BIDNTVRFD - board ID not verified (identity/whitelist)
! CTORGNMMIS - organisation name mismatch
! NOZTPSP - no ZTP service provider found
!
Edge# show sdwan control statistics
Reaching vBond, which is plain routing
Before any of the SD-WAN machinery matters, the edge needs an address on its transport interface and a route towards vBond. That part is ordinary networking and it is worth confirming first, because a firewall rule or a missing default route produces exactly the same symptom as a certificate problem — a control connection that will not come up.
! Ordinary connectivity, before anything SD-WAN specific
Edge# show ip interface brief
Edge# show ip route 0.0.0.0
Edge# ping vrf 0 203.0.113.10
! ^ VPN 0 is the transport VRF. The controllers live there.
!
! What the tunnel interface will and will not accept
Edge# show sdwan running-config sdwan | section tunnel-interface
! ^ allow-service lines decide whether ping, ssh and dhcp
! are answered on the transport - a common reason a test
! ping fails while the overlay is perfectly healthy
What the certificate actually proves
Mutual authentication means both sides prove who they are, and it is worth being precise about what that establishes. The certificate proves the device is the specific hardware it claims to be and that a trusted authority vouched for it. It does not, by itself, prove the device belongs in this overlay — that is what the authorised list adds. Two separate checks, and confusing them is why a device with a perfectly valid certificate can still be refused.
The practical consequence is that the two failures need different fixes. A certificate problem is resolved by installing the right root or reissuing the device certificate. An authorisation problem is resolved by adding an identity to a list and distributing it, with the certificate untouched throughout. The connection history distinguishes them, which is the reason to read it before doing anything else.
Zero-touch provisioning
An edge with no configuration can find its vBond through a hosted provisioning service rather than being told manually, which is what makes shipping a router directly to a branch practical. It requires the device to be claimed against the organisation in that service, to have internet access with name resolution, and to be in the authorised list. When it does not work, the reason is almost always one of those three rather than anything on the device.
| Onboarding method | Needs | Suits | Usual failure |
|---|---|---|---|
| Manual bootstrap | Console access, minimal config typed | Lab, controlled deployments | Typo in the vBond address or organisation |
| Bootstrap file | A generated file placed on the device | Staging before shipping | File not read at boot |
| Zero-touch | Claimed device, DNS, internet | Shipping direct to site | Not claimed, or DNS blocked |
A minimal bootstrap configuration
! The smallest thing that will join an overlay
system
system-ip 10.255.1.11
site-id 111
organization-name SPOTO-LAB
vbond 203.0.113.10
!
! A transport interface in VPN 0 with a tunnel on it
vrf definition 0
!
interface GigabitEthernet1
ip address dhcp
!
sdwan
interface GigabitEthernet1
tunnel-interface
encapsulation ipsec
color biz-internet
allow-service dhcp
allow-service dns
allow-service icmp
exit
exit
!
! Then confirm, in this order
Edge# show sdwan control local-properties
Edge# show sdwan control connections
Edge# show sdwan omp peers
How Do the Four Planes Map Onto the Components?
Why does the mapping matter?
Because it tells you what an outage of any one component actually costs. Orchestration down means no new devices join. Management down means no changes and no visibility. Control down means no new routes and no policy updates, with existing forwarding continuing. Data plane down means traffic stops. Only the last is an outage in the sense a user would recognise, and conflating the four is what produces incident calls where nobody can say whether anything is actually broken.
A Deeper Dive into the Planes
What each outage actually costs
| Component down | Traffic | New sites | Config changes | Route changes |
|---|---|---|---|---|
| vBond | Unaffected | Blocked | Unaffected | Unaffected |
| vManage | Unaffected | Blocked | Blocked | Unaffected |
| vSmart | Unaffected while state holds | Blocked | Policy blocked | Blocked |
| One WAN Edge | That site only | — | — | Its routes withdrawn |
Why the control plane outage is the subtle one
Losing vSmart does not drop traffic, because each edge holds the routes and policy it already received and continues to forward on them. What stops is adaptation: a site that fails cannot have its routes withdrawn from everyone else, a new policy cannot be pushed, and a new tunnel cannot be established. The overlay is frozen rather than broken, and it degrades only when something changes that it can no longer react to.
! What an edge still holds with no controller reachable
Edge# show sdwan omp routes | count
Edge# show sdwan bfd sessions | include up | count
Edge# show sdwan control connections
! ^ connections down, and yet:
Edge# show sdwan policy from-vsmart
! ^ the policy it last received is still in effect
!
! The graceful restart timer governs how long it holds
Edge# show sdwan omp summary | include graceful|timer
Where policy lives and where it is applied
Centralised policy is configured on vManage, activated onto vSmart, and applied by vSmart as it reflects routes — which means the edge receives an already-filtered view rather than applying the policy itself. Localised policy is pushed to the edge and applied there. Knowing which kind a rule is tells you where to look when it does not behave: the vSmart side for anything shaping what a site learns, the edge side for anything acting on packets as they arrive.
! What this edge received after vSmart applied policy
Edge# show sdwan policy from-vsmart
Edge# show sdwan omp routes vpn 10 | include received|installed
!
! Versus what is configured locally on the edge
Edge# show sdwan policy access-list-counters
Edge# show sdwan running-config policy
!
! And on vSmart, what it is actually enforcing
vSmart# show running-config policy
vSmart# show omp routes vpn 10 detail
Stating impact during an incident
The most useful thing to establish in the first two minutes is which plane is affected, because that determines both the urgency and who needs to be involved. A useful form of words is to state what still works rather than what is broken: traffic is flowing and we cannot make changes, or new sites cannot join and everything existing is unaffected, or this one site is down and the rest of the estate is fine.
That framing also prevents the common escalation error, which is treating a controller outage as a network outage and pulling in people who have nothing to contribute. Three commands on any edge — control connections, OMP peers, BFD sessions — produce the evidence for that statement, and running them before the first update makes the update accurate.
The transport VRF
Transport interfaces and the controller connections live in a dedicated routing context, separate from every user network. That separation is what allows the underlay routing to be entirely different from the overlay, and it is why a ping test has to specify which context it is running in. A test that appears to fail because it was issued in the wrong context is a routine early confusion. Reading this once is not the same as being able to do it under time pressure, which is what repetition against realistic CCIE lab practice scenarios is for.
How Do I Design for Redundancy and Scale?
What needs to be redundant?
All three controller roles, and for different reasons. vBond because onboarding stops without it and because a new site cannot wait. vSmart because losing it freezes the control plane, and because policy changes are the mechanism by which incidents get resolved. vManage because it holds the configuration database and the historical data, which is the one thing in the system that cannot be reconstructed from the network itself.
A Deeper Dive into Design
Multiple vBonds behind one name
Edges are configured with a vBond address, and pointing that at a name that resolves to several addresses is what makes vBond redundant — the edge tries them in turn. This costs nothing and it is the difference between one vBond failure blocking every new site and it being invisible. Configuring a literal address instead is the common shortcut that gets discovered during the first outage.
! Point at a name, not an address
system
vbond vbond.example.com
!
! The edge resolves it and will try each answer
Edge# show sdwan control local-properties | include vbond
Edge# show sdwan control connections-history | include vbond
!
! Confirm resolution works from the transport context
Edge# ping vrf 0 vbond.example.com
How many vSmarts, and how edges use them
Two is the minimum and each edge maintains sessions to more than one, so a controller loss is not a control-plane event for the sites connected to the survivor. The number of sessions an edge maintains is configurable, and raising it costs controller capacity while lowering it costs resilience. The setting worth understanding is the one limiting sessions per edge, because at scale it decides how much load each controller carries.
! How many controller sessions this edge will hold
sdwan
interface GigabitEthernet1
tunnel-interface
max-control-connections 2
! ^ per tunnel interface
exit
exit
!
system
max-omp-sessions 2
!
! Verify the edge really has more than one
Edge# show sdwan control connections | include vsmart
Edge# show sdwan omp peers
! ^ two vSmart peers in Up state is the target
Templates, and why they are part of the redundancy story
Configuration in this architecture is normally applied through templates rather than typed per device, and that has a resilience consequence beyond convenience. A site rebuilt from a template comes back identical to how it was, in minutes, without anyone needing to remember what was on it. A site configured by hand comes back as whatever the person rebuilding it can reconstruct.
That makes the template library part of the recovery plan rather than merely an efficiency tool, and it is another argument for protecting the configuration database. The network survives losing vManage; the ability to rebuild a failed site quickly does not, unless the templates are backed up somewhere independent of it.
vManage clustering
A single vManage is a single point of failure for configuration and for every piece of historical data the system has collected. Clustering distributes the database and the services across several nodes so that losing one is survivable. It is more involved to build and operate than a single node, and the argument for it is not availability of the network — which is unaffected — but availability of the configuration record.
Where to place controllers
| Placement | Suits | Advantage | Consideration |
|---|---|---|---|
| Cloud-hosted by the vendor | Most deployments | No infrastructure to run | Dependency on internet reachability |
| Self-hosted, one data centre | Regulatory constraints | Full control | The data centre becomes a dependency |
| Self-hosted, two data centres | Large self-managed estates | Survives a site loss | More to build and keep in step |
What to monitor
Four things, and only one of them is about traffic. Control connection count per edge, alerting when it falls below the designed number rather than when it reaches zero, because the gap between two sessions and one is the warning and the gap between one and none is the incident. Tunnel session count per edge against its expected peer list. Certificate expiry dates across every device, which is a calendar item rather than a threshold. And the authorised list count against the device inventory.
The certificate one deserves emphasis because it is the failure that takes down an entire overlay at once and does so on a date that was known years in advance. Controller certificates expiring simultaneously is a recognised way to lose everything, and the only defence is a diary entry and a renewal process that someone owns.
! The four, collected on a schedule
!
Edge# show sdwan control connections | count up
! ^ compare against the designed number, not against zero
!
Edge# show sdwan bfd sessions | count up
! ^ compare against the expected peer count for this site
!
Edge# show sdwan certificate validity
vManage# show certificate validity
! ^ a calendar item; expiry takes the whole overlay down
!
vBond# show orchestrator valid-vedges | count
! ^ compare against the device inventory
Scaling the data plane rather than the controllers
The number that grows fastest in a large overlay is not controller load but tunnel count, because a full mesh of sites each with two transports grows quadratically. Controlling that is a policy exercise: a hub-and-spoke or regional-mesh topology built with centralised policy means each edge holds tunnels to a handful of peers rather than to everyone. This is the main reason topology policy exists and it is a capacity decision rather than an aesthetic one.
! How many tunnels is this edge actually maintaining?
Edge# show sdwan bfd sessions | count up
Edge# show sdwan omp tlocs | count
!
! And the effect of a topology policy limiting them
Edge# show sdwan policy from-vsmart | include topology|tloc
!
! Watch these as the estate grows - they are the
! numbers that bind long before controller capacity does
Edge# show platform resources
Which Onboarding Failures Look Like Network Problems?
What are the failures worth memorising?
Four, and three of them are identity rather than connectivity. An organisation name that does not match. A device missing from the authorised list. A certificate that does not verify against the installed root. And, the genuinely network-shaped one, a firewall permitting the wrong ports or a colour mismatch that prevents two sites from building a tunnel even though both reach the controllers perfectly.
A Deeper Dive into the Failure Catalogue
The identity mismatch
show sdwan control connections-history shows a mismatch reason code; compare the value on the edge against vBond and vManage character by character. Fix: correct the value on the device — and note that it is not something to guess at, since the authoritative version is the one embedded in the certificates.The device nobody authorised
show orchestrator valid-vedges on vBond does not list the device's identity; the connection history shows an identity verification failure. Fix: add the device to the list and push it to the controllers, then confirm on vBond rather than assuming the push landed.The colour mismatch
show sdwan bfd sessions shows no session between the pair; show sdwan omp tlocs shows the remote TLOC with a private address. Fix: use a public colour on any transport that crosses translation, keeping private colours for genuinely private circuits.The transport that blocks the ports
Control connections and data tunnels use a defined port range, and a transport provider or an enterprise firewall permitting only part of it produces selective failures — controllers reachable and tunnels not, or one colour working and another not. Because the symptom is partial, it reads as an overlay problem rather than as a filtering one, and the fastest way to distinguish them is to test the specific path rather than the overlay as a whole.
! Test the specific path rather than the overlay
Edge# ping vrf 0 <remote-transport-ip>
Edge# show sdwan bfd sessions
Edge# show sdwan bfd history | include <remote-system-ip>
!
! Which local address and port is being used
Edge# show sdwan control local-properties | include port|public|private
!
! And whether the tunnel interface is even allowing it
Edge# show sdwan running-config sdwan | section tunnel-interface
A diagnostic order that works
! Identity first, then reachability, then the overlay
!
! 1. Does this device have a valid identity?
Edge# show sdwan control local-properties | include organization|certificate|root-ca
!
! 2. Was it ever admitted?
vBond# show orchestrator valid-vedges | include <system-ip>
!
! 3. Why did the attempts fail?
Edge# show sdwan control connections-history
!
! 4. Plain connectivity to vBond
Edge# ping vrf 0 <vbond>
!
! 5. Only now the overlay itself
Edge# show sdwan control connections
Edge# show sdwan omp peers
Edge# show sdwan bfd sessions
Onboarding an estate rather than a device
The difference between onboarding one router and onboarding two hundred is entirely in the preparation. Every device added to the authorised list before any hardware ships, the organisation name and root certificate settled and never touched again, a template validated on one site before it is attached to the rest, and the transport firewall rules confirmed once per provider rather than per site. Done in that order, each site becomes a few minutes of work.
Done in the other order — shipping first and resolving identity per site as problems arise — every site becomes an investigation, and they are all the same investigation. The failures in the catalogue above are not site-specific; they are estate-wide conditions that happen to be discovered one site at a time. Fixing them once, before the first shipment, is the whole difference.
Blueprint framing
The CCIE Enterprise Infrastructure v1.1 blueprint places SD-WAN in the software-defined infrastructure domain, and the questions concentrate on the architecture rather than on the interface: what each controller does, in what order a device joins, what a TLOC is made of, and which failures stop traffic as opposed to stopping change. Being able to state the four components and the onboarding sequence covers most of what is asked.
Conclusion
Four components, four jobs, and the separation is what makes the architecture comprehensible. vBond introduces devices and is the only one that must be publicly reachable. vManage holds configuration and history. vSmart distributes routes and policy as a reflector. The edge builds the tunnels and forwards traffic. Every question about impact starts by identifying which of the four is involved.
The property worth internalising is that none of the controllers is in the data path. An overlay whose every controller is unreachable continues forwarding on the state it already holds — it is frozen, not broken, and it degrades only when something changes that it can no longer react to. That distinction is the difference between an incident that needs a response tonight and one that needs a response now, and the two get reported in identical language.
Onboarding is where most early time is lost, and mostly on things that are not networking. The organisation name must match exactly, the device must appear in the authorised list, and the certificate must verify against the installed root. All three fail identically to a reachability problem and all three are identified in one command — the connection history, whose reason codes name the actual cause rather than the symptom. More CCIE Enterprise Infrastructure material — labs, protocol breakdowns and study guides — is collected on the SPOTO CCIE site.
External Links
- RFC 6347 — Datagram Transport Layer Security Version 1.2
- RFC 4301 — Security Architecture for the Internet Protocol
- RFC 5880 — Bidirectional Forwarding Detection (BFD)
- RFC 5280 — Internet X.509 Public Key Infrastructure Certificate and CRL Profile
- Cisco — Cisco SD-WAN Getting Started Guide
- Cisco Design Zone — WAN and Branch Design Guides
- Cisco Learning Network — CCIE Enterprise Infrastructure
Reference Notes
- Cisco SD-WAN documentation describes the four planes — orchestration, management, control and data — and maps them to vBond, vManage, vSmart and the WAN Edge respectively.
- Cisco documentation describes vBond as the orchestrator that authenticates devices and informs them of the vManage and vSmart addresses, and notes that it requires a publicly reachable address.
- Cisco documentation describes vSmart as distributing OMP routes and centralised policy, and states that it does not participate in data forwarding.
- Cisco documentation defines a TLOC as the combination of system IP address, colour and encapsulation, identifying a transport tunnel endpoint.
- Cisco documentation distinguishes private colours, which use the pre-NAT address, from public colours, which use the post-NAT address.
- Cisco documentation states that WAN Edge devices at the same site identifier do not form data plane tunnels with one another by default.
- RFC 6347 specifies DTLS, one of the two transports used for SD-WAN control connections; TLS is the alternative.
- RFC 4301 defines the IPsec security architecture underlying the encrypted data plane tunnels between WAN Edge devices.
- RFC 5880 specifies BFD, which SD-WAN uses between WAN Edge devices to detect data plane tunnel liveness and measure path characteristics.
- RFC 5280 defines the X.509 certificate profile underlying the mutual authentication performed during device onboarding.
- Cisco documentation describes the authorised device list distributed by vManage to the controllers, without which a device cannot establish control connections.
- The CCIE Enterprise Infrastructure v1.1 unified exam topics include SD-WAN within the software-defined infrastructure domain.