Application-aware routing is the feature people buy SD-WAN for and the one most often configured in a way that cannot deliver what was expected. The idea is simple: measure each path continuously, and put each application on a path that currently meets its requirements. The difficulty is entirely in the details of that sentence — what is measured, how often, over what window, and what happens when nothing qualifies.
The default measurement window is the detail that surprises people most. Out of the box the decision is made from a long average, which is deliberate and makes the overlay stable, and which also means a path that degrades will carry traffic for a considerable time before anything moves. Anyone expecting a reaction within seconds of a circuit going bad is expecting behaviour the defaults do not provide.
This article covers what is measured and by what mechanism, how a service-level class is defined and attached to traffic, what happens when no path qualifies — which is a design decision with two opposite answers — how to stop the whole thing oscillating, and the behaviours that get reported as faults and are the feature working correctly. It is written for the lab rather than for the written exam, and sits alongside the rest of the CCIE Enterprise Infrastructure lab certification track.
The liveness probes carry the measurements. The decision walks preferred colour, any qualifying path, the named backup, and finally either the least-bad path or nothing at all.
What Does Application-Aware Routing Measure, and How?
Where do the numbers come from?
From the same probes that detect whether a tunnel is alive. Each tunnel between two endpoints is continuously probed, and those probes yield loss, latency and jitter as a side effect of establishing liveness. No separate measurement traffic is generated, which is why the feature costs almost nothing to run and why its measurements reflect the actual tunnel rather than a synthetic path alongside it.
A Deeper Dive into Measurement
Per tunnel, not per site
Measurements exist for each pair of endpoints, which means a site with two transports talking to a site with two transports has four separately measured paths. The decision is made among those, per application, which is what allows one application to use the private circuit to a destination while another uses the internet circuit to the same destination at the same moment.
! The measurements, per tunnel
Edge# show sdwan app-route stats
!
Edge# show sdwan bfd sessions
SYSTEM IP SITE ID STATE SOURCE TLOC REMOTE TLOC ...
10.255.1.12 112 up mpls mpls
10.255.1.12 112 up mpls biz-internet
10.255.1.12 112 up biz-internet mpls
10.255.1.12 112 up biz-internet biz-internet
! ^ four paths to one site, each measured separately
!
Edge# show sdwan app-route sla-class
The window, which is the thing to understand
Measurements are collected into buckets and a decision is made from the average of several buckets. The bucket length and the number averaged are both configurable, and their product is the effective window — how much history the decision reflects and therefore how long a change takes to influence it. The defaults produce a window measured in tens of minutes, which is stable and slow.
! The two settings whose product is the window
bfd
app-route poll-interval 120000
! ^ milliseconds. Default is 600000 (10 minutes).
app-route multiplier 5
! ^ buckets averaged. Default is 6.
!
! Default: 10 min x 6 = ~60 minutes of history
! Above: 2 min x 5 = ~10 minutes of history
!
Edge# show sdwan bfd summary | include poll|multiplier
Liveness and quality are different reactions
A tunnel that fails outright is detected in seconds by the liveness mechanism and traffic moves immediately — that has nothing to do with the measurement window. The window governs decisions based on a path being poor rather than dead. Conflating the two is the source of the expectation that this feature reacts instantly, and it is worth separating explicitly when discussing what the network will do.
| Event | Detected by | Reaction time | Governed by |
|---|---|---|---|
| Tunnel fails completely | Liveness probes | Seconds | Probe interval and multiplier |
| Path becomes lossy | Quality measurement | The window | Poll interval × multiplier |
| Path recovers | Quality measurement | The window | Same settings |
| Circuit physically down | Interface state | Immediate | Nothing configurable |
Why measuring the tunnel is the right choice
An alternative design would measure the circuit — its utilisation, its error counters, its provider-reported availability — and steer on that. Measuring the tunnel instead captures everything that actually affects the application: the circuit, the provider's network, any congestion at either end, and the far site's own handling. A circuit that is lightly loaded and reaching its destination through a congested provider core measures badly here and looks perfect in the circuit statistics.
It also means the measurement is inherently directional and pairwise. A transport that performs well towards one site and poorly towards another produces exactly that picture, which is both accurate and initially surprising to anyone who expects a circuit to have a single quality figure. The practical benefit is that steering decisions are made per destination rather than per circuit, which is what applications actually experience rather than what a circuit report describes.
The probe settings underneath
! Liveness probing, per colour
bfd color mpls
hello-interval 1000
multiplier 7
pmtu-discovery
!
bfd color biz-internet
hello-interval 1000
multiplier 7
!
! ^ this decides how fast a DEAD tunnel is noticed.
! The app-route settings decide how fast a POOR one is.
!
Edge# show sdwan bfd sessions | include DETECT|hello
How Is an SLA Class Defined and Applied?
What is a class?
Three thresholds — loss, latency and jitter — that a path must satisfy simultaneously to qualify. A path exceeding any one of them does not qualify, however good the other two are. The class is then attached to traffic by a policy sequence that also names a preferred transport, and the combination of those two is what produces the behaviour: use this transport if it is good enough, otherwise anything else that is.
A Deeper Dive into Classes
Defining classes that mean something
The thresholds should come from what the application actually requires rather than from what sounds strict. A class demanding a latency no path can achieve means the application never has a qualifying path and always lands in the fallback, which is the opposite of what was intended. Starting from the application's documented requirements, and checking them against the measurements the network is already producing, avoids writing a class nothing can satisfy. The measurements needed for that check are already being collected before any policy exists.
! Classes, with thresholds that reflect real requirements
policy
sla-class VOICE
loss 1
latency 150
jitter 50
!
sla-class BUSINESS-CRITICAL
loss 2
latency 250
jitter 100
!
sla-class BULK
loss 5
latency 500
!
!
! Check them against what paths actually achieve
Edge# show sdwan app-route stats
! ^ if no path ever meets VOICE, the class is wrong,
! not the network
Start with fewer applications than you plan to end with
The temptation on first deployment is to classify everything, which produces a policy with many sequences whose behaviour nobody has observed. Starting with the two or three applications that genuinely have quality requirements — typically voice, video and one business system — gives a policy small enough to verify and a set of observations that inform every later addition.
It also surfaces the measurement reality early. Seeing what the paths actually achieve for real traffic tends to change the thresholds people intended to use, and discovering that on three sequences is considerably less work than discovering it on twenty. Additional applications can then be added against classes that are known to behave.
Attaching a class to traffic
! The policy that connects traffic to a class
policy
app-route-policy BRANCH-AAR
vpn-list CORPORATE
sequence 10
match
app-list VOICE-APPS
action
sla-class VOICE preferred-color mpls
count VOICE-MATCHED
!
!
sequence 20
match
app-list BUSINESS-CRITICAL-APPS
action
sla-class BUSINESS-CRITICAL preferred-color-group PRIMARY
!
!
sequence 30
match
dscp 8
action
sla-class BULK preferred-color biz-internet
!
!
default-action
!
!
!
apply-policy
site-list BRANCHES
app-route-policy BRANCH-AAR
!
!
Preferred colour versus requirement
The preferred colour is a preference, not a constraint. If it meets the class, it is used; if it does not, the traffic moves to something that does. That is the entire point of the feature and it is occasionally misread as a guarantee that traffic will use that transport, which it is not. Where a transport genuinely must be used regardless of quality, that is a data policy pinning rather than an application-aware routing preference.
| Requirement | Mechanism | Behaviour when the path degrades |
|---|---|---|
| Use the private circuit if it is healthy | Preferred colour with an SLA class | Moves to another qualifying path |
| Always use the private circuit | Data policy TLOC pinning | Stays, however bad it gets |
| Never use the internet circuit for this | Class plus a restricted colour set | Falls back within the allowed set only |
| Drop rather than degrade | Strict mode | Traffic stops |
Where this policy sits relative to the others
Application-aware routing runs alongside data policy rather than instead of it, and the two can both have an opinion about which path a packet takes. A data policy that pins traffic to a transport removes the choice this feature exists to make, so the two working on the same traffic is usually a configuration accident rather than a design. Establishing which policy owns which traffic keeps both comprehensible.
The division that works is to let application-aware routing own everything where the requirement is about quality, and data policy own everything where the requirement is about identity or security — steering through a service, blocking, direct internet access. Where both must apply to the same traffic, the interaction is worth testing explicitly rather than reasoning about, because the outcome depends on release behaviour that is not obvious from either policy.
Verifying the policy is doing something
Two checks. The counter on the sequence proves traffic is being matched, and the per-class state shows which paths currently qualify. A busy counter with no qualifying paths means the fallback is in use, which is a different situation from a zero counter — the first is the feature working under bad conditions, the second is the policy not matching.
! Is the sequence matching?
Edge# show sdwan policy app-route-policy-filter
!
! Which paths currently qualify for each class?
Edge# show sdwan app-route sla-class
!
! And what the policy actually received from the controller
Edge# show sdwan policy from-vsmart | include app-route|sla
!
! Busy counter + no qualifying paths = fallback in use
! Zero counter = not matching at all
show sdwan app-route stats for the tunnels in question, compared against the class thresholds. Fix: set thresholds from measured reality with headroom, rather than from an aspiration — a class that never qualifies provides no steering at all.What Happens When No Path Meets the SLA?
What are the options?
Three, in order. A named backup transport can be used, which is a deliberate second choice rather than an arbitrary one. Failing that, either the traffic goes on the least-bad available path, or it is discarded. Which of those last two happens is a configuration choice, and it is the same discard-or-degrade decision that appears anywhere a quality guarantee meets reality.
A Deeper Dive into Fallback
The named backup
Specifying a backup transport gives the fallback a defined destination rather than leaving it to whatever measures best. That matters when the transports differ in cost or in policy — falling back from a private circuit to a cellular connection that is billed by the gigabyte is a decision somebody should make deliberately rather than something that happens because the measurements pointed there.
! A deliberate second choice, rather than whatever wins
sequence 10
match
app-list VOICE-APPS
action
sla-class VOICE preferred-color mpls
backup-sla-preferred-color biz-internet
! ^ used when nothing meets the class
!
!
! Note what this does NOT say: it does not add the
! backup to the qualifying set. It names where to go
! when the qualifying set is empty.
Strict, and what it costs
Marking a class strict means traffic is discarded when no path qualifies. For a service where degraded quality is worse than no service — some voice and video deployments take this view — it is the correct choice and it needs to be understood as converting a quality problem into an outage. For everything else, degrading is preferable, and the default behaviour of carrying on is what most traffic should do.
! Strict: no qualifying path means the traffic stops
sequence 10
match
app-list VOICE-APPS
action
sla-class VOICE strict preferred-color mpls
!
!
! Without 'strict', the same situation means the
! traffic uses the best available path and the users
! experience poor quality rather than none.
!
Edge# show sdwan policy from-vsmart | include strict
Which to choose
| Traffic | Setting | Rationale |
|---|---|---|
| Interactive voice | Strict, arguably | A call that fails is better than one nobody can use |
| Video conferencing | Usually not strict | Degrades gracefully; users tolerate it |
| Business applications | Not strict | Slow is far better than unavailable |
| Bulk transfer | Not strict | Quality is irrelevant to the outcome |
| Anything without an owner to ask | Not strict | Do not convert degradation into an outage by default |
Fallback is not failover
The two words get used interchangeably and describe different events here. Failover is what happens when a tunnel dies: liveness detects it in seconds and traffic moves because the path no longer exists. Fallback is what happens when every surviving path is alive and none is good enough: the traffic is placed somewhere by the rules above, on a path that is working but does not meet the target.
The distinction matters when writing a runbook, because the two have different causes and different fixes. Failover points at something broken and usually resolves itself when the circuit returns. Fallback points at quality, which may mean congestion, a provider problem, or a threshold that was always too tight — and none of those resolves by waiting.
Making the choice visible
Strict mode produces an outage for a specific application under conditions that are not a network failure, which is a difficult thing to diagnose if nobody knows the setting exists. It belongs in the runbook for that application, phrased as the observable symptom: this application stops working when no path meets its quality target, by design, and the fix is to restore path quality rather than to look for a fault. Reading this once is not the same as being able to do it under time pressure, which is what repetition against realistic CCIE lab practice scenarios is for.
! During an incident, the two commands that explain it
Edge# show sdwan app-route sla-class
! ^ which classes currently have qualifying paths
!
Edge# show sdwan app-route stats
! ^ the actual loss, latency and jitter per tunnel
!
! If a class has no qualifying paths and is strict,
! the application is down by design and the fix is
! upstream in the transport, not in the overlay.
How Do I Stop It Oscillating?
Why does it oscillate?
Because a path sitting near a threshold crosses it repeatedly, and each crossing moves traffic. Moving traffic changes the load on both paths, which changes their measurements, which can move the traffic back. Shortening the measurement window to get faster reaction makes this more likely, which is the central tension in tuning this feature: responsiveness and stability pull in opposite directions.
A Deeper Dive into Stability
The window is the main control
A longer window averages away brief excursions and produces a decision that changes rarely. A shorter one reacts to real degradation quickly and also to noise. The useful target is a window long enough that a transient does not move traffic and short enough that a genuine problem is acted on before users have finished reporting it — which in most deployments is minutes rather than either seconds or the default hour. Landing somewhere in that range deliberately is better than accepting either extreme by default.
! A middle ground that suits most deployments
bfd
app-route poll-interval 120000
app-route multiplier 5
! ^ ~10 minutes of history: a two-minute transient
! moves the average very little, a sustained problem
! moves it decisively
!
! Watch the effect before and after a change
Edge# show sdwan app-route stats
Edge# show sdwan app-route sla-class
! ^ run repeatedly; a class whose qualifying set keeps
! changing is the definition of oscillation
Thresholds with headroom
A threshold set at exactly the typical performance of a path guarantees that path will cross it constantly. Setting thresholds with margin — comfortably above what the good path normally achieves and comfortably below what a degraded one does — creates a gap that noise does not bridge. That is cheaper and more effective than any timer tuning, and it is the first thing to adjust when a class is unstable. It also has no cost in responsiveness, which timer changes always do.
! Measure first, then set with margin
Edge# show sdwan app-route stats
! suppose the private circuit normally shows
! loss 0.1%, latency 40ms, jitter 8ms
!
! A class set at loss 0 / latency 45 / jitter 10 will flap.
! A class set with headroom will not:
policy
sla-class BUSINESS-CRITICAL
loss 2
latency 150
jitter 40
!
!
! Now normal variation stays well inside the class, and
! only genuine degradation falls outside it.
The feedback loop nobody designs
Moving traffic changes the conditions that caused it to move. A congested internet circuit improves when the voice traffic leaves it, which can bring it back within the class, which brings the traffic back, which congests it again. This loop is not hypothetical on sites where the WAN traffic is a significant fraction of the circuit's capacity, and it is invisible from any single measurement.
The defence is the same as for any oscillation: a window long enough that the loop cannot complete within it, and thresholds far enough apart that the traffic's own contribution does not bridge the gap. Recognising the pattern matters because the instinct when traffic keeps moving is to shorten the window and react faster, which makes this particular cause considerably worse.
Fewer classes, further apart
Several classes with thresholds close together produce traffic that shuffles between them as conditions vary slightly. Three well-separated classes — something like strict, normal and tolerant — cover the great majority of requirements and behave predictably. The instinct to define a class per application produces a policy that is both harder to reason about and less stable, for no observable benefit. Applications with genuinely distinct requirements are rarer than that instinct suggests.
Tuning by observation rather than by target
Choosing window and threshold values by deciding what reaction time sounds acceptable produces numbers unconnected to the network they will run on. The productive order is the reverse: leave the defaults, watch the measurements for a week, and then set values that would have produced sensible behaviour against the variation actually observed.
That week of observation is also the only reliable way to discover what normal looks like per site. A branch on a domestic broadband service and one on a metro fibre circuit have different normal ranges, and a single set of thresholds across both will be wrong somewhere. Whether that matters enough to warrant different classes per site type is a judgement the observation informs and nothing else does.
Watching for it
| Observation | Means | Action |
|---|---|---|
| Qualifying set changes every few minutes | Oscillation | Lengthen the window or widen the thresholds |
| Qualifying set never changes | Stable, or the class is unsatisfiable | Check the class is ever met |
| Changes only during business hours | Real congestion | A capacity question, not a tuning one |
| One tunnel flaps, others steady | That path or its far end | Investigate the specific circuit |
Which Behaviours Look Like Faults and Are Not?
What gets reported incorrectly?
Four things. Traffic on the internet circuit when everyone expects it on the private one, which is usually the feature working. A delay before traffic moves after a circuit degrades, which is the window. An application that stops entirely during degradation, which is strict mode. And two applications to the same destination taking different paths, which is the whole point and reads as an inconsistency.
A Deeper Dive into Misread Behaviour
Traffic on the "wrong" transport
show sdwan app-route stats for the private path, compared against the class thresholds — one of the three will be exceeded. Fix: nothing in the overlay. The finding is about the private circuit's quality, and the correct action is to pursue that with whoever provides it.The delay before anything happens
Different paths for different applications
A diagnostic order that works
! Is the traffic matched, and what does it require?
Edge# show sdwan policy app-route-policy-filter
Edge# show sdwan policy from-vsmart | include sla-class|preferred
!
! What do the paths currently measure?
Edge# show sdwan app-route stats
!
! Which paths qualify for the relevant class?
Edge# show sdwan app-route sla-class
!
! Are the tunnels themselves healthy?
Edge# show sdwan bfd sessions
!
! Most reports resolve at step two or three: the path
! is measurably worse than the class requires, and the
! overlay is responding correctly to that fact.
Setting expectations before deployment
Most of the misreadings in this section are expectation problems rather than technical ones, and they are cheapest to prevent before the feature is switched on. Three statements cover it: traffic will sometimes be on the transport you did not expect and that is correct; quality-based moves take minutes rather than seconds; and two applications between the same sites may legitimately take different paths.
Saying that once, to the people who watch the dashboards and answer the phone, removes a recurring class of ticket. It also makes the genuine problems easier to spot, because a report that does not fit those three patterns is more likely to be real and gets the attention it deserves rather than being filed alongside the false ones.
What to monitor
The measurements themselves, as a trend rather than an alarm, because they are the best available evidence about transport quality and they exist whether or not anybody looks. Alongside them, the count of qualifying paths per class per site — a class with no qualifying paths is either a genuine problem or an unsatisfiable threshold, and both are worth knowing. Neither of these is covered by circuit monitoring, which reports a circuit as up throughout its worst performance.
What this feature cannot fix
It selects among the paths that exist and creates none. A site with one transport gains nothing from it beyond visibility, because there is nowhere for traffic to move to. A site whose two transports both cross the same congested provider network will find both degrading together, and the feature will correctly report that neither qualifies while having no better option to offer.
That is worth saying because the feature is occasionally expected to compensate for insufficient or insufficiently diverse capacity. It cannot, and its measurements are the clearest possible evidence of that when the conversation about more capacity or genuinely diverse transports needs to be had. Used that way it is valuable even where it never moves a packet.
Blueprint framing
The CCIE Enterprise Infrastructure v1.1 blueprint covers SD-WAN policy within the software-defined infrastructure domain. Application-aware routing questions are about the model: what is measured and by what mechanism, what an SLA class consists of, the order in which a path is chosen, and what happens when nothing qualifies. Being able to state the three thresholds and the four-step decision covers the substance.
Conclusion
Application-aware routing measures every tunnel continuously using the same probes that detect liveness, and places traffic on a path that meets a defined loss, latency and jitter target. The mechanism costs nothing to run and the difficulty is entirely in the parameters: what the thresholds are, how long the measurement window is, and what happens when nothing qualifies.
The window is the parameter most worth understanding, because the defaults produce a decision made from a long average and therefore a reaction measured in tens of minutes. That is stable and deliberate, and it is not what most people believe they have configured. Liveness failure still moves traffic in seconds; it is quality-based decisions that are slow, and separating those two in any discussion of expected behaviour prevents a recurring misunderstanding.
The rest is judgement. Thresholds need headroom or the traffic oscillates, fewer and better-separated classes behave more predictably than many narrow ones, and strict mode converts degradation into an outage under conditions that are not a network failure — a decision that belongs to whoever owns the application. And three of the four behaviours most often reported as faults are the feature working correctly, which makes explaining it to the people watching the dashboards the highest-value thing to do after configuring it. More CCIE Enterprise Infrastructure material — labs, protocol breakdowns and study guides — is collected on the SPOTO CCIE site.
External Links
- RFC 5880 — Bidirectional Forwarding Detection (BFD)
- RFC 2474 — Definition of the Differentiated Services Field
- RFC 3550 — RTP: A Transport Protocol for Real-Time Applications
- RFC 4301 — Security Architecture for the Internet Protocol
- Cisco — Cisco SD-WAN Policies Configuration Guide
- Cisco Design Zone — WAN and Branch Design Guides
- Cisco Learning Network — CCIE Enterprise Infrastructure
Reference Notes
- Cisco SD-WAN documentation describes application-aware routing as using BFD between tunnel endpoints to measure loss, latency and jitter on each tunnel.
- Cisco documentation describes the app-route poll interval and multiplier, whose product determines the period of history over which a path's statistics are averaged.
- Cisco documentation gives the default app-route poll interval as 600000 milliseconds and the default multiplier as 6.
- Cisco documentation describes an SLA class as a combination of loss, latency and jitter thresholds that a tunnel must satisfy.
- Cisco documentation describes the
preferred-coloroption, which selects a transport when it meets the SLA class, and the fallback to other qualifying tunnels when it does not. - Cisco documentation describes
backup-sla-preferred-color, used when no tunnel meets the configured SLA class. - Cisco documentation describes the
strictoption on an SLA class, which causes matching traffic to be dropped when no tunnel meets the class. - RFC 5880 specifies BFD, the mechanism underlying both tunnel liveness detection and the per-tunnel measurements used by application-aware routing.
- Cisco documentation describes separate BFD hello interval and multiplier settings per colour, which govern liveness detection independently of the app-route averaging window.
- RFC 2474 defines the Differentiated Services field, which an application-aware routing policy can match on.
- Cisco documentation describes
show sdwan app-route sla-classandshow sdwan app-route stats, which report qualifying tunnels per class and per-tunnel measurements respectively. - The CCIE Enterprise Infrastructure v1.1 unified exam topics include SD-WAN policy within the software-defined infrastructure domain.