Flow accounting is the cheapest visibility a network offers. The router is already looking at every packet to forward it, so grouping those packets into conversations and counting them costs very little, and the result answers the questions that matter operationally — who is talking to whom, how much, over which path, and when it changed.
The flexible form of it separates the definition of a flow from the export of it from the cache that holds it, which means one router can run several different accounting regimes at once and a definition written for one purpose can be reused everywhere. That separation is the useful part and it is not where deployments go wrong. They go wrong on a timer whose default was chosen for a different era, and which silently makes the whole thing up to half an hour late.
This article covers the four objects and which one carries the design, how key fields decide what a flow even is, which timers decide when you find out about it, where to apply it and in which direction, and why a correctly configured exporter can leave a collector showing nothing at all. It is written for the lab rather than for the written exam, and sits alongside the rest of the CCIE Enterprise Infrastructure lab certification track.
Only the record is a design decision; the match fields define what counts as a distinct flow and therefore set the cache size, and the active timer decides how long a running flow goes unreported.What Are the Four Objects?
How do they fit together?
A record defines what a flow is and what is counted about it. An exporter says where records are sent and how. A monitor ties a record to an exporter and owns the cache in which flows accumulate. The monitor is applied to an interface in a direction. Only the record involves design thought; the other three are plumbing, and a deployment that spends its effort on the exporter and accepts a predefined record has skipped the part that decides what the data is worth.
A Deeper Dive into the Objects
Why the separation is useful
One record can feed several monitors with different caches and different destinations. One exporter can receive from several monitors. A router can run one accounting regime for security analysis and another for capacity planning, with different fields and different timers, without either interfering with the other.
That flexibility is the point of the flexible form and most deployments use a fraction of it. Using a fraction is fine — a single well-designed record applied everywhere is a good outcome — provided the record was designed rather than inherited.
The predefined records
Several ready-made records exist, reproducing what the older fixed-format accounting collected. They work, they are one line, and they collect a field set chosen for a network of a different shape.
Using one to get started is reasonable. Leaving it in place indefinitely means the accounting answers the questions somebody had decades ago rather than the ones being asked now, and the difference is usually a couple of fields.
! The record: what a flow is, and what is counted
R1(config)# flow record FR-IPV4
R1(config-flow-record)# match ipv4 protocol
R1(config-flow-record)# match ipv4 source address
R1(config-flow-record)# match ipv4 destination address
R1(config-flow-record)# match transport source-port
R1(config-flow-record)# match transport destination-port
R1(config-flow-record)# match ipv4 tos
R1(config-flow-record)# match interface input
R1(config-flow-record)# collect counter bytes long
R1(config-flow-record)# collect counter packets long
R1(config-flow-record)# collect timestamp absolute first
R1(config-flow-record)# collect timestamp absolute last
R1(config-flow-record)# collect interface output
R1(config-flow-record)# collect routing next-hop address ipv4
R1(config-flow-record)# collect transport tcp flags
The exporter
A destination, a source, a port and a format. The source matters for the same reason it matters in any exchange with a central system: the collector identifies the device by the address the records arrive from, and a device whose records arrive from whichever interface routing chose will be identified inconsistently.
The format is a choice between the established version with templates and the standardised successor. Both work; the successor is the direction of travel and supports field types the older one does not. Where the collector supports it, it is the better choice.
! The exporter: where records go and in what format
R1(config)# flow exporter FE-COLLECTOR
R1(config-flow-exporter)# destination 10.200.0.50
R1(config-flow-exporter)# source Loopback0
R1(config-flow-exporter)# transport udp 2055
R1(config-flow-exporter)# export-protocol netflow-v9
R1(config-flow-exporter)# template data timeout 60
R1(config-flow-exporter)# option interface-table timeout 300
R1(config-flow-exporter)# option application-table timeout 300
The monitor and its cache
The monitor combines the two and adds the cache: how many flows can be held, how long they live, and what happens when it fills. The cache is where the timers live and it is the object most worth thinking about after the record.
Three cache behaviours exist. The ordinary one accumulates flows and exports them on a timer. One exports every packet as its own record, which is enormously expensive and exists for specific purposes. One never expires entries, which turns the cache into a set of counters to be read rather than exported. The ordinary one is right unless there is a specific reason.
! The monitor: record, exporter, and the cache that matters
R1(config)# flow monitor FM-IPV4
R1(config-flow-monitor)# record FR-IPV4
R1(config-flow-monitor)# exporter FE-COLLECTOR
R1(config-flow-monitor)# cache timeout active 60
R1(config-flow-monitor)# cache timeout inactive 15
R1(config-flow-monitor)# cache entries 200000
R1(config-flow-monitor)# cache type normal
!
R1(config)# interface GigabitEthernet0/1
R1(config-if)# ip flow monitor FM-IPV4 input
What to verify immediately
That the cache is filling, that records are being exported, and that the exporter reports no failures. Three commands, and together they distinguish a working deployment from one that is collecting nothing or collecting and failing to send.
! Three commands, immediately after applying it
R1# show flow monitor FM-IPV4 cache | include Current entries
Current entries: 4821
!
R1# show flow exporter FE-COLLECTOR statistics | include sent|dropped
Packets sent: 1842
Packets dropped: 0
!
R1# show flow interface GigabitEthernet0/1
FNF: monitor: FM-IPV4 direction: Input traffic(ip): on
| Object | Decides | Reused across | Design effort |
|---|---|---|---|
| Record | What a flow is and what is counted | Every monitor | All of it |
| Exporter | Where records go | Every monitor | Minimal |
| Monitor | The cache and its timers | Every interface | The timers |
| Interface application | Which traffic is seen | — | Direction only |
How Do Key Fields Decide What a Flow Is?
What is the rule?
Two packets belong to the same flow if every key field is identical, and to different flows if any one differs. So the key fields are the definition of a flow, and each one multiplies how many distinct flows exist. The conventional five — protocol, both addresses, both ports — describe a conversation. Adding more makes the data richer and the cache larger, sometimes dramatically, and the second effect is not obvious until the cache is full.
A Deeper Dive into Field Selection
The conventional set, and why it works
Protocol, source and destination address, source and destination port. That combination identifies one conversation between two endpoints, which is the unit almost every question is asked about. It is the right starting point and most designs should add one or two fields rather than rethinking it.
The useful additions are the ingress interface, which tells you where traffic entered and therefore which path it took, and the marking, which tells you how it was classified. Both are low cardinality — a router has a handful of interfaces and a design has a handful of classes — so neither multiplies the cache by much.
Fields that explode the cache
Anything with high cardinality that varies within a conversation. The most common mistake is including a field that changes per packet or per short interval, which turns one flow into thousands.
The arithmetic is worth doing before applying a record: multiply the expected distinct values of each key field. A record with five fields at modest cardinality is fine; adding a sixth with thousands of distinct values multiplies everything by thousands.
show flow monitor FM-IPV4 statistics shows the cache at its configured size with a high count of entries added and aged. Fix: reduce the key field count, or raise the cache size if the platform allows — and calculate the expected cardinality rather than discovering it.Collected fields, which are free by comparison
A collected field is recorded alongside the flow without participating in its identity. The byte and packet counters, the timestamps, the egress interface, the next hop, the connection flags — all of these are collected, none creates additional flows, and each costs only bytes per entry.
So the way to make a record richer without making the cache larger is to move things from key to collected wherever the field's value is stable within a conversation. The egress interface is the clearest example: it is the same for the life of a flow, so collecting it gives the same information as matching it at no cardinality cost.
What to collect that is often missed
The connection flags, which distinguish a completed conversation from an attempt. The timestamps of the first and last packet, which are what make duration computable. And the next hop, which reveals a routing change without needing to look at routing.
Those three turn a volume report into something that can answer operational questions. Without them, the data says how much traffic went where and very little else.
! Cardinality, computed before applying
! protocol ~5 in practice
! source addr ~2000 hosts
! dest addr ~5000 destinations
! source port high, but bounded per host
! dest port ~200 services
! interface input ~8
!
! Then check what actually happened
R1# show flow monitor FM-IPV4 statistics
Cache type: Normal (Platform cache)
Cache size: 200000
Current entries: 48211
High Watermark: 93044
Flows added: 18405212
Flows aged: 18357001
Separate records for separate purposes
A security analysis wants connection flags and every conversation. A capacity plan wants volume by class and path and does not care about individual conversations. Those are different records, and running both is cheaper than a single record with the union of their fields, because the union's cardinality is the product rather than the sum.
Two monitors with two records and two caches on the same interface is supported and is the right structure when the purposes genuinely differ.
Matching on what the platform supports
Switching platforms implement this in hardware with a limited set of fields. A record naming a field the hardware cannot key on is either rejected or causes the traffic to be handled in software, which on a switch means a performance problem rather than an error message.
Checking the platform's supported field list before designing a record avoids producing one that works on the routers and behaves badly on the switches. It is documentation reading rather than testing and it takes ten minutes.
| Field | Key or collect | Cardinality | Worth it |
|---|---|---|---|
| Protocol, addresses, ports | Key | The baseline | Always |
| Ingress interface | Key | Very low | Yes |
| Marking | Key | Low | Yes, if classes matter |
| Egress interface | Collect | — | Yes — same value all flow |
| Next hop | Collect | — | Yes — reveals path changes |
| Connection flags | Collect | — | Yes — attempts against sessions |
Which Timers Decide When You Find Out?
What are they?
Two. The inactive timer exports a flow that has stopped, and its default of a few seconds is sensible. The active timer exports a flow that is still running, so that a long conversation is reported periodically rather than only when it ends — and its default is half an hour. A long-lived flow therefore produces no record at all for thirty minutes after it starts, which makes every dashboard built on this data up to half an hour stale for exactly the traffic that matters most.
A Deeper Dive into the Timers
Why the default is what it is
It was chosen when export volume was expensive and flows were short. Reporting a long-lived flow every half hour kept the record count low, and the flows that dominated a network then finished quickly anyway.
Neither premise holds now. A backup, a video stream, a database replication session or a large transfer runs for hours, and a monitoring system that hears about it twice an hour cannot show anything useful about current utilisation.
What to set it to
Sixty seconds is the usual answer and it is a large improvement for a modest increase in record volume. Thirty seconds is reasonable where the monitoring is expected to be near real time. Below that, the record volume rises without the data becoming more useful, because the collector is usually aggregating into minute buckets anyway.
The record volume increase is proportional to how many long flows exist, which on most networks is a small fraction of the total. The commonly feared explosion in export traffic does not materialise.
show flow monitor shows the active timeout at its default. Fix: set the active timeout to sixty seconds on every monitor, and treat it as a mandatory part of the configuration rather than a tuning option.The inactive timer
It decides how long a flow with no traffic is held before being considered finished. Its default of a few seconds is appropriate: shorter and a conversation with a natural pause is split into several flows; longer and finished flows occupy the cache unnecessarily.
The one reason to change it is a cache under pressure, where reducing it frees entries sooner. That is treating a symptom — the real answer is fewer key fields or a larger cache — but it is a legitimate short-term measure.
Connection teardown
A connection-oriented flow whose termination is observed is exported immediately rather than waiting for the inactive timer. That is a useful property and it applies only where the flags are visible, which means it does not help for connectionless traffic or for a conversation whose end is never seen.
It is also the reason collecting the connection flags is worthwhile: without them the record cannot distinguish a completed conversation from one that was abandoned, which is a distinction security analysis depends on entirely.
Adding the delays up
The time between something happening and a collector knowing about it is the active timer, plus the export interval, plus whatever the collector takes to process. The first dominates by a wide margin at its default and is comparable to the others once set sensibly.
That arithmetic is worth doing when somebody asks how current the data is, because the honest answer at defaults is "up to thirty-two minutes old" and almost nobody expects that.
! The three settings that decide freshness
R1(config)# flow monitor FM-IPV4
R1(config-flow-monitor)# cache timeout active 60
R1(config-flow-monitor)# cache timeout inactive 15
!
R1(config)# flow exporter FE-COLLECTOR
R1(config-flow-exporter)# template data timeout 60
!
! Confirm what is in force, not what was typed
R1# show flow monitor FM-IPV4 | include timeout|Cache
Cache timeout active: 60 secs
Cache timeout inactive: 15 secs
Sampling, and when it is the answer
Where the platform cannot account for every packet at line rate, a sampler processes one packet in a configured number and the collector scales the result. That reduces the load proportionally and introduces statistical error that is small for large flows and large for small ones.
It is appropriate for capacity planning, where the large flows are what matter. It is a poor basis for security analysis, where a single connection is significant and sampling may miss it entirely. Where both are needed, sample for one and account fully for the other, on different interfaces if necessary. Reading this once is not the same as being able to do it under time pressure, which is what repetition against realistic CCIE lab practice scenarios is for.
! Sampling, where full accounting is not affordable
R1(config)# sampler SAMP-100
R1(config-sampler)# mode random 1 out-of 100
!
R1(config)# interface TenGigabitEthernet0/1
R1(config-if)# ip flow monitor FM-IPV4 sampler SAMP-100 input
!
! The collector must know the rate to scale correctly
R1# show sampler SAMP-100
R1# show flow interface TenGigabitEthernet0/1
Where Should It Be Applied?
Which interfaces and which direction?
Ingress, on every interface. A packet crossing a router enters exactly once and leaves exactly once, so accounting on ingress everywhere counts each packet exactly once and covers every path through the device. Accounting on both directions counts every transit packet twice, and nothing warns that the totals are doubled. Accounting on egress only misses everything the router discarded, which is frequently the traffic worth knowing about.
A Deeper Dive into Placement
Why ingress everywhere is the answer
It is complete and it does not double count. Every packet handled by the device is seen exactly once, including packets that are discarded by a filter or by a routing decision, because ingress accounting happens before those.
The exception is a design deliberately measuring what left rather than what arrived, which is a narrower question. For the general question — what is this device handling — ingress everywhere is both the simplest and the most accurate arrangement.
The double counting trap
Applying the monitor in both directions on the same interface, or on ingress at one interface and egress at another, produces two records for one packet. The collector has no way to know and reports twice the real volume.
It is a quiet failure because the numbers are plausible. A link reported at twice its real utilisation looks like a capacity problem rather than a configuration error, and the resulting capacity decision is expensive.
show flow interface across every interface, looking for any applied in both directions. Fix: apply ingress only, on every interface, so that each packet is accounted exactly once wherever it entered.Sub-interfaces and tunnels
A monitor can be applied to a logical interface as well as a physical one, and applying it to both counts the traffic twice again. Deciding at which layer accounting happens — the physical port or the logical sub-interfaces on it — and applying it consistently is the requirement.
Sub-interfaces are usually the more useful layer, because the logical interface corresponds to a customer, a site or a service, and the physical port is merely where several of them share a cable.
Which interfaces to leave out
Interfaces whose traffic is already accounted for elsewhere, and interfaces where the volume is high and the value is low. A link between two switches in the same rack carrying aggregated traffic that is already measured at its edges adds records and no information.
That is a judgement rather than a rule. The default should be everywhere, with exclusions justified individually, because an interface nobody is accounting for is the one traffic disappears into during an investigation.
! Ingress everywhere, consistently
R1(config)# interface range GigabitEthernet0/0 - 3
R1(config-if-range)# ip flow monitor FM-IPV4 input
!
! And the IPv6 half, which needs its own monitor
R1(config-if-range)# ipv6 flow monitor FM-IPV6 input
!
! Audit: which interfaces, which direction
R1# show flow interface
Interface GigabitEthernet0/0
FNF: monitor: FM-IPV4
direction: Input
traffic(ip): on
Both address families
A monitor for one family does not see the other. A dual-stack network with accounting for only one has a blind spot that grows as the second family carries more traffic, and it is invisible because the reports look complete.
Two records, two monitors, both applied. It is twice the configuration and it is the only way the accounting matches reality on a dual-stack network.
Layer 2 accounting
On a switch, traffic that is switched rather than routed can also be accounted for, with a record keyed on the addresses at that layer. It answers a different set of questions — which device is generating traffic within a segment — and it is worth knowing about for investigating a segment rather than for general visibility.
The field support is platform dependent and narrower than at the network layer, which again argues for checking before designing.
| Placement | Counts each packet | Misses | Use |
|---|---|---|---|
| Ingress, every interface | Exactly once | Nothing | The default answer |
| Egress, every interface | Once, if it left | Everything discarded | Rarely |
| Both directions | Twice | Nothing, but doubled | Never |
| Ingress, some interfaces | Once, where applied | Paths not covered | Only with justification |
| Physical and sub-interface | Twice | — | Never both |
Why Does the Collector Show Nothing?
What are the causes?
Four. The cache is not filling, which means the monitor is not applied or the traffic is not crossing where it is applied. The cache is filling and nothing is being exported, which is an exporter problem. Records are being sent and not arriving, which is a path or filtering problem. Or records are arriving and cannot be decoded, because the template describing them has not been received — which happens whenever the collector restarts and does not resolve until the next template is sent.
A Deeper Dive into the Silence
Working from the cache outward
Look at the cache first. If it is empty, nothing further downstream can possibly be working and the problem is placement or traffic. If it is filling, look at the exporter's statistics, which report what was sent and what failed. If those show records leaving, the problem is between here and the collector.
Four stages, each with a command, and the order eliminates the stages below it. Starting at the collector and working backward is the instinct and it is considerably slower.
! Four stages, outward from the cache
!
! 1. Is the cache filling?
R1# show flow monitor FM-IPV4 cache | include entries|Current
!
! 2. Is anything being exported?
R1# show flow exporter FE-COLLECTOR statistics
Packets sent: 184052
Bytes sent: 184052000
Packets dropped: 0
!
! 3. Can we reach the collector from the export source?
R1# ping 10.200.0.50 source Loopback0
!
! 4. Templates - has the collector had one recently?
R1# show flow exporter FE-COLLECTOR | include template|Option
The template problem
The records themselves contain only values; the meaning of those values is carried in a separate template message sent periodically. A collector that starts without having received a template cannot interpret anything until the next one arrives.
With a long template interval that can be a substantial gap, during which the collector receives data and discards it. Setting the interval to a minute means a restarted collector is useful again within a minute, which costs a negligible amount of traffic.
The path and the filtering
Records travel as ordinary traffic to a destination and are subject to everything that affects ordinary traffic. A filter that permits the collector's management protocols and not the export port discards them, and the router reports a successful send because it did send.
The export source address matters here too: a filter written against the expected source will discard records arriving from an unexpected one, which is the same argument for pinning the source as in any other central-system exchange.
Export volume
A large network exporting records for every flow generates a substantial stream toward one collector. That is fine until it is not, and the failure is at the collector rather than on the network — records arriving faster than they can be processed are discarded there, silently.
The remedies are sampling on the high-volume interfaces, a longer active timer on the monitors where freshness matters less, or more collectors. Which is right depends on what the data is for, and all three are better than a collector quietly dropping a third of what it receives.
What to monitor about the monitoring
The cache occupancy against its size, whose approach to the limit predicts silent loss. The exporter's dropped count, which should be zero. And the collector's own received-against-processed figures, which is where the losses that the network cannot see appear.
The third is the one that requires cooperation with whoever runs the collector, and it is where the interesting failures live on a large deployment.
! The numbers worth collecting about the collection
R1# show flow monitor FM-IPV4 statistics | include Current entries|High Watermark
R1# show flow exporter statistics | include dropped|sent
!
! And a look at what is actually in the cache right now
R1# show flow monitor FM-IPV4 cache format table | head 20
Reading the cache directly
The cache can be displayed on the router, which is the fastest way to answer a question about current traffic without involving the collector at all. During an incident that is frequently what is wanted: which conversations are consuming this link, right now.
It is also the check that proves the accounting is working when the collector is in doubt. A cache with sensible entries and a collector showing nothing localises the problem immediately.
Blueprint framing
The CCIE Enterprise Infrastructure v1.1 blueprint covers network assurance within its infrastructure services domain, and flow accounting is the main mechanism in it. What is examined is generally the object model and the distinction between key and non-key fields, rather than a full deployment.
| Symptom | Stage | Command |
|---|---|---|
| Cache empty | Placement or traffic | show flow interface |
| Cache filling, nothing sent | Exporter | show flow exporter statistics |
| Sent, not arriving | Path or filtering | Ping from the export source |
| Arriving, not decoded | Template not received | Template timeout setting |
| Partial data | Collector overloaded | The collector's own counters |
| Totals doubled | Applied both directions | show flow interface, every one |
Conclusion
Four objects, and only one of them is a design decision. The record defines what a flow is, and the key fields are that definition — two packets differing in any of them are different flows, so every key field multiplies the cache. Anything whose value is stable for the life of a conversation belongs in the collected set instead, which gives the same information at no cardinality cost. The exporter and the monitor are a dozen lines that look the same everywhere.
The setting that matters most is a timer nobody looks at. The active timeout decides how long a running flow goes unreported, and its default of half an hour was chosen when export was expensive and flows were short. Neither is true on a network carrying backups and streaming, so a deployment left at defaults produces a monitoring system that is up to thirty minutes behind on exactly the long-lived traffic that saturates links. Sixty seconds costs very little and fixes it.
Apply it on ingress, on every interface, because a packet enters a device exactly once and is therefore counted exactly once — including the packets the device discarded, which egress accounting never sees. Applying it in both directions doubles every number in a way that looks plausible and leads to a capacity decision rather than an investigation. And when the collector shows nothing, work outward from the cache: empty cache, filling cache with no export, export with no arrival, arrival with no template. Four stages, one command each, and the order rules out everything below it. More CCIE Enterprise Infrastructure material — labs, protocol breakdowns and study guides — is collected on the SPOTO CCIE site.
External Links
- RFC 7011 — IP Flow Information Export (IPFIX) Protocol
- RFC 7012 — Information Model for IPFIX
- RFC 3954 — Cisco Systems NetFlow Services Export Version 9
- RFC 5470 — Architecture for IP Flow Information Export
- RFC 5475 — Sampling and Filtering Techniques for IP Packet Selection
- RFC 7015 — Flow Aggregation for the IPFIX Protocol
- Cisco Learning Network — CCIE Enterprise Infrastructure
Reference Notes
- RFC 7011 specifies IPFIX, including the template mechanism by which the structure of exported data records is described separately from the records themselves.
- RFC 7011 states that a collector cannot interpret data records until it has received the corresponding template, and recommends periodic retransmission of templates over unreliable transport.
- RFC 7012 defines the IPFIX information model, including the distinction between fields used to identify a flow and fields reported about it.
- RFC 3954 specifies NetFlow version 9, the template-based export format preceding IPFIX, and likewise requires templates to be sent periodically.
- RFC 5470 describes the IPFIX architecture, including the metering process that maintains flow records and the exporting process that transmits them.
- RFC 5470 describes flow expiration, including expiry after a period of inactivity and periodic expiry of long-lived flows so that they are reported before they end.
- RFC 5475 describes packet selection techniques including random sampling, and the accuracy implications of sampling for flows of different sizes.
- RFC 7015 describes flow aggregation, in which records are combined according to a reduced set of key fields.
- Cisco documentation describes Flexible NetFlow, comprising flow records, flow exporters, flow monitors and samplers as separate configuration objects.
- Cisco documentation describes match statements as defining key fields and collect statements as defining non-key fields, with key fields determining flow identity.
- Cisco documentation states that the default active flow timeout is 1800 seconds and the default inactive timeout is 15 seconds.
- The CCIE Enterprise Infrastructure v1.1 unified exam topics include network assurance within the infrastructure services domain.