What Is ACI Cisco?

Cisco Application Centric Infrastructure (ACI) is Cisco’s flagship Software-Defined Networking (SDN) solution built specifically for enterprise data centers and private cloud architectures. Unlike traditional networks that rely on manual, switch-by-switch CLI configurations and VLAN-centric segmentation, Cisco ACI introduces a centralized, policy-driven model. It automates fabric management, enforces zero-trust security postures, and accelerates workload deployment by abstracting physical network operations into programmable software constructs.

By treating the data center network as an integrated fabric rather than a collection of independent switches, Cisco ACI aligns network behavior directly with application requirements, delivering consistent connectivity, automated segmentation, and unified visibility across physical, virtualized, and multi-cloud environments.

Cisco ACI Architecture: How Spine-Leaf and APIC Form the Policy-Driven SDN Model

Understanding Cisco ACI requires examining how its hardware underlay, protocol overlays, and centralized controller interact to create an automated network fabric. The architecture follows a five-layer structural model:

1. The Physical Clos Fabric (Spine-Leaf Topology)
   ACI discards traditional three-tier architectures in favor of a two-tier spine-and-leaf topology based on Clos switching designs. In an ACI fabric, every leaf switch (typically Cisco Nexus 9300 series) connects to every spine switch (Cisco Nexus 9300 or 9500 modular series). Crucially, leaf switches never connect to other leaf switches, and spine switches never connect to other spine switches. All physical servers, hypervisors, storage systems, and external Layer 3 routers connect exclusively to leaf switches. This ensures that any packet traveling between any two servers across the fabric traverses exactly two hops, providing non-blocking, deterministic, and low-latency east-west traffic flow.

2. The Automated Underlay Control Plane
   The underlying physical network in ACI is completely self-configuring. When leaf and spine nodes power on, they use Cisco Discovery Protocol (CDP) and Link Layer Discovery Protocol (LLDP) to discover neighbors. An internal, unnumbered Intermediate System-to-Intermediate System (IS-IS) protocol automatically builds the Layer 3 routed fabric using an isolated infrastructure VLAN (VLAN 4093/4094). Equal-Cost Multi-Path (ECMP) routing balances traffic over all available spine links. To track workload locations dynamically, spine switches run the Council of Oracles Protocol (COOP), an internal protocol that synchronizes endpoint IP and MAC address databases across all fabric nodes.

3. The VXLAN Overlay and Group-Based Policy
   ACI decouples logical network definitions from the physical infrastructure using Virtual Extensible LAN (VXLAN) encapsulation. Every packet that enters a leaf switch is encapsulated with an outer IP header and an ACI-specific VXLAN header. This header contains a 24-bit Virtual Network Identifier (VNID) that encodes not just the Layer 2 or Layer 3 boundary, but also the security identity of the workload (its Endpoint Group). Because encapsulation and decapsulation happen at line rate inside custom Cisco Cloud Scale ASICs, the fabric routes and bridges traffic across the overlay with zero performance penalty.

4. The Centralized Management Plane (Cisco APIC)
   The Cisco Application Policy Infrastructure Controller (APIC) is the central orchestration engine and unified repository for all fabric policies. APIC is deployed as a clustered appliance (requiring a minimum of three physical or virtual nodes for high availability and quorum). Operating strictly in the management plane, APIC exposes a comprehensive REST API and GUI. APIC pushes declarative policies down to leaf and spine switches; it does not process data packets. If all APIC nodes are disconnected or powered down, the fabric continues to forward data traffic, route packets, and enforce existing security rules without interruption.

5. The Declarative Intent-Based Policy Model
   Rather than requiring engineers to execute imperative CLI commands on individual switches (such as configuring VLANs, access control lists, and interface trunks), ACI relies on a declarative object model. Administrators define high-level intent—specifying which application tiers exist and which services may communicate—using Endpoint Groups (EPGs) and Contracts. The APIC automatically translates this intent into the underlying hardware programming and provisions the necessary state across the relevant leaf switches.

Section Takeaway: Cisco ACI decouples network policy from physical topology by pairing an automated IS-IS/VXLAN Clos fabric with an out-of-band APIC cluster that distributes declarative intent directly to switch ASICs.

Cisco ACI vs. Traditional Data Center Networking and Standalone VXLAN EVPN

Evaluating Cisco ACI requires comparing its controller-based fabric approach against both legacy three-tier network architectures and standalone, standards-based VXLAN EVPN deployments.

While standalone NX-OS VXLAN EVPN provides vendor neutrality and relies on open standards, it requires network engineers to design, build, and troubleshoot complex BGP overlays across every fabric node. In contrast, Cisco ACI delivers an integrated, turnkey platform that automates the underlay and overlay, trading open-standard protocol independence for declarative security, fabric-wide consistency, and centralized policy control.

Section Takeaway: Cisco ACI automates the operational and protocol complexities of VXLAN EVPN, substituting distributed BGP configurations with centralized, policy-driven microsegmentation at the expense of hardware flexibility.

Step-by-Step Configuration: Deploying Core ACI Constructs

Deploying workloads inside Cisco ACI involves structuring logical objects within the APIC Management Information Tree (MIT). The following sequential guide walks through provisioning an end-to-end network path from the administrative boundary down to policy enforcement.

+-------------------------------------------------------+
|                       Tenant                          |
|  +-------------------------------------------------+  |
|  |                      VRF                        |  |
|  |  +-------------------------------------------+  |  |
|  |  |              Bridge Domain (BD)           |  |  |
|  |  |               (Subnet / Gateway)          |  |  |
|  |  +-------------------------------------------+  |  |
|  +-------------------------------------------------+  |
|                                                       |
|  +-------------------------------------------------+  |
|  |          Application Network Profile (ANP)      |  |
|  |  +--------------------+   +--------------------+ |  |
|  |  |      Web EPG       |   |      App EPG       | |  |
|  |  +---------+----------+   +---------^----------+ |  |
|  |            |     Provided/Consumed  |            |  |
|  |            +----------Contract------+            |  |
|  +-------------------------------------------------+  |
+-------------------------------------------------------+

Step 1: Create the Tenant

A Tenant is the primary administrative container in ACI that isolates policies, resources, and access controls.
– In the APIC UI, navigate to Tenants > Add Tenant.
– Enter a name (e.g., Production_Tenant).
– Cisco ACI includes preconfigured system tenants: mgmt (for in-band/out-of-band management), infra (for internal fabric operations), and common (for globally shared policies and external connections). For custom enterprise applications, always create dedicated user tenants.

Step 2: Establish the VRF (Virtual Routing and Forwarding)

The VRF represents a private Layer 3 routing domain and contains independent routing tables, ensuring full IP address space isolation.
– Within Production_Tenant, navigate to Networking > VRFs > Create VRF.
– Provide a name (e.g., Production_VRF).
– Configure the Policy Enforcement Preference to Enforced (default zero-trust mode, which drops all inter-endpoint traffic unless permitted by a contract) or Unenforced (for initial migrations where unrestricted intra-VRF communication is temporarily required).

Step 3: Define the Bridge Domain (BD) and Subnet

A Bridge Domain is a Layer 2 forwarding construct tied to a specific VRF. Unlike traditional VLANs, a Bridge Domain decouples the Layer 2 broadcast boundary from physical interfaces and can contain multiple IP subnets.
– Navigate to Networking > Bridge Domains > Create Bridge Domain.
– Name the domain (e.g., App_BD) and associate it with Production_VRF.
– Under the BD, add a Subnet (e.g., 10.1.10.1/24). This IP acts as the pervasive, distributed anycast default gateway across every leaf switch in the fabric.
– Select forwarding parameters:
– Unicast Routing: Enabled (required for the BD default gateway to route traffic).
– L2 Unknown Unicast: Set to Hardware Proxy (uses spine COOP databases to forward traffic, minimizing fabric flooding) or Flood (behaves like a traditional VLAN).

Step 4: Construct the Application Network Profile (ANP) and Endpoint Groups (EPGs)

An Application Network Profile groups the logical tiers that comprise a business service. Inside the ANP, you define Endpoint Groups (EPGs)—collections of endpoints (virtual machines, bare-metal servers, containers) that share identical security and connectivity rules.
– Navigate to Application Profiles > Create Application Profile and name it (e.g., ECommerce_ANP).
– Right-click the profile and create an Application EPG (e.g., Web_EPG).
– Associate Web_EPG with App_BD. This grants endpoints within the EPG access to the default gateway and Layer 2 broadcast policies configured in Step 3.

Step 5: Map Physical Domains and Static Switch Interfaces

To link physical host traffic to an EPG, connect the logical model to the underlying leaf ports:
– Navigate to your EPG (Web_EPG) and locate Domains (VMs and Bare-Metals).
– Associate a Physical Domain (or Virtual Machine Manager / VMM Domain for VMware vSphere). This binds an approved VLAN Pool to the EPG.
– Under Static Ports, bind the physical leaf ports or Virtual Port Channels (vPCs) where workloads connect (e.g., Node-101, eth1/10), specifying the encapsulation VLAN tag (e.g., VLAN 100) that hosts will use to hit the leaf.

Step 6: Define Contracts and Subjects for Inter-EPG Security

In an enforced VRF, endpoints inside Web_EPG cannot communicate with endpoints in another EPG (e.g., DB_EPG) by default, even if they belong to the same Bridge Domain or VRF.
– Navigate to Contracts > Standard > Create Contract (e.g., Web_to_App_Contract).
– Create a Subject inside the contract and bind a Filter specifying protocols and ports (e.g., Permit_HTTP_TCP_80).
– Return to ECommerce_ANP:
– Set Web_EPG as the Consumer of the contract.
– Set App_EPG as the Provider of the contract.
– The APIC immediately compiles these rules and programs ternary content-addressable memory (TCAM) zoning tables on all leaf switches hosting endpoints for those EPGs.

Section Takeaway: Provisioning in ACI follows a strict hierarchical object path (Tenant → VRF → Bridge Domain → EPG → Contract), mapping logical application communication policies onto physical infrastructure without manual port-by-port network reconfiguration.

Troubleshooting Common Fabric Faults and Operational Anomalies

Operational issues in Cisco ACI typically manifest as health score reductions, communication drops, or endpoint synchronization discrepancies across leaf nodes.

Why has the APIC fabric health score dropped, and how do I trace the root cause?

The APIC assigns every Managed Object (MO), switch node, pod, and tenant a real-time Health Score from 0 to 100. When system events occur, the APIC logs a Fault containing a specific error code (F-code) and deducts points based on severity (Critical, Major, Minor, or Warning).

To diagnose a score drop:
1. In the APIC UI, click the degraded health score badge or navigate to System > Faults.
2. Filter the active faults by Severity (e.g., Critical and Major) and sort by transition time.
3. Inspect the fault’s code (e.g., F0467 for physical interface state changes or F1425 for unresolvable configuration relationships) and identify the dn (Distinguished Name) of the impacted Managed Object.
4. Verify whether the fault is in a “soaking” or “raised” lifecycle phase to determine if it represents an ongoing hardware failure or a transient interface flap.

What causes Endpoint Flapping, and how does Rogue Endpoint Control resolve it?

Endpoint flapping occurs when the ACI fabric observes an identical IP or MAC address rapidly moving between different leaf switches or switch interfaces. This is commonly caused by Layer 2 loops outside the fabric (such as an unmanaged switch looping packets), misconfigured hypervisor virtual switches with active-active teaming on non-bonded uplinks, or duplicate IP assignments.

To troubleshoot and remediate:
– Open the CLI of the affected leaf switch and check the endpoint table using:
show endpoint ip <ip-address>
show system internal epmc endpoint ip <ip-address>
– Identify whether the endpoint is bouncing between local interfaces or reporting remote leaf transitions via COOP.
– Remediation: Enable Rogue Endpoint Control under System > System Settings > Endpoint Controls > Rogue EP Control. Configure an aggressive detection threshold (e.g., 4 moves within 60 seconds). When an endpoint violates this limit, ACI temporarily freezes the endpoint on its current interface and stops learning it elsewhere, isolating the flapping host and shielding the fabric’s spine COOP control plane from route churn.

How do you diagnose leaf-to-spine link state drops and fabric discovery failures?

When a newly added leaf or spine switch fails to join the fabric or shows an inactive link state:
– Verify physical and LLDP health from the switch CLI using show lldp neighbors. If LLDP neighbors are missing, check optical transceiver power levels via show interface transceiver details.
– Inspect fabric discovery status on the APIC under Fabric > Inventory > Fabric Membership. A switch will not be operational until it receives an underlay IP address from the APIC’s internal DHCP pool (communicated over the infrastructure VLAN).
– Check the underlay IS-IS adjacency state on the leaf switch using show isis adjacency. If the adjacency is down, confirm that the switch is running an authorized, compatible ACI image version matching the APIC cluster.

Why are workloads in different EPGs unable to communicate despite an applied contract?

When communication fails between two EPGs governed by an active contract, the issue typically stems from incorrect contract scope, filter directionality, or routing configuration:
1. Contract Scope: Ensure the Contract Scope is set properly. If both EPGs reside in the same VRF, set the scope to VRF. If they reside in different VRFs or different Tenants, the scope must be set to Tenant or Global, and subnet routes must be explicitly exported (“Shared between VRFs”).
2. Filter Directionality: Verify whether the contract filter has Reverse Filter Ports enabled. When enabled, return traffic for standard TCP handshakes is automatically permitted without creating separate return rules.
3. Hardware TCAM Verification: Log into the leaf switch hosting the consuming endpoint and verify hardware zoning rules:
show zoning-rule | grep <source-epg-id>
show platform internal hal counters tcam entries
Confirm that the source and destination EPG security tags (pcTags) match the active contract and that packet drop counters are not actively incrementing on the programmed rule ID.

Section Takeaway: ACI troubleshooting centers on resolving APIC fault codes, containing endpoint motion through Rogue Endpoint Control, and verifying hardware-level TCAM zoning rules directly on leaf switches.

Cisco ACI Licensing Model and Subscription Tiers

Cisco licenses its data center switching infrastructure under the Cisco Data Center Networking (DCN) software subscription framework. Licenses are applied per physical Cisco Nexus 9000 switch (both leaf and spine nodes) and managed centrally through Cisco Smart Software Licensing (CSSM).

The licensing portfolio is structured into three progressive subscription tiers: Essentials, Advantage, and Premier.

Licensing Term Structures and Purchasing Considerations

• Subscription Terms: Cisco DCN licenses are subscription-based, available in 3-year, 5-year, and 7-year terms.
• Enterprise Agreements (EA): For organizations deploying large footprints (typically exceeding $100,000 in software contract value), enrolling in a Cisco Data Center Enterprise Agreement provides significant discounts, license portability between switch hardware generations, and simplified co-terming.
• Hardware vs. Software Separation: DCN subscription licenses cover only the operating system features, fabric control planes, and management tools. Physical switch chassis, power supplies, fan trays, and APIC hardware appliances are purchased separately.
• APIC Controller Licensing: APIC management software is included with the purchase of leaf and spine switch software tiers; there is no separate per-APIC software license, although the physical or virtual appliance itself requires its own base platform SKU.

Section Takeaway: Single-site and multi-pod fabrics run completely on DCN Essentials, while multi-site federation and remote leaf extensions require DCN Advantage, and full day-2 assurance and predictive analytics necessitate DCN Premier.

Authoritative Cisco ACI Design Guidelines and Official Documentation

Architecting an enterprise-grade Cisco ACI deployment requires consulting Cisco’s official engineering manuals, verified scale limits, and validated design recommendations. These documents define tested architectural limits and hardware best practices.

1. Cisco Validated Designs (CVD) and Architectural Whitepapers

• Cisco ACI Fabric Hardware Design Guide: Establishes standard spine-and-leaf oversubscription guidelines. Cisco recommends maintaining a bandwidth oversubscription ratio between 2:1 and 3:1 for leaf-to-spine uplinks in general enterprise compute clusters. This guide also outlines border-leaf pairing, multi-pod Inter-Pod Network (IPN) topologies, and high-density 100G/400G fabric backbones.
• Cisco ACI Multi-Pod Design White Paper: Defines requirements for extending a single APIC management domain across geographically distributed halls or data centers. Mandates that the Inter-Pod Network (IPN) support:
• Minimum MTU of 1600 bytes (to accommodate the 50-byte VXLAN encapsulation overhead without packet fragmentation).
• Bidirectional Protocol Independent Multicast (PIM-Bidir) or ingress replication for handling multi-destination BUM (Broadcast, Unknown Unicast, Multicast) traffic between pods.
• Quality of Service (QoS) mappings to preserve DSCP markings across service provider links.
• Cisco ACI Multi-Site Architecture Whitepaper: Specifies rules for interconnecting distinct, autonomous ACI fabrics using the Nexus Dashboard Orchestrator (NDO). Each site maintains an independent APIC cluster, isolating fault domains while synchronizing global application policies across sites over an untrusted Layer 3 Inter-Site Network (ISN).

2. Cisco APIC Verified Scalability Guides

Before designing an ACI fabric or staging major software upgrades, engineers must cross-reference their deployment scale against the Cisco APIC Verified Scalability Guide specific to their target software release. Critical maximum parameters verified by Cisco engineering include:
– Maximum Leaf Nodes per Fabric: 400 leaf switches per APIC cluster (in large cluster sizes with 5 or 7 APIC nodes).
– Maximum Endpoints: Up to 180,000 total endpoints (combined MAC and IP addresses) per fabric.
– Maximum Tenants and VRFs: Up to 3,000 Tenants and 3,000 VRF instances per fabric.
– Hardware TCAM Limits: Documented thresholds for total active contract rules (typically 64,000 entries per leaf ASIC slice) before rules begin spilling over or failing to program.

3. Official Compatibility and Deployment References

• Cisco Nexus 9000 Series Hardware Installation Guides: Authoritative wiring pinouts, environmental requirements, and power-consumption metrics for all leaf and spine models.
• Cisco ACI Virtual Machine Manager (VMM) Compatibility Matrix: Identifies supported hypervisor versions (VMware vCenter/ESXi, Red Hat OpenShift, Microsoft SCVMM) compatible with specific APIC software releases.
• Cisco Software Compatibility Matrix for ACI: Specifies certified upgrade paths between APIC controller builds and switch firmware versions, preventing catastrophic upgrade locks and fabric disconnects.

Section Takeaway: Safe, supportable production deployments require strict adherence to Cisco Validated Design topology guidelines and formal verification of scale ceilings against the target release’s Verified Scalability Guide.

Transforming the Modern Data Center Fabric

Cisco ACI transitions data center engineering from a manual, device-centric model to an automated, policy-driven programmable fabric. By pairing a physical Clos architecture with an automated VXLAN overlay, ACI eliminates the performance constraints of legacy Spanning Tree topologies while giving operations teams centralized control through the APIC.

Whether your organization deploys a single-site data center on DCN Essentials or manages globally federated hybrid environments with DCN Advantage and Premier, mastering the core object hierarchy—Tenants, VRFs, Bridge Domains, EPGs, and Contracts—is essential. When backed by systematic fault-code troubleshooting and alignment with Cisco Validated Designs, Cisco ACI delivers an agile, resilient foundation that dynamically scales to meet evolving application demands.

Please follow and like us: