Hardware & Platform Flexibility<\/strong><\/td>\n| Vendor-agnostic; functions on commodity Layer 2\/3 enterprise switches.<\/td>\n | Open-standard; supported on diverse Cisco Nexus switches and third-party multi-vendor network platforms.<\/td>\n | Vendor-locked; requires Cisco Nexus 9000 switches running in ACI mode paired with an APIC cluster.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n While standalone NX-OS VXLAN EVPN provides vendor neutrality and relies on open standards, it requires network engineers to design, build, and troubleshoot complex BGP overlays across every fabric node. In contrast, Cisco ACI delivers an integrated, turnkey platform that automates the underlay and overlay, trading open-standard protocol independence for declarative security, fabric-wide consistency, and centralized policy control.<\/p>\n Section Takeaway:<\/strong> Cisco ACI automates the operational and protocol complexities of VXLAN EVPN, substituting distributed BGP configurations with centralized, policy-driven microsegmentation at the expense of hardware flexibility.<\/p>\n<\/span>Step-by-Step Configuration: Deploying Core ACI Constructs<\/span><\/h2>\nDeploying workloads inside Cisco ACI involves structuring logical objects within the APIC Management Information Tree (MIT). The following sequential guide walks through provisioning an end-to-end network path from the administrative boundary down to policy enforcement.<\/p>\n +-------------------------------------------------------+\n| Tenant |\n| +-------------------------------------------------+ |\n| | VRF | |\n| | +-------------------------------------------+ | |\n| | | Bridge Domain (BD) | | |\n| | | (Subnet \/ Gateway) | | |\n| | +-------------------------------------------+ | |\n| +-------------------------------------------------+ |\n| |\n| +-------------------------------------------------+ |\n| | Application Network Profile (ANP) | |\n| | +--------------------+ +--------------------+ | |\n| | | Web EPG | | App EPG | | |\n| | +---------+----------+ +---------^----------+ | |\n| | | Provided\/Consumed | | |\n| | +----------Contract------+ | |\n| +-------------------------------------------------+ |\n+-------------------------------------------------------+\n<\/code><\/pre>\n<\/span>Step 1: Create the Tenant<\/span><\/h3>\nA Tenant is the primary administrative container in ACI that isolates policies, resources, and access controls.
\n– In the APIC UI, navigate to Tenants > Add Tenant<\/strong>.
\n– Enter a name (e.g., Production_Tenant<\/code>).
\n– Cisco ACI includes preconfigured system tenants: mgmt<\/code> (for in-band\/out-of-band management), infra<\/code> (for internal fabric operations), and common<\/code> (for globally shared policies and external connections). For custom enterprise applications, always create dedicated user tenants.<\/p>\n<\/span>Step 2: Establish the VRF (Virtual Routing and Forwarding)<\/span><\/h3>\nThe VRF represents a private Layer 3 routing domain and contains independent routing tables, ensuring full IP address space isolation.
\n– Within Production_Tenant<\/code>, navigate to Networking > VRFs > Create VRF<\/strong>.
\n– Provide a name (e.g., Production_VRF<\/code>).
\n– Configure the Policy Enforcement Preference to Enforced<\/strong> (default zero-trust mode, which drops all inter-endpoint traffic unless permitted by a contract) or Unenforced<\/strong> (for initial migrations where unrestricted intra-VRF communication is temporarily required).<\/p>\n<\/span>Step 3: Define the Bridge Domain (BD) and Subnet<\/span><\/h3>\nA Bridge Domain is a Layer 2 forwarding construct tied to a specific VRF. Unlike traditional VLANs, a Bridge Domain decouples the Layer 2 broadcast boundary from physical interfaces and can contain multiple IP subnets.
\n– Navigate to Networking > Bridge Domains > Create Bridge Domain<\/strong>.
\n– Name the domain (e.g., App_BD<\/code>) and associate it with Production_VRF<\/code>.
\n– Under the BD, add a Subnet (e.g., 10.1.10.1\/24<\/code>). This IP acts as the pervasive, distributed anycast default gateway across every leaf switch in the fabric.
\n– Select forwarding parameters:
\n – Unicast Routing<\/strong>: Enabled (required for the BD default gateway to route traffic).
\n – L2 Unknown Unicast<\/strong>: Set to Hardware Proxy<\/em> (uses spine COOP databases to forward traffic, minimizing fabric flooding) or Flood<\/em> (behaves like a traditional VLAN).<\/p>\n<\/span>Step 4: Construct the Application Network Profile (ANP) and Endpoint Groups (EPGs)<\/span><\/h3>\nAn Application Network Profile groups the logical tiers that comprise a business service. Inside the ANP, you define Endpoint Groups (EPGs)\u2014collections of endpoints (virtual machines, bare-metal servers, containers) that share identical security and connectivity rules.
\n– Navigate to Application Profiles > Create Application Profile<\/strong> and name it (e.g., ECommerce_ANP<\/code>).
\n– Right-click the profile and create an Application EPG (e.g., Web_EPG<\/code>).
\n– Associate Web_EPG<\/code> with App_BD<\/code>. This grants endpoints within the EPG access to the default gateway and Layer 2 broadcast policies configured in Step 3.<\/p>\n<\/span>Step 5: Map Physical Domains and Static Switch Interfaces<\/span><\/h3>\nTo link physical host traffic to an EPG, connect the logical model to the underlying leaf ports:
\n– Navigate to your EPG (Web_EPG<\/code>) and locate Domains (VMs and Bare-Metals)<\/strong>.
\n– Associate a Physical Domain (or Virtual Machine Manager \/ VMM Domain for VMware vSphere). This binds an approved VLAN Pool to the EPG.
\n– Under Static Ports<\/strong>, bind the physical leaf ports or Virtual Port Channels (vPCs) where workloads connect (e.g., Node-101, eth1\/10<\/code>), specifying the encapsulation VLAN tag (e.g., VLAN 100<\/code>) that hosts will use to hit the leaf.<\/p>\n<\/span>Step 6: Define Contracts and Subjects for Inter-EPG Security<\/span><\/h3>\nIn an enforced VRF, endpoints inside Web_EPG<\/code> cannot communicate with endpoints in another EPG (e.g., DB_EPG<\/code>) by default, even if they belong to the same Bridge Domain or VRF.
\n– Navigate to Contracts > Standard > Create Contract<\/strong> (e.g., Web_to_App_Contract<\/code>).
\n– Create a Subject<\/strong> inside the contract and bind a Filter<\/strong> specifying protocols and ports (e.g., Permit_HTTP_TCP_80<\/code>).
\n– Return to ECommerce_ANP<\/code>:
\n – Set Web_EPG<\/code> as the Consumer<\/strong> of the contract.
\n – Set App_EPG<\/code> as the Provider<\/strong> of the contract.
\n– The APIC immediately compiles these rules and programs ternary content-addressable memory (TCAM) zoning tables on all leaf switches hosting endpoints for those EPGs.<\/p>\nSection Takeaway:<\/strong> Provisioning in ACI follows a strict hierarchical object path (Tenant \u2192 VRF \u2192 Bridge Domain \u2192 EPG \u2192 Contract), mapping logical application communication policies onto physical infrastructure without manual port-by-port network reconfiguration.<\/p>\n<\/span>Troubleshooting Common Fabric Faults and Operational Anomalies<\/span><\/h2>\nOperational issues in Cisco ACI typically manifest as health score reductions, communication drops, or endpoint synchronization discrepancies across leaf nodes.<\/p>\n <\/span>Why has the APIC fabric health score dropped, and how do I trace the root cause?<\/span><\/h3>\nThe APIC assigns every Managed Object (MO), switch node, pod, and tenant a real-time Health Score from 0 to 100. When system events occur, the APIC logs a Fault containing a specific error code (F-code) and deducts points based on severity (Critical, Major, Minor, or Warning).<\/p>\n To diagnose a score drop:
\n1. In the APIC UI, click the degraded health score badge or navigate to System > Faults<\/strong>.
\n2. Filter the active faults by Severity<\/strong> (e.g., Critical and Major) and sort by transition time.
\n3. Inspect the fault’s code<\/code> (e.g., F0467<\/code> for physical interface state changes or F1425<\/code> for unresolvable configuration relationships) and identify the dn<\/code> (Distinguished Name) of the impacted Managed Object.
\n4. Verify whether the fault is in a “soaking” or “raised” lifecycle phase to determine if it represents an ongoing hardware failure or a transient interface flap.<\/p>\n<\/span>What causes Endpoint Flapping, and how does Rogue Endpoint Control resolve it?<\/span><\/h3>\nEndpoint flapping occurs when the ACI fabric observes an identical IP or MAC address rapidly moving between different leaf switches or switch interfaces. This is commonly caused by Layer 2 loops outside the fabric (such as an unmanaged switch looping packets), misconfigured hypervisor virtual switches with active-active teaming on non-bonded uplinks, or duplicate IP assignments.<\/p>\n To troubleshoot and remediate:
\n– Open the CLI of the affected leaf switch and check the endpoint table using:
\n show endpoint ip <ip-address>
\n show system internal epmc endpoint ip <ip-address><\/code>
\n– Identify whether the endpoint is bouncing between local interfaces or reporting remote leaf transitions via COOP.
\n– Remediation<\/strong>: Enable Rogue Endpoint Control<\/strong> under System > System Settings > Endpoint Controls > Rogue EP Control<\/strong>. Configure an aggressive detection threshold (e.g., 4 moves within 60 seconds). When an endpoint violates this limit, ACI temporarily freezes the endpoint on its current interface and stops learning it elsewhere, isolating the flapping host and shielding the fabric’s spine COOP control plane from route churn.<\/p>\n<\/span>How do you diagnose leaf-to-spine link state drops and fabric discovery failures?<\/span><\/h3>\nWhen a newly added leaf or spine switch fails to join the fabric or shows an inactive link state:
\n– Verify physical and LLDP health from the switch CLI using show lldp neighbors<\/code>. If LLDP neighbors are missing, check optical transceiver power levels via show interface transceiver details<\/code>.
\n– Inspect fabric discovery status on the APIC under Fabric > Inventory > Fabric Membership<\/strong>. A switch will not be operational until it receives an underlay IP address from the APIC’s internal DHCP pool (communicated over the infrastructure VLAN).
\n– Check the underlay IS-IS adjacency state on the leaf switch using show isis adjacency<\/code>. If the adjacency is down, confirm that the switch is running an authorized, compatible ACI image version matching the APIC cluster.<\/p>\n<\/span>Why are workloads in different EPGs unable to communicate despite an applied contract?<\/span><\/h3>\nWhen communication fails between two EPGs governed by an active contract, the issue typically stems from incorrect contract scope, filter directionality, or routing configuration:
\n1. Contract Scope<\/strong>: Ensure the Contract Scope is set properly. If both EPGs reside in the same VRF, set the scope to VRF<\/code>. If they reside in different VRFs or different Tenants, the scope must be set to Tenant<\/code> or Global<\/code>, and subnet routes must be explicitly exported (“Shared between VRFs”).
\n2. Filter Directionality<\/strong>: Verify whether the contract filter has Reverse Filter Ports<\/strong> enabled. When enabled, return traffic for standard TCP handshakes is automatically permitted without creating separate return rules.
\n3. Hardware TCAM Verification<\/strong>: Log into the leaf switch hosting the consuming endpoint and verify hardware zoning rules:
\n show zoning-rule | grep <source-epg-id>
\n show platform internal hal counters tcam entries<\/code>
\n Confirm that the source and destination EPG security tags (pcTags) match the active contract and that packet drop counters are not actively incrementing on the programmed rule ID.<\/p>\nSection Takeaway:<\/strong> ACI troubleshooting centers on resolving APIC fault codes, containing endpoint motion through Rogue Endpoint Control, and verifying hardware-level TCAM zoning rules directly on leaf switches.<\/p>\n<\/span>Cisco ACI Licensing Model and Subscription Tiers<\/span><\/h2>\nCisco licenses its data center switching infrastructure under the Cisco Data Center Networking (DCN)<\/strong> software subscription framework. Licenses are applied per physical Cisco Nexus 9000 switch (both leaf and spine nodes) and managed centrally through Cisco Smart Software Licensing (CSSM). <\/p>\nThe licensing portfolio is structured into three progressive subscription tiers: Essentials, Advantage, and Premier.<\/p>\n \n\n\n| Feature \/ Architectural Capability<\/th>\n | DCN Essentials<\/th>\n | DCN Advantage<\/th>\n | DCN Premier<\/th>\n<\/tr>\n<\/thead>\n | \n\nBase Fabric Automation (Spine-Leaf, IS-IS underlay, VXLAN)<\/strong><\/td>\n| Included<\/td>\n | Included<\/td>\n | Included<\/td>\n<\/tr>\n | \nCore Policy Model (Tenants, VRFs, Bridge Domains, EPGs, Contracts)<\/strong><\/td>\n| Included<\/td>\n | Included<\/td>\n | Included<\/td>\n<\/tr>\n | \nStandard Layer 3 Routing (Static, OSPF, BGP, EIGRP L3Outs)<\/strong><\/td>\n| Included<\/td>\n | Included<\/td>\n | Included<\/td>\n<\/tr>\n | \nVirtual Machine Manager (VMM) Integrations (vSphere, OpenShift)<\/strong><\/td>\n| Included<\/td>\n | Included<\/td>\n | Included<\/td>\n<\/tr>\n | \nMulti-Pod Architecture (Single APIC cluster across multiple pods)<\/strong><\/td>\n| Included<\/td>\n | Included<\/td>\n | Included<\/td>\n<\/tr>\n | \nMulti-Site Orchestration (Nexus Dashboard Orchestrator – NDO)<\/strong><\/td>\n| Not Supported<\/td>\n | Included<\/td>\n | Included<\/td>\n<\/tr>\n | \nPhysical Remote Leaf Support<\/strong><\/td>\n| Not Supported<\/td>\n | Included<\/td>\n | Included<\/td>\n<\/tr>\n | \nNexus Dashboard Insights (NDI) & Flow Telemetry<\/strong><\/td>\n| Not Supported<\/td>\n | Not Supported<\/td>\n | Included<\/td>\n<\/tr>\n | \nNetwork Assurance Engine (NAE – Automated Policy & Pre-Change Verification)<\/strong><\/td>\n| Not Supported<\/td>\n | Not Supported<\/td>\n | Included<\/td>\n<\/tr>\n | \n | | | | | | | | |
|