Table Of ContentVerifiedScalabilityGuideforCiscoAPIC,Release3.1(1i)andCiscoNexus
9000 Series ACI-Mode Switches, Release 13.1(1i)
Overview 2
GeneralScalabilityLimits 2
MultipleFabricOptionsScalabilityLimits 4
CiscoACIMulti-Site 5
FabricTopology,SPAN,Tenants,Contexts(VRFs),ExternalEPGs,BridgeDomains,Endpoints,andContracts
ScalabilityLimits 6
VMMScalabilityLimits 17
Layer4-Layer7ScalabilityLimits 19
AD,TACACS,RBACScalabilityLimits 19
QoSScalabilityLimits 20
Revised:May11,2022
Overview
ThisguidecontainsthemaximumverifiedscalabilitylimitsforACIparametersfortheCiscoAPICRelease3.1.1iandCiscoNexus
9000SeriesACI-ModeSwitches,Release13.1.1i.Thesevaluesarebasedonaprofilewhereeachfeaturewasscaledtothenumbers
specifiedinthetables.ThesenumbersdonotrepresentthetheoreticallypossibleACIfabricscale.
General Scalability Limits
•L2Fabric:InLegacymodethereisnorouting,L3context,norcontractenabledintheL2fabricprofile. Atenantinthisprofile
doesnotneedtobemappedtoonededicatedACItenant. AtenantcanberepresentedbyasetofEPGsinstead.Toimprovethe
loadsharingamongAPICcontrollernodes,youmustdistributeEPGsandBDsacrossanACItenant.
•L3Fabric:TheACIL3fabricsolutionprovidesafeature-richhighlyscalablesolutionforpubliccloudandlargeenterprise.
Withthisdesign,almostallsupportedfeaturesaredeployedatthesametimeandaretestedasasolution.Thescalabilitynumbers
listedinthissectionaremulti-dimensionalscalabilitynumbers.Thefabricscalabilitynumbersrepresenttheoverallnumberof
objectscreatedonthefabric.Theper-leafscalenumbersaretheobjectscreatedandpresentedonanindividualleafswitch.The
fabriclevelscalabilitynumbersrepresentAPICclusterscalabilityandthetestedupperlimits.Someoftheper-leafscalability
numbersaresubjecttohardwarerestrictions.Theper-leafscalabilitynumbersarethemaximumlimitstestedandsupportedby
leafswitchhardware.Thisdoesnotnecessarilymeanthateveryleafswitchinthefabricwastestedwithmaximumscalenumbers.
•StretchedFabric:Stretchedfabricallowsmultiplefabrics(upto3)distributedinmultiplelocationstobeconnectedasasingle
fabricwithasinglemanagementdomain.Thescalefortheentirestretchedfabricremainsthesameasforasinglesitefabric.
ForexampleaL3stretchedfabricwillsupportupto200leafstotalwhichisthemaximumnumberofleafssupportedonasingle
sitefabric.Parametersonlyrelevanttostretchedfabricarementionedinthetablesbelow.
•Multi-Pod:Multipodenablesprovisioningamorefault-tolerantfabriccomprisedofmultiplepodswithisolatedcontrolplane
protocols.Also,multipodprovidesmoreflexibilitywithregardtothefullmeshcablingbetweenleafandspineswitches.For
example,ifleafswitchesarespreadacrossdifferentfloorsordifferentbuildings,multipodenablesprovisioningmultiplepods
perfloororbuildingandprovidingconnectivitybetweenpodsthroughspineswitches.
MultipodusesasingleAPICclusterforallthepods;allthepodsactasasinglefabric.IndividualAPICcontrollersareplaced
acrossthepodsbuttheyareallpartofasingleAPICcluster.
•Multi-Site:Multi-SiteisthearchitectureinterconnectingandextendingthepolicydomainacrossmultipleAPICclusterdomains.
Assuch,Multi-SitecouldalsobenamedasMulti-Fabric,sinceinterconnectsseparateAvailabilityZones(Fabrics)eachdeployed
asasinglePodinthisreleaseandmanagedbyanindependentAPICcontrollercluster.AnACIMulti-Sitepolicymanageris
partofthearchitectureandisusedtocommunicatewiththedifferentAPICdomainstosimplifythemanagementofthearchitecture
andthedefinitionofinter-sitepolicies.
NOTE:Themaximumnumberofleafswitchesoverallis400perfabricscale.Themaximumnumberofphysicalportsis19,200per
fabric.Themaximumnumberofremoteleafswitchesis30perfabric
Feature L2Fabric L3Fabric LargeL3Fabric
NumberofAPICcontrollers 3 Minimum3(4also 5
supported)
Numberofleafswitches 80 80 200
2
Feature L2Fabric L3Fabric LargeL3Fabric
Numberofspines Maximumspinesperpod:6. Maximumspinesperpod:6. Maximumspinesperpod:6.
Totalspines24. Totalspines24. Totalspines24.
NumberofFEXs N/A 20perleaf,320ports/leaf, N/A
650perfabric
Numberoftenants N/A 1000 3000
NumberofLayer3(L3)contexts N/A 1000 3000
(VRFs)
Numberofcontracts/filters N/A •10,000contracts •10,000contracts
•10,000filters •10,000filters
Numberofendpointgroups(EPGs) 21,000(500maximumper 15,000(500maximumper 15,000(500maximumper
tenant,or4,000inasingle tenant,or4,000inasingle tenant,or4,000inasingle
tenantinafabric) tenantinafabric) tenantinafabric)
NumberofIsolationenabledEPGs 400 400 400
Numberofbridgedomains(BDs) 21,000 15,000 15,000
NumberofBGP+numberofOSPF N/A 3,000 3,000
sessions+EIGRP(forexternal
connection)
NumberofMulticastgroups N/A 8000 8000
NumberofMulticastgroupsperVRF N/A 8000 8000
Numberofstaticroutestoasingle N/A 5,000 10,000
SVI/VRF
Numberofstaticroutesonasingleleaf N/A 5,000 10,000
switch
NumberofvCenters N/A •200VDS •200VDS
•50AVS •50AVS
•50CiscoACIVirtual •50CiscoACIVirtual
Edge Edge
NumberofServiceChains N/A 1000 1000
NumberofL4-L7devices N/A 30physical,1,200virtual 30physical,1,200virtual
(1,200maximumperfabric) (1,200maximumperfabric)
NumberofESXihosts-VDS N/A 3200 3200
NumberofESXihosts-AVS N/A 3200(Only1AVSinstance 3200(Only1AVSinstance
perhost) perhost)
3
Feature L2Fabric L3Fabric LargeL3Fabric
NumberofESXihosts-AVE N/A 3200(Only1AVEinstance 3200(Only1AVEinstance
perhost) perhost)
NumberofVMs N/A Dependsuponserverscale Dependsuponserverscale
Numberofconfigurationzonesper 30 30 30
fabric
NumberofBFDsessionsperleaf 256 256 256
switch
MinimumBFDtimer MinimumBFDtimer MinimumBFDtimerrequired
requiredtosupportthisscale: requiredtosupportthisscale: tosupportthisscale:
•minTx:50 •minTx:50 •minTx:50
•minRx:50 •minRx:50 •minRx:50
•multiplier:3 •multiplier:3 •multiplier:3
Multi-Pod 3*or4nodeAPICcluster,6 3*or4nodeAPICcluster,6 •5*or6nodeAPIC
pods,80leafswitchesoverall pods,80leafswitchesoverall cluster,6pods,200leaf
NOTE:*ispreferredclustersize
switchesmaxperpod,
300leafswitchesmax
overall
•7nodeAPICcluster,12
pods,200leafswitches
maxperpod,400leaf
switchesmaxoverall
L3EVPNServicesoverFabricWAN N/A 1000VRFs,60,000routesin 1000VRFs,60,000routesin
-GOLF(withandwithoutOpFlex) afabric afabric
Layer3Multicastroutes N/A 8,000 8,000
NumberofRoutesinOverlay-1VRF 1,000 1,000 1,000
Multiple Fabric Options Scalability Limits
ConfigurableOptions PerLeafScale PerFabricScale
StretchedFabric
Maximumnumberoffabricsthatcanbeastretched N/A 3
fabric
MaximumnumberofRouteReflectors N/A 6
Multi-Pod
MaximumnumberofPODs N/A 12
4
ConfigurableOptions PerLeafScale PerFabricScale
MaximumnumberofleafswitchesperPOD N/A 200
Maximumnumberofleafswitchesoverall N/A 400
Cisco ACI Multi-Site
Stretchedvs.nonStretched—Stretchedinmulti-sitemeansthemulti-sitefabrichasstretchedobjectssuchasEPG,BD,VRFor
SubnetacrossmultiplesitesorhascrosssitecontractsbetweenEPGs.Thescaleparametersforthisscenarioisdescribedinthe
"Stretched(Multi-Site)"column.Non-Stretchedinmulti-sitemeansallobjects(ie.EPG,contract,BD)arelocaltoasiteonlyanddo
notcrossthelocal-siteboundary.Thescaleparametersforthisscenarioisdescribedinthe"Non-Stretched(APIC)"columninthe
tablebelow:
NOTE:Formaximumscalemulti-siteconfigurationswithmanyfeaturesenabledsimultaneously,itisrecommendedthatthose
configurationsbetestedinalabpriortodeployment.
Multi-SiteGeneralScalabilityLimits
ConfigurableOptions Scale
Sites 8
Leafswitchespersite 200
Multi-SiteObjectScale
ConfigurableOptions Scale
PolicyObjectsperSchema 500
TemplatesperSchema 5
Multi-SiteOrchestratorUsers(nonparallel*) 50
*Multi-SiteOrchestratorprocessesrequestssequentiallyfrom
multipleuserseveniftheyaredeployingdifferentschemas.
Multi-SiteScalabilityLimitsforStretchedObjects
ScalingItem Stretched(Multi-Site) Non-Stretched(APIC)
Tenants 200 2500
VRFs 400 3000
BDs 2000 10,000
Contracts 2000 2000
5
ScalingItem Stretched(Multi-Site) Non-Stretched(APIC)
EndPoints 50,000 100,000including:
•50,000-stretchedfromothersites
•50,000-locallylearnedinsite-local
EPGs 2000 10,000
IsolatedEPGs 400 400
MicroSegmentEPGs 400 400
IGMPSnooping 8000 8000
L3OutexternalEPGs 500 2400
Subnets 2000 10,000
Fabric Topology, SPAN, Tenants, Contexts (VRFs), External EPGs, Bridge
Domains, Endpoints, and Contracts Scalability Limits
Thetablebelowshowsthemappingofthe"ALE/LSEType"tothecorrespondingTORswitches.Thisinformationishelpfulto
determinewhichTORswitchisaffectedwhenweusethetermsALEv1,ALEv2,LSEorLSE2inremainingsections.
ALE/LSEType ACI-SupportedTORs
ALEv1 •N9K-C9396PX+N9K-M12PQ
•N9K-C93128TX+N9K-M12PQ
•N9K-C9396TX+N9K-M12PQ
ALEv2 •N9K-C9396TX+N9K-M6PQ
•N9K-C93128TX+N9K-M6PQ
•N9K-C9396PX+N9K-M6PQ
•N9K-C9372TX64K
•N9K-C9332PQ
•N9K-C9372PX
LSE N9K-C93108TC-EX+N9K-C93180YC-EX
LSE2 N9K-C93108TC-FX+N9K-C93180YC-FX
NOTE:Unlessexplicitlycalledout,LSErepresentsbothLSEandLSE2andALErepresentsbothALEv1andALEv2intablebelow:
6
ConfigurableOptions PerLeafScale PerFabricScale
FabricTopology
NumberofPCs,vPCs 320(withFEXHIF) N/A
Numberofencapsperaccessport,PC,vPC(non 1750 N/A
FEXHIF)
NumberofencapsperFEXHIF,PC,vPC 20 N/A
NumberofmemberlinksperPC,vPC* 8 N/A
*vPCtotalports=16,8perleaf
NumberofportsxVLANS(globalscopeandno 64,000 N/A
FEXHIF)
168,000(whenusinglegacyBDmode)
NumberofportsxVLANS(FEXHIFsand/orlocal ForALEv1andv2:9,000 N/A
scope)
ForLSEandLSE2:
10,000
Numberofstaticportbindings ForALEv1andv2:30,000 400,000
ForLSEandLSE2:
60,000
STP AllVLANs N/A
Mis-CablingProtocol(MCP) 256VLANsperinterface N/A
2000logicalports(portxVLAN)per
leaf
7
ConfigurableOptions PerLeafScale PerFabricScale
Maximumnumberofendpoints(EPs) Defaultprofile(Dualstack)- Max.360,000ProxyDatabaseEntries
inthefabric,whichcanbetranslated
•ForALEv1andv2:
intoanyoneofthefollowing:
•MAC:12,000
•360,000MAC-onlyEPs(eachEP
•IPv4:12,000or withoneMAConly)
•IPv6:6000or •180,000IPv4EPs(eachEPwith
oneMACandoneIPv4)
•IPv4:4000,IPv6:4000
•120,000dual-stackEPs(eachEP
•ForLSEorLSE2: withoneMAC,oneIPv4andone
IPv6)
•MAC:24,000
Theformulatocalculateinmixed
•IPv4:24,000
modeisasfollows:
•IPv6:12,000
#MAC+#IPv4+#IPv6<=360,000
NOTE:Fourfabricmodulesare
IPv4Scaleprofile—
requiredonallspinesinthefabricto
•ForLSEandLSE2: supportabovescale.
•MAC:48,000
Fixedspineswitches(N9K-C9364C):
•IPv4:48,000 Max.180,000ProxyDatabaseEntries
inthefabric,whichcanbetranslated
•IPv6:Notsupported
intoanyoneofthefollowing:
•ForALEv1andv2:Notsupported •180,000MAC-onlyEPs(eachEP
withoneMAConly)
HighDualStackScaleprofile—
•90,000IPv4EPs(eachEPwith
•ForLSEandLSE2: oneMACandoneIPv4)
•MAC:64,000 •60,000dual-stackEPs(eachEP
withoneMAC,oneIPv4,and
•IPv4:64,000
oneIPv6)
•IPv6:24,000
Theformulatocalculateinmixed
•ForALEv1andv2:Notsupported modeisasfollows:
#MAC+#IPv4+#IPv6<=180,000
NumberofMACEPGs N/A 125
NumberofMulticastGroups Defaultprofile(Dualstack):8,000 8000
IPv4Scaleprofile:8,000
HighDualstackScaleprofile:0
NumberofMulticastGroupsperVRF 8000 8000
NumberofIPsperMAC 1024 1024
8
ConfigurableOptions PerLeafScale PerFabricScale
SPAN ALEbasedToRs: N/A
•4uni-directionalor2bi-directional
access/tenantsessions
•4uni-directionalor2bi-directional
fabricsessions
LSEbasedToRs:
•8uni-directionalor4bi-directional
sessions(fabric,access,ortenant)
NumberofportsperSPANsession •Allleafaccessportscouldbeinone N/A
session
•Allleaffabricportscouldbeinone
session
NOTE:ForLSE/LSE2only:30(total
numberofuniqueports(fabric+access)
acrossalltypesofspansessions)
NumberofsourceEPGsintenantSPANsessions ALE-basedToRswitches: N/A
Note Thenumberslistedinthisrowassume •230ingressdirection+50egress
thatonlytenantSPANisconfigured. direction
Ifboth,AccessandTenantSPANare
LSE-basedToRswitches:
configured,thefollowingformula
appliesforbothingressandegress •230bidirectional
SPAN:
•460unidirectional(230ingress+
E + P + E*P + EPP + v6FePP + 230egress)
0.5*v4FePP <= 230
Where:
•E—NumberofsourceEPGsin
TenantSPAN
•P—NumberofsourcePortsin
accessSPANwithoutanyfilters
•EPP—Numberof(Epg,Port)Pairs
inaccessSPANwithEPGfilter
only(nofiltergroup)
•v4FePP—Numberof(v4filter
entry,Port)PairsinaccessSPAN
withfiltergroup
•v6FePP—Numberof(v6Filter
entry,Port)PairsinaccessSPAN
withfiltergroup
9
ConfigurableOptions PerLeafScale PerFabricScale
Commonpervasivegateway 256virtualIPsperBridgeDomain N/A
MaximumnumberofDataPlanepolicersat ALE: N/A
interfacelevel
•64ingresspolicers
•64egresspolicers
LSEandLSE2:
•7ingresspolicers
•3egresspolicers
MaximumnumberofDataPlanepolicersatEPG 128ingresspolicers N/A
andinterfacelevel
MaximumnumberofinterfaceswithPer-Protocol 63 N/A
Per-Interface(PPPI)CoPP
MaximumnumberofTCAMentriesfor 256 N/A
Per-ProtocolPer-Interface(PPPI)CoPP
OnePPPICoPPconfigurationmayuse
morethanoneTCAMentry.Thenumber
ofTCAMentriesusedforeach
configurationvariesineachprotocoland
leafplatform.Usevsh_lc -c 'show
system internal aclqos pppi copp
tcam-usage'commandtocheckon
LSE/LSE2platforms
MaximumnumberofSNMPtrapreceivers 10 10
FirstHopSecurity(FHS)* 2000endpoints N/A
WithanycombinationofBDs/EPGs/EPswithin 1000bridgedomains
thesupportedlimit
MaximumnumberofQ-in-Qtunnels 1980 N/A
(bothQinQcoreandedgecombined)
MaximumnumberofTEP-to-TEPatomiccounters N/A 1600
Tenants
NumberofContexts(VRFs)pertenant ALE:50 ALE:50
LSE:128 LSE:128
Numberofapplicationprofilespertenant(orper N/A N/A
Context(VRF))
Contexts(Allnumbersapplicabletodualstackunlessexplicitlycalledout)
10
Description:as a single Pod in this release and managed by an independent APIC controller cluster. An ACI Multi-Site policy manager is part of the architecture and is used to communicate with the different APIC domains to simplify the management of the architecture and the definition of inter-site policies.
