Table Of ContentSun StorEdge 3000 Family
™
Best Practices Manual
Sun StorEdge 3510 FC Array
Sun StorEdge 3511 FC Array with SATA
Sun Microsystems, Inc.
www.sun.com
Part No. 816-7325-15
June 2004, Revision A
Submit comments about this document at: http://www.sun.com/hwdocs/feedback
Copyright©2004DotHillSystemsCorporation,6305ElCaminoReal,Carlsbad,California92009,USA.Allrightsreserved.
SunMicrosystems,Inc.andDotHillSystemsCorporationmayhaveintellectualpropertyrightsrelatingtotechnologyembodiedinthisproduct
ordocument.Inparticular,andwithoutlimitation,theseintellectualpropertyrightsmayincludeoneormoreoftheU.S.patentslistedat
http://www.sun.com/patentsandoneormoreadditionalpatentsorpendingpatentapplicationsintheU.S.andothercountries.
Thisproductordocumentisdistributedunderlicensesrestrictingitsuse,copyingdistribution,anddecompilation.Nopartofthisproductor
documentmaybereproducedinanyformbyanymeanswithoutpriorwrittenauthorizationofSunanditslicensors,ifany.
Third-partysoftwareiscopyrightedandlicensedfromSunsuppliers.
PartsoftheproductmaybederivedfromBerkeleyBSDsystems,licensedfromtheUniversityofCalifornia.UNIXisaregisteredtrademarkin
theU.S.andinothercountries,exclusivelylicensedthroughX/OpenCompany,Ltd.
Sun,SunMicrosystems,theSunlogo,SunStorEdge,AnswerBook2,docs.sun.com,andSolarisaretrademarksorregisteredtrademarksofSun
Microsystems,Inc.intheU.S.andinothercountries.
U.S.GovernmentRights—Commercialuse.GovernmentusersaresubjecttotheSunMicrosystems,Inc.standardlicenseagreementand
applicableprovisionsoftheFARanditssupplements.
DOCUMENTATIONISPROVIDED“ASIS”ANDALLEXPRESSORIMPLIEDCONDITIONS,REPRESENTATIONSANDWARRANTIES,
INCLUDINGANYIMPLIEDWARRANTYOFMERCHANTABILITY,FITNESSFORAPARTICULARPURPOSEORNONINFRINGEMENT,
AREDISCLAIMED,EXCEPTTOTHEEXTENTTHATSUCHDISCLAIMERSAREHELDTOBELEGALLYINVALID.
Copyright©2004DotHillSystemsCorporation,6305ElCaminoReal,Carlsbad,Californie92009,Etats-Unis.Tousdroitsréservés.
SunMicrosystems,Inc.etDotHillSystemsCorporationpeuventavoirlesdroitsdepropriétéintellectuelsrelatantsàlatechnologieincorporée
dansleproduitquiestdécritdanscedocument.Enparticulier,etsanslalimitation,cesdroitsdepropriétéintellectuelspeuventinclureunou
plusdesbrevetsaméricainsénumérésàhttp://www.sun.com/patentsetunoulesbrevetsplussupplémentairesoulesapplicationsdebrevet
enattentedanslesEtats-Unisetdanslesautrespays.
Ceproduitoudocumentestprotégéparuncopyrightetdistribuéavecdeslicencesquienrestreignentl’utilisation,lacopie,ladistribution,etla
décompilation.Aucunepartiedeceproduitoudocumentnepeutêtrereproduitesousaucuneforme,parquelquemoyenquecesoit,sans
l'autorisationpréalableetécritedeSunetdesesbailleursdelicence,s’ilyena.
Lelogicieldétenupardestiers,etquicomprendlatechnologierelativeauxpolicesdecaractères,estprotégéparuncopyrightetlicenciépardes
fournisseursdeSun.
DespartiesdeceproduitpourrontêtredérivéesdessystèmesBerkeleyBSDlicenciésparl’UniversitédeCalifornie.UNIXestunemarque
déposéeauxEtats-Unisetdansd’autrespaysetlicenciéeexclusivementparX/OpenCompany,Ltd.
Sun,SunMicrosystems,lelogoSun,SunStorEdge,AnswerBook2,docs.sun.com,etSolarissontdesmarquesdefabriqueoudesmarques
déposéesdeSunMicrosystems,Inc.auxEtats-Unisetdansd’autrespays.
LADOCUMENTATIONESTFOURNIE“ENL’ÉTAT”ETTOUTESAUTRESCONDITIONS,DECLARATIONSETGARANTIESEXPRESSES
OUTACITESSONTFORMELLEMENTEXCLUES,DANSLAMESUREAUTORISEEPARLALOIAPPLICABLE,YCOMPRISNOTAMMENT
TOUTEGARANTIEIMPLICITERELATIVEALAQUALITEMARCHANDE,AL'APTITUDEAUNEUTILISATIONPARTICULIEREOUA
L’ABSENCEDECONTREFAÇON.
Please
Recycle
Contents
Overview 1
Comparison of the Sun StorEdge 3510 FC Array and the Sun StorEdge 3511
SATA Array 3
Sun StorEdge 3500 Series Architecture 4
DAS and SAN Storage Architectures 5
Fibre Channel Protocols 6
Supported RAID Levels 7
Logical Drives 7
Cache Optimization 9
Array Management Tools 11
Saving and Restoring Configuration Information 12
Direct-Attached Storage 12
Storage Area Networking 13
Scaling Capacity 14
First Steps 16
General Configuration Considerations 17
Configuring an Array’s RCCOM Channel 18
Cautions 20
iii
Non-Redundant DAS Configurations 21
Non-Redundant DAS Tips and Techniques 22
Non-Redundant DAS Setup Details 23
High-Availability DAS Configurations 26
High-Availability DAS Tips and Techniques 27
Standard High-Availability DAS Setup Details 28
Six Node High-Availability DAS Setup Details 31
Full-Fabric SAN Configurations 33
Full-Fabric SAN Tips and Techniques 34
Full-Fabric SAN Setup Details 35
High-Performance SAN Configurations 38
High-Performance SAN Tips and Techniques 39
High-Performance SAN Setup Details 40
Scaling Arrays Into High Capacity Configurations 43
Limitations 43
Sun StorEdge 3510 FC Array High Capacity Configurations 44
Sun StorEdge 3511 SATA Array High Capacity Configurations 53
Summary 59
iv Sun StorEdge 3000 Family Best Practices Manual • June 2004
Best Practices for the Sun StorEdge
3510 and 3511 FC Arrays
This Best Practices Manual describes both the Sun StorEdge™ 3510 FC Array and
Sun StorEdge 3511 FC Array with SATA, and complements the Sun StorEdge 3000
Family Installation, Operation, and Service Manual for these products.
TheSunStorEdge3511FCArraywithSATAisalsoknownastheSunStorEdge3511
SATA array which will be the term used throughout most of this document.
Procedures for configuration of the Sun StorEdge 3500 series arrays are identical
except where specifically noted in this manual.
Note – The primary use of the two arrays is very different due to different disk
technologies. Be sure to read the following Overview section and the “Comparison
of the Sun StorEdge 3510 FC Array and the Sun StorEdge 3511 SATA Array” on
page3 for optimum use of each array.
Overview
Today’s business, educational and consumer environments continue to create a
seemingly inexhaustible amount of information. In addition, there is a constant
strain on the trade-offs between performance and cost when architecting and
deploying a storage solution. Storage technology has provided a multitude of
options to appropriately store information based upon retrieval characteristics such
as:frequency,random/sequentialtypeofaccess,criticality,time-sensitivityandcost
basis.
1
Both the Sun StorEdge 3500 series arrays feature full 2Gb host Fibre Channel (FC)
connections and are next-generation storage systems designed to provide direct
attachedstorage(DAS)toentry-level,mid-range,andenterpriseservers,ortoserve
as the disk storage within a storage area network (SAN).
Modernbusinessesemployserversofallsizesrequiringstoragetomatchthetypeof
data storage and processing being used by the applications involved, as well as
considerthecostoftheserver.TheSunStorEdge3500productfamilyisdesignedto
address network application priorities and cost considerations by applying the
appropriate disk technology as follows:
(cid:1) Applications demanding transaction-based random access patterns with high
duty cycles are better suited by the FC-disk solution of the StorEdge 3510 FC
array.
The StorEdge 3510 FC array is designed for business-critical applications and
theirhigh-valuedata.Thesecriticalapplicationsrequirehigheravailability(nota
trait of SATA-disk based solutions).
(cid:1) Applicationsrequiringstreamingorsequentiallyaccesseddataorthosewithcost-
sensitive duty cycles such as backup and archival are better served by the Serial
ATA-disk solution of the StorEdge 3511 SATA array.
The StorEdge 3511 SATA array is designed for applications requiring lower-cost
bulk storage (not a trait of FC-disk based solutions), where application or data
availability is not a primary concern.
TheStorEdge3511SATAarrayisonlyusedforarchival,backup,orstaticstorage;
it is not used to store single instances of data, and cannot serve as a boot device.
Designedforsmallerdirect-attachedstorage(DAS)needsorforconsolidatedstorage
applicationsusingaStorageAreaNetwork(SAN),matchingthearrayreliabilityand
disk performance to the application availability requirements has never been easier.
This document gives a high level overview of the Sun StorEdge 3500 series arrays
andoutlinesfivesamplestoragesolutionsforentry-level,mid-range,andenterprise
servers. Use these solutions as-is or tailor them to fit your exact needs. Examples of
customization opportunities include adding disks, enclosures and software or even
combining configurations. Choosing the solution that best matches your particular
environment will provide the best results.
The benefits and purpose of each array varies according to network requirements:
(cid:1) The Sun StorEdge 3510 FC array uses Fibre Channel (FC) disk technology and is
ideal for performance-sensitive, intense I/O applications.
(cid:1) The Sun StorEdge 3511 SATA array uses Serial ATA (SATA) disk technology, and
is ideal for cost-sensitive archival, static storage, and backup applications.
2 Sun StorEdge 3000 Family Best Practices Manual • June 2004
Comparison of the Sun StorEdge 3510 FC Array
and the Sun StorEdge 3511 SATA Array
Before installing and configuring your array, please review the key differences
between the SunStorEdge3510FCarrayandtheSunStorEdge3511SATAarray. The
following list provides a comparison of the SunStorEdge3510FCarrayandtheSun
StorEdge3511SATAarrayfeatures.
Caution – Althoughthetwoproductsareverysimilarinappearanceandsetup,the
configurations have very important differences. While the StorEdge 3510 FC Array
can be used for all applications, its higher cost makes it a poor choice for certain
applications. The StorEdge 3511 SATA array should only be used where indicated
below. Its features are a subset of the StorEdge 3510 FC array’s and therefore may
notbeusedeverywheretheStorEdge3510FCarraycanbeused.UsingtheStorEdge
3511 SATA array where a FC disk-based solution should be used may result in the
loss of data access or even, in extreme cases, data corruption.
TABLE1 ComparisonofSunStorEdge3510FCArrayandtheSunStorEdge3511SATAArrayFeatures
SunStorEdge3510FCArray SunStorEdge3511SATAArray
Applications Bestsuitedforproduction Bestsuitedforaninexpensivesecondary
applicationswherethesuperior storageapplicationwherehighercapacity
featuresofFCtechnical drivesareneededwhicharenotmission-
characteristicsandperformance critical,andwherelowerperformanceand
isarequirement.Thisincludes lessthan7/24availabilityisanoption.
thebelowonlineapplications: Thisincludesthebelownear-line
• Database applications:
• DecisionSupport • Informationlifecyclemanagement
• Datawarehousing • Contentaddressablestorage
• Electroniccommerce • Backupandrestore
• Enterpriseresourceplanning • SecondarySANstorage
• Messaging,fileandprint • Near-lineDASstorage
• Staticreferencedatastorage
Disks FibreChanneldisks: SATAdisks:
36,73,or146GBat10krpm 250GBat7200rpm
36or73GBat15krpm
MaximumFCHostPortsper 4 6
I/OControllerModule (oneSFPporteachforchannels (twoSFPportseachforchannels1and0;
0,1,4,and5) oneSFPporteachforchannels4and5)
MaximumNumberof 8 5
ExpansionUnitsConnected
toaRAIDArray
Best Practices for the Sun StorEdge 3510 and 3511 FC Arrays 3
TABLE1 ComparisonofSunStorEdge3510FCArrayandtheSunStorEdge3511SATAArrayFeatures
SunStorEdge3510FCArray SunStorEdge3511SATAArray
MaximumNumberofDisks 108 (1RAIDarray+8expansion 72 (1RAIDarray+5expansionunits)
perConfiguration units)
MaximumNumberof 8logicaldrives 8logicaldrives
LogicalDrives
MaximumTotalStorage 15.75TB 18.0TB
Capacity
MaximumUsableStorage 14.0TBRAID5 14.0TBRAID5
Capacity 15.1TBRAID0 16.0TBRaid0
JBODArraySupport OneJBODperserver Notsupported
Sun StorEdge 3500 Series Architecture
AllmembersoftheStorEdge3500seriesarrayssharethesamearchitectureelements.
Thissectiondiscussesthoseelements,makingnoteofthefewdifferencesinhowthe
architecture was implemented differently in the StorEdge 3510 FC and 3511 SATA
products.
The Sun StorEdge 3500 series RAID controllers have six FC channels. RAID
controller channels 0, 1, 4, and 5 are normally designated for connection to hosts or
Fibre Channel switches. RAID controller channels 2 and 3 are dedicated drive
channelsthatconnecttodisks.Eachchannelhasasingleportconnectionexceptthe
Sun StorEdge 3511 SATA array which has two extra ports (two connections for
channels 0 and 1).
In a dual RAID controller configuration, the architecture of the loops within the
chassisprovidesbothRAIDcontrollersthesamehostchanneldesignators.Eachhost
channelofthetopRAIDcontrollersharesaloopwiththematchinghostchannelon
the bottom RAID controller. For example, channel 0 of the top RAID controller
sharesthesameloopaschannel0ofthebottomRAIDcontroller.Thisprovidesfour
distinct loops for connectivity. The individual loops provide LUN failover without
causing HBA path failover in the event of a controller failure.
AsingleRAIDcontrollerconfigurationisslightlydifferent.ThelowerI/Oboardhas
drive channels but does not have host channels. Overall, the same number of loops
are available, but with only half as many host channel ports.
Sun StorEdge 3510 FC Array. RAID controller channels 0, 1, 4, and 5 are normally
designated host channels. Any of the host channels can be configured as a drive
channel. In a dual controller configuration, each host loop includes two ports per
loop, one port on the top controller and one port on the bottom controller.
4 Sun StorEdge 3000 Family Best Practices Manual • June 2004
All six fibre channels in an I/O controller module support 1-Gbit or 2-Gbit data
transfer speeds.
RAID controller channels 2 and 3 are dedicated drive channels that connect to
expansionunits.EachI/Oboardhastwoportsdesignatedasdiskdriveloops.These
ports connect to the internal dual-ported FC disk drives and are used to add
expansion units to the configuration. The two drive loop ports on the upper I/O
boardformFCloop 2(channel 2)while thetwo drive ports onthelower I/Oboard
form FC loop 3 (channel 3). FC loop 2 provides a data path from both RAID
controllerstotheAloopoftheinternaldiskdrives,whileFCloop3providesadata
path from both RAID controllers to the B loop of the internal disk drives.
Sun StorEdge 3511 SATA Array. RAID controller channels 0 and 1 are dedicated
host channels. Channels 4 and 5 are normally designated host channels but can be
configured as drive channels. RAID controller channels 2 and 3 are dedicated drive
channels that connect to expansion units.
How the channels are exposed on the StorEdge 3511 RAID controller is one of the
key differences between the StorEdge 3510 and 3511. Host channels 0 and 1 include
four ports per loop (two ports on the upper controller, and two ports on the lower
controller). Channels 0 and 1 support 1 Gbit or 2 Gbit data transfer speed.
Channels4and5providetwoportsperloop(oneportoneachcontroller).Channels
4 and 5 only support 2 Gbit data transfer speed.
Each RAID controller has two ports designated as disk drive loops. The drive ports
support only 2 Gbit data transfer speed. These ports connect to the internal Serial
ATAdiskdrivesusinginternalFC-SATAroutingtechnology.Driveportsareusedto
add expansion chassis to the configuration. Like the host channels, each drive
channelofthetopRAIDcontrollersharesaloopwiththematchingdrivechannelon
thebottomRAIDcontroller.Forexample,drivechannel2ofthetopRAIDcontroller
shares the same loop as channel 2 of the bottom RAID controller.
DAS and SAN Storage Architectures
There are two popular methods for connecting storage to servers.
(cid:1) Direct attached storage (DAS) uses a direct connection between a server and its
storage system. The DAS solution of connecting each server to its own dedicated
storage system is straight-forward, and the absence of storage switches can
minimize costs in some instances.
(cid:1) A storage area network (SAN) places a Fibre Channel (FC) storage switch
between network servers and storage systems. A SAN solution shares a storage
system among multiple servers using FC storage switches, and reduces the total
number of storage systems required for a particular environment at the cost of
additional element management (FC switches) and FC path complexity.
Best Practices for the Sun StorEdge 3510 and 3511 FC Arrays 5
Direct-Attached Storage (DAS)
Storage Area Network (SAN)
FIGURE1 DASandSANStorageArchitectures
Selecting the best storage architecture for a particular environment can be a
confusing exercise. In general, some environments are well-suited for DAS while
others will benefit greatly from SAN.
ThechallengeofselectingbetweenDASandSANisoftenfurthercomplicatedbythe
needtochoosebetweendifferentstoragesystems, onedesigned forDASoranother
intended for SAN. Fortunately, Sun StorEdge 3500 series arrays inherently supports
both DAS and SAN.
Fibre Channel Protocols
The Sun StorEdge 3500 series arrays support point-to-point and Fibre Channel–
ArbitratedLoops(FC–AL)protocols.Usingthepoint-to-pointprotocolwiththeSun
StorEdge 3500 series arrays requires a switched fabric network (SAN), whereas
selecting FC-AL mode enables the arrays to be used in either DAS or SAN
environments.Usingpoint-to-pointprotocolenablesfull-duplexuseoftheavailable
channel bandwidth, whereas selecting FC-AL mode limits host channels to half-
duplex communication.
In a point-to-point configuration, only one ID can be assigned to each host channel.
If more than one ID is assigned, the point-to-point protocol rules are violated. Any
host channel with more than one ID will not be able to log in to an FC switch in
fabricmode.This“one-ID-per-channel”requirementistrueinbothsingle-controller
anddual-controllerconfigurations.Thus,indual-controllerconfigurations,eitherthe
primary or the secondary controller can have an ID assigned, but not both. This
yields:
6 Sun StorEdge 3000 Family Best Practices Manual • June 2004
