Amazon Simple Storage Service

153 Pages·2012·0.81 MB·English

Amazon Simple Storage
Service
Developer Guide
API Version 2006-03-01
AmazonSimpleStorageServiceDeveloperGuide
AmazonSimpleStorageService:DeveloperGuide
Copyright©2008AmazonWebServicesLLCoritsaffiliates.Allrightsreserved.
AmazonSimpleStorageServiceDeveloperGuide
Table of Contents
What'sNew......................................................................................................................................1
WelcometoAmazonS3..................................................................................................................3
IntroductiontoAmazonS3..............................................................................................................6
CoreConcepts..................................................................................................................................7
ComponentsofAmazonS3..........................................................................................7
Operations.....................................................................................................................8
AmazonS3ApplicationProgrammingInterfaces(API)..............................................8
AmazonS3DataConsistencyModel...........................................................................9
PayingforAmazonS3................................................................................................10
UsingAmazonS3..........................................................................................................................11
WorkingwithAmazonS3Components.....................................................................11
WorkingwithAmazonS3Buckets...................................................................11
BucketRestrictionsandLimitations........................................................12
BucketConfigurationOptions.................................................................13
BucketsandAccessControl....................................................................14
BillingandReportingofBuckets............................................................14
WorkingwithAmazonS3Objects....................................................................14
Keys.........................................................................................................15
ListingKeys...................................................................................15
CommonListRequestParameters.................................................16
CommonListResponseElements.................................................16
IteratingThroughMulti-PageResults............................................18
ListingKeysHierarchicallyusingPrefixandDelimiter................19
Metadata..................................................................................................19
GettingObjects........................................................................................20
StandardDownloads......................................................................20
ChunkedandResumableDownloads............................................21
AuthenticationandAccessControl............................................................................21
Authentication...................................................................................................22
AccessControlLists..........................................................................................25
QueryStringAuthentication.............................................................................30
RequestRouting..........................................................................................................31
RequestRedirectionandtheRESTAPI...........................................................31
DNSConsiderations..........................................................................................36
PerformanceOptimization..........................................................................................36
TCPWindowScaling........................................................................................36
TCPSelectiveAcknowledgement.....................................................................37
UsingAmazonDevPaywithAmazonS3...................................................................37
AmazonS3CustomerDataIsolation................................................................37
AmazonDevPayTokenMechanism.................................................................38
AmazonS3andAmazonDevPayAuthentication............................................39
AmazonS3BucketLimitation..........................................................................39
AmazonS3andAmazonDevPayProcess........................................................40
AdditionalInformation......................................................................................40
WorkingwithErrors...................................................................................................40
AmazonS3ErrorBestPractices.......................................................................40
ErrorResponse..................................................................................................41
ErrorCode...............................................................................................41
ErrorMessage..........................................................................................42
FurtherDetails.........................................................................................42
ListofErrorCodes..................................................................................42
ServerAccessLogging...............................................................................................46
ServerAccessLoggingConfigurationAPI.......................................................47
AmazonSimpleStorageServiceDeveloperGuide
DeliveryofServerAccessLogs........................................................................49
ServerAccessLogFormat................................................................................50
SettingUpServerAccessLogging...................................................................53
UsingtheRESTAPI......................................................................................................................57
CommonRESTAPIElements...................................................................................58
TheRESTErrorResponse..........................................................................................59
AuthenticatingRESTRequests..................................................................................60
SettingAccessPolicywithREST...............................................................................70
VirtualHostingofBuckets.........................................................................................71
RequestRedirectionandtheRESTAPI.....................................................................74
Browser-BasedUploadsUsingPOST........................................................................77
Introduction.......................................................................................................77
HTMLForms....................................................................................................78
Examples...........................................................................................................86
POSTwithAdobeFlash....................................................................................94
OperationsontheService...........................................................................................95
GETOperation..................................................................................................95
OperationsonBuckets................................................................................................96
PUTBucket.......................................................................................................97
GETBucket.......................................................................................................98
GETBucketLocation......................................................................................100
DELETEBucket.............................................................................................100
POSTObject...................................................................................................101
OperationsonObjects...............................................................................................107
PUTObject......................................................................................................107
GETObject.....................................................................................................109
HEADObject..................................................................................................114
DELETEObject..............................................................................................114
UsingtheSOAPAPI....................................................................................................................116
CommonSOAPAPIElements.................................................................................116
TheSOAPErrorResponse.......................................................................................117
AuthenticatingSOAPRequests................................................................................117
SettingAccessPolicywithSOAP............................................................................118
OperationsontheService.........................................................................................119
ListAllMyBuckets...........................................................................................119
OperationsonBuckets..............................................................................................120
CreateBucket...................................................................................................121
DeleteBucket...................................................................................................121
ListBucket.......................................................................................................122
GetBucketAccessControlPolicy......................................................................124
SetBucketAccessControlPolicy.......................................................................124
GetBucketLoggingStatus................................................................................125
SetBucketLoggingStatus.................................................................................126
OperationsonObjects...............................................................................................127
PutObjectInline................................................................................................127
PutObject.........................................................................................................129
GetObject........................................................................................................131
GetObjectExtended.........................................................................................136
DeleteObject....................................................................................................136
GetObjectAccessControlPolicy.......................................................................137
SetObjectAccessControlPolicy.......................................................................138
UsingBitTorrent™withAmazonS3..........................................................................................140
HowYouareChargedforBitTorrentDelivery........................................................140
UsingBitTorrenttoRetrieveObjectsStoredinAmazonS3....................................141
PublishingContentUsingAmazonS3andBitTorrent.............................................141
Glossary.......................................................................................................................................143
DocumentConventions................................................................................................................144
Index.............................................................................................................................................147
AmazonSimpleStorageServiceDeveloperGuide
What's New
ThisWhat'sNewisassociatedwiththe2006-03-01releaseofAmazonS3.Thisguidewaslastupdated
onApril09,2008.
ThefollowingtabledescribestheimportantchangessincethelastreleaseoftheAmazonS3Developer
Guide.
Change Description ReleaseDate
LoggingChanges AmazonS3nowenablesyoutoautomaticallygrantaccessto 7April2008
logswithinabuckettousersotherthanthebucketowner.For
moreinformation,seeSettingUpServerAccessLogging.
TCPWindowScaling AmazonS3nowsupportsTCPwindowscalingandTCP 3March2008
selectiveacknowledgementwhichenablesyoutooptimize
networkperformance.Formoreinformation,seePerformance
Optimization.
Chunkedand Theguidewasupdatedtodescribehowtoperformchunked 11January
Resumable andresumabledownloads.Formoreinformation,see 2008
Downloads ChunkedandResumableDownloads.
HTTPPOSTchanges Theredirectfieldwaschangedtosuccess_action_redirectand 31December
thesuccess_action_statusfieldwasadded.Formore 2007
information,seeBrowser-BasedUploadsUsingPOST.
DevPay AmazonDevPayenablesyoutochargecustomersforusing 18December
yourAmazonS3productthroughAmazon'sauthentication 2007
andbillinginfrastructure.Youcanchargeanyamountfor
yourproductincludingusagecharges(storage,transactions,
andbandwidth),monthlyfixedcharges,andaone-time
charge.Formoreinformation,seeUsingAmazonDevPay
withAmazonS3.
HTTPPOST AmazonS3nowsupportsbrowser-baseduploadsusing 17December
POST,whichallowsyouruserstouploadcontentdirectlyto 2007
AmazonS3.Formoreinformation,seeBrowser-Based
UploadsUsingPOST.
Restructuringand Theintroductorysectionsofthedocumentwererestructured 17December
VariousEdits andnumerouseditsweremadebasedoncustomerinputfrom 2007
APIVersion2006-03-01
1
AmazonSimpleStorageServiceDeveloperGuide
Change Description ReleaseDate
theFeedbacklinkandforums.
LocationConstraints AmazonS3nowsupportslocationconstraints,whichallow 15October
youtospecifywheretostoredata.Formoreinformation,see 2007
LocationSelection.
SupportforRedirects IfDNSinformationforabucketisnotpropagatedthroughout 15October
theInternet,clientswillreceivea307redirect.Ifyouattempt 2007
useapath-stylerequesttoaccessanobjectwithinabucket
thatwascreatedusing<CreateBucketConfiguration>,
youwillreceiveapermanent301redirect.Formore
informationonredirects,soyoucanoptimizeyourcode,see
LocationSelection.
BucketLocation AmazonS3supportsanewoperationforgettingthelocation 15October
ofabucket.Formoreinformation,seeGETBucketLocation. 2007
NewAuthentication Theauthenticationsectionwasrewrittentoclarifyquestions 10September
Section thatappearedintheforums.Formoreinformation,see 2007
AuthenticationandAccessControl.
Feedback Youcannowprovidefeedbackcommentsonanytopicinthe 10September
HTMLversionofthisguide.Toprovidefeedback,simply 2007
clicktheFeedbacklinkatthetopofthepage.
MinorEdits Minoreditsweremadethroughoutthedocumenttoclarify 10September
issuesthatappearedintheforumsandtoimproveoverall 2007
documentquality.
AmazonDevPay AmazonDevPayisanewAmazonservicethatenablesyouto 10September
chargecustomersforuseofyourAmazonS3productthrough 2007
theAmazonauthenticationandbillinginfrastructure.For
moreinformation,seeUsingAmazonDevPaywith
AmazonS3.
NewBucketLimit Inadditiontothe100bucketlimitassociatedwithyourAWS 10September
account,eachofyourcustomerscanhaveupto100buckets 2007
foreachAmazonDevPayproductthatyousell.Formore
information,seeUsingAmazonDevPaywithAmazonS3.
APIVersion2006-03-01
2
AmazonSimpleStorageServiceDeveloperGuide
Audience
Welcome to Amazon S3
Topics
• Audience
• HowThisGuideIsOrganized
• RelatedResources
ThankyouforyourinterestinAmazonS3.
Thissectiondescribeswhoshouldreadthisguide,howtheguideisorganized,andotherresources
relatedtoAmazonS3.
AmazonS3willoccasionallybereferredtowithinthisguideassimply"S3";allcopyrightsandlegal
protectionsstillapply.
Wehopeyoufindtheservicetobeeasy-to-use,reliable,andinexpensive.Ifyouwanttoprovide
feedbacktotheAmazonS3developmentteam,pleasepostamessagetotheAmazonS3Developer
Forum.
Audience
ThisguidedescribestheAmazonS3interfacesandfunctionalityindetailandisintendedfordevelopers
whoarebuildingapplicationandservicesthatneedtostoreandretrieveanyamountofdata,atanytime,
fromanywhereontheweb.
Required Knowledge and Skills
Useofthisguideassumesyouarefamiliarwiththefollowing:
• XML(SeeW3SchoolsXMLTutorial)
• Basicunderstandingofwebservices(SeeW3SchoolsWebServicesTutorial))
• Aprogramminglanguageforconsumingawebserviceandanyrelatedtools
YoushouldalsohavereadtheAmazonS3GettingStartedGuide.
APIVersion2006-03-01
3
AmazonSimpleStorageServiceDeveloperGuide
RelatedResources
How This Guide Is Organized
Thisguideisorganizedintoseveralmajorsectionsdescribedinthetablebelow.
Information RelevantSections
GeneralinformationaboutAmazonS3 • IntroductiontoAmazonS3
Conceptualinformationabout • CoreConcepts
AmazonS3
InformationaboutusingAmazonS3 • UsingAmazonS3
APIInformation • UsingtheRESTAPI
• UsingtheSOAPAPI
InformationaboutBitTorrent • UsingBitTorrent™withAmazonS3
Typographicandsymbolconventions • DocumentConventions
Eachsectioniswrittentostandonitsown,soyoushouldbeabletolookuptheinformationyouneed
andgobacktowork.However,youcanalsoreadthroughthemajorsectionssequentiallytogetin-depth
knowledgeabouttheAmazonS3.
Related Resources
Thetablebelowlistsrelatedresourcesthatyou'llfindusefulasyouworkwiththisservice.
Resource Description
AmazonS3GettingStartedGuide TheGettingStartedGuideprovidesaquicktutorialofthe
servicebasedonasimpleusecase.Examplesand
instructionsforJava,Perl,PHP,C#,Python,andRubyare
included.
AmazonS3DeveloperGuide TheDeveloperGuide(whichyouarereading)providesa
detaileddiscussionoftheservice.Itincludesanoverview,
programmingreference,andAPIreference.
AmazonS3ReleaseNotes TheReleaseNotesgiveahigh-leveloverviewofthecurrent
release.Theyspecificallynoteanynewfeatures,corrections,
andknownissues.
AWSDeveloperResourceCenter Acentralstartingpointtofinddocumentation,code
samples,releasenotes,andotherinformationtohelpyou
buildinnovativeapplicationswithAWS.
APIVersion2006-03-01
4
AmazonSimpleStorageServiceDeveloperGuide
RelatedResources
Resource Description
AmazonS3productinformation TheprimarywebpageforinformationaboutAmazonS3.
DiscussionForums Acommunity-basedforumfordeveloperstodiscuss
technicalquestionsrelatedtoAmazonWebServices.
E-mailaddressforquestionsrelatedto Thise-mailaddressisonlyforaccountquestions.For
yourAWSaccount: technicalquestions,usetheDiscussionForums.
<[email protected]>
APIVersion2006-03-01
5
AmazonSimpleStorageServiceDeveloperGuide
OverviewofAmazonS3
Introduction to Amazon S3
Topics
• OverviewofAmazonS3
• ServiceFeatures
ThisintroductiontoAmazonS3isintendedtogiveyouadetailedsummaryofthiswebservice.After
readingthissection,youshouldhaveagoodideaofwhatitoffersandhowitcanfitinwithyour
business.
Overview of Amazon S3
AmazonS3isstoragefortheInternet.Itisdesignedtomakeweb-scalecomputingeasierfordevelopers.
AmazonS3hasasimplewebservicesinterfacethatcanbeusedtostoreandretrieveanyamountof
data,atanytime,fromanywhereontheweb.Itgivesanydeveloperaccesstothesamehighlyscalable,
reliable,fast,inexpensivedatastorageinfrastructurethatAmazonusestorunitsownglobalnetworkof
websites.Theserviceaimstomaximizebenefitsofscaleandtopassthosebenefitstodevelopers.
Service Features
AmazonS3isintentionallybuiltwithaminimalfeaturesetthatfocusesonsimplicityand
robustness.ThefollowingaresomeoffeaturesoftheAmazonS3service:
• Write,read,anddeleteobjectsfrom1byteto5gigabytesinsizewithaccompanyingmetadata.There
isnofixedlimitonthenumberofobjectsyoucanstore.
• Astraightforwardflatobjectstoremodel,whereeachobjectisstoredandretrievedusingaunique
developer-assignedkey
• Authenticationmechanismsareprovidedtoensurethatdataiskeptsecurefromunauthorizedaccess.
Objectscanbemadeprivateorpublicandrightscanbegrantedtospecificusers.
• Standards-basedRESTandSOAPinterfacesdesignedtoworkwithanyInternet-developmenttoolkit.
APIVersion2006-03-01
6