Kilim: A server framework with lightweight actors, isolation types

127 Pages·2010·1.3 MB·English

by  Sriram Srinivasan

Technical Report
UCAM-CL-TR-769
ISSN 1476-2986
Number 769
Computer Laboratory
Kilim: A server framework with
lightweight actors, isolation types
and zero-copy messaging
Sriram Srinivasan
February 2010
15 JJ Thomson Avenue
Cambridge CB3 0FD
United Kingdom
phone [phone]
http://www.cl.cam.ac.uk/
(cid:13)c 2010 Sriram Srinivasan
This technical report is based on a dissertation submitted
February 2010 by the author for the degree of Doctor of
Philosophy to the University of Cambridge, King’s College.
Technical reports published by the University of Cambridge
Computer Laboratory are freely available via the Internet:
http://www.cl.cam.ac.uk/techreports/
ISSN 1476-2986
Abstract
Kilim: A Server Framework
with Lightweight Actors, Isolation Types & Zero-copy Messaging
SriramSrinivasan
Internetservicesareimplementedashierarchicalaggregatesofcommunicatingcomponents: networks
ofdatacenters,networksofclustersinadatacenter,connectedserversinacluster,andmultiplevirtual
machinesonaservermachine,eachcontainingseveraloperatingsystemsprocesses.
Thisdissertationarguesforextendingthisstructuretotheintra-processlevel,withnetworksofcom-
municating actors. An actor is a single-threaded state machine with a private heap and a thread of its
own. It communicates with other actors using well-defined and explicit messaging protocols. Actors
must be light enough to comfortably match the inherent concurrency in the problem space, and to
exploit all available parallelism. Our aims are two-fold: (a) treat SMP systems as they really are: dis-
tributed systems with eventual consistency, and (b) recognize from the outset that a server is always
partofalargercollectionofcommunicatingcomponents,thuseliminatingthemindsetmismatchbe-
tweenconcurrentprogramminganddistributedprogramming.
Althoughtheactorparadigmisbynomeansnew,ourdesignpointsareinformedbydrawingparallels
between the macro and micro levels. As with components in a distributed system, we expect that
actorsmustbeisolatableinanumberofways:memoryisolation,faultisolation,upgradeisolation,and
execution isolation. The application should be able to have a say in actor placement and scheduling,
andactorsmustbeeasilymonitorable.
Ourprimarycontributionisinshowingthattheserequirementscanbesatisfiedinalanguageanden-
vironmentsuchasJava,withoutchangestothesourcelanguageortothevirtualmachine,andwithout
leavingmuchoftheidiomaticambitofJava, withitsmindsetofpointersandmutablestate. Inother
words, one does not have to move to a concurrency-oriented language or to an entirely immutable
objectparadigm.
We demonstrate an open-source toolkit called Kilim that provides (a) ultra-lightweight actors (faster
andlighterthanextantenvironmentssuchasErlang),(b)atypesystemthatguaranteesmemoryisola-
tionbetweenthreadsbyseparatinginternalobjectsfromexportablemessagesandbyenforcingowner-
shipandstructuralconstraintsonthelatter(linearityandtree-structure,respectively)and,(c)alibrary
withI/Osupportandcustomizablesynchronizationconstructsandschedulers.
Weshowthatthissolutionissimplertoprogramthanextantsolutions,yetstaticallyguaranteedtobe
freeoflow-leveldataraces. Itisalsofaster,morescalableandmoresteady(inincreasingscale)intwo
industrial strength evaluations, interactive web services (comparing Kilim Web Server to Jetty) and
databases(comparingBerkeleyDBtoaKilimvariantofit).

Acknowledgments
ToAlka,mydearest. Life’sawonderfuljourney,andIhavethebesttravelcompanion.
ToAmmaandAppa, fortheeasylaughter, loveandencouragement, andtheearlymorningsandlate
nights they have lavished on me. This dissertation is dedicated to Amma. She would have exclaimed
“hai”!
Tomyparents-in-law,forbeinganadditionalsetofparents. Oneshouldbesolucky.
To my advisors. Prof. Jean Bacon, for making this excursion possible, for allowing me to gradually
discover my interests and strengths without pressure, and for her gentle and selfless guidance to col-
laborate with other groups. To Prof. Alan Mycroft, for setting distant Bézier control points that so
significantlyinfluencedthearcofmylearning,forhisexhortationsandforhisunwaveringbeliefthat
Iwouldgetthere. ToDr. KenMoody,forhiscriticalreadingofmypapersandthisdocument,forhis
cheerfulandenthusiasticcross-pollinationofideas,andtoursofbeersandwines.
To Reto Kramer, for his friendship, guidance, technical knowledge, and a brain the size of a planet.
There are a few dissertations lurking in there. To Lauri Pesonen and David Pearce, for the critical
reviews, for the long coffee sessions packed with techie talk, and for sharing my enthusiasm for pro-
gramminglanguages. ToEikoYoneki,PedroBrandão,DavidEyers,DaveEvans,MinorGordonand
NikoMatsakisforsharingtheirimpressiveknowledge,opinionsandgoodhumor,forchallengingmy
biasesandassumptions,andforthegentleproddingtogetonwithit.
ToBorisFeiginandKathrynGrayforintroducingmetotheworldoftypesandsemantics,andholding
meaccountabletorigor.
To seminal influences in my professional life: Ed Felt, Anno Langen, Uday Rangaswamy, Gopalan
Sankaran, Jolly Chen, Paul Ambrose and Terence Parr. I am a professional learner because of them.
To Tony Hoare, Simon Peyton Jones, Mooly Sagiv and Uday Khedker for giving me their valuable
timeandforjusttherightmixofhardquestionsandencouragement. ToTimHarris,MartinRichards,
CliffClick,ChrisNewcombeandtheBerkeleyDBteamforlendingtheirear,timeandpatience.
To Samuel Kounev, Anil Madhavapeddy, Sandy & Elisabeth Fraser for their viral enthusiasm and
constructiveencouragement. Ihopetopassthaton.
TovariouscafesinBerkeleyandCambridge: theBeanery,Roma,NeroandKing’s,whereIwasboth
pupaandpupil!
And finally, in a dissertation that speaks of networks, load-sharing and fault-tolerance, it seems only
fittingtoacknowledgemyownwonderfulnetwork,onethatsharesmyloadandtoleratesmyfoibles:
mysisterVidyaandlittleMalavikaforbeingarockofsupport;Praveen&Shailaandthecuties,Anjali
andSanjana;Lizette,forthefood,coffee,theWireandthegoodcheer,andtoallourdearfriends: Ajit,
Alka (Raghuram), Amita, Anil, Arjuni, Ashwini, Aslam, Athula, Banny, Caroline, Conor, Deepak,
Flavia, Geetanjali, Hanna, Haresh, Hiroo, Jennifer, Kavita, Kinkini, Kirtana, Krishna, Larry, Lata,
Lise, Maithili, Mark, Mayank, Namrata, Naren, Nasreen, Nishanth, Niteen, Nupur, Peggy, Prasad,
Raghu, Raman, Rao, Sanjay, Santosh Jain, Sarita, Satish, Sarah, Savita, Scarlet, Shaheen, Shobana,
Simona,Tara,Tom,Trish,UshaJain,Uma,Vandana,Viji,Vinod,Vipul.
Thankyouall.

Contents
1 Introduction 11
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2 ThesisContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.3 Kilim: CommunicatingActorsforJava . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.5 DissertationOutline. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2 Background 21
2.1 Present-dayserverarchitectureissues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1.1 Dataplane: Statetransformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1.2 Dataplane: Immutableobjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1.3 Controlplane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Hardwaretrendsandserver-sidedesignopportunities . . . . . . . . . . . . . . . . . . . . . . 31
2.3 CommunicatingActors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.4 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4.1 ControlPlane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4.2 DataPlane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.4.3 Isolation: notanisolatedquest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3 Kilim Abstract Machine 39
3.1 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 OperationalSemantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4 Type System 47
4.1 DesignChoices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.1.1 Puttingthemalltogether. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.2 Phase-1: BaseTypeInference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.2.1 Phase-1TerminationandSoundness: Proofsketch . . . . . . . . . . . . . . . . . . . 53
4.3 TypestatesandHeapAbstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.3.1 Objecttypestate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
8 CONTENTS
4.3.2 Shapegraphs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3.3 States+Shapegraph=Shape-stategraph . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.3.4 PropertiesofShape-Stategraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.4 Phase-2: Messageverification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.4.1 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.4.2 SemanticRules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.5 Soundnessofmessageverification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.6 MappingtoJava. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.6.1 Sharabletypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.6.2 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.6.3 CollectionsandIterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.7 Relatedwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5 Kilim architecture and implementation 81
5.1 ActorsandFibers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.1.1 Continuations: Anoverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.1.2 AutomaticStackManagement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.1.3 JVMconstraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.1.4 Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.1.5 CooperativePreemption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.1.6 LightweightThreads: RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2 CommunicationandCoordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.3 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.3.1 SchedulerHopping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.3.2 Scheduling: RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.4 NetworkSupport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.4.1 TheKilimI/Oarchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.4.2 NetworkSupport: RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6 Evaluation and Performance 97
6.1 In-corebenchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.2 Networkbenchmark: KilimI/Ovs. Thread-per-request . . . . . . . . . . . . . . . . . . . . 98
6.2.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
CONTENTS 9
6.3 CPU+Networkbenchmark: KilimWebServer . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.3.1 KWSarchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.3.2 KWSapplication: auto-completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.4 CPU+Disk: BerkeleyDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.4.1 BerkeleyDB:originalarchitecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.4.2 BerkeleyDB:Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.4.3 BerkeleyDB:BenchmarkandResults . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.5 QualitativeEvaluation: Kiliminpractice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7 Conclusions and Future Work 109
7.1 FutureWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
8 Notation and Nomenclature 113
Bibliography 115