Table Of ContentTransactional Partitioning: A New Abstraction for
Main-Memory Databases
Vivek Shah
(supervised by Marcos Vaz Salles)
UniversityofCopenhagen
Copenhagen,Denmark
[email protected]
7
1
0 ABSTRACT Stonebraker et al. have shown that OLTP systems can
2 be deployed in main-memory using shared-nothing cluster
The growth in variety and volume of OLTP (Online Trans-
of machines at modest cost [21]. This leads to order-of-
n action Processing) applications poses a challenge to OLTP
magnitudegainsinperformanceoverclassicalrelationaldatabase
a systemstomeetperformanceandcostdemandsintheexist-
J ing hardware landscape. These applications are highly in- managementsystems(RDBMS).Withthereductioninprices
ofcommodity hardwareandgrowthof cloudcomputing[1],
6 teractive(latencysensitive)andrequireupdateconsistency.
a cluster-based infrastructure consisting of shared-memory
1 They target commodity hardware for deployment and de-
commodityhardwarenodesincreasinglyprovidesflexiblemain-
mand scalability in throughput with increasing clients and
memory deployment options to meet HICCUP application
] data. Currently, OLTP systems used by these applications
B requirements.
provide trade-offs in performance and ease of development
D overavarietyofapplications. Inordertobridgethegapbe- In order to build HICCUP applications for cluster-based
architectures, there are currently two dominant program-
. tween performanceandease ofdevelopment,weproposean
s intuitive,high-levelprogrammingmodelwhichallowsOLTP ming models exposed to the application developer. The
c first approach abstracts the distributed architecture into a
applications to be modeled as a cluster of application logic
[
uniform shared-memory space, which is then presented us-
units. By extending transactions guaranteeing full ACID
1 semantics to providetheproposed model, wemaintain ease ing a unified data model, e.g., the relational model under
v of application development. The model allows the applica- strongconsistencysemantics(ACIDtransactions). Thesec-
9 tion developer to reason about program performance, and ond approach abstracts the distributed architecture into a
3 toinfluenceitwithouttheinvolvementofOLTPsystemde- low-level distributed storage-oriented model, e.g., key value
3 signers (database designers) and/or DBAs. Asa result, the stores with no/loose consistency semantics.
4 Althoughthefirstapproachprovideseaseofdevelopment,
databasedesignerisfreetofocusonefficientrunningofpro-
0 performance varies with the variety of applications. This
grams to ensureoptimal cluster resource utilization.
. is because of the sensitivity of the approach to data and
1
codepartitioningtoreducetheimpact ofdistributedtrans-
0 1. INTRODUCTION
actions [11]. In the second approach, performance is con-
7
OLTPapplicationshavegrowninvolume,varietyandper- trolledbytheapplication developer,butdevelopmentisex-
1
: formance demands [20]. In addition to the classic banking, tremelyhardduetono/looseconsistencysemanticsandthe
v reservation, and order entry systems, novel applications in- low-level, storage-oriented programming model. There are
i clude massively multiplayer online worlds (MMOW) or vir- in-betweenapproachesthatpartitiontheunifieddatamodel
X
tualworlds,massivelymultiplayeronlineroleplayinggames and provideconsistency semantics within partitions [2, 19].
r (MMORPG), financial applications (e.g., online trading), However, in these systems, the clients must specify these
a
telecommunication applications and information visualiza- partitions and handle consistency across partitions. This
tions. These applications demand scalability in through- increasesapplicationcomplexity,especiallyforHICCUPap-
put with growing data and clients. They exhibit high in- plications which cannot always be architected to eliminate
teractivity(latencysensitive)whilemaintainingconsistency inter-partition accesses.
on updates. Furthermore, they typically target commodity Wewanttoaddressthisgapbyexploringamiddleground
hardware for deployment. We will refer to them as HIC- betweenthetwoextremeapproaches. Strongconsistencyse-
CUP(Highly-InteractiveCommodity-HardwareConsistent- mantics(ACIDtransactions)provideeaseofapplicationde-
on-Update) applications. velopment. Sacrificing consistency semantics is not enough
to guarantee OLTP application performance [10]. We want
to build a transactional system maintaining global consis-
tencysemanticsintheclustertoenableeaseofdevelopment
while guaranteeing optimal cluster performance. In order
todoso,weproposetoextendthetransactionalabstraction
This work is licensed under the Creative Commons Attribution-
with a logical distribution abstraction. This enables the
NonCommercial-NoDerivs3.0UnportedLicense.Toviewacopyofthisli-
cense,visithttp://creativecommons.org/licenses/by-nc-nd/3.0/. Obtainper- application developers to reason about the performance of
mission prior to any use beyond those covered by the license. Contact their application in terms of distributed transactional logic
[email protected]. units and thuswrite efficient programs.
Proceedings of the VLDB 2014 PhD Workshop.
1
While the logical distribution abstraction hides the low- 2.2 State ofthe Art
level distributed architecture, it allows application develop- In recent times there have been numerous implementa-
erstounderstandprogramperformancebehaviorinacluster tionsofsystemsthatcanbeusedbyHICCUPapplications.
setup and to influence it. Database designers can then fo- There has been an explosion in the implementation of key-
cus on running the programs efficiently for optimal cluster value stores, which provide varying performance, scalabil-
resource utilization. This approach ensures that the appli- ity and availability characteristics. Amazon Dynamo [8],
cation developers can participate in improving the perfor- Google BigTable [3], Yahoo! PNUTS [5], Amazon HBase
mance of the application. The approach also reduces the and Cassandra [15], Amazon S3, Google Cloud Storage,
criticality of data and code partitioning. As a result, it Windows Azure Storage are some examples of distributed
removesdatabasedesignersfrom thecriticalpathofperfor- key-valuestores. Thesesystemstargetcommodityhardware
mance improvement over a variety of applications by sep- in acluster-based architectureandprovidehigh availability
arating concerns of writing efficient programs (application andscalabilityatthecostoflowerconsistencysemanticsand
developers) and running them efficiently (database design- a low-level storage-oriented programming model. In order
ers). Global consistency guarantees in the cluster free the to build HICCUP applications using these systems, appli-
developer from worrying about consistency issues, thus re- cations must implement multi-key transactions and man-
ducing program complexity and development costs. age the low-level key-value based infrastructure, which is
The remainder of the paper is organized as follows. In extremely complex and error prone, thus limiting ease of
Section 2,weprovideabackgroundoftheproject bylaying development [22].
downthedesignrequirementsforOLTPsystemsandreview- In order to fix these problems, transactional capabilities
ing the state of the art with respect to these requirements. were introduced in distributed data stores, such as Google
InSection3,wedescribeourapproachingreaterdetailwith Spanner [6] and Megastore [2], or exposed using client li-
1
an example OLTP application (the TPC-C benchmark). braries [9, 18]. Megastore [2] provides full ACID semantics
Finally,inSection4,weoutlinethechallengesandroadmap butonlywithinpartitionsofthedata,whichlimitstheease
overthecourseof theproject with some initial thoughtson of application developmentand flexibledeployments. Warp
how to tackle them. [9] and Percolator [18] provide multi-key transactions us-
ing client libraries but their storage-oriented programming
2. BACKGROUND modelandclient-basedexecutionmakeitdifficulttoreason
aboutapplicationperformanceforvaryingcluster-basedde-
In this section, we provide a background of the Ph.D.
ployments.
project by first isolating the OLTP design requirements in
On the other end, classical RDBMS provide a high-level
the current hardware and software landscape and then re-
uniform shared-memory abstraction, which makes applica-
viewing the state of the art with respect to these design
tion development extremely easy. However, these systems
requirements.
face performance scalability issues. In order to meet these
challenges, Shore-MT [13] and DORA [16] target shared-
2.1 OLTP DesignRequirements
memory architectures for high performance. Shore-MT op-
timizes a classical RDBMS engine by reducing or removing
1. In order to meet the requirements of a growing vari- contention points for multi-core and multi-processor hard-
ety of HICCUPapplications, OLTPsystems must ex- ware. DORA proposes a novel methodology to partition
pose a programming model which allows application data amongst theprocessing threads and migrates transac-
developers to reason about program performance in a tions between threads based on the partitioning. Although
cluster-basedsetupandtoimprovetheperformanceof boththesesystemseaseapplication development,reasoning
their architected applications (programs). aboutprogramperformanceremainsextremelyhard. These
systemsarealsonotsuitedforcluster-baseddeploymentun-
2. Lack of global consistency semantics and a low-level less a shared-memory layer is built underneaththem.
distributedstorage-orientedprogrammingmodelmake H-Store was one of the first systems to target cluster-
application development extremely expensive, error- based deployments for OLTP applications [14]. In order
proneandlimittheadoptabilityofsuchasystem[22]. to providea high-level relational abstraction, H-Storeis re-
OLTPsystemsmustprovideahigh-level,intuitivepro- liantoninferringoptimaldatapartitioningfromapplication
gramming model with strong global consistency se- logic tominimizetheimpactofdistributedtransactions[7].
mantics over the distributed cluster setup to ease ap- This makes it extremely difficult for application develop-
plication development. ers to reason about program performance over a variety of
applications and increases the performance reliance of the
3. Totakeadvantageofadvancesincommodityhardware system on automatic data [17] and code partitioning [4],
andthegrowthofcloudcomputing[1],OLTPsystems or database administrators (DBAs). This brings database
must target a hardware ecosystem of shared-memory designers and/or DBAs in the critical path of performance
nodes in a shared-nothing setup, allowing flexible de- improvementsofvaryingOLTPapplicationsandignoresap-
ployments varying in performance and cost. OLTP plication developers as a valuable resource.
systems must also be designed to utilize the multi- Our approach is to provide a high-level, intuitive pro-
processor, multi-coreprocessor and multi-tieredcache gramming model over a cluster-based architecture where
architecture in individual cluster nodes to ensure op- the uniform shared-memory space is partitioned by appli-
timal hardware performance [13]. cation logic. This partitioning is done by the application
developers, which allows them to reason about the perfor-
1http://www.tpc.org/tpcc/spec/tpcc_current.pdf manceoftheirprogramsandtoimprovethem. Byextending
2
transactions to provide this abstraction with global ACID txnnew order(w id,d id, c id, order) {
semantics, our approach maintains ease of application de- //Get the order id from the district and insert the
velopment. //next order id to be used in it
<wh,dist,cust>=gen order id(w id,did, c id, order);
3. APPROACH
total =0;
Inthissection, weexplainourapproach indetailwith an for(orditem in order.items) {
//Get the amount for the number of items ordered
exampleHICCUPapplication(theTPC-Cbenchmark)and
amount=get amount(orditem);
demonstrate how it meets thedesign requirements outlined
total +=amount;
in Section 2.
//Get the district information of the stock from the
3.1 Logical Partitions
//supplier warehouse
In a cluster-based architecture, the programming model stock info = get distinfo stock(orditem);
mustprovideanabstractiontobalanceeaseofdevelopment
//Update the stock of the supplier warehouse
and utilization of cluster resources. We want to explore
update stock(orditem,amount);
the middle ground between the two current dominant ap-
proachesbyallowing programmerstoreason abouttheper- add(”order line”, dist.order id, w id, d id, stock info,
formance of their programs in the distributed setup while amount, ...);
maintainingeaseofuse. Wedosobyintroducingthenotion }
total pay =(1+wh.tax+dist.tax)∗ total ∗ (1 −cust.
of logical partitions. A logical partition forms the unit of
discount);
logical code isolation. A programmer architects his appli- returntotal pay;
cation to consist of multiple application logic units, which }
communicate with each other, and establishes control flow
dependenciesbetweenthem. Anapplicationlogicunitmust
Figure 1: New order transaction
be associated with a logical partition by the programmer.
Communication between the application logic units is ex-
pensive,whichencouragesthedeveloperstothinkaboutthe
Modelinghardtopartitionapplications. Themodelis
performanceoftheconstructedprogramandtomakeitmore
ideally suited for applications wherepartitioning is implicit
efficient. Inordertoprovideeaseofdevelopment,ACIDse-
intheapplicationlogic. Fortunately,thispropertyholdsfor
manticsaremaintainedgloballyacrosslogicalpartitionsvia
a large class of HICCUP applications. Although applica-
transactions, which form the unit of code construction in
tions that are hard to partition, e.g., social networks would
this model .
not bea compelling usecase for ourmodel, thereisnorea-
In our model, a program (transaction) begins execution
sonforthemtoperformworseusingourmodelcomparedto
onalogicalpartition. Duringitsexecutiontheprogramcan
RDBMS and key-value stores. On the contrary, the model
only access data on that logical partition. If it needs to
providesananalyticalplaygroundtoapplication developers
access data on another logical partition, it must explicitly
to experiment with different partition layouts. This allows
do so by invoking a program on another logical partition
themtoadopttheappropriatemodel(relationalorkey-value
which is executed as a subtransaction [25, p. 253]. When
stores) for their application depending on the performance
a transaction or subtransaction is invoked, the logical par-
needsanddevelopmentcosts. Thegoalofthemodelisnotto
tition where it must be executed must be specified as well.
infer a partitioning layout for non-partitioned transactions
Program code is pre-compiled on each logical partition and
but to provide a framework to the application developer to
executed within thedatabase process for performance.
specify transactions with respect to a partitioning layout.
Incontrast topartitioning data,ourmodel forces theap-
plication developer to partition his application logic. The
System Initialization. The model deterministically as-
programmer must construct his application in terms of dis-
signs transactions to logical partitions. Since the model al-
tributedlogicalpartitionsbyassociatingapplicationlogicto
ways starts with an initial empty database state, the user
them. Data partitioning is a consequence of the code par-
applicationmustprovideinitializationtransactionstocreate
titioning formulated by the application developer. Existing
aninitial databasestate. Sincethosetransactionswould be
OLTP systemseitherprovidetheview of asingle datapar-
created using the proposed model, data distribution would
tition or fragmented data partitions, but not partitioning
always be deterministic and unambiguous. It is important
in terms of application logic. This increases the reliance
tomentionthatcommandlogging toprovidedurabilitycan
of these systems on optimal code and data partition lay-
beinterpretedasanautomaticprogram togenerateagiven
out, thus making it hard for application developers to rea-
desirable initial databasestate. Thishighlights thefocus of
sonabouttheperformanceofavarietyofapplicationsusing
theapproach to logically partition code and not data.
these systems.
Insummary,ourmodelallowstheapplicationdeveloperto
3.2 TPC-C Example
reason about program performance in terms of distributed
application logic units by exposing coordination costs be- The ease of partitioning application logic will determine
tween logical partitions. The database designer can focus the ease of applicability of this approach. HICCUP appli-
onprovidingefficientmappingoflogicalpartitionstophysi- cations that have simple application logic are well suited
calpartitions(machines)foroptimalclusterresourceutiliza- for this approach as we demonstrate by explaining our ap-
tion. Ourmodelallowseaseofconstructionanddeployment proachusingthenew ordertransactionclassfromtheTPC-
ofdifferentHICCUPapplicationsacrossvariousclustercon- C benchmark as an example. We will not go into the de-
figurations. tailedexplanationofthebenchmarkwhichisavailableinits
3
txnnew order update stock(order){ txnnew order(w id,d id, c id, order) {
result =<>; <wh,dist,cust>=gen order id(w id,did, c id, order);
for(orditem in order.items) { results[order.num s w id]; i=0;
amount=get amount(orditem);
stock info = get dist infostock(ord item); for(s id in order.supplier w id) {
update stock(orditem,amount); //Execute the remote transaction for all the order
append(result,<stock info,amount>); // items requested of the remote supplier warehouse
} results[i++]=EXECnew order update stock
returnresult; (subset(order, s id)) ONPARTITION(s id);
} }
//Use results to add order line and compute total pay
Figure 2: Transaction to read and update stock in-
total =0;
formation for(result in results) {
for(item result in result) {
PARTITIONING FUNCTIONmap(w id){returnw id;}; total +=item result.amount;
new orderPARTITIONMAPPER map; add(”order line”, dist.order id, w id, did,
new order update stockPARTITIONMAPPER map; item result, ...) ;
}
}
Figure 3: Mappingtransactions to logical partitions total pay =(1+wh.tax+dist.tax)∗total∗(1−cust.discount);
returntotal pay;
specification2. }
We begin by expressing the new order application logic
without any notion of partitioning. This is followed by the
Figure 4: New order transaction partitioned by
transformation of the program into the proposed model by
warehouses
partitioning the new order transaction by warehouses. Fi-
nally,weshow howdeveloperscan reason about theperfor- logiciswarehouse-affineandbothdataandnumberofclient
mance ofthistransaction and rewrite it in ordertomakeit transactions increase with additional warehouses. In order
more efficient. to allow orders to be entirely supplied by a customer ware-
house, each warehouse replicates theitem relation.
3.2.1 New OrderApplicationLogic
Underthislogicalpartitioningmodel,thenew ordertrans-
Thepseudocodeforthenew ordertransactionisoutlined actioncodeismostlylocaltothecustomerwarehouse(w id).
in Figure 1. The pseudocode preserves the data dependen- Inordertoexecutetheget dist info stockandupdate stock
ciesinthetransactionclass,butabstractsawayotherdetails methods,thetransactionneedstoaccessaremotepartition,
of the implementation including data types. In the trans- sincethesemethodsoperateonthesupplierwarehouse. We
action logic, the order is first sanity checked and an order needtogroupthesemethodsunderanew order update stock
id is generated. For each item in the order, the amount transactionasshowninFigure2,sothattheycanbeinvoked
for the number of ordered items is computed. The stock on theremote supplier warehouse logical partition.
of the supplier warehouse is then updated and an order In the partitioned model, each logical partition is identi-
line is inserted for each item along with the district infor- fied by auniquelogical partition identifier. Without loss of
mation of the stock. Finally, the total amount to be paid generality,thelogical partitionscan beidentifiedbyunique
by the customer is computed and returned by the transac- natural numbers. When a transaction is invoked, the logi-
tion. Itisimportanttonotethatthefieldsretrieved bythe cal partition where it would be executed must be specified,
get dist info stock method from the stock relation, which which establishes the relationship between the application
areusedtoinsertarowintheorder linerelation,arenever logic unitandalogical partition inthemodel. This isdone
updated by the update stock method. The price for each by using a mapping function that operates on the call pa-
item is present in theitem relation, which is neverupdated rameters or any available context and returns an appropri-
and nevergrows as more warehouses are added. ate partition identifier. Each transaction class declares the
TheTPC-Cbenchmarkwasdesignedtoscaleinthenum- mapping function it uses and defines it; however, all map-
berofwarehouses,i.e.,withadditionalwarehouses,thereare pingfunctionsoutputpartitionnumbersinthesamedomain
more customers who request greater number of new order of logical partitions. For our example, warehouse id can be
transactions. Warehouses form the unit of distribution and used as the logical partition identifier. The mapping func-
scale-up in the benchmark. Many HICCUP applications tion is declared and defined as shown in Figure 3. Now, a
have similar properties where a distribution unit leads to new order transaction can be invokedas:
growth in both data and client requests and importantly, EXEC new order (w id, ...) ON PARTITION (w id)
application logic is mostly affine to the distribution unit. Finally, we re-partition the original new order transac-
Existing programming models fail to take advantage of this tion asshown in Figure 4, so that it is aware of logical par-
property. Our approach, which provides a high-level, intu- titions by invokingnew order update stock on remote sup-
itive, partitioned programming model with ACID guaran- plier warehouse logical partitions. Since transactions guar-
tees, is ideally suited for this class of applications. anteefullACIDsemanticsevenacrosslogicalpartitions,the
programmersdonothavetoworryaboutconsistencyissues.
3.2.2 CodePartitioningbywarehouses
IntheTPC-Cbenchmark,awarehouseistheunitoflogi- 3.2.3 FurtherOptimizations
calpartitioning. Thisfollowsintuitivelybecausetransaction
A closer look at the partitioned new order transaction
2http://www.tpc.org/tpcc/spec/tpcc_current.pdf shown in Figure 4 exposes several performance issues ow-
4
txnnew order(w id,did, c id, order) { reason about program performance. HICCUP applications,
<wh,dist,cust>=gen order id(w id,d id, c id, order); which have an implicit distribution unit, can be intuitively
modeled in the proposed programming model as shown in
//Execute the remote transaction for all the order items
the TPC-C example. Strong global consistency semantics
//requested of the supplier warehouse in parallel
PARALLELEXECnew order update stock ensure ease of program development.
(subset(order, s id)) ONPARTITION(s id)
for s id in order.supplier w id; 4. CHALLENGES AND ROADMAP
In this section, we outline the challenges that need to
total =0;
for(orditem in order.items) { tackled during the course of the project. We also provide
//Compute the pay amount some initial thoughts on how to address them.
amount=get amount(orditem);
4.1 PhysicalImplementation andEvaluation
total +=amount;
//Get the district information of the stock using the
Logical to Physical Mapping. In order to deploy a set
//replicated stock table
of logical partitions over a set of physical partitions (phys-
stock info = get dist infostock(ord item);
add(”order line”, dist.order id, w id, did, stock info, ical machines with varying hardware characteristics), the
amount, ...); logical partitions need to be mapped to the set of physical
} partitions. Each physical partition needs to run a partition
total pay =(1+wh.tax+dist.tax)∗total∗(1−cust.discount); executorthat is responsible for handlingtransactions. This
returntotal pay;
gives rise to the following sub-challenges, which we plan to
}
investigate: (1) How to reuse a transaction processing sys-
tem which meets performance demands on modern main-
Figure 5: New order transaction with further opti-
memory multi-core machines and extend it with thelogical
mizations
programming abstraction? (2) How to map a set of logical
partitions to a set of physical partitions to ensure optimal
resource usage of thecluster?
ing to control flow dependencies upon remote transaction
In order to meet the first challenge, we plan to use a
results. There is a dependency on the results from the re-
state-of-the-art transaction processing system designed for
motenew order update stock transaction inordertoinsert
main-memory multi-core machines [24], and extend it to
order line entries. Since the item relation is already repli-
operate in a distributed setup to provide the logical par-
catedoneachlogicalpartition,theamountcanbecomputed
titioning abstraction. In order to meet the second chal-
locally. The real dependency of order line entries is to get
lenge, we plan to investigate cost models that attempt to
the district information of the stock from the remote sup-
approximatetransactionexecutioncostsinaphysicalcluster
plier warehouse. The district information fields are never
setup. This involves identifying and characterizing metrics
updated so these fields can be replicated on each logical
inthecost model. Thecost modelcanthenbeusedtopro-
partition. Then, the new order update stock transaction
duce an initial mapping, which can be changed if workload
can be modified so that it does not have to return a result
characteristics change. We also plan to investigate trans-
andthenew ordertransactioncanberewrittenasshownin
action scheduling strategies without violating performance
Figure 5. In order to invoke subtransactions in parallel, we
constraints, whichwouldhelpinbetterresourceutilization.
providePARALLELEXECconstruct. Inordertousethe
LocalConcurrencyControlandGlobalCommit. One
construct,thetransactiontobeexecutedandtheiteratorto
ofthesimplestmodelstorunmultipletransactionsistorun
beusedmustbespecified. UsingthisconstructinFigure5,
them serially, which is what H-Store advocates [14]. This
“EXEC new order update stock (order) ON PARTITION
simple model will not work for distributed transactions un-
(s id)” is invoked in parallel for all iterator elements (s id)
less a global schedule is pre-decided [23]. Runningtransac-
in order.supplier w id list.
tions serially would be a waste of compute power in multi-
Sincenew ordertransactionsconstitute∼43%ofthework-
core architectures of today. There is also a need to hide
load mix, this optimization might be worthwhile. As the
memory latencies, network latencies and commit latencies.
number of warehouses increase, however, the size of the
However,multi-threadingwithpessimisticconcurrencycon-
replicated stock relation with thedistrict information fields
trolleadstocontentionbottlenecksinthelockmanager[13]
increases as well. For a large number of warehouses, this
and multi-phase commits, which hurtperformance.
optimization option may thus not be viable. Nevertheless,
Inapartitionedmodel,distributedoptimisticconcurrency
it is important to note that the model allowed the applica-
control holds promise owing to its parallel nature, lack of
tion developer to reason about the performance of the de-
long critical sections and multi-phase commits [12]. It pro-
velopedprograms(new orderprograms) andimprovethem
videsaglobalcommitorderinthedistributedsetupusinglo-
using the mentioned optimizations. Consequently, OLTP
calcommitorderingdecisionswithoutnecessitatingamulti-
systems exposing a logical partitioning model can provide
phaseprotocolandhinderinglocaltransactions. Wewantto
performanceguaranteesovervaryingclassesofapplications.
investigate theperformance of adistributedoptimistic con-
currency control mechanism in order to support the trans-
3.2.4 Summary
actional semantics of our approach. We also want to in-
In a cluster-based setup, it is important to separate the vestigate how transaction aborts hurt performance under
concernsofwritingefficientprogramsandensuringtheiref- contention,andwhetherhybridconcurrencycontroland/or
ficient execution by mapping to physical machines. This program repartitioningwouldimprovesystemperformance.
is done by exposing a programming model based on logi- Evaluation. In order to evaluate the system implemen-
cal partitioning, which allows the application developers to tation, we plan to measure the scalability in transaction
5
throughput, latency, abort rates and cost per transaction [7] C. Curino, et al. Schism: a workload-driven approach
with growing data and number of clients. We are particu- to database replication and partitioning. PVLDB,
larlyinterestedinevaluatingthesemetricsforvariousphys- 3(1):48–57, 2010.
icalmachineconfigurationsandlogical-to-physicalpartition [8] G. DeCandia, et al. Dynamo: amazon’s highly
mappings. WeplantoevaluatethesystemusingtheTPC-C available key-valuestore. In SOSP, pages 205–220.
benchmark. Weare also interested in looking at other non- ACM, 2007.
standardapplicationbenchmarks,whichcouldprovideclues [9] R.Escriva, B. Wong, and E. G. Sirer. Warp:
onusabilityoftheproposedmodel. Weplantocomparethe Lightweight multi-keytransactions for key-value
resultswiththatofH-Store,acommercialRDBMSandSilo stores. Technical report, Cornell University,Ithaca,
tocharacterizethesystemperformanceandunderstandthe 2013.
architecture design trade-offs.
[10] A.Floratou, et al. Can theelephants handlethe nosql
onslaught? PVLDB, 5(12):1712–1723, 2012.
4.2 Cloud Integration
[11] P. Helland. Life beyond distributed transactions: an
The cloud computing infrastructure provides a flexible apostate’s opinion. In CIDR,pages 132–141, 2007.
ecosystem for deployment of the system. In order to tar- [12] M. Herlihy.Apologizing versusasking permission:
get the cloud computing infrastructure and satisfy varying Optimistic concurrency control for abstract data
HICCUP application requirements, we plan to investigate types.ACM Trans. Database Syst., 15(1):96–124,
the necessary set of self-tuning tools to make the system 1990.
cloud-ready. Deploymentadvisorsthatself-managediverse, [13] R.Johnson, et al. Shore-mt: a scalable storage
changing resources [26], adapt to the cloud infrastructure manager for themulticore era. In EDBT, volume 360,
[27], have received a lot of attention lately. We plan to de- pages 24–35. ACM, 2009.
velop a deployment advisor that generates an optimal clus-
[14] R.Kallman, et al. H-store: a high-performance,
terconfiguration inthecloudfrom anapplication’srequire-
distributed main memory transaction processing
ments(e.g.,throughput,latency,costconstraintsandgoals)
system. PVLDB, 1(2):1496–1499, 2008.
and programs by constructing and solving an optimization
[15] A.Lakshman and P. Malik. Cassandra: a
problem. Weplantoinvestigatetechniquestoallowthede-
decentralized structured storage system. Operating
ploymentadvisortoadapttothevarianceandworkloadmix
Systems Review, 44(2):35–40, 2010.
in thecloud infrastructure.
[16] I. Pandis, et al. Data-oriented transaction execution.
PVLDB, 3(1):928–939, 2010.
5. CONCLUSION [17] A.Pavlo, C. Curino, and S. B. Zdonik. Skew-aware
automatic database partitioning in shared-nothing,
Designingmain-memoryOLTPsystemsthatprovidehigh
parallel oltp systems. In SIGMOD Conference, pages
performance, scalability and ease ofdevelopment overa va-
61–72, 2012.
riety of HICCUP applications is an open challenge. In this
[18] D. Pengand F. Dabek.Large-scale incremental
paper, we identify the need to allow application developers
processing using distributed transactions and
to reason about the performance of their programs while
notifications. In OSDI,pages 251–264. USENIX
maintaining ease of use. We propose an extension to the
Association, 2010.
transactional abstraction to provide a high-level, intuitive,
distributed logical programming model with strong global [19] J. Shute,et al. F1: A distributed sqldatabase that
consistencysemanticstotheapplicationdeveloper. Wehave scales. PVLDB, 6(11):1068–1079, 2013.
also shown how the model can be intuitively used by the [20] M. Stonebraker. Newopportunities for new sql.
application developer by using TPC-C as an example. We Commun. ACM,55(11):10–11, 2012.
plan to evaluate the potential of this approach to meet the [21] M. Stonebraker, et al. The end of an architectural era
requirements of a variety of HICCUPapplications. (it’s time for a complete rewrite). In VLDB, pages
1150–1160. ACM, 2007.
[22] D. Terry.Replicated data consistency explained
6. REFERENCES
through baseball. Commun. ACM, 56(12):82–89, 2013.
[23] A.Thomson, et al. Calvin: fast distributed
[1] M. Armbrust,et al. A view of cloud computing.
transactions for partitioned database systems. In
Commun. ACM, 53(4):50–58, 2010.
SIGMOD Conference, pages 1–12, 2012.
[2] J. Baker, et al. Megastore: Providing scalable, highly
[24] S. Tu,et al. Speedy transactions in multicore
available storage for interactiveservices. In CIDR,
in-memory databases. In SOSP, pages 18–32. ACM,
pages 223–234, 2011.
2013.
[3] F. Chang, et al. Bigtable: A distributed storage
[25] G. Weikum and G. Vossen. Transactional Information
system for structured data. ACM Trans. Comput.
Systems: Theory, Algorithms, and the Practice of
Syst., 26(2), 2008.
Concurrency Control and Recovery. Morgan
[4] A. Cheung, et al. Automatic partitioning of database
Kaufmann, San Francisco, CA, USA,2002.
applications. PVLDB, 5(11):1471–1482, 2012.
[26] Q. Yin,et al. Rhizoma: A runtime for self-deploying,
[5] B. F. Cooper, et al. Pnuts: Yahoo!’s hosted data
self-managing overlays. In Middleware, volume 5896,
serving platform. PVLDB, 1(2):1277–1288, 2008.
pages 184–204. Springer, 2009.
[6] J. C. Corbett, et al. Spanner: Google’s
[27] T. Zou, et al. Cloudia: A deployment advisor for
globally-distributed database. In OSDI,pages
publicclouds. PVLDB, 6(2):109–120, 2012.
261–264. USENIXAssociation, 2012.
6
