ArchIS: an XML-based approach to transaction-time temporal database systems

19 Pages·2007·1.43 MB·English

TheVLDBJournal
DOI10.1007/s00778-007-0086-6
REGULAR PAPER
ArchIS: an XML-based approach to transaction-time
temporal database systems
FushengWang · CarloZaniolo · XinZhou
Received:29August2006/Revised:17September2007/Accepted:7October2007
©Springer-Verlag2007
Abstract Effective support for temporal applications by assure excellent storage efficiency by providing compres-
databasesystemsrepresentsanimportanttechnicalobjective sionasanoption.Thisapproachachievesfull-functionality
thatisdifficulttoachievesinceitrequiresanintegratedsolu- transaction-timedatabaseswithoutrequiringtemporalexten-
tionforseveralproblems,including(i)expressivetemporal sions in XML or database standards, and provides critical
representationsanddatamodels,(ii)powerfullanguagesfor supporttoemergingapplicationareassuchasRFID.
temporal queries and snapshot queries, (iii) indexing, clus-
teringandqueryoptimizationtechniquesformanagingtem- Keywords Temporal database · XML database ·
poralinformationefficiently,and(iv)architecturesthatbring Temporalgrouping·Temporalquery·XQuery
together the different pieces of enabling technology into a
robust system. In this paper, we present the ArchIS system
that achieves these objectives by supporting a temporally 1 Introduction
grouped data model on top of RDBMS. ArchIS’ architec-
ture uses (a) XML to support temporally grouped (virtual) Theinterestinanduserdemandfortemporaldatabaseshave
representationsofthedatabasehistory,(b)XQuerytoexpress only increased with time [1]. For example, the emerging
powerfultemporalqueriesonsuchviews,(c)temporalclus- RFIDapplicationsgeneratelargeamountofhistorydatafor
teringandindexingtechniquesformanagingtheactualhis- trackingandmonitoringphysicalobjects[2].Unfortunately,
torical data in a relational database, and (d) SQL/XML for DBMSvendorsandstandardgroupshavenotmovedaggres-
executingthequeriesontheXMLviewsasequivalentque- sively to extend their systems with support for transaction-
riesontherelationaldatabase.Theperformancestudiespre- timeorvalid-time.Giventhestrongapplicationdemandand
sented in the paper show that ArchIS is quite effective at thesignificantresearcheffortsspentontheseproblems[3],
storing and retrieving under complex query conditions the the lack of viable solutions suggests that (i) the technical
transaction-timehistoryofrelationaldatabases,andcanalso challenges posed by the problem are many and severe, (ii)
their severity is exacerbated by the inflexibility of the rela-
tionaldatamodelandthelackofextensibilityofSQL,and
B
F.Wang( )
(iii) the addition of temporal extensions to SQL standards
IntegratedDataSystemDepartment,SiemensCorporateResearch,
is not to be expected any time soon. In this paper, we ins-
Princeton,NJ08540,USA
e-mail:[email protected] tead introduce a novel low-cost solution, by showing how
XMLanditsquerylanguagescanbeusedtoovercomemost
C.Zaniolo
ofthesedifficulties,andproposetransaction-timeextensions
ComputerScienceDepartment,UniversityofCalifornia,
fordatabasesystemsthatrequirenomodificationofexisting
LosAngeles,LosAngeles,CA90095,USA
e-mail:[email protected] standards. Indeed, unlike the relational model, XML pro-
videsexcellentsupportfortemporallygroupeddatamodels,
X.Zhou
which have long been advocated as the most natural and
TeradataDivision,NCRCorporation,
effectiverepresentationsoftemporalinformation[4].Moreo-
LosAngeles,CA90245,USA
e-mail:[email protected] ver,unlikeSQL,XQuery[5]isTuring-completeandnatively
123
F.Wangetal.
extensible [6]. Thus many additional constructs needed for Table1 Thesnapshothistoryofemployees
temporal queries can be defined in XQuery itself, without
Id Name Salary Title Deptno Start End
having to depend on difficult-to-obtain extensions by stan-
dardcommittees.However,inthispaperweshowthatXML/ 1001 Bob 60000 Engineer d01 1995-01-01 1995-05-31
XQuery can play a very useful role in bringing temporal 1001 Bob 70000 Engineer d01 1995-06-01 1995-09-30
support to database systems while retaining current stan- 1001 Bob 70000 SrEngineer d02 1995-10-01 1996-01-31
dards, and such extensions are needed in emerging appli- 1001 Bob 70000 TechLeader d02 1996-02-01 1996-12-31
cation areas, such as the management of RFID data [2].
Becauseofthesenewapplications,andsincedatabaseven-
dors who were torpid on temporal extensions for RDBMS
are now moving feverishly to add support for XML and
XQuerytotheirsystems,weseeawindowofopportunityfor
the approach proposed in this paper. In fact, most database
systems now support the publishing and viewing of data-
basetablesthroughXMLviews,whichcanbequeriedusing
XQueryandotherlanguages.Thesequeriesarethensuppor-
ted by mapping them into equivalent queries on the under-
lyingdatabase[7,8].Databasevendorsandstandardgroups
areaddingthesecapabilitiestoSQLthroughtheSQL/XML
initiative[9].
Inthispaper,weproposeaveryusefulgeneralizationof Fig. 1 Temporallygroupedhistoryofemployees
thisidea,byshowingthattheevolutionhistoryofarelational
databasecanalsobeviewednaturallyusingXMLandque-
ofthetable and remains invariant inthehistory.1 Withthis
riedeffectivelyusingXQuery.Moreover,theArchISsystem
approach,anychangeofanattributevaluewillleadtoanew
discussed inthispaper demonstrates thatthetemporal data
history tuple, thus causes redundancy information between
and temporal queries can be supported efficiently via the
tuples, e.g., the name of Bob appeared the same but was
data-compression, clustering, indexing and query-mapping
storedinallthetuples.Moreover,temporalqueriesneedto
techniquesdiscussedinthepaper.
frequentlycoalescetuples.Temporalcoalescingisasourceof
Thepaperisorganizedasfollows:inSect.2weshowthat
complicationsintemporaldatabases,whichiscomplexand
XMLprovidesanaturalvehicleforimplementingatempo-
hardtoscaleinRDBMS.Forinstance,atemporalcoalescing
rallygroupeddatamodelforrepresentingtheevolutionhis-
querycantakemorethan20linesofSQLwithSQL92,and
toryofarelationaldatabase.Infact,inSect.3weshowthat
thebestperformanceofcoalescingonRDBMSisquadratic
complextemporalandsnapshotqueriescanbeexpressedon
[11].
suchviewsusingXQuery.Next,inSect.3.4,weshowthatthe
These problems can be overcome using a representation
approachcanbeusedquitenaturallytosupportRFIDappli-
where the timestamped history of each attribute is grou-
cations. In Sects. 4 and 5, we focus on the efficient imple-
ped under the attribute (Fig. 1)[4], i.e., with the values of
mentationofsuchqueriesonRDBMS,wherequeriesonthe
eachattribute,weassociateasetoftimeintervalsdenoting
XMLviewsaretranslatedintoSQL/XMLqueriesontherela-
theirvalidity—intervalsthatareadjacentoroverlapforthe
tionaltables,andvariousindexing/clusteringtechniquesare
sameattributevaluearecoalesced.Whilethisnestedrepre-
usedtomaketheexecutionofthesequeriesefficient.Query
sentation is hard to be represented in a flat table, it can be
performance study in Section 6 shows that ArchIS is quite
naturally represented by an XML-based hierarchical view
effective, and in Sect. 7, we propose database compression
showninFig.2.WecalltheseH-documentsorH-viewswhen
asanoption,andpresentasimplebuteffectivetechniquefor
thesearevirtualrepresentations.Hereforsimplicity,weonly
compressingarchiveddatabases.Relatedworkisdiscussed
show examples for employee table, other tables like dept
inSect.8,followedbyconclusionsandfuturework.
table can be modeled in similar way. The root element in
anH-documentrepresentsthecorrespondingtable’shistory
(thecreationanddeletionofatable),anditschildelements
2 ViewingrelationhistoryinXML
1 Intheremainderofthispaper,ourgranularityfortimeisaday;how-
ever,allthetechniqueswepresentareequallyvalidforanygranularity
Table 1 describes the history of employees as they would
usedbytheapplication.Forfinergranularity,techniquesin[10]canbe
beviewedintraditionaltransaction-timedatabases[3]using
used.Furthermore,throughoutthispaper,weassumethatrelationkeys
atemporallyungroupedrepresentation,whereidisthekey remaininvariant.
123
ArchIS
representations and support queries on the whole view and
itsprojectionswithoutrequiringneitherjoinsnorcoalescing
operations.Ontheotherhand,thesebenefitsarerestrictedto
non-keyattributes:coalescingisstillrequiredwhenwepro-
jectoutattributesfromtheprimarykey.Forinstance,aquery
such as “For how long has department d01 been without a
Sr. Engineer?”, would require projecting out name, salary,
titleandid—andacoalescingoperationwillthenbeneeded
becauseoftheeliminationofthekeyattributeid.
However, even for such cases, other XML’s advantages
remain,particularlysincewecannowuseXQuery—aquery
Fig. 2 The history of the employee table is viewed as languagethatissignificantlymorepowerfulthanSQL[6]—
employees.xml toexpresscoalescingandothertemporalqueries.Infact,as
discussednext,complexhistoricalqueriescanbeexpressed
in standard XQuery, thus avoiding the need for a special-
represent the grouped history of attribute values. Each ele- purposetemporalquerylanguage.
mentinanH-documentisassignedtwoattributeststart
andtend,torepresenttheinclusivetime-intervaloftheele-
ment. The value of tend can be set to now, to denote the 3 TemporalqueriesusingXQuery
ever-increasingcurrenttime.(Thiswillbefurtherdiscussed
inSect.3.3.)Notethatthereisatemporalcoveringconstraint Thekeyadvantageofourapproachisthatpowerfultempo-
thattheintervalofaparentnode(tablehistory)alwayscovers ral queries can be expressed in XQuery without requiring
thatofitschildnodes(attributehistories). theintroductionofnewconstructsinthelanguage.Wenext
In general, consider a relation R and let K denotes its show how to express the main classes of temporal queries
primarykey. Then thecorresponding H-document contains asdiscussedin[14,15]:temporalprojection,temporalsnap-
an entity E for each maximal period of validity of a value shot, temporal slicing, temporal join, temporal aggregate,
of K. Thus in Fig. 2, id is our key and the period of vali- andrestructuring,onemployees.xmldocument(Fig.2)
dity of id = 1001 is T = (1995-01-01–1996-12-31]. Thus, anddepts.xmldocument(whichistheH-documentofthe
our H-document contains an entity timestamped with this departmenttable,includingthehistoriesofattributesdeptno,
validityperiod.Thenthesuccessivevaluesoftheattributes deptname,andmgrno).
for tuples with id = 1001 are represented as subelements
QUERY1:TemporalProjection.Retrievethetitlehistoryof
of E timestampedwiththeirperiodsofmaximalvalidityfor
employee“Bob”:
the values of the corresponding attributes. The attribute in
element title_history{
K always has one period of validity, while the others have
for $t in doc("employees.xml")/employees/
oneormore;inourexample,Bobhasonlyoneperiod,salary
employee[name="Bob"]/title
anddeptnohavetwoandtitlehasthree.Thus,H-documents return $t}
haveasimpleandwell-definedschemawhichcanbeeasily
This query returns all title elements of employee
generatedfromthatoftherelationwhosehistorytheywant
“Bob”, using a single XPath /employees/employee/
topreserve.
title,withapredicateonthenameelement.Thisquery
Our H-documents use a temporally grouped data model
illustrates the benefits of avoiding the need for coalescing
[4]. Clifford et al. [4,12,13] show that temporally grouped
the query results. Since the history of titles is grouped, the
models are more natural and powerful than temporally
projected result is already coalesced—whereas coalescing
ungrouped ones, and can significantly reduce the need for
on the results would have to be performed in a temporally
coalescing,sinceanattributehistoryisalreadygrouped.For
ungroupedrepresentation.
instance,inFig.2wecanprojectoutthesalaryentrieswithout
QUERY 2: Temporal Snapshot. Retrieve the managers on
having to change the title ones, and vice versa. If we ins-
1994-05-06:
teadusearelationalrepresentation,suchasthatofTable1,
we will have to perform a coalescing operation after these for $m in doc("depts.xml")/depts/dept/mgrno
projections.Usingtherelationalmodel,analternativesolu- [tstart(.)<=xs:date("1994-05-06") and
tend(.) >= xs:date("1994-05-06")]
tionconsistsindecomposingTable1intofourtablessharing
return $m
theidcolumn.Thissolutionwillavoidcoalescing,inasmuch
as H-documents do, but requires frequent joins on such Thisqueryreturnsallmanagersinthehistorywhoseinter-
tables. Therefore, H-views combine the benefits of both val overlap with the timestamp 1994-05-06. Note that
123
F.Wangetal.
xs is the namespace of XML Schema (the declaration of OthertemporalaggregatessuchasTSQL2rising[15]and
namespacesisignoredhere).tstart($e)andtend($e) moving window aggregates, can also be supported through
areuser-definedfunctionsthatrespectivelyextractthestar- user-definedfunctions.
ting date and ending date of an element. By using such QUERY6:Restructuring.Findthemaximumlengthoftime
functions, the implementation details of our timestamping during which Bob worked continuously without changing
schemescanbemadetransparenttousers.Thiswillbefur- titleordepartment:
therdiscussedinSect.3.2.
for $e in doc("emp.xml")/employees/
QUERY 3: Temporal Slicing. Find employees who worked
employee[name="Bob"]
atanytimebetween1994-05-06and1995-05-06: let $d := $e/dept
let $t := $e/title
for $e in doc("employees.xml")/employees let $overlaps := restructure($d, $t)
/employee[ toverlaps(., return max($overlaps)
telement( xs:date("1994-05-06"),
xs:date("1995-05-06")))] Theuser-definedfunctionrestructuretakestwolists
return $e/name
andreturnsalltheiroverlappingintervals.
Thisquerychecksiftheintervalofanemployeeelement
3.1 Morecomplexqueries
overlapped with a given interval. Here, toverlaps
($a, $b) is a function that returns true if the temporal
Here, we discuss more advanced temporal queries, such as
intervals of elements a and b overlap, and false otherwise;
until,since,andcontain,whichareoftenusedasatestforthe
telement($x, $y)constructsanelementwithxandy
expressivepoweroftemporallanguages[16].Forinstance,
asitsattributes.
thefollowingisasincequery:
QUERY4:TemporalJoin.Findthehistoryofemployeesthat
QUERY 7: A Since B. Find the employee who has been a
eachmanagermanages:
SeniorEngineerindept“d001”sincehe/shejoinedthedept:
element manages{
for $e in doc("employees.xml")/employees/employee
for $d in doc("depts.xml")/depts/dept
let $m:= $e/title[.="Sr Engineer" and
for $m in $d/mgrno
tend(.)=current-date()]
return
let $d:=$e/deptno[.="d001" and
element manage {$d/deptno, $m,
tend(.)=current-date() and
element employees {
tcontains($m, .)]
for $e in doc("employees.xml")/employees
where not empty($d) and not empty($m)
/employee/deptno
return <employee>
where $e = $d/deptno and not(
{$e/id, $e/name}</employee>
empty(overlapinterval($e, $m)))
return ($e/../name, overlapinterval ($e, $m))
}}} Heretcontains($p,$c)isauser-definedfunctionto
checkifoneintervalcontainsanother.Thefirstletclause
Thisqueryjoinsthedepts.xmlandemployees.xml returns current “Sr Engineers”, and the second let clause
documents and generates a hierarchical XML document returnsthosecurrentlyindept“d001”,withtheperiodcontai-
grouped by dept and manager: for each mgrno in ningthetitleperiod.
depts.xml,allemployeesinemployees.xmlareretur- QUERY 8: Period Containment. Find employees with the
nediftheywereinthesamedepartmentasandduringperiods sameemploymenthistoryasemployee“Bob”,i.e.,theywor-
thatoverlappedwiththoseofthemanager.Here,overlap- ked in the same department(s) as employee “Bob” and
interval($a, $b) is a user-defined function that forexactlythesameperiods:
returns an interval element describing the overlapping
period.Ifthisperiodisempty,thentheXQuerybuilt-infunc- for $e1 in doc("employees.xml")/employees
/employee[name = "Bob"]
tionempty($e)issatisfiedandnothingisreturned.
for $e2 in doc("employees.xml")/employees
QUERY 5: Temporal Aggregate. Retrieve the history of the /employee[name != "Bob"]
averagesalary: where every $d1 in $e1/deptno satisfies
some $d2 in $e2/deptno satisfies
(string($d1)=string($d2)
let $s := document("emp.xml")/employees/
and tequals($d2,$d1))
employee/salary
and every $d2 in $e2/deptno satisfies
return tavg($s)
some $d1 in $e1/deptno satisfies
(string($d2)=string( $d1)
Here tavg($s) is a temporal aggregate function that and tequals($d1,$d2))
canbeefficientlycomputedinasinglescanofthedatabase. return <employee>{$e2/name}</employee>
123
ArchIS
Here tequals($d1,$d2) is a user-defined function 3.3 Supportfor“now”
that checks whether two nodes have the same time periods
(i.e.,sametstartandtend).Forthisquery,wecheckthat An important issue in temporal databases is how to handle
for each department Bob worked, if there is a person who now or UC (until changed) [17,18]. In a transaction-time
onceworkedinthatdepartmentaswell,andwithexactlythe database,nowmeansthatthevaluesinthetuplearestillcur-
sameperiod,andviceversa. rentatthetimethequeryisasked.Inourstrategy,wereplace
the symbol “now” with the value current_timestamp
(or current_date, depending on the used time granu-
3.2 Temporalfunctions
larity). Such instantiation is performed conservatively only
whenneeded.
The use of functions such as tequals($d1,$d2),
Internally, we use the “end-of-time” value (e.g., “9999-
tstart($e), and tend($e) offers the advantage of
12-31”fordate)todenotethe“now”symbol;thisrepresen-
divorcingtemporalqueriesfromthelow-leveldetailsusedin
tation can assure that the current search techniques based
representingtime,e.g.,iftheintervalisclosedattheend,or
on indexes and temporal ordering can be used without any
hownowisrepresented.Otherusefulfunctionspredefinedin
change.Theuserdoesnotaccessthisvaluedirectly,he/she
oursystemincludeintervalfunctionsasdefinedAllen’stem-
willaccessitthroughbuilt-infunctionststart($e)and
poraloperators;during/date/timefunctionsthatwillextract
tend($e). While the function tstart($e) returns the
timefromthedatabasebyhidingtheimplementationdetails;
startoftheinterval,thetend($e)functionreturnsitsend,
functionstohandlenow,andrestructuringfunctionforhand-
ifthisisdifferentfrom“9999-12-31”andcurrent_date
ling coalescing, which is very common for temporal data-
otherwise.Also,manysituationscallforadifferentsolution.
bases.
Forinstance,theXMLnodesreturnedbyQUERY1,usethe
Restructuring functions: coalesce($l) will coalesce a
“9999-12-31”internalrepresentationfornow,whichmakes
listofnodes,andrestructure($a,$b)willreturnall
iteasiertousethisasinputinothertemporalqueries.
theoverlappedintervalsontwosetofnodes.
However,fordatareturnedtotheend-user,twodifferent
Intervalfunctions:tequals($a,$b),tmeets($a,$b), representations are preferable. One is to return the
toverlaps($a,$b), tprecedes($a,$b) and current_date by applying function rtend($e) that,
tcontains($a,$b)returntrueorfalseaccordingtothe recursively,replacesalltheoccurrenceof“9999-12-31”with
positions of the two intervals. The overlapinterval thevalueofcurrent_date.Theotheristoreturnaspe-
($a,$b) returns the overlapping portion of the intervals, cialstring,suchas“now”or“until-changed”tobedisplayed
ifthisisnotempty,withthefollowingform:<interval on the end-user screen. As discussed in [17], this is often
tstart= "d1" tend="d2"/>. moreintuitiveandappealing forusers,andissupportedby
Durationanddate/timefunctions: ourbuilt-infunctionexternalnow($e)thatdoesthatfor
timespan($e)returnsthescalartimespanofanode; thenodeeanditssub-nodes.
tstart($e)returnsthestarttimeofanode;
tend($e) returnstheendtimeofanode; 3.4 RFIDapplications
tinterval($e) returnstheintervalofanode;
telement($Ts, $Te) constructs an empty element The need for supporting temporal queries is pervasive
telementwithattributeststartandtend; in many new and important application areas, and in the
rtend($e) recursively replaces all the occurrence of previous section, we have shown how the classical queries
“9999-12-31”withthevalueofcurrent_date; from the temporal database literature can be effectively
externalnow($e)recursivelyreplacesalltheoccurrence expressed in ArchIS. In this section, we will focus on the
of“9999-12-31”withthestring“now”. emergingapplicationareaofRFIDdatamanagement,since
Allthesetemporalfunctionsmustalsosupportthespecial temporalqueriesplayakeyroleinthisimportantapplication
‘now’timestamp,whichisdiscussedinSect.3.3,below. domain[2].
Inparticular,becauseoftheinternalrepresentationweuse RFID(radiofrequencyidentification)isakeytechnology
for ‘now’, the function tcontains($a,$b) can simply forautomaticidentificationanddatacapturethatusesradio
bedefinedasfollows: frequencywavestotransferdatabetweenreadersandEPC.2
UsingRFID,objectscanbeautomaticallyidentified,catego-
define function tcontains($a, $b){ rized,andtracked,withoutrequiringalineofsightorcontact
if($a/@tstart<= $b/@tstart and
$a/@tend >= $b/@tend)
then true() 2 (EPC—ElectronicProductCode—isanidentificationschemeforuni-
else false() versallyidentifyingphysicalobjects,definedbystandardcommittees
} [19].)
123
F.Wangetal.
(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1) 11 (cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1) 22 tThhreues,dthimee<nlsioocnsatofiolonc>atihoins,tocroynotafionumreonbt,jeacntdidoewntnifieresdhibpy.
(cid:1)(cid:1)(cid:1)(cid:1) (cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1) theEPCtagA1B2C3isgeneratedbythesteps1–4ofFig.3;
S12u::(cid:1)(cid:1) pCPpaalilseleert s Wl op(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)aa(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)a(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)dr(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)c(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)eekhdeo dou nsoetnot ot rpuaclklet 33 (cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1) (cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1) (cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1) 44 (cid:1)(cid:1)(cid:1)(cid:1) fitihtrses<tsitcneotponsatschaaoiswennaenirndptFhcieg>n.h3ini.stotoarypainllsettetahdarteaclosordosctchuerpreladcdeumrienngt
43.: CPaaslleest ucnhleocakdeedd o tuot aret traeigl issttoerre (cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1)(cid:1) Next, we show that powerful RFID queries for tracking
Retail Store andmonitoringcanbesupportedinArchISwithXQuery.
Fig. 3 LocationdrivenchangesinRFID-enabledapplications
3.4.2 TemporalRFIDquerieswithArchIS
QUERY R1: Track the location history of object with EPC
betweenreadersandtaggedobjects.Becauseofitssignificant
“A1B2C3”:
advantages,RFIDtechnologyisbeinggraduallyadoptedand
deployedinawideareaofapplications. element locationhistory{
ThereareseveralimportantentitiesinRFIDapplications, for $l in doc("rfid.xml")/objects/
object[epc="A1B2C3"]/location
includingEPC-taggedobjects,RFIDreaders,andlocations.
return $l }
Figure3demonstratesanexampleofanRFIDenabledsup-
plychain,wherecasesarepackedontopalletsandshipped QUERYR2:Thiscase(withEPC“A1B2C3”)ofproductis
throughtruckstoretailstorestobesoldtocustomers.During damaged.Whatothercaseshaveeverbeenputinthesame
the process, these RFID-tagged entities interact with each palletwiththiscaseatthesametime?
otherandexperiencevariousevents,suchaschangeofloca- for $s in doc("rfid.xml")/objects/object
tions, and change of containers. There are also other types [epc="A1B2C3"]
of events, such as business operations, and change of legal for $c in $s/containerepc
for $o in doc("rfid.xml")/objects/object
owners. Thus RFID data are temporally oriented [2]. One
[epc != "A1B2C3" and containerepc=$c]
majorgoalofRFIDapplicationsistotrackandmonitorphy- where toverlaps($c, overlapinterval($s, $o))
sicalobjects,thusatemporaldatabaseisessentialtosupport return $o
suchapplications.Temporalsupport,however,ismissingin
This query checks if there are other objects in the same
pastRFIDmiddlewaresystems[20,21].Next,weshowthat
containerofobject“A1B2C3”andwithoverlappingperiods.
ourapproachisaperfectfitfortrackingandmonitoringRFID
QUERY R3: Find how many items were returned by custo-
objects.
mers(withlocation“sold”)tothestore(withlocation“retai-
lerW”)on2005-11-11:
3.4.1 RepresentingRFIDdatainXML
let $o := doc("rfid.xml")/objects/object
[location ="retailerW" and tstart(location)
ThehistoryofRFIDobjectscanbedirectlyrepresentedinan =xs:date("2005-11-11")]
H-view,asshowninanexampleinFig.4.Inthisexample, where every $i in $o satisfies
fn:exists($o[location = "sold" and
the H-view represents the history of an object’s along the
tend(location)=xs:date("2005-11-11")])
return count($o)
Since items sold to customers would begin a new loca-
<objectststart="2005-08-01"tend="now">
<object tstart="2005-08-01"tend="now"> tion,thequerychecksallobjectswhoselocationsstartedat
<epc tstart="2005-08-01"tend="now">A1B2C3</epc>
location“retailerW”on2005-11-11.
<locationtstart="2005-08-01"tend="2005-08-05">warehouse1</location>
<locationtstart="2005-08-06"tend="2005-08-09">routeA</location> QUERY R4: find missing objects at current location “L”,
<locationtstart="2005-08-10"tend="2005-08-15">retailerW</location>
whichwereavailableatlocation“S”attime“T”.
<locationtstart="2005-08-16"tend="2005-08-17">sold</location>
<locationtstart="2005-08-18"tend="2005-08-19">retailerW</location>
for $o in doc("rfid.xml")/objects/object
<locationtstart="2005-08-20"tend="UC">sold</location>
<containerepctstart="2005-08-01"tend="2005-08-05">A1EFGD</containerepc> [location="S" and tcontains(location,
<containerepctstart="2005-08-06"tend="2005-08-09">A1C3DF</containerepc> telement(xs:date("T"), xs:date("T")) )]
<owner tstart="2005-08-01"tend="2005-08-04">SupplyACo.</owner> where empty($o/location[. ="L" and
<owner tstart="2005-08-05"tend="2005-08-15">RetailWCo.</owner>
tend($o/location) = "UC"]
<ownertstart="2005-08-16"tend="2005-08-17">Customer</owner>
<ownertstart="2005-08-18"tend="2005-08-19">RetailWCo.</owner> return $o
<ownertstart="2005-08-20"tend="UC">Customer</owner>
</object> Thisquerywillsearchobjectswhichpassedthroughloca-
<!--more...--> tion“S”attime“T”,butarenotavailableon“L”atpresent.
</objects>
The XML-based representation and query support also
Fig. 4 RFIDobjecthistoryinXML(rfid.xml) facilitate the exchange of RFID data in distributed RFID
123
ArchIS
applications,suchaslargedistributedsupplychainsystems. attribute history table is built for each attribute to store the
RFIDobservationsarefrequentlyexchangedamongmultiple history of such attribute. A key table is built for the key.
siteswithPML(PhysicalMarkupLanguage)[22];therefore, Eachtablewillincludetwoattributeststartandtendto
theRFIDdatahistorycanbeeasilyqueriedandexchanged represent the valid interval of that tuple. Besides, a global
inourH-documentformat. relationtableisusedtorecordthehistoryofrelations.
Forexample,wehavethefollowingrelationinthecurrent
database:
4 TheArchISsystem employee(id, name, salary, title, deptno)
whereidisthekey.Thehistoryofthetableisviewedasan
Two approaches are possible for storing and querying
H-document,whichisthenstoredas(i)akeytable,(ii)the
H-documents: one is to use a native XML DBMS such as
attributehistorytables,andaglobalrelationtable:
TaminoXMLServer[23];theotheristouseRDBMSsand
Thekeytable:
providemappingsofqueriesandqueryresultsbetweenthe
employee_id(id, tstart, tend)
XMLviewsandtheunderlyingdatabasesystems.Thequery
performanceandthestorageefficiencyofthetwoapproaches Since id will not change along the history, the interval
arecomparedinSect.6. (tstart, tend)inthekeytablealsorepresentsthevalid
Themaindesignissuesthatmustbeaddressedforaneffi- intervaloftheemployee.
cientrealizationofthesecondapproachincludei)howtomap For composite keys, for example, (supplierno,
(shred) the XML views representing the H-documents into itemno), we build a key table as lineitem_id(id,
tables(whichwecallH-tables),ii)howtotranslatequeries supplierno, itemno, tstart, tend),whereid
ontheXMLviewstotheH-tables,andiii)whichindexing, isauniquevaluegeneratedfrom(supplierno,itemno).
clusteringandquerymappingtechniquesshouldbeusedfor Theuseofkeysisforeasyjoiningofallattributehistoriesof
highperformance. anobjectsuchasanemployee.
Wewillnextdiscussthesolutionsoftheseproblemsused Attributehistorytables:
inourArchivalInformationSystem—ArchIS,whichusesthe employee_name(id,name,tstart,tend)
employee_deptno(id, deptno, tstart,tend)
RDBMS-basedapproach(ArchIS-DB2onDB2andArchIS–
employee_salary(id, salary, tstart,tend)
ATLaS on ATLaS [24]—a compact implementation of an
employee_title(id, title, tstart,tend)
RDBMSatUCLA.ThearchitectureofArchISisshownin
Thevaluesofidsintheabovetablesarethecorresponding
Fig. 5. In our implementation, the “current database” and
keyvalues,thusindexesonsuchidscanefficientlyjointhese
H-tablesareimplementedastablesinasamedatabase,but
relations.
theresultsareeasilygeneralizedtothesituationswherethese
Asamplecontentoftheemployee_salarytableis
twoareseparate,oreventhecasewherethecurrentdatabase
isaviewcontainingthenowsnapshotoftheH-tables. ID SALARY TSTART TEND
===================================
100022 40000 02/20/1988 02/19/1989
4.1 H-tables 100022 42010 02/20/1989 02/04/1990
100022 42525 02/20/1990 02/04/1991
100022 42327 02/20/1991 02/19/1992
InArchIS,eachH-documentisstoredinthedatabaseasinter-
...
nalH-tables.Foreachtableinthecurrentrelationaldatabase 100023 43162 07/13/1988 07/13/1989
westoreakeytableandseveralattributehistorytables.An ...
Whenanewtupleisinserted,theTSTARTforthenewtupleis
settothecurrenttimestamp,andTENDissettonow.When
there is a delete on a current tuple, we simply change the
TEND value in that tuple as current timestamp. An update
canbeviewedasadeletefollowedbyaninsert.
Globalrelationtable:
relations(relationname, tstart, tend)
willrecordalltherelationshistoryinthedatabaseschema,
i.e.,thetimespanscoveredbythevarioustablesinthedata-
base.ThiscorrespondstotherootelementsofH-documents.
Ourdesignbuildsontheassumptionthatkeys(e.g.,id)
remain invariant in the history. Otherwise, a system-
Fig. 5 ArchIS:ArchivalInformationSystem generatedsurrogatekeycanbeused.
123
F.Wangetal.
Algorithm1XQuerytoSQL/XMLtranslationalgorithm the employees hired after 02/04/2003, we can map the
1: foreachvariable$vi inforandletclauseofXQuerydo XQuerytothefollowingSQL/XMLquery:
2: FindtableTi andColumnAi accordingtoschemamapping
3: endfor select XMLElement (Name "new_employees",
4: foranyvariablevj whichisdefinedbyarelativeXPathfromvi, XMLAttributes ("02/04/2003" as "start"),
suchasvj :=vi/salarydo XMLAgg(XMLElement(Name "employee", e.name))
5: GeneratejoinconditionCij−id :Vi.id=Vj.id from employee_name as e
6: endfor where e.tstart >= "02/04/2003"
7: foreveryconditionVi opVj inwhereclauseofXQuerydo
8: GenerateconditionCij−where:Ti.Ai opTj.Aj Assuming that only Bob and Jack were hired after
9: endfor
10: foreveryfunctioninXQuery fn($vi,$vj)do 02/04/2003,thepreviousqueryreturnsthefollowingoutput:
11: Generate function fnij(Ti.Ai, Ti.tstart, Ti.tend, Tj.Aj, <new_employees start="02/04/2003">
Tj.tstart,Tj.tend) <employee>Bob</employee>
12: endfor
13: forevery$vi inreturnclausedo <employee>Jack</employee>
14: GenerateEi:XMLElement(Namevi’selementname,XMLAt- </new_employees>
tributes(Ti.tstartas“tstart”,Ti.tendas“tend”),Ti.Ai)
15: endfor TheseSQL/XMLconstructssimplifythetranslationfrom
16: foreveryparent-childrelationshipvi(vj)inreturnclausedo queries expressed on H-views to equivalent queries on
17: trGibeunteersa(teTiE.tisjt:aXrtMasL“Etslteamrte”n,tT(iN.teanmdeavsi’“steenledm”)e,nEtnj,aTmie.,AXi)MLAt- H-tables.Forinstance,theSQL/XMLtranslationofQUERY
18: endfor 1inSect.3isshownbelow:
19: GenerateoutputSQL
select XMLElement(Name "title_history",
XMLAgg(XMLElement(Name "title",
XMLAttributes(T.tstart as "tstart",
T.tend as "tend"), T.title)))
4.2 Updatingtablehistories
from employee_title as T,employee_name as N
where N.id = T.id and N.name = "Bob"
Changes in the current database can be tracked with either group by N.id
updatelogsortriggers.ForourtestingonArchIS-DB2,we
NoticethattheN.id = T.idconditioninthewhere
build triggers that successfully track changes in the cur-
clauseisgeneratedduetothe[name="bob"]predicatein
rent database and archive them into H-tables. For ArchIS–
the XPath expression. A group by clause is also added
ATLaS,forbetterperformance,weuseupdatelogstotrack
to group all titles of an id into an element through the
andarchivechanges.
XMLAgg()function.
Asanotherexample,QUERY3willbetranslatedto:
4.3 Querymapping select XMLElement (Name "emp",
XMLElement (Name "id", XMLAttributes (
e.tstart as "tstart",e.tend as "tend"),e.id),
VerygeneraltranslationmechanismsfromXMLdocuments
XMLElement (Name "name", XMLAttributes(
toRDBMShavebeenstudiedin[25]andmiddlewaresuch n.tstart as "tstart",n.tend as "tend"),n.name))
as XTABLES [7] can be used to publish our H-tables into from employee_id e, employee_name n
XMLandsupportqueriesonsuchtables.Afterinvestigating where e.id = n.id and toverlaps( e.tstart,
e.tend, "1994-05-06", "1995-05-06")
these general-purpose translation techniques and software,
for ArchIS we developed specialized techniques and soft- Hereajoinconditionisneededtojoine.idwithn.id,
ware,thusobtainingsignificantperformanceimprovements which is implied in the XPath expression $e/name. The
(seeSect.6.2). translation of UDF (user-defined function) toverlaps
Therefore, ArchIS implements XQuery on H-views, by takes in the tstart and tend values, and returns true or
translating them into equivalent SQL/XML expressions on false. More on built-in function translation is discussed in
H-tables. SQL/XML [9] is now a standard efficiently sup- Sect.4.4.
portedincommercialDBMS.Wealsoimplementeditinour The mapping of queries on H-views to H-tables can be
experimentalsystemATLaSusingthetechniquesdescribed summarizedasfivemainsteps:
in[26].
TheexpressionsonH-tablesusetheSQL/XMLconstructs – Identificationofvariablerange:Foreachvariabledefined
XMLElement,XMLAttributes,andXMLAgg,whichare byafororletexpressionintheoriginalquery,weiden-
discussednext.TheXMLElementandXMLAttributes tifywhether,intheunderlyingH-tables,thiscorresponds
constructs are used to return elements and their attributes. to(i)atuplevariablerangingoverakeyrelation,or(ii)
XMLAggisanaggregatefunction,whichconstructsanXML atuplevariablerangingoveranattributetable,or(iii)an
valuefromacollectionofXMLvalueexpressions.Forins- attributevariablesuchasT.AwhereTisatuplevariable
tance,toreturnanew_employeeselementcontainingall over a key table or an attribute table, and A denotes an
123
ArchIS
attributeinsuchtables.Foreachdistincttuplevariablein bemodifiedaccordingtothedifferentsituationsofnodetypes
the original query, a distinct tuple variable is created in inH-views,discussednext.
thefromclauseoftheSQL/XMLquery. Leafnodes:Forleafnodessuchasidorsalary,wecan
Forinstance,QUERY1identifiestwoattributevariables, identifyatuplevariableTovereitherakeytableoranattri-
from tables employee_title and employee_ butehistorytable.Asaresult,wecantaketheT.tstart
name.Therefore,thefromclauseoftheSQL/XMLsta- and T.tend attribute variables as input to the UDF, and
tement contains such two tuple variables with aliases T implementthefunctionalityoftheUDF.
andN. Parent nodes of leaf nodes: For example, the employee
– Generation of join conditions: There is a join condition nodeinourH-documentisaparentnodeofleafnodes.Since
T.idandN.idforanypairofdistincttuplevariables. thetstartandtendattributevaluesarethesameasthose
– Generationofthewhereconditions:thesearethecondi- oftheiridchildnodes,wecanidentifythetuplevariableT
tions that are contained in the where clause of the over their key table, and pass T.tstart and T.tend as
XQueryorspecifiedinthepathexpression(e.g., theinputoftheUDF.
[name="Bob"]inQUERY1). Extending ArchIS via a library of user-defined function
– Translation of built-in functions: Temporal functions imported into the DSMS is similar to the approach widely
(suchastoverlaps($a,$b))aresimplymappedinto usedincommercialDBMS—underababelofnamessuchas
the corresponding built-ins we have implemented for DBextenders,snapins,cartridge,etc.Thepopularityofsuch
ArchIS.Wewillhavemorediscussionforfunctionmap- libraries is due to the fact that they enable software houses
pinginthenextsection. anddomainspecialiststoextendtheDBMSwithoutrequiring
– Outputgeneration:Thisisachievedthroughtheuseofthe changestothestandards.ArchISusesasimilarapproachto
XMLElement and the XMLAgg constructs previously bringabouttransaction-timedatabases.3
described. Through expression of these constructs the
ArchIS compiler supports simple expressions, such as
return $t of QUERY 1, and more complex expres-
5 Temporalclusteringandindexing
sionssuchasQUERY4.Meanwhile,usershavetheopi-
tontospecifyatableconstructinthereturnclauseto
In our current RDBMS-based archiving scheme, tuples are
bypasstheSQL/XMLtransformation,sotheresultscan
storedinatemporallygroupedorder(i.e.,thesalaryhistoryof
bereturnedastables.
anemployeebeforethatofthenextemployee).Performance
onsnapshotqueriescanbeimprovedwithamoreeffective
Algorithm 4.3 summarizes the overall translation algo- temporalclusteringscheme.Thus,weuseasegment-based
rithm,whichisimplementedonGalax[27],anopensource archivingschemewhichhasbettertemporalclustering,and
parser of XQuery. The SQL generation at step 19 of willboosttheperformanceofmosttemporalqueries,andis
Algorithm4.3producesastatementwhichlistsinthefrom alsoamendabletocompressiontechniques.
clausealltheT tables,andlistsintheselectclauseallthe
i
entities E and subentities E generated in the algorithms,
i ij
with the where clauses including the concatenation of all 5.1 Usefulness-basedclustering
conditionsfrombothCij−id andCij−where conditions.
The translated SQL/XML queries on the H-tables often Assumethatanattributehistoryisstoredinasegment.For
contain many natural joins such as N.id = T.id. These each segment, we define its usefulness as U = Nlive/Nall,
joins execute very fast (in linear time) since every table is where Nlive isthecountoflive(orcurrent)tuplesand Nall
alreadysortedonitsidattribute. isthecountofalltuples.U beginswith100%anddecreases
withupdates.Wealsodefineaminimumtolerableusefulness
U .
min
4.4 Functionmapping
Initially all tuples in an attribute history table are archi-
vedinalivesegment SEG withusefulnessU = 100%.
live
The temporal functions discussed in Sect. 3.2 were imple-
Updateswillbeperformedonthelivesegment,andwhenU
mentedasC++user-definedfunctionsimportedintotheSQL
dropsbelowU ,weperformthefollowingoperations:
min
DBMStroughtheirstandardUDFmechanism.Thesefunc-
tions are collected in a special library registered with the
ArchIScompilerwhich,therefore,embedscallstothesefunc-
3 InourTamino-basedimplementationweinsteadusedtemporalfunc-
tionsintheequivalent SQL/XMLcode generated bytrans-
tionswrittenforandimportedintoTamino.Tolimittheimpactofthe
lating the temporal XQuery statements. This simplifies the
differentimplementations,onlyoneofthequeriesofourexperiments
mappingfromXQuerytoSQL.However,themappingmust usesafunction.
123
F.Wangetal.
Fig. 6 Segment-based
Update Update Update Update
archivingofhistorydata
copy live copy live
SEGlive U<=Umin SEGlive SEGlive U<=Umin SEGlive
live tuple
non-active tuple copyall copy all
Si Si+1
[segstarti, segendi] [segstarti+1, segendi+1]
1. Anewsegment S isallocated; There are several advantages for segment-based
i
2. The interval of this segment is recorded in the table clustering:First,thecurrentlivesegmentalwayshasahigh
segment(segno, segstart, segend), where usefulness,whichassureseffectiveupdates;second,records
segstartandsegendrecordthestartingandending are globally temporally clustered on segments; third, for
timeforthesegmentrespectively; snapshot queries, only one segment is used, and for tem-
3. All tuples in SEG are copied into the new segment poralslicingqueries,onlysegmentsinvolvedareused,thus
live
S ,sortedbyID; suchqueriescanbemoreefficient,asdiscussedinSect.6.3;
i
4. All live tuples in SEG are copied into a new live and last, we have the flexibility to control the number of
live
segment SEGlive(cid:1),andtheoldlivesegmentisdropped. redundanttuplesinsegmentsbyUmin,asdiscussednext.
Aftertheseoperationsarecompleted,segment SEGlive(cid:1) 5.2 Storageusage
becomesthenewlivesegmentforupdates,andtheprocess
repeats.TheprocessisillustratedinFig.6. Assume all segments have usefulness Umin, thus the total
As an example, the segment-based scheme for number of invalid (non-current) tuples of all segments are
employee_salarytablewillbeclusteredonsegno,as (1 -Umin)× Nseg, where Nseg is the total number of tuples
shownfollows: inarchivedsegments.Assumefortheworstcase,alltuples
(N ) in the original relation (without segmentation)
noseg
SegNo ID SALARY TSTART TEND
becomeinvalid,then N ≥(1−U )×N ,or:
========================================= noseg min seg
001 100022 40000 02/20/1988 02/19/1989
N 1
001 100022 42010 02/20/1989 02/04/1990 seg ≤ (3)
001 100022 42525 02/20/1990 02/04/1991 Nnoseg 1−Umin
001 100022 42727 02/20/1991 12/31/9999
002 100022 42727 02/20/1991 02/19/1992 Figure7showstheratioofstoragesizewithdifferentUmin,
comparedtothatwithoutsegmentation.
Indexing Sincethetableisgeneratedbybulk-copyingsor-
ted by SegNo and ID, the physical clustering of the data
is guaranteed. By building traditional B+-Tree indexes on
SegNoandID,weobtainaclusteredB+-Treeindexonthe
segmented table. In this way, for history queries, only seg-
mentsinvolvedinthequeriesneedtoberetrieved.
Andthecontentinsegmenttablewillbe
SegNo segstart segend
========================================
001 01/01/1985 10/17/1991
002 10/18/1991 07/08/1995
...
An important feature of this usefulness-based clustering
is,thefollowingtwoconditionsarealwayssatisfiedforany
tupleinasegment:
tstart ≥segstart (1)
tuple SEG
tendtuple ≤segendSEG (2) Fig. 7 StoragesizefordifferentUmin
123