Table Of ContentWelcome to VLDB'04
The international conference series on Very Large Data Bases (VLDB) was launched in
1975 in Framingham MA, about 20 miles from Boston. The conference was a huge
success for its time, attracting almost 100 papers and more than 150 participants.
We have come a long way since then! VLDB conferences attract today hundreds of
submissions and participants. Thanks to the efforts of programme committees, authors,
and the VLDB Endowment over the years, VLDB conferences constitute today a
prestigious scientific forum for the presentation and exchange of research results and
practical experiences among members of the international Databases community.
VLDB conference regulars are familiar with Canada. This is the third time the conference
is visiting, after Montreal (1980) and Vancouver (1992.) In all three occasions, the
Canadian Databases community served as backbone for the organizing and the
programme committees. At the same time, as with other years, the committees that put
together this conference were international, with participation from all regions of the
globe. We are grateful to all members of these committees for their time and efforts.
Special thanks to Tamer Ozsu, the general programme chair of this year's conference;
also Kyu-Young Whang, who served as the VLDB Endowment liaison. Their spirit of
cooperation throughout was invaluable.
Of course, the technical programme is not the only attraction of this year's conference.
The conference hotel is located in the core downtown area of Toronto, within walking
distance of museums, parks, shopping areas and tourist attractions such as the CN
tower, the Harbourfront or Lake Ontario. Toronto has been called the most multicultural
city in the world and while here, you will have the opportunity to visit neighborhoods that
have the distinctive ethnic flavour of parts of the Far East and Europe. More than that,
Toronto is a well-run, safe city that visitors enjoy visiting and revisiting.
Welcome to VLDB'04 and Toronto. We hope that you enjoy both the technical
programme and the city!!
John Mylopoulos
General Chair, VLDB'04
Foreword from the Program Chairs
Welcome to the 30th International Conference on Very Large Databases (VLDB'04).
VLDB Conferences are among the premier database meetings for dissemination of
research results and for the exchange of latest ideas in the development and practice of
database technology. The program includes two keynote talks, a 10-year award
presentation, 81 research papers, 26 industrial papers (9 of which are invited), 5
tutorials, 2 panels and 34 demonstrations. It is a very rich program indeed.
This year we witnessed a significant jump of submissions. There were 504 research and
industrial paper submissions, accounting for about 10% increase over last year and
about 7% increase over 2002. Consequently the competition was fierce with an
acceptance rate of 16.1% for research papers and about 40% for industrial papers.
The first keynote talk is by David Yach, who is the Senior Vice President of Software at
Research in Motion (RIM). RIM is a leading designer, manufacturer and marketer of
innovative wireless solutions for the worldwide mobile communications market. Their
best-known product is the Blackberry line of wireless handhelds. David oversees and
manages the development of all lines of software at RIM. In his talk entitled "Databases
in a Wireless World", David addresses the emerging environment where "information is
stored not only in these central databases, but on a myriad of computers and computer-
based devices in addition to the central storage. These range from desktop and laptop
computers to PDA's and wireless devices such as cellular phones and BlackBerry's.
The combination of large centralized databases with a large number and variety of
associated edge databases effectively form a large distributed database, but one where
many of the traditional rules and assumptions for distributed databases are no longer
true." His talk discusses some of the new and challenging attributes of this new
environment, particularly focusing on the challenges of wireless and occasionally
connected devices.
Alon Halevy of the University of Washington gives the second keynote talk. His talk is
entitled "Structures, Semantics and Statistics" and addresses the issues in integrating
data from multiple sources. This is a problem that has occupied our community for a
long time and has gained renewed importance with the emergence of the World Wide
Web and the very many (and diverse) data sources that have become available on the
Web. Alon's talk begins by "highlighting some of the significant recent achievements in
the field of data integration, both in research and in industry." He then focuses on the
main challenge going forward, namely, large-scale reconciliation of semantic
heterogeneity, and on-the-fly information integration.
The ten-year best paper award this year goes to and Ramakrishan Srikant for their
paper entitled "Fast Algorithms for Mining Association Rules in Large Databases" that
appeared in the 1994 VLDB Conference Proceedings. The Awards Committee
(consisting of Masaru Kitsuregawa, Johann-Christoph Freytag, Raghu Ramakrishnan,
Anastassia Ailamaki, Paolo Atzeni, and Limsoon Wong and chaired by Tamer Ozsu)
considered this to be one of the seminal papers in data mining. The paper identifies
association rule mining and they discover a very nice property (a priori) that helps in
pruning candidates in association rule mining. Rakesh and Ramakrishnan present a talk
at this year's conference that focuses on the future of data mining.
The same committee has selected the paper "Model-Driven Data Acquisition in Sensor
Networks" by Amol Deshpande, Carlos Guestrin, Samuel R. Madden, Joseph M.
Hellerstein, and Wei Hong as the best paper.
The five tutorials that are scheduled cover a wide range of topics including core
database topics as well as emerging issues in data management. The tutorials are the
following:
• Database Architectures for New Hardware by Anastassia Ailamaki of Carnegie
Mellon University
• Security of Shared Data in Large Systems by Arnon Rosenthal of Mitre
Corporation and Marianne Winslett of University of Illinois at Urbana Champaign
• Self-Managing Technology in Database Management Systems by Surajit
Chaudhuri of Microsoft Research, Benoit Dageville of Oracle, and Guy Lohman
of IBM Almaden Research Lab.
• Architectures and Algorithms for Internet-Scale (P2P) Data Management by
Joseph M. Hellerstein of University of California, Berkeley and Intel Research
Berkeley
• The Continued Saga of DB-IR Integration by Ricardo Baeza-Yates of University
of Chile and Mariano Consens of University of Toronto
There are two panels scheduled at this year's conference. The first panel is moderated
by Thodoros Topaloglou of MDS Proteomics and is on "Biological Data Management:
Research, Practice and Opportunities". The panel focuses on the data management
problems that arise in the field of biological research. The panelists (Susan B. Davidson,
H. V. Jagadish, Victor M. Markowitz, Evan W. Steeg, and Mike Tyers) discuss the ways
in which database researchers can better serve the needs of biomedical research.
The second panel is entitled "Where is Business Intelligence taking today's database
systems" and is moderated by William O'Connell of IBM Canada. The panelists are Andy
Witkowski, Ramesh Bhashyam, Surajit Chauduri, Nigel Campbell. The panel addresses
issues that arise in the production level deployment of business intelligence solutions
(e.g., data mining, OLAP) over relational systems.
The technical program is the result of efforts by a large group of people. Three Program
Committees were formed along themes (core database, infrastructure for information
systems, and industrial and applications) consisting of 137 colleagues, each of whom
reviewed about 13 papers. Raymond Ng and Matthias Jarke handled the tutorials, Jarek
Gryz and Fred Lochovsky selected the panels, Bettina Kemme and David Toman
assembled the demonstrations program. We thank them all for helping us put together
an exciting program. We also thank Mario Nascimento for the excellent work he has
done in putting together these Proceedings. We, along with Mario, also want to extend
our thanks to DCC/UFAM in Brazil for the local support they have provided during the
compilation of these proceedings while he was in Brazil.
M. Tamer Ozsu
Donald Kossmann
Renee J. Miller
Jose A. Blakeley
Berni Schiefer
Databases in a Wireless World
David Yach
Research in Motion
Waterloo, Ontario
Canada N2L 3W8
Abstract
The traditional view of distributed databases
is based on a number of database servers with
regular communication. Today information is
stored not only in these central databases, but
on a myriad of computers and computer-based
devices in addition to the central storage.
These range from desktop and laptop com
puters to PDA's and wireless devices such as
cellular phones and BlackBerry's. The com
bination of large centralized databases with a
large number and variety of associated edge
databases effectively form a large distributed
database, but one where many of the tradi
tional rules and assumptions for distributed
databases are no longer true.
This keynote will discuss some of the new
and challenging attributes of this new envi
ronment, particularly focusing on the chal
lenges of wireless and occasionally connected
devices. It will look at the new constraints,
how these impact the traditional distributed
database model, the techniques and heuristics
being used to work within these constraints,
and identify the potential areas where future
research might help tackle these difficult is
sues.
Permission to copy without fee all or part of this material is
granted provided that the copies are not made or distributed for
direct commercial advantage, the VLDB copyright notice and
the title of the publication and its date appear, and notice is
given that copying is by permission of the Very Large Data Base
Endowment. To copy otherwise, or to republish, requires a fee
and/or special permission from the Endowment.
Proceedings of the 30th VLDB Conference,
Toronto, Canada, 2004
Structures, Semantics and Statistics
Alon Y. Halevy
University of Washington, Seattle
[email protected]
Abstract Structures and Semantics
At a fundamental level, the key challenge in data integra There are many factors that make data integration a
tion is to reconcile the semantics of disparate data sets, hard problem, not all of which are purely technical.
each expressed with a different database structure. I argue Some of these include query processing across multi
that computing statistics over a large number of structures ple autonomous systems, processing XML documents
offers a powerful methodology for producing semantic map (and other semi-structured data) streaming from the
pings, the expressions that specify such reconciliation. In network, managing data ownership and privacy across
essence, the statistics offer hints about the semantics of the organizational boundaries, and in some cases, even
symbols in the structures, thereby enabling the detection capturing or locating the data needed for particular
of semantically similar concepts. The same methodology applications, or transforming it into machine process-
can be applied to several other data management tasks able form. However, the most notable and unique chal
that involve search in a space of complex structures and lenge in data integration is reconciling the semantic
in enabling the next-generation on-the-fly data integration heterogeneity of the sources being integrated.
systems.
The fundamental reason that makes semantic het
erogeneity so hard is that the data sets were developed
Data Integration independently, and therefore varying structures were
used to represent the same or overlapping concepts.
Data integration is a pervasive challenge faced in data
By structures, I mean both the choice of data model
management applications that need to query across
(relational, XML, object-oriented, ontology formal
multiple data sources. Data integration is crucial in
ism) and the particular choices made in designing the
large enterprises that own a multitude of data sources,
schema (naming of relations, attributes or tags, choices
for progress in large-scale scientific projects, where
of data types, decomposition, and nesting structure).
data sets are being produced independently by multi
The presence of a variety of structures is unavoidable
ple researchers, for better cooperation among govern
both because humans think differently from one an
ment agencies, each with their own data sources, and
other and because the applications these data sets were
in searching the deep web, the part of the web that is
designed for have different needs. Efforts to circum
hidden behind web forms. The emergence of XML and
vent this problem by imposing standardized schemas
web services as technologies for sharing data and for
have met limited success at best.
accessing remote data sources have further fueled the
desire of organizations to share data. The many appli As a first step toward reconciling semantic hetero
cations of data integration have led to a very fruitful geneity, researchers developed languages for describing
line of research in the Database and Artificial Intel semantic mappings, expressions that relate the seman
ligence Communities, and recently to a budding in tics of data expressed in different structures [6, 9, 15].
dustry, known as Enterprise Information Integration These languages typically relate different structures
(EII) [1]. with a variety of query and constraint expressions.
With these languages, researchers have developed al
gorithms for query reformulation, which translate a
Permission to copy without fee all or part of this material is
granted provided that the copies are not made or distributed for query posed over one schema into a set of queries
direct commercial advantage, the VLDB copyright notice and over other schemas. More generally, recent research
the title of the publication and its date appear, and notice is
on model management investigates a general algebra
given that copying is by permission of the Very Large Data Base
for manipulating structures (called models) and map
Endowment. To copy otherwise, or to republish, requires a fee
and/or special permission from the Endowment. pings between them [2, 13]. The algebra includes oper
Proceedings of the 30th VLDB Conference, ations such as merging and applying transformations
Toronto, Canada, 2004 on models, and for composing and inverting mappings.
structures and Statistics are manifested in the representation. For exam
ple, we can find that the Books table typically in
Given the languages for expressing semantic mappings
cludes an ISBN column and a foreign key into an
between disparate structures, the bottleneck is to cre
Availability table, but that ISBN never appears in
ate and maintain these mappings. Writing these map
a Warehouse table. These relationships are use
pings is very tedious and error prone, and often very
ful in order to prune candidate schema matches
repetitive. In fact, in many integration projects, more
that appear less likely. They can also be used to
than half of the resources are spent on these tasks.
build a system that provides advice in designing
Clearly, completely automating the creation of seman
new schemas.
tic mappings is unlikely. Hence, the focus of research
has been on reducing the human effort needed in the • Domain constraints: We can leverage a corpus
process (see [14] for a recent survey). to find integrity constraints on the domain and
This is where statistics come into play. A pow its representations. For example, we can observe
erful approach for discovering semantic mappings is that ISBN is a foreign key into multiple tables
based on analyzing a large number of structures and involving books, and hence possibly an identifier
mappings in a particular domain. The intuition be for books, or discover likely data types for certain
hind this approach is that statistics computed over fields (e.g., address, price). Constraints may have
large number of structures can be used to provide hints to do with ordering of attributes. For example, in
about the semantics of the symbols used in these struc a corpus of web forms about cars for sale, we may
tures. Therefore, these statistics can be leveraged to discover that the make attribute is always placed
predict when two symbols, from disparate structures, before the model and price attribute, but occurs
are meant to represent the same domain concept. after the new/used attribute.
In a sense, the goal of this approach is to mirror the
Typically, constraints we discover in this way are
success of statistical analysis of large corpora of texts
soft constraints^ in the sense that they are some
in the field of Information Retrieval (IR) and of the
times violated, but can still be taken as rules of
recent significant advances made in the field of Nat
thumb about the domain. Therefore, they are ex
ural Language Processing by analyzing large corpora
tremely useful in resolving ambiguous situations,
of annotated sentences [12]. However, the analogy to
such as selecting among several candidate schema
these fields also highlights the unique challenges we
matches [3, 11].
face here.
In the IR context, text documents typically contain
It is important to note that in all of these exam
a significant amount of information and high level of
ples there is a close interplay between properties of the
redundancy. Hence, IR techniques can be eff^ective by
underlying domain (e.g., books, warehouses and their
abstracting a document as a bag of words. By contrast,
properties) and of the representations of the domain
in our context, schema descriptions are very terse and
(e.g., the particular relational structures in schemas).
the underlying semantics are very rich. Hence, the bag
In fact, this interplay is the reason this technique is so
of words abstraction does not suffice.
powerful.
As a consequence, to exploit a corpus of schemas
Several works have already applied this approach
and mappings, we need statistics that provide hints
in various contexts [3, 5, 7, 8, 10]. Doan et al. [3] ad
about deeper domain concepts and at a finer granular
dress the problem of matching schemas of data sources
ity. The following are a few examples:
to a single mediated schema. [3] uses Machine Learn
ing techniques to compute models of the elements in
• Domain concepts and their representa
the mediated schema from a set of manually provided
tional variations: As a first step, we can an
mappings. These models are then used to recognize
alyze a corpus to identify the main concepts in
the mediated schema elements in the schemas of un
the domain. For example, in a corpus of book in
mapped data sources. He and Chang [7] generate a
ventory schemas, we may identify the concept of
mediated schema for a domain based on analyzing a
book and warehouse and a cluster of price-related
corpus of web forms in that domain. Madhavan et
elements. Even more importantly, we will dis
al. [11] leverage a corpus of schemas and mappings to
cover variations on how these concepts are rep
match between two unseen schemas. In doing so, [11]
resented. The variations may differ on naming of
learns from the corpus models for elements of the do
schema elements, grouping attributes into tables
main and constraints on the domain.
or the granularity of modehng a particular con
Another application of this paradigm is search for
cept. Knowledge of these variations will be lever
web services [4]: locating web services (or operations
aged when we match two schemas in the domain.
within them) that are relevant to a particular need.
• Relationships between concepts: Given a set Simple keyword search does not suffice in this context
of concepts, we can discover relationships between because keywords (or parameter names) do not cap
them, and the ways in which these relationships ture the underlying semantics of the web service. Dong
et al. [4] show how to analyze a corpus of web services hai Doan, Pedro Domingos, Luna Dong, Oren Etzioni,
and cluster parameter names into semantically mean Zack Ives, Jayant Madhavan, Luke McDowell, Peter
ingful concepts. These concepts are used to predict Mork, Rachel Pottinger, Dan Suciu, Peter Tarczy-
when two web service operations have similar func Hornoch and Igor Tatarinov. The work has been sup
tionality. ported by NSF ITR grant IIS-0205635, NSF CAREER
Searching for web services is an instance of a gen Grant IIS-9985114 and a gift from Microsoft Research.
eral class of search problems, where the objects being
References
searched have rich semantics, but the descriptions of
these objects (e.g., schema definitions or WSDL de [1] Aberdeen Group. Enterprise information integration
scriptions) are terse and do not fully capture their se - the new way to leverage e-information. Aberdeen
mantics. Other examples of such search problems are Group, Boston, Mass., 2003.
in trying to locate web forms that are relevant to a [2] P. A. Bernstein. Applying Model Management to
particular information need, or locating relevant data Classical Meta Data Problems. In Proceedings of
sources within an enterprise. In all of these examples, the Conference on Innovative Data Systems Research
simple keyword search does not suffice. Analyzing a (CIDR), 2003.
corpus of such objects, and using the statistics to glean [3] A. Doan, P. Domingos, and A. Y. Halevy. Reconcil
hints about the semantics of the objects offers a pow ing Schemas of Disparate Data Sources: A Machine
erful supplement to keyword search. I now outhne a Learning Approach. In Proceedings of the ACM SIG-
major challenge for the field of data integration which MOD Conference, 200L
will benefit significantly from this general approach. [4] X. L. Dong, A. Y. Halevy, J. Madhavan, E. Nemes,
and J. Zhang. Similarity search for web services. In
A Data Integration Challenge
Proc. of VLDB, 2004.
Despite the inmiense progress, building a data inte
[5] A. Halevy, O. Etzioni, A. Doan, Z. Ives, J. Madhavan,
gration application is still a major undertaking that
L. McDowell, and I. Tatarinov. Crossing the structure
requires significant resources, upfront effort and tech chasm. In Proceedings of the First Biennial Confer
nical expertise. As a result, data integration systems ence on Innovative Data Systems Research (CIDR),
have two major drawbacks. First, evolving the system 2003.
as the requirements in the organization change is hard.
[6] A. Y. Halevy. Answering Queries Using Views: A
Second, many smaller-scale and more transient infor Survey. VLDB Journal, 10(4), 2001.
mation integration tasks that we face on a daily basis
[7] B. He and K. C.-C. Chang. Statistical Schema Match
are not supported.
ing across Web Query Interfaces. In Proceedings of the
Hence, a challenge to our community is to funda
ACMSIGMOD Conference, 2003.
mentally change the cost-benefit equation associated
[8] A. Hess and N. Kushmerick. Learning to Attach Se
with integrating data sources. Our goal should be to
mantic Metadata to Web Services. In Proceedings of
enable on-the-fly data integration, thereby faciUtating
the International Semantic Web Conference, 2003.
the evolution of data integration appUcations and en-
[9] M. Lenzerini. Data Integration: A Theoretical Per
abUng individuals to easily integrate information for
spective. In In Proceedings of PODS, 2002.
their personal, possibly transient, needs.
To achieve this goal, I believe a data integration en [10] J. Madhavan, P. Bernstein, K. Chen, A. Halevy, and
P. Shenoy. Matching schemes by learning from others.
vironment should incorporate the following two prin
In Working notes of the IJCAI-03 workshop on Data
ciples. First, as data integration tasks are performed,
Integration on the Web, 2003.
the system should accumulate and analyze them, and
then leverage prior tasks when facing a new task. Sec [11] J. Madhavan, P. Bernstein, A. Doan, and A. Halevy.
Corpus-based schema matching. Technical Report
ond, the data integration environment should be a nat
2004-06-04, University of Washington, 2004.
ural extension of the user's personal information space,
i.e., the information one stores on the desktop. In that [12] C. Manning and H. Schutze. Foundations of Statistical
way, a user can extend her personal data space with Natural Language Processing. MIT Press, 1999.
public data sources, and seamlessly integrate personal [13] S. Melnik, E. Rahm, and P. Bernstein. Rondo: A pro
information (e.g., spreadsheets, contacts lists, personal gramming platform for generic model management. In
databases) with organizational resources. Achieving Proc. ofSIGMOD, 2003.
these goals will substantially increase the perception [14] E. Rahm and P. A. Bernstein. A survey of ap
of data management systems and their impact on our proaches to automatic schema matching. VLDB Jour
daily lives. nal, 10(4):334-350, 2001.
[15] J. D. UUman. Information Integration using Logical
Acknowledgements
Views. In Proceedings of the International Conference
The ideas espoused in this paper have benefited from on Database Theory (ICDT), 1997.
many discussions and hard work by my colleagues
and students. I'd hke to thank Phil Bernstein, An-
Whither Data Mining?
Rakesh Agrawal Ramakrishnan Srikant
IBM Almaden Research Center
650 Harry Road, San Jose, CA 95120
Abstract
The last decade has witnessed tremendous ad
vances in data mining. We take a retrospective
look at these developments, focusing on associ
ation rules discovery, and discuss the challenges
and opportunities ahead.
The full version of the paper will be
made available at http://www.almaden.ibm.
com/software/quest.
Permission to copy without fee all or part of this material is granted pro
vided that the copies are not made or distributed for direct commercial
advantage, the VLDB copyright notice and the title of the publication and
its date appear, and notice is given that copying is by permission of the
Very Large Data Base Endowment. To copy otherwise, or to republish,
requires a fee and/or special permission from the Endowment.
Proceedings of the 30th VLDB Conference,
Toronto, Canada, 2004
Compressing Large Boolean Matrices Using Reordering
Techniques
David Johnson Shankar Krishnan Jatin Chhugani
AT&T Labs - Research AT&T Labs - Research Johns Hopkins University
dsj Oresearch.att.com krishnasOresearch.att.com j atinchQcs.j hu.edu
Subodh Kumar Suresh Venkatasubramanian
Johns Hopkins University AT&T Labs - Research
subodhQcs.jhu.edu sureshSresearch.att.com
Abstract sualization and telephone call data we obtain
significant improvement in access time over
Large boolean matrices are a basic represen standard techniques, and for the visuahzation
tational unit in a variety of applications, with application we also make significant improve
some notable examples being interactive visu ments in compression.
alization systems, mining large graph struc
tures, and association rule mining. Designing 1 Introduction
space and time efficient scalable storage and
query mechanisms for such large matrices is a Consider the following three problems:
challenging problem.
• You are visualizing a large and complex three-
We present a lossless compression strategy to
dimensional geometric model and you would hke
store and access such large matrices efficiently
to have a real-time walkthrough (> 20 frames/s
on disk. Our approach is based on viewing the
update). In order to do this, you need to deter
columns of the matrix as points in a very high
mine quickly what parts of the model can be seen
dimensional Hamming space, and then for
from a region of space (cell) bounding your cur
mulating an appropriate optimization prob
rent location.
lem that reduces to solving an instance of the
Traveling Salesman Problem on this space.
• You work for a major phone company, and you
Finding good solutions to large TSP's in high have access to data that tells you which numbers
dimensional Hamming spaces is itself a chal call which numbers. You would like to manage
lenging and little-explored problem - we can this data to develop graph models of communities
not readily exploit geometry to avoid the need of interest.
to examine all A^^ inter-city distances and in
stances can be too large for standard TSP • You have large volumes of data describing various
codes to run in main memory Our multi- purchases that people make, and you'd hke to in
faceted approach adapts classical TSP heuris fer association rules from this very large database.
tics by means of instance-partitioning and
In all of the above problems, the basic unit of data
sampling, and may be of independent inter
is a large, disk-resident matrix of ones and zeros. In
est. For instances derived from interactive vi-
the first case, rows correspond to transitions between
view cells, and columns are the primitives (typically
Permission to copy without fee all or part of this material is
granted provided that the copies are not made or distributed for collections of triangles) that become visible in moving
direct commercial advantage, the VLDB copyright notice and from one cell to the next. These matrices are very
the title of the publication and its date appear, and notice is
large, having of the order of hundreds of thousands of
given that copying is by permission of the Very Large Data Base
rows and colunms. Representing them and querying
Endowment. To copy otherwise, or to republish, requires a fee
and/or special permission from the Endowment. them efficiently is a non-trivial problem. In the second
Proceedings of the 30th VLDB Conference, case, rows and columns are individual customers, and
Toronto, Canada, 2004 each entry of the matrix represents a call made from
13
