Table Of ContentPerformance
Evaluation
and
Benchmarking
Performance
Evaluation
and
Benchmarking
Edited by
Lizy Kurian John
Lieven Eeckhout
Boca Raton London New York
A CRC title, part of the Taylor & Francis imprint, a member of the
Taylor & Francis Group, the academic division of T&F Informa plc.
3622_Discl.fm Page 1 Friday, August 5, 2005 8:59 AM
Published in 2006 by
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2006 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number-10: 0-8493-3622-8 (Hardcover)
International Standard Book Number-13: 978-0-8493-3622-5 (Hardcover)
Library of Congress Card Number 2005047021
This book contains information obtained from authentic and highly regarded sources. Reprinted material is
quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts
have been made to publish reliable data and information, but the author and the publisher cannot assume
responsibility for the validity of all materials or for the consequences of their use.
No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic,
mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and
recording, or in any information storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com
(http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive,
Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration
for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate
system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only
for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
John, Lizy Kurian.
Performance evaluation and benchmarking / Lizy Kurian John and Lieven Eeckhout.
p. cm.
Includes bibliographical references and index.
ISBN 0-8493-3622-8 (alk. paper)
1. Electronic digital computers--Evaluation. I. Eeckhout, Lieven. II. Title.
QA76.9.E94J64 2005
004.2'4--dc22 2005047021
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
Taylor & Francis Group and the CRC Press Web site at
is the Academic Division of T&F Informa plc. http://www.crcpress.com
Preface
It is a real pleasure and honor for us to present you this book titled Perfor-
mance Evaluation and Benchmarking. Performance evaluation and benchmark-
ing is at the heart of computer architecture research and development. With-
out a deep understanding of benchmarks’ behavior on a microprocessor and
without efficient and accurate performance evaluation techniques, it is
impossible to design next-generation microprocessors. Because this research
field is growing and has gained interest and importance over the last few
years, we thought it would be appropriate to collect a number of these
important recent advances in the field into a research book. This book deals
with a large variety of state-of-the-art performance evaluation and bench-
marking techniques. The subjects in this book range from simulation models
to real hardware performance evaluation, from analytical modeling to fast
simulation techniques and detailed simulation models, from single-number
performance measurements to the use of statistics for dealing with large data
sets, from existing benchmark suites to the conception of representative
benchmark suites, from program analysis and workload characterization to
its impact on performance evaluation, and other interesting topics. We expect
it to be useful to graduate students in computer architecture and to computer
architects and designers in the industry.
This book was not entirely written by us. We invited several leading
experts in the field to write a chapter on their recent research efforts in the
field of performance evaluation and benchmarking. We would like to thank
Prof. David J. Lilja from the University of Minnesota, Prof. Tom Conte from
North Carolina State University, Prof. Brad Calder from the University of
California San Diego, Prof. Chita Das from Penn State, Prof. Brinkley Sprunt
from Bucknell University, Alex Mericas from IBM, and Dr. Kishore Menezes
from Intel Corporation for accepting our invitation. We thank them and their
co-authors for contributing. Special thanks to Dr. Joshua J. Yi from Freescale
Semiconductor Inc., Paul D. Bryan from North Carolina State University,
Erez Perelman from the University of California San Diego, Prof. Timothy
Sherwood from the University of California at Santa Barbara, Prof. Greg
Hamerly from Baylor University, Prof. Eun Jung Kim from Texas A&M
University, Prof. Ki HwanYum from the University of Texas at San Antonio,
Dr. Rumi Zahir from Intel Corporation, and Dr. Susith Fernando from Intel
Corporation for contributing. Many authors went beyond their call to adjust
their chapters according to the other chapters. Without their hard work, it
would have been impossible to create this book.
We hope you will enjoy reading this book.
Prof. L. K. John
The University of Texas at Austin, USA
Dr. L. Eeckhout
Ghent University, Belgium
Editors
Lizy Kurian John is an associate professor and Engineering Foundation
Centennial Teaching Fellow in the electrical and computer engineering
department at the University of Texas at Austin. She received her Ph.D. in
computer engineering from Pennsylvania State University in 1993. She
joined the faculty at the University of Texas at Austin in fall 1996. She was
on the faculty at University of South Florida, from 1993 to 1996. Her current
research interests are computer architecture, high-performance microproces-
sors and computer systems, high-performance memory systems, workload
characterization, performance evaluation, compiler optimization techniques,
reconfigurable computer architectures, and similar topics. She has received
several awards including the 2004 Texas Exes teaching award, the 2001 UT
Austin Engineering Foundation Faculty award, the 1999 Halliburton Young
Faculty award, and the NSF CAREER award. She is a member of IEEE, IEEE
Computer Society, ACM, and ACM SIGARCH. She is also a member of Eta
Kappa Nu, Tau Beta Pi, and Phi Kappa Phi Honor Societies.
Lieven Eeckhout obtained his master’s and Ph.D degrees in computer sci-
ence and engineering from Ghent University in Belgium in 1998 and 2002,
respectively. He is currently working as a postdoctoral researcher at the same
university through a grant from the Fund for Scientific Research—Flanders
(FWO Vlaanderen). His research interests include computer architecture,
performance evaluation, and workload characterization.
Contributors
Paul D. Bryan is a research assistant in the TINKER group, Center for
Embedded Systems Research, North Carolina State University. He received
his B.S. and M.S. degrees in computer engineering from North Carolina State
University in 2002 and 2003, respectively. In addition to his academic work,
he also worked as an engineer in the IBM PowerPC Embedded Processor
Solutions group from 1999 to 2003.
Brad Calder is a professor of computer science and engineering at the Uni-
versity of California at San Diego. He co-founded the International Sympo-
sium on Code Generation and Optimization (CGO) and the ACM Transac-
tions on Architecture and Code Optimization (TACO). Brad Calder received
his Ph.D. in computer science from the University of Colorado at Boulder
in 1995. He obtained a B.S. in computer science and a B.S. in mathematics
from the University of Washington in 1991. He is a recipient of an NSF
CAREER Award.
Thomas M. Conte is professor of electrical and computer engineering and
director for the Center for Embedded Systems Research at North Carolina
State University. He received his M.S. and Ph.D. degrees in electrical engi-
neering from the University of Illinois at Urbana-Champaign in 1988 and
1992, respectively. In addition to academia, he’s consulted for numerous
companies, including AT&T, IBM, SGI, and Qualcomm, and spent some time
in industry as the chief microarchitect of DSP vendor BOPS, Inc. Conte is
chair of the IEEE Computer Society Technical Committee on Microprogram-
ming and Microarchitecture (TC-uARCH) as well as a fellow of the IEEE.
Chita R. Das received the M.Sc. degree in electrical engineering from the
Regional Engineering College, Rourkela, India, in 1981, and the Ph.D. degree
in computer science from the Center for Advanced Computer Studies at the
University of Louisiana at Lafayette in 1986. Since 1986, he has been working
at Pennsylvania State University, where he is currently a professor in the
Department of Computer Science and Engineering. His main areas of interest
are parallel and distributed computer architectures, cluster systems, com-
munication networks, resource management in parallel systems, mobile
computing, performance evaluation, and fault-tolerant computing. He has
