Compiling Parallel Loops for High Performance Computers: Partitioning, Data Assignment and Remapping

170 Pages·1993·4.237 MB·English

by  David E. Hudak, Santosh G. Abraham (auth.)

COMPILING PARALLEL LOOPS
FOR HIGH PERFORMANCE
COMPUTERS:
Partitioning, Data Assignment
and Remapping
THE KLUWER INTERNATIONAL SERIES
IN ENGINEERING AND COMPUTER SCIENCE
COMPILING PARALLEL LOOPS
FOR HIGH PERFORMANCE
COMPUTERS:
Partitioning, Data Assignment
and Remapping
by
David E. Hudak
Ohio Northern University
Santosh G. Abraham
The University of Michigan
~.
"
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
Library of Congress Cataloging-in-Publication Data
Hudak, David E.
Compiling parallelloops for high performance computers
partitioning, da ta assignment, and remapping / by David E. Hudak,
Santosh G. Abraham.
p. cm. -- (The Kluwer international series in engineering and
computer science ; SECS 200)
Includes bibliographical references and index.
ISBN 978-1-4613-6386-6 ISBN 978-1-4615-3164-7 (eBook)
DOI 10.1007/978-1-4615-3164-7
1. ParaIJel computers. 2. Compiling (Electronic computers)
I. Abraham, Santosh G. 11. Title. III. Series.
QA76. 58. H83 1993
005 . 4 ' 53--dc20 92-30509
CIP
Copyright © 1993 by Springer Science+Business Media New York
Origina11y published by Kluwer Academic Publishers in 1993
Softcover reprint ofthe hardcover 1st edition 1993
All rights reserved. No part of this publication may be reproduced, stored in a retrieval
system or transmitted in any form or by any means, mechanical, photo-copying, recording,
or otherwise, without the prior written permission of the publisher,
Springer Science+Business Media, LLC.
Printed on acid-free paper.
To my beloved parents, Paul and Carol Hudak, for their patience and
inspiration. As always, for Cathy.
To Koshy and Mary Gheevarghese
and to Susan and Aditi
TABLE OF CONTENTS
DEDICATION . . .. v
LIST OF FIGURES ix
LIST OF TABLES Xll
PREFACE. xiii
CHAPTER
1 INTRODUCTION 1
1.1 Model Assumptions. 3
1.2 Related Work 10
1.3 Overview....... 13
2 CONTIGUOUS LOOP PARTITIONS FOR NEIGHBOR-
HOOD COMMUNICATION 19
2.1 Introduction............. 19
2.2 Quantifying Communication. . . . 21
2.3 Compensation for Cache Line Size 31
2.4 Partition Construction . . . . . . 35
2.5 Experimental Evaluation of ADP 44
2.6 Conclusion............. 53
3 CONTIGUOUS DATA ASSIGNMENTS FOR NEIGH-
BORHOOD COMMUNICATION 55
3.1 Introduction.... 55
3.2 Data Assignments. . . 58
3.3 Exploiting Overlap . . 68
3.4 Software Redundancy. 71
3.5 ADAPT . . . . . . . . 73
3.6 Experimental Results. 83
3.7 Conclusion....... 90
4 CYCLIC LOOP PARTITIONS FOR LINEARLY VARY
ING LOOPS. . . . . . . . . . . . . . . . . . . . . . . . . .. 93
viii
4.1 Introduction ................ . 93
4.2 Code Segments .............. . 96
4.3 Determining Communication Parameters . 99
4.4 Multicast Communication Overhead · 103
4.5 Partitioning ..... . · 103
4.6 Experimental Results . 117
4.7 Conclusion....... · 121
5 COLLECTIVE PARTITIONING AND REMAPPING FOR
MULTIPLE LOOP NESTS 125
5.1 Introduction........ . 125
5.2 Program Enclosure Trees. . 128
5.3 The CPR Algorithm . . 132
5.4 Experimental Results. . 141
5.5 Conclusion. . 146
BIBLIOGRAPHY. 149
INDEX . ...... . 157
LIST OF FIGURES
Figure
1.1 The Butterfly Architecture. . ....... . 5
1.2 Example of an iterative data-parallel loop .. 7
1.3 Contiguous tiling and assignment of an iteration space. 13
2.1 Communication along a line segment. .. 24
2.2 Access pattern for the access offset, (3,2). 25
2.3 Decomposing an access vector along an orthogonal basis set of
vectors. . ................ . 26
2.4 An analysis of communication patterns. 29
2.5 Decomposing a vector along two separate basis sets of vectors. 31
2.6 Cache lines aligning with borders. 33
2.7 Cache lines not aligned with borders. 34
2.8 nh is the difference of nd and nb. 42
2.9 nh is the sum of nd and nb. 42
2.10 The ADAPT system. 44
2.11 Code segment used in experiments. . 46
2.12 Execution rates for various partitions. 47
2.13 Execution time of partitions on Multimax. 48
2.14 Performance increase as processing power increases. 49
2.15 Percentage miss ratios for various aspect ratios and line sizes. 51
3.1 The partition exterior is heavily referenced. 59
3.2 The regions of a data set. . ........ . 66
3.3 Example of a deterministic iterative data parallel loop .. 75
3.4 The PAT interface to the ADAPT system ........ . 76
x
3.5 A loop restructured by ADAPT .... 77
3.6 The preamble generated by ADAPT .. 78
3.7 Partitioning an N x M iteration space into q row classes and r
column classes. . . . . . . . . . . . . . . . . . . 79
3.8 A cache line straddles two parts of a partition. 81
3.9 The ratio of (rm,rm) to (rm,c) times as N increases. 86
3.10 Execution time in J-lS vs. degree of overlap. . . . . . 87
3.11 Execution time of SHOPF with P = 16, aspect ratio of 0.25. 89
3.12 Execution time of SHOPF with N = 200, aspect ratio of 0.25. 90
4.1 Applications with various computation and communication char-
acteristics. .............................. 94
4.2 Partition classes associated with computation and communica-
tion characteristics. ......... 94
4.3 A sample loop for LU factorization. 96
4.4 The grammar for the code segment being optimized. 97
4.5 Size of iteration space decreases through time. 98
4.6 Code segment obtained from trfd. 99
4.7 Code segment obtained from tred2. 100
4.8 A contiguous partitioning where each chunk is 2 x 2 .. 105
4.9 A cyclic partitioning with four tiles. . ..... . 105
4.10 A k x k iteration space divided into four groups. 109
4.11 The execution times without pivoting .... 119
4.12 The execution times with column pivoting. 119
4.13 The execution times for N = 128. 121
4.14 The execution times with pivoting, relative to slowest partition .. 122
5.1 A code segment with neighborhood communication. . . . . .. 129
5.2 A sample loop for matrix multiplication and LU factorization. 130
5.3 The program enclosure tree associated with Figure 5.2. 131
5.4 The CPR algorithm. . . . . . . . . . . . . . . . . . 133
5.5 The algorithm for determining remapping points. . 136
5.6 The algorithm for determining collective partitions. 138
Xl
5.7 Collective partitioning of loop lists ... 140
5.8 The algorithm for merging partitions. 142
5.9 The execution times for varying matrix sizes. 144
5.10 The execution times for N = 128. . ..... 145
5.11 The execution times as communication varies .. 147