Table Of ContentPALGRAVE STUDIES IN DIGITAL
BUSINESS & ENABLING
TECHNOLOGIES
Series Editor: Theo Lynn
HETEROGENEITY,
HIGH PERFORMANCE
COMPUTING,
SELF-ORGANIZATION
AND THE CLOUD
Edited by
Theo Lynn, John P. Morrison
and David Kenny
Palgrave Studies in Digital Business & Enabling
Technologies
Series Editor
Theo Lynn
Irish Centre for Cloud Computing (IC4)
Dublin City University
Dublin, Ireland
This multi-disciplinary series will provide a comprehensive and coherent
account of cloud computing, social media, mobile, big data, and other
enabling technologies that are transforming how society operates and how
people interact with each other. Each publication in the series will focus on
a discrete but critical topic within business and computer science, covering
existing research alongside cutting edge ideas. Volumes will be written by
field experts on topics such as cloud migration, measuring the business
value of the cloud, trust and data protection, fintech, and the Internet of
Things. Each book has global reach and is relevant to faculty, researchers
and students in digital business and computer science with an interest in
the decisions and enabling technologies shaping society.
More information about this series at
http://www.palgrave.com/gp/series/16004
Theo Lynn • John P. Morrison
David Kenny
Editors
Heterogeneity, High
Performance
Computing, Self-
Organization and the
Cloud
Editors
Theo Lynn John P. Morrison
Irish Centre for Cloud Computing Department of Computer Science
(IC4) University College Cork
Dublin City University Cork, Ireland
Dublin, Ireland
David Kenny
Dublin City University
Dublin, Dublin, Ireland
Palgrave Studies in Digital Business & Enabling Technologies
ISBN 978-3-319-76037-7 ISBN 978-3-319-76038-4 (eBook)
DOI 10.1007/978-3-319-76038-4
Library of Congress Control Number: 2018941797
© The Editor(s) (if applicable) and The Author(s) 2018 This book is an open access publication.
Open Access This book is licensed under the terms of the Creative Commons Attribution-
NonCommercial- NoDerivatives 4.0 International License (http://creativecommons.org/
licenses/by-nc-nd/4.0/), which permits any noncommercial use, sharing, distribution and repro-
duction in any medium or format, as long as you give appropriate credit to the original author(s)
and the source, provide a link to the Creative Commons license and indicate if you modified the
licensed material. You do not have permission under this license to share adapted material derived
from this book or parts of it.
The images or other third party material in this book are included in the book’s Creative Commons
license, unless indicated otherwise in a credit line to the material. If material is not included in the
book’s Creative Commons license and your intended use is not permitted by statutory regulation
or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
This work is subject to copyright. All commercial rights are reserved by the author(s), whether the
whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by
similar or dissimilar methodology now known or hereafter developed. Regarding these commercial
rights a non-exclusive license has been granted to the publisher.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with
regard to jurisdictional claims in published maps and institutional affiliations.
Cover pattern © Melisa Hasan
Printed on acid-free paper
This Palgrave Macmillan imprint is published by the registered company Springer International
Publishing AG part of Springer Nature.
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
P
reface
This is the first book in the series, “Advances in Digital Business and
Enabling Technologies”, which aims to contribute to multi-disciplinary
research on digital business and enabling technologies, such as cloud com-
puting, social media, big data analytics, mobile technologies, and the
Internet of Things, in Europe. This first volume focuses on research that
extends conventional thinking on cloud computing architecture design to
greater support High Performance Computing (HPC). Meeting the needs
of HPC users provides significant challenges to cloud service providers,
both technically and culturally, and this book provides a novel approach
and indicates a future direction for cloud computing architecture research
that may address a significant portion of these challenges. Given the sig-
nificant role that HPC plays in scientific advancement and the increasing
dominance of cloud computing as a global enterprise computing para-
digm, this book has value to university educators and researchers, indus-
try, and policy makers.
The content of the book is based on contributions from researchers on
the CloudLightning project, a European Union project funded under
Horizon 2020 (www.cloudlightning.eu). CloudLightning commenced in
2015 and brought together eight project partners from five countries
across Europe to create a new way to provision heterogeneous cloud
resources to deliver services, specified by the user, using a bespoke service
description language. The goal of CloudLightning is to address energy
inefficiencies, particularly in the use of resources, and consequently to
deliver savings to the cloud service provider and the cloud consumer in
v
vi PREFACE
terms of reduced power consumption and improved service delivery, with
hyperscale systems particularly in mind. This book is an output of this
joint research.
The chapters in the book are organised around key research contribu-
tions from CloudLightning. Chapter 1 provides a context for HPC and the
cloud, and discusses how heterogeneous cloud computing might provide
a solution for certain classes of HPC users. While heterogeneous resources
can help address performance concerns of HPC users, it also introduces
complexity into an already complex feature space. As such, Chapter 1 also
introduces four key design principles used by CloudLightning to address
complexity—emergent behaviour, self-organisation, self-m anagement, and
the separation of concerns. Chapter 2 presents CloudLightning, a novel
heterogeneous cloud computing architecture. Chapters 3 and 4 outline
how approaches to resource management, based on self-organisation, self-
management, and separation of concerns, help to manage the complexity
of the heterogeneous cloud. HPC users are not the only stakeholders
whose needs must be met. While HPC users require performance at orders
of magnitude greater than the norm, modern cloud service providers
require scalability at so-called hyperscale. Chapter 5 discusses the chal-
lenges of evaluating the performance of heterogeneous cloud computing
architectures at hyperscale and presents a simulation of the proposed solu-
tion. The book concludes with a brief discussion of the disruptive potential
of the CloudLightning approach both for high performance computing
and for hyperscale cloud computing in general.
Dublin, Ireland Theo Lynn
Cork, Ireland John P. Morrison
Dublin, Ireland David Kenny
a
cknowledgements
This book was partially funded by the European Union’s Horizon 2020
Research and Innovation Programme through the CloudLightning proj-
ect (http://www.cloudlightning.eu) under Grant Agreement Number
643946, and by the Irish Centre for Cloud Computing and Commerce,
an Enterprise Ireland and IDA funded technology centre.
vii
c
ontents
1 Addressing the Complexity of HPC in the Cloud:
Emergence, Self-Organisation, Self-Management,
and the Separation of Concerns 1
Theo Lynn
2 Cloud Architectures and Management Approaches 31
Dapeng Dong, Huanhuan Xiong, Gabriel G. Castañe, and
John P. Morrison
3 Self-Organising, Self-Managing Frameworks and Strategies 63
Huanhuan Xiong, Christos Filelis-Papadopoulos, Gabriel G.
Castañe, Dapeng Dong, and John P. Morrison
4 Application Blueprints and Service Description 89
Ioan Dragan, Teodor-Florin Fortiș, Marian Neagul, Dana
Petcu, Teodora Selea, and Adrian Spataru
ix
x CONTENTS
5 Simulating Heterogeneous Clouds at Scale 119
Christos K. Filelis-Papadopoulos, Konstantinos M.
Giannoutakis, George A. Gravvanis, Charalampos S.
Kouzinopoulos, Antonios T. Makaratzis, and Dimitrios
Tzovaras
6 Concluding Remarks 151
Theo Lynn and John P. Morrison
Index 157
