Computational Approaches in the Transfer of Aesthetic Values from Paintings to Photographs : Beyond Red, Green and Blue

209 Pages·2018·14.28 MB·English

by  Chan

Xiaoyan Zhang · Martin Constable
Kap Luk Chan · Jinze Yu
Wang Junyan
Computational
Approaches in the
Transfer of Aesthetic
Values from Paintings
to Photographs
Beyond Red, Green and Blue
Computational Approaches in the Transfer
of Aesthetic Values from Paintings to Photographs
Xiaoyan Zhang Martin Constable
(cid:129)
Kap Luk Chan Jinze Yu
(cid:129)
Wang Junyan
Computational Approaches
in the Transfer of Aesthetic
Values from Paintings
to Photographs
Beyond Red, Green and Blue
123
Xiaoyan Zhang Jinze Yu
Collegeof Computer Science andSoftware Institute of Industrial Science
Engineering TheUniversity of Tokyo
Shenzhen University Tokyo
Shenzhen Japan
China
WangJunyan
Martin Constable University of SouthernCalifornia
RoyalMelbourneInstitute of Technology LosAngeles, CA
HoChiMinh USA
Vietnam
Kap LukChan
Tolendata SingaporeR&D CentrePvt. Ltd.
Singapore
Singapore
ISBN978-981-10-3559-3 ISBN978-981-10-3561-6 (eBook)
DOI 10.1007/978-981-10-3561-6
LibraryofCongressControlNumber:2017944299
©SpringerNatureSingaporePteLtd.2018
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerNatureSingaporePteLtd.
Theregisteredcompanyaddressis:152BeachRoad,#21-01/04GatewayEast,Singapore189721,Singapore
Colour is uncontainable. It effortlessly
reveals the limits of language and evades our
best attempts to impose a rational order on
it… To work with colour is to become acutely
aware of the insufficiency of language and
theory - which is both disturbing and
pleasurable.
David Batchelor, ‘Chromophobia’, 1997
Preface
Paintingsareassumedtobeaspecialclassofimageforthequalityoftheiraesthetic
appeal. In a painting, little is left to chance, with colour, form, composition and
suchlikeallbeinggivencarefulconsiderationbythepainter,whoisahighlytrained
individual. In this way, a painting offers an aesthetic experience that is unlikely to
be found in a photograph.
Newgenerationsofimagecapturedevices,suchasGoogleGogglesandthelight
field camera, promise a future in which the formal attributes of a captured scene
may be made available for editing to a degree that has hitherto been the exclusive
territoryofpainting.Inthissense,paintingsandphotographsareconverging,andit
therefore seems an opportune time to address them in comparison to each other.
This book details cutting-edge work demonstrating how some of the aesthetic
attributes of a painting may be transferred to a photograph using the latest com-
putational approaches.
The research described in this book was the result of a collaboration between
engineers and an artist and was a substantial learning experience for all. For the
artist,theengineeringcontributedanempiricaldimensiontotheclassicliteratureon
thetopic,mostofwhichhailsfromatimebeforeanimagecouldbesubjecttosuch
exact evaluation. For the engineers, the complex and multidimensional way that
artists fabricate images was a source of constant surprise.
For its emphasis on technical process, this book will be of primary appeal to
engineers.However,wecanimagineitalsobeingofrelevancetoartistsseekingan
empirical understanding of the aesthetic image.
A Quick Tour of the Contents
Part I (Part I on p. 1) discusses the image attributes and aesthetic aspects of
paintings. Introduced in Chap. 1 on p. 3 are the two painting genres that are the
focus of this book: the portrait and the landscape. In Chap. 2 on p. 9 the topic of
colour is introduced. This introduces the six structural contrasts that are the
vii
viii Preface
foundational core of our research: global contrast, local contrast, centre-corner
contrast,regionalcontrast,neighbouringregionalcontrastanddepth-awarecontrast.
In Chap. 3 on p. 33 is presented a summary of ways in which contrast may be
computationally modelled. In Chap. 4 on p. 39 the topic of composition is
addressed, with a special focus on vignetting.
Part II (Part II on p. 53) details a number of methods by which the aesthetic
values of paintings may be transferred to photographs. These are as follows:
(cid:129) Transfer of the global contrast (Chap. 5 on p. 55). In this early work, we
analyse the global hue, saturation and lightness contrasts of paintings by two
well-known artists: Vincent Van Gogh (Dutch 1853–1890) and Sir Lawrence
Alma-Tadema (English 1836–1912). The style of a painting is considered as a
high-level organisation of these contrasts. This organisation is transferred to a
photographicimagebyhistogrammatchingofsaturationandlightnessandalso
by dominant hue alignment and spread stretch along the RYB colour wheel.
(cid:129) Transfer of the atmospheric perspective effect (Chap. 6 on p. 71). The
atmosphericperspectiveeffectisanaturalphenomenon,relatingtotheeffectsof
atmospheric distortion upon the appearance of distant objects. We show that
artists exaggerate this phenomenoninorder toimprove thevisualappeal ofthe
paintingandtheillusionofdepth.Thecontrastmanipulationisformulatedasan
optimisation problem that simultaneously considers the desired inter-contrast,
intra-contrast and specified gradient constraints.
(cid:129) Transfer of regional contrast from portrait paintings (Chap. 7 on p. 107).
The function of a portrait is to draw attention of the viewer to the sitter’s face.
We show that painters serve this function through attention to the inter- and
intra-regional contrast values of the painting. In transferring this contrast
structurefrompaintingstophotographs,weemployanovelpiecewisenonlinear
transformation curve.
(cid:129) Transfer of the pose-related composition in portraits (Chap. 8 on p. 137).
Paintersexhibitsubstantialdiscretioninthemannerinwhichtheyframeafigure
within a portrait. Pose, face direction and space around the figure are the main
elements considered in portrait composition. A graph model is proposed that
describes these attributes, the purpose of which is to select a painting with
similar geometrical organisation. Space cropping technique is used to improve
the composition of the photograph by aligning its composition to that of the
selected painting.
(cid:129) Transfer of the vignetting effect (Chap. 9 on p. 155). Emulating the natural
fade-off of human vision, vignetting manifests as a darkening of the edges and
cornersofa painting, and a lightening of its centre.We compare thevignetting
effect in paintings with that of photographs. The observation from the analysis
showsthatthevignettingeffectinpaintingsismorepurposelyorganised.Based
on the observation, an algorithm is explored to transfer the lightness weighting
from an example painting to a similar photograph.
Preface ix
(cid:129) Defininghuecontrast(Chap. 10onp.179).Thischapteraddressesthethorny
subject of global hue contrast. There are well-understood methods to define
globallightnessandsaturationcontrast,whichallreferencesuchintensity-based
values asaverage, least and most. Hue cannot be understood in such terms and
isthereforenotdefinableusingthesameapproach.UsingasreferenceJohannes
Itten’scolourcontrasts,wepresentanapproachbywhichtheglobalhuecontrast
of an image may be quantified. It is particularly suitable to visualise the global
hue contrast of large sets of images. We apply this definition on sets of aes-
thetically related images, thereby expressing their high-level hue organisation.
(cid:129) Interactive local hue contrast (Chap. 11 on p. 191). Hue has an inherent
semanticdimension,thehueofanobjectfrequentlybeinglinkedtoitsmeaning.
For this reason, it is very perilous to edit the hue of an image. This chapter
proposes a way in which the hue of an image may be manipulated without its
semantic meaning being damaged. The colour of an object frequently presents
notasasinglehue,butratherarangeofhues.Henceagreentreemayconsistof
yellow-greens and blue-greens. Taking the cue from the traditional painters
practiceof‘breaking’theircolours,wepresentameansbywhichthisrangemay
be compressed or stretched. We develop a tool based on superpixels that can
interactively select a range of hues, to make them available for this
manipulation.
Shenzhen, China Xiaoyan Zhang
Ho Chi Minh, Vietnam Martin Constable
Singapore, Singapore Kap Luk Chan
Tokyo, Japan Jinze Yu
Los Angeles, USA Wang Junyan
2017
Acknowledgements
Wewouldliketoacknowledgeallthefantasticeditorsandtheirsupportstaff,who
worked those long hours and toiled so very hard on this book.
Thanks also to the Institute for Media Innovation, Nanyang Technological
University (NTU), Singapore, for providing the PhD grant to complete part of the
work. A special thanks to Prof. Michael S. Brown, of the National University of
Singapore(NUS)forallhissupportandguidance,andtoProf.MartinReiserforhis
support and valuable input.
xi
Contents
Part I Image Attributes and Aesthetic Aspects
1 The Portrait and Landscape Genres .... .... .... .... ..... .... 3
1.1 The Portrait Genre . ..... .... .... .... .... .... ..... .... 3
1.2 The Landscape Genre.... .... .... .... .... .... ..... .... 5
1.3 Concluding Remarks..... .... .... .... .... .... ..... .... 8
References. .... .... .... ..... .... .... .... .... .... ..... .... 8
2 The Colour Attributes of Paintings . .... .... .... .... ..... .... 9
2.1 Painters, Paint and Colour Systems . .... .... .... ..... .... 9
2.2 Colour Spaces. .... ..... .... .... .... .... .... ..... .... 10
2.3 Colour Contrast ... ..... .... .... .... .... .... ..... .... 16
2.3.1 Itten’s Contrasts.. .... .... .... .... .... ..... .... 17
2.3.2 Structural Contrasts... .... .... .... .... ..... .... 18
2.3.3 Hue and Harmony.... .... .... .... .... ..... .... 25
2.4 Concluding Remarks..... .... .... .... .... .... ..... .... 30
References. .... .... .... ..... .... .... .... .... .... ..... .... 31
3 Computational Models of Colour Contrast... .... .... ..... .... 33
3.1 Contrast Models for Simple Images . .... .... .... ..... .... 33
3.2 Contrast Models for Complex Images.... .... .... ..... .... 34
3.3 Contrast Models for Hue Contrast .. .... .... .... ..... .... 36
References. .... .... .... ..... .... .... .... .... .... ..... .... 37
4 The Geometric Attributes of Paintings .. .... .... .... ..... .... 39
4.1 Composition in Paintings . .... .... .... .... .... ..... .... 42
4.1.1 The Rule of Thirds ... .... .... .... .... ..... .... 42
4.1.2 Region of Interest and the Horizon... .... ..... .... 44
4.2 Vignetting.... .... ..... .... .... .... .... .... ..... .... 46
4.3 Concluding Remarks..... .... .... .... .... .... ..... .... 51
References. .... .... .... ..... .... .... .... .... .... ..... .... 51
xiii