Patient-specific anatomical illustration via model-guided texture

196 Pages·2012·43.26 MB·English

by  Ilknur Kaynar Kabul

Patient-specific anatomical illustration via
model-guided texture synthesis
Ilknur Kaynar Kabul
A dissertation submitted to the faculty of the University of North Carolina at Chapel
Hill in partial fulfillment of the requirements for the degree of Doctor of Philosophy in
the Department of Computer Science.
Chapel Hill
2012
Approved by:
Stephen M. Pizer
Russell Taylor
Julian Rosenman
Marc Niethammer
Svetlana Lazebnik
(cid:13)c 2012
Ilknur Kaynar Kabul
ALL RIGHTS RESERVED
ii
Abstract
ILKNUR KAYNAR KABUL: Patient-specific anatomical illustration via
model-guided texture synthesis.
(Under the direction of Stephen M. Pizer.)
Medical illustrations can make powerful use of textures to attractively, effectively, and effi-
ciently visualize the appearance of the surface or cut surface of anatomic structures. It can do
this by implying the anatomic structure’s physical composition and clarifying its identity and
3-D shape. Current visualization methods are only capable of conveying detailed information
about the orientation, internal structure, and other local properties of the anatomical objects
for a typical individual, not for a particular patient. Although one can derive the shape of
the individual patient‘s object from CT or MRI, it is important to apply these illustrative
techniques to those particular shapes. In this research patient-specific anatomical illustrations
are created by model-guided texture synthesis (MGTS).
Given 2D exemplar textures and model-based guidance information as input, MGTS uses
exemplar-based texture synthesis techniques to create patient-specific surface and solid tex-
tures. It consists of three main components. The first component includes a novel texture
metamorphosis approach for creating interpolated exemplar textures given two exemplar tex-
tures. This component uses an energy optimization scheme derived from optimal control prin-
ciples that utilizes intensity and structure information in obtaining the transformation. The
second component consists of creating the model-based guidance information, such as direc-
tions and layers, for that specific model. This component uses coordinates implied by discrete
medial 3D anatomical models (“m-reps”). The last component accomplishes exemplar-based
texture synthesis by textures whose characteristics are spatially variant on and inside the 3D
models. It considers the exemplar textures from the first component and guidance informa-
tion from the second component in synthesizing high-quality, high-resolution solid and surface
textures.
iii
Patient-specificillustrationswithavarietyoftexturesfordifferentanatomicalmodels,such
as muscles and bones, are shown to be useful for our clinician to comprehend the shape of the
models under radiation dose and to distinguish the models from one another.
iv
In honor of
my parents, Mukaddes and Ihsan Kaynar,
and
my husband, Onur Kabul
v
Acknowledgments
I would like to express my gratitude to people who have been important to this work and
to my life as a graduate student at UNC. So many people have helped and inspired me both
in research and in life and it has been a privilege and honor for me to be associated with all
these people.
FirstandforemostIamthankfultoStevePizer,mydoctoraladvisor. Withouthisguidance
and encouragement I do not think I would have been able to finish this dissertation. He
taught me scientific principles in the medical image analysis and visualization fields, helped
me to achieve all the doctoral milestones and inspired me with his outstanding passion and
dedication in research. It has been a privilege to be one of his doctoral advisees. He has been
a great mentor to me.
I am grateful to my committee members for their comments and suggestions. Their re-
search expertise and critical thinking were extremely valuable and helped me in clarifying
specific research aims at different stages of the research. Julian Rosenman has been a vision-
ary leader and a generous patron of the “Netterly Visualization” project. He has been actively
involved in my dissertation research and has been of great help in developing major ideas in
this dissertation. Marc Niethammer has lent me freely his valuable knowledge about image
registration for solving the texture metamorphosis in this dissertation. I want to thank Russell
Taylor for his critical comments and advices as a visualization researcher. He has been equally
constructive and helpful in numerous discussions regarding my dissertation research. I also
want to thank Svetlana Lazebnik for her feedback and advice.
The entire MIDAG group has been supportive and helpful over those years. In particular,
I want to thank Derek Merck for being a supportive and charitable partner on my dissertation
work. Last but not least, my office mate during the most productive years of my dissertation,
Xiaoxiao Liu , has been a very good companion at work. Positive energy in the office definitely
helped me to handle dissertation writing stress.
vi
I would like to thank my husband, Onur, and my daughter, Nilufer, for their invaluable
support during this long journey. My parents, Ihsan and Mukaddes Kaynar, and my sister,
Gulnur Unal, have all contributed hugely to this project by being unconditionally supportive
and boundlessly patient for what I chose to accomplish.
vii
Table of Contents
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Clinical Application Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Illustrative Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3 Overview of Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Thesis and Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.5 Overview of Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1 Texture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.1 Texture categories . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.2 Texture features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1.3 Sources of texture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.1.4 Applications of textures . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1.5 Texture mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.6 Texture layering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2 Texture Metamorphosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.3 Texture Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3.1 2D texture synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.1.1 Pixel-based texture synthesis . . . . . . . . . . . . . . . . 36
viii
2.3.1.2 Patch-based texture synthesis . . . . . . . . . . . . . . . . 39
2.3.1.3 Optimization-based texture synthesis . . . . . . . . . . . . 40
2.3.2 Feature-guided (controllable) texture synthesis . . . . . . . . . . . 41
2.3.3 Texture synthesis on surfaces . . . . . . . . . . . . . . . . . . . . . 42
2.3.3.1 Computation of guidance information on the surface . . . 43
2.3.3.2 Synthesizing texture on the surface . . . . . . . . . . . . . 44
2.3.4 Solid texture synthesis . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.3.4.1 Shape-independent solid textures . . . . . . . . . . . . . . 46
2.3.4.2 Shape-dependent solid textures . . . . . . . . . . . . . . . 48
2.3.4.3 Hybrid methods . . . . . . . . . . . . . . . . . . . . . . . 51
2.3.4.4 Solid texture synthesis for medical visualization . . . . . . 51
2.4 Object Representations in 3D Space . . . . . . . . . . . . . . . . . . . . . . 52
2.4.1 Surface representations . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.4.2 Volumetric (solid) representations . . . . . . . . . . . . . . . . . . . 54
2.4.2.1 Volumetric data structures . . . . . . . . . . . . . . . . . . 54
2.4.2.2 User-guided volumetric modeling . . . . . . . . . . . . . . 56
2.4.3 Model-based coordinate systems . . . . . . . . . . . . . . . . . . . . 57
2.4.3.1 Discrete medial representation . . . . . . . . . . . . . . . . 59
2.4.3.2 Single figure coordinate system . . . . . . . . . . . . . . . 59
3 Texture Metamorphosis . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.2 Image Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3 Texture Metamorphosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.4 Solving the Texture Metamorphosis Equation . . . . . . . . . . . . . . . . 70
3.4.1 Adjoint solution method . . . . . . . . . . . . . . . . . . . . . . . . 73
3.4.2 Combining forward and backward transitions
in metamorphosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.5 Metamorphosis for Color Textures . . . . . . . . . . . . . . . . . . . . . . . 79
3.6 Metamorphosis for Structured Textures . . . . . . . . . . . . . . . . . . . . 80
3.7 Results of Texture Metamorphosis Variants . . . . . . . . . . . . . . . . . . 83
ix
3.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.9 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.9.1 Optimality conditions . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.9.2 Source term . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4 Model-based Guidance Computation . . . . . . . . . . . . . . . . . . . 92
4.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.2 Model-based Coordinate Generation . . . . . . . . . . . . . . . . . . . . . . 94
4.3 Material Transition Field Computation . . . . . . . . . . . . . . . . . . . . 96
4.3.1 Adding randomness to the transition regions . . . . . . . . . . . . . 97
4.4 Scalar Field Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.5 Vector Field Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.5.1 Surface vector field . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.5.2 Volumetric vector field . . . . . . . . . . . . . . . . . . . . . . . . . 101
5 Model-based Surface Texture Synthesis . . . . . . . . . . . . . . . . . . 104
5.1 Surface Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.2 Projection of Surface Patches . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.3 Texture Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6 Model-based Solid Texture Synthesis . . . . . . . . . . . . . . . . . . . 117
6.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.2 Texture Features For Solid Texture Synthesis . . . . . . . . . . . . . . . . . 121
6.2.1 Textons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.2.2 Signed distance field . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.2.3 Medial representation of textons . . . . . . . . . . . . . . . . . . . . 122
6.2.4 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
x