Detection of Temporal Events and Abnormal Images

87 Pages·2013·5.43 MB·English

by  Nawarathna, Ruwan D.

DETECTION OF TEMPORAL EVENTS AND ABNORMAL IMAGES FOR QUALITY

ANALYSIS IN ENDOSCOPY VIDEOS

Ruwan D. Nawarathna

Dissertation Prepared for the Degree of

DOCTOR OF PHILOSOPHY

UNIVERSITY OF NORTH TEXAS

Augus t 2013

APPROVED:

JungHwan Oh, Major Professor
Bill Buckles, Committee Member
Yan Huang, Committee Member
Song Fu, Committee Member
Barrett Bryant, Chair of the Department of
Computer Science and Engineering
Costas Tsatsoulis, Dean of the College of
Engineering
Mark Wardell, Dean of the Toulouse Graduate
School
Nawarathna, Ruwan D. Detection of Temporal Events and Abnormal Images for Quality
Analysis in Endoscopy Videos. Doctor of Philosophy (Computer Science and Engineering),
August 2013, 77 pp., 14 tables, bibliography, 50 titles.
Recent reports suggest that measuring the objective quality is very essential towards
the success of colonoscopy. Several quality indicators (i.e. metrics) proposed in recent studies
are implemented in software systems that compute real-time quality scores for routine
screening colonoscopy. Most quality metrics are derived based on various temporal events
occurred during the colonoscopy procedure. The location of the phase boundary between the
insertion and the withdrawal phases and the amount of circumferential inspection are two such
important temporal events. These two temporal events can be determined by analyzing various
camera motions of the colonoscope. This dissertation put forward a novel method to estimate
X, Y and Z directional motions of the colonoscope using motion vector templates.
Since abnormalities of a WCE or a colonoscopy video can be found in a small number of
frames (around 5% out of total frames), it is very helpful if a computer system can decide
whether a frame has any mucosal abnormalities. Also, the number of detected abnormal
lesions during a procedure is used as a quality indicator. Majority of the existing abnormal
detection methods focus on detecting only one type of abnormality or the overall accuracies
are somewhat low if the method tries to detect multiple abnormalities. Most abnormalities in
endoscopy images have unique textures which are clearly distinguishable from normal textures.
In this dissertation a new method is proposed that achieves the objective of detecting multiple
abnormalities with a higher accuracy using a multi-texture analysis technique. The multi-texture
analysis method is designed by representing WCE and colonoscopy image textures as textons.
Copyright 2013
by
Ruwan D. Nawarathna
ii
ACKNOWLEDGEMENT
I must first acknowledge my parents for their unconditional love and continuous
support. I gratefully acknowledge my dissertation supervisor, Dr. JungHwan Oh, for guiding me
throughout this dissertation work. He has been an excellent teacher, mentor, a wonderful guide
and a truly helpful person. He awarded me with a research assistantship throughout the PhD
program which enabled me to concentrate on my research making significant contributions and
also to stay financially stable. I believe that this dissertation would not have been possible
without his support. I express my sincere thanks to the committee members of this
dissertation, Dr. Bill Buckles, Dr. Yan Huang, and Dr. Song Fu for monitoring my progress in
research and providing valuable suggestions. I am equally grateful to all my professors at the
Department of Computer Science and Engineering, University of North Texas who have trained
me in computer science. I present my sincere thanks to my lab mates Mr. Jayantha
Muthukudage and Mr. Ashok Dahal for their constant support and encouragement throughout
my research work. In addition, my two brothers, too, have been very supportive and have
contributed to my success throughout the PhD program. Finally, I would like to thank my wife
Lakshika. Her encouragement, support, quiet patience and resolute love were unquestionably
the bedrock upon which the past five years of my life have been built. Her tolerance of my
occasional rough moods is a testament in itself of her unyielding devotion and love.
iii
TABLE OF CONTENTS
Page
ACKNOWLEDGEMENT .................................................................................................................... iii
LIST OF TABLES .............................................................................................................................. vii
LIST OF FIGURES ........................................................................................................................... viii
CHAPTER 1 INTRODUCTION ............................................................................................................ 1
1.1 Colonoscopy .......................................................................................................................... 1
1.2 Wireless Capsule Endoscopy ................................................................................................. 2
1.3 Motivation ............................................................................................................................. 3
1.4 Problem 1: Real-time Temporal Event Detection in Colonoscopy ....................................... 4
1.5 Problem 2: Abnormal Image Detection in Endoscopy Videos .............................................. 5
1.6 Organization of the Dissertation ........................................................................................... 6
CHAPTER 2 RELATED WORK ............................................................................................................ 7
2.1 Related Work on Temporal Event Detection in Colonoscopy .............................................. 7
2.2 Related Work on Abnormal Image Detection in WCE and Colonoscopy Video ................... 9
CHAPTER 3 REAL-TIME TEMPORAL EVENT DETECTION IN COLONOSCOPY ................................. 11
3.1 Introduction to Real-time Temporal Event Detection in Colonoscopy using Camera
Motion Analysis......................................................................................................................... 11
3.2 Proposed Method ............................................................................................................... 13
iv
3.2.1 Frame Preprocessing ................................................................................................... 14
3.2.2 Motion Vector Generation using Optical Flow Block Matching .................................. 16
3.2.3 Camera Motion Estimation using Motion Vector Templates ...................................... 18
3.2.4 Estimating the Values of Camera Motions using the Motion Contribution Algorithm 23
3.2.5 Detection of Temporal Events using the Estimated Motions ...................................... 27
3.3. Experimental Setup and Results ........................................................................................ 30
3.3.1 Effectiveness of the Camera Motion Estimation by the Proposed Method................ 31
3.3.2 Effectiveness of the Real-time Phase Boundary Detection Algorithm ........................ 33
3.3.3 Effectiveness of the Real-time Spiral Score Calculation Algorithm ............................. 35
CHAPTER 4 ABNORMAL IMAGE DETECTION IN ENDOSCOPY VIDEOS .......................................... 37
4.1 Introduction to Abnormal Image Detection in WCE and Colonoscopy using Filter Banks
and Local Binary Pattern ........................................................................................................... 37
4.2. Proposed Method .............................................................................................................. 40
4.2.1 WCE and Colonoscopy Textures .................................................................................. 42
4.2.2 Training Stage 1: Textons and their Dictionary Generation ........................................ 43
4.2.3 Training Stage 2: Generating the Image Block Dictionary (IBD) .................................. 49
4.2.4 Detecting Stage: Evaluating Images for Abnormal Detection ..................................... 52
4.3. Experimental Setup and Results ........................................................................................ 54
4.3.1 Experiments with WCE Videos ..................................................................................... 58
v
4.3.2 Experiments with Colonoscopy Videos ........................................................................ 64
CHAPTER 5 DISCUSSION AND CONCLUSIONS ............................................................................... 69
BIBLIOGRAPHY .............................................................................................................................. 73

vi
LIST OF TABLES
Page
Table 3.1 Comparison of the three methods for camera motion estimation ............................. 32
Table 3.2 Real colonoscopy test sets ........................................................................................... 34
Table 3.3 Effectiveness of the proposed phase boundary detection method for test set 1 ....... 34
Table 3.4 Effectiveness of the proposed phase boundary detection method for test set 2 ....... 34
Table 3.5 Average execution time of the three methods ............................................................ 35
Table 4.1 Normal and abnormal textures of WCE and colonoscopy ........................................... 42
Table 4.2 Generated image block dictionaries (IBDs) .................................................................. 56
Table 4.3 Accuracy metrics .......................................................................................................... 57
Table 4.4 Images, image regions, and image blocks used for the WCE experiment ................... 58
Table 4.5 Average performance of the 36 WCE IBDs .................................................................. 60
Table 4.6 Evaluation of WCE abnormal image detection ............................................................ 64
Table 4.7 Images, image regions, and image blocks used for the colonoscopy experiment ...... 65
Table 4.8 Performance of 36 colonoscopy IBDs with polyp texture as the positive class .......... 66
Table 4.9 Evaluation of colonoscopy abnormal image detection ................................................ 68

vii
LIST OF FIGURES
Page
Figure 1.1. Sample colonoscopy images. ........................................................................................ 2
Figure 1.2. Sample WCE images. ..................................................................................................... 3
Figure 1.3. Illustration of the phase bounadry and the circumferential inspection of
colonoscopy. ................................................................................................................................... 5
Figure 3.1. Steps in the proposed temporal event detection method. ........................................ 14
Figure 3.2. Frame preprocessing. ................................................................................................. 16
Figure 3.3. Comparison of the motion vector generation algorithms. ......................................... 17
Figure 3.4. Generated motion vectors from a frame pair and their TMVs in the 4-template
approach. ...................................................................................................................................... 19
Figure 3.5. TMVs of perfect camera motions in the 4-template approach .................................. 20
Figure 3.6. A motion conflict in 4-template approach.................................................................. 21
Figure 3.7. Generated motion vectors from a frame pair and their TMVs in the 4-template
Approach ....................................................................................................................................... 22
Figure 3.8. TMVs of perfect camera motions in the 8-template approach. ................................. 24
Figure 3.9. Supporting regions of each template for each motion. ............................................. 25
Figure 3.10. Example of a total CM calculation ............................................................................ 27
Figure 3.11. A CDCM plot of a full colonoscopy video stream for phase boundary detection .... 28
Figure 3.12 Comparison of the spiral score between the existing and the new methods. ......... 36
Figure 4.1. Sample abnormal and normal images of WCE and colonoscopy. .............................. 38
Figure 4.2. A summary of the proposed abnormal image detection method. ............................. 41
viii
Figure 4.3. Filters in the LM filter bank. ........................................................................................ 43
Figure 4.4. A set of LBPs with four LBPs with different P and R values ........................................ 44
Figure 4.5. Extracted image regions for texton generation from each texture ........................... 46
Figure 4.6. Training stage 1: Texons and their dictionary generation. ......................................... 48
Figure 4.7. Two examples of abnormal image block extraction. .................................................. 49
Figure 4.8. Training stage 2: Process of generating the IBD. ........................................................ 51
Figure 4.9. Detecting stage of the proposed method. ................................................................. 53
Figure 4.10. Average G-mean and F-measure analysis of 36 WCE IBDs. ...................................... 60
Figure 4.11. Comparison of G-mean values of each texture for 36 WCE IBDs. ............................ 62
Figure 4.12. Comparison of F-measure values of each texture for 36 WCE IBDs. ....................... 63
Figure 4.13. Comparison of G-mean and F-measure values of polyp texture for each IBD. ........ 67

ix