vision-based detection and distance estimation of micro unmanned aerial vehicles a thesis ...

119 Pages·2015·8.34 MB·English

VISION-BASEDDETECTIONANDDISTANCEESTIMATIONOFMICRO
UNMANNEDAERIALVEHICLES
ATHESISSUBMITTEDTO
THEGRADUATESCHOOLOFNATURALANDAPPLIEDSCIENCES
OF
MIDDLEEASTTECHNICALUNIVERSITY
BY
FAT˙IHGÖKÇE
INPARTIALFULFILLMENTOFTHEREQUIREMENTS
FOR
THEDEGREEOFDOCTOROFPHILOSOPHY
IN
COMPUTERENGINEERING
SEPTEMBER2015

Approvalofthethesis:
VISION-BASEDDETECTIONANDDISTANCEESTIMATIONOFMICRO
UNMANNEDAERIALVEHICLES
submittedbyFATI˙HGÖKÇEinpartialfulfillmentoftherequirementsforthedegree
of Doctor of Philosophy in Computer Engineering Department, Middle East
TechnicalUniversityby,
Prof. Dr. GülbinDuralÜnver
Dean,GraduateSchoolofNaturalandAppliedSciences
Prof. Dr. AdnanYazıcı
HeadofDepartment,ComputerEngineering
Prof. Dr. GöktürkÜçoluk
Supervisor,ComputerEngineeringDepartment,METU
ExaminingCommitteeMembers:
Assoc. Prof. Dr. ErolS¸ahin
ComputerEngineeringDepartment,METU
Prof. Dr. GöktürkÜçoluk
ComputerEngineeringDepartment,METU
Assist. Prof. Dr. SinanKalkan
ComputerEngineeringDepartment,METU
Assist. Prof. Dr. A.Bug˘raKoku
MechanicalEngineeringDepartment,METU
Assist. Prof. Dr. KutlukBilgeArıkan
MechatronicsEngineeringDepartment,AtılımUniversity
Date:
I hereby declare that all information in this document has been obtained and
presented in accordance with academic rules and ethical conduct. I also declare
that,asrequiredbytheserulesandconduct,Ihavefullycitedandreferencedall
materialandresultsthatarenotoriginaltothiswork.
Name,LastName: FAT˙IHGÖKÇE
Signature :
iv
ABSTRACT
VISION-BASEDDETECTIONANDDISTANCEESTIMATIONOFMICRO
UNMANNEDAERIALVEHICLES
Gökçe,Fatih
Ph.D.,DepartmentofComputerEngineering
Supervisor : Prof. Dr. GöktürkÜçoluk
September2015,95pages
In this thesis, we study visual detection and distance estimation of Micro Unmanned
Aerial Vehicles (mUAVs), a crucial problem for (i) intrusion detection of mUAVs in
protected environments, (ii) sense and avoid purposes on mUAVs or on other aerial
vehicles and (iii) multi-mUAV control scenarios such as environmental monitoring,
surveillance and exploration. The problem is challenging since (i) a real-time so-
lution is required, a burden when computational power is limited by the hardware
carried by an mUAV, (ii) non-convex structure of the mUAVs causes the bounding
box of mUAVs to include very different background patterns, (iii) background pat-
terns from indoor or outdoor are very complex with different characteristics and can
includemovingobjects,(iv)mUAVstiltandrotateunavoidablyresultinginverylarge
changes in their appearances, (v) when the camera is not stationary, motion blur is a
problem, and (vi) illumination direction and brightness changes cause different im-
ages. We evaluate vision algorithms for this problem, since other sensing modalities
limittheenvironmentorthedistancebetweenthemUAVs. WetestHaar-likefeatures,
Local Binary Patterns (LBP) and Histogram of Gradients (HOG) using boosted cas-
caded classifiers. We also integrate a distance estimation method utilizing geometric
cues with Support Vector Regressors. We evaluated each method on indoor and out-
doorvideoscollectedsystematicallyandonvideoswithmotionblur. Ourexperiments
show that, using boosted cascaded classifiers with LBP, near real-time detection and
v
distanceestimationofmUAVsarepossibleinabout60msindoors(1032×778reso-
lution) and 150 ms outdoors (1280×720 resolution) per frame, with a detection rate
of0.96F-Score. However,classifiersofHaar-likefeaturesleadtobetterdistanceesti-
mationsincetheypositiontheboundingboxesonmUAVsmoreaccurately. Ourtime
analysisyieldsthatclassifiersofHOGtrainandrunfasterthantheotheralgorithms.
Keywords: micro UAV, computer vision, detection, distance estimation, cascaded
boostedclassifiers,Haar-likefeatures,LBP,HOG
vi
ÖZ
M˙IKRO˙INSANSIZHAVAARAÇLARININB˙ILG˙ISAYARLIGÖRMETABANLI
ALGILANMASIVEMESAFEKEST˙IR˙IM˙I
Gökçe,Fatih
Doktora,BilgisayarMühendislig˘iBölümü
TezYöneticisi :Prof.Dr.GöktürkÜçoluk
Eylül2015,95sayfa
Bu tezde, mikro insansız hava araçlarının (m˙IHA) görsel olarak algılanması ve me-
safekestirimiüzerindeçalıs¸tık.Buproblem,korunmasıgerekenalanlaraizinsizgiren
mUAV lerin algılanması, m˙IHA lar veya dig˘er hava araçlarının algılama ve kaçınma
sistemleri ve çoklu m˙IHA kontrol senaryoları için önemlidir. S¸u nedenler bu prob-
lemi zorlas¸tırmaktadır: (i) Gerçek zamanlı bir çözüm gerekmektedir. m˙IHA ların ta-
s¸ıyabilecekleriyle kısıtlı donanımlar düs¸ünüldüg˘ünde bu oldukça zordur. (ii) m˙IHA
ların konveks olmayan yapıları sebebiyle, m˙IHA ları içine alan görüntü pencere-
sinde, deg˘is¸ik arka plan görüntüleri de bulunur. (iii) Arka plan görüntüleri karma-
s¸ıktır ve hareketli nesne içerebilir. (iv) m˙IHA ların eg˘ilmeleri ve dönmeleri görü-
nümlerinideg˘is¸tirir.(v)Kamerasabitdeg˘ilse,hareketbulanıklıg˘ıbirproblemdir.(vi)
Aydınlatmayönündekiveparlaklıktakideg˘is¸iklikler,görüntülerdebüyükfarklılıklara
sebep olur. Dig˘er yöntemler, ortam ve mesafeyi sınırlandırdıg˘ı için, problemimizin
çözümünde görsel verilerin kullanımını deg˘erlendirdik. Bu amaçla, HAAR benzeri
öznitelikler,lokalikiliörüntü(LBP)veyönlügradyanhistogramları(HOG)yöntem-
lerini kademeli sınıflandırıcılarla test ettik. Sistemimize aynı zamanda, mesafe kes-
tirimi için destek vektör regresörü tabanlı bir yöntemi ekledik. Herbir yöntemi, iç
ve dıs¸ ortamlarda sistematik s¸ekilde topladıg˘ımız görüntülerle ve hareket bulanık-
lıg˘ı içeren görüntülerle test ettik. Testlerimiz, kademeli sınıflandırıcıların LBP ile
kullanımıyla gerçek zamanlı çalıs¸maya yakın hızda (iç ortam:60ms@1032×778,
vii
dıs¸ ortam:150ms@1280×720) ve yüksek hassasiyette (0.96 F-ölçütü) algılama ve
mesafe kestiriminin mümkün oldug˘unu göstermektedir. HAAR benzeri öznitelikle-
rin kullanımı, m˙IHA ların görüntü içerisinde bulundug˘u alanı daha hassas s¸ekilde
konumlandırdıg˘ı için, daha iyi mesafe kestirimi sag˘lamaktadır. Zaman analizlerimiz
HOGyöntemininög˘renmeveçalıs¸mazamanlarıaçısındandig˘eralgoritmalardandaha
hızlıçalıs¸tıg˘ınıgöstermektedir.
AnahtarKelimeler:mikro˙IHA,bilgisayarlıgörme,algılama,mesafekestirimi,kade-
melikuvvetlendirilmis¸ sınıflandırıcılar,Haar-benzeriöznitelikler,LBP,HOG
viii
TomyoneandonlyYasemin,oursonMustafaandourfamilies
ix
ACKNOWLEDGMENTS
First of all, I want to express my immeasurable appreciation and sincere gratitude to
ErolS¸ahin,GöktürkÜçolukandSinanKalkan(Inalphabeticalorderbynames,since
I cannot make any separation among them.) without whom this study could not be
possible. I am deeply thankful to Erol S¸ahin for his support, guidance, encourage-
ment, invaluable comments and especially for preventing me from getting lost. I am
extremely grateful to my supervisor Göktürk Üçoluk for his guidance, supervision
and pushing me always forward with his immense wisdom; for helping me gener-
ously whenever I needed in every aspect of the study, especially for his great support
during the outdoor experiments we performed starting very early in the mornings;
and most importantly for trusting me more than myself even at the hardest times. I
am immensely thankful to Sinan Kalkan for his never-ending support; for his guid-
ance at every detail of this study with his great expertise; for his visionary advices
when I faced struggles; for devoting his invaluable time generously for me whenever
I needed; for keeping me fresh and strong with his encouragement. I want to also
express my special thanks to these three great people for creating KOVAN Research
Laboratoryandforgivingmethechancetoworkwiththem.
I am very thankful to my committee members A. Bug˘ra Koku and Kutluk Bilge
Arıkanfortheirsupports,suggestions,valuablecommentsandencouragement.
I would like to express my special thanks to two great people: Hande Çelikkanat, for
alwaysbeingthesourceofenergyinthelaboratory,forlendingahandtomewithher
expertise whenever I need, for helping me during indoor and outdoor experiments,
and for being a great companion during the long hour studies in the laboratory. Ali
Emre Turgut, for his incredibly generous helps and supports and for being one of the
biggest source of hope in my life. Great thanks to you for being such nice symbolic
personsoftruefriendship.
I owe a debt of gratitude to Fatos¸ Tünay Yarman Vural for her supports and encour-
agementthroughoutmyeducationinMETU.
IamverygratefultoallfacultymembersoftheDepartmentofComputerEngineering
of METU, especially to Adnan Yazıcı, Uluç Saranlı, Attila Özgit, ˙I. Hakkı Toroslu,
Volkan Atalay, Cevat S¸ener, Veysi ˙Is¸ler, Selim Temizer, Sibel Tarı, Ahmet Ços¸ar,
˙IsmailSengörAltıngövde,AhmetOg˘uzAkyüz,AliHikmetDog˘ru,HalitOg˘uztüzün,
Faruk Polat, Tolga Can, Onur Tolga S¸ehitog˘lu, Pınar Karagöz and Murat Manguog˘lu
for helping me whenever I needed and for always encouraging me. It was a great
x