Table Of ContentCamera-aided Inertial Navigation for
Unmanned Aerial Vehicles using
Non-linear Observers
LORENZO FUSINI
DOCTORAL THESIS
SUBMITTED FOR THE PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
NTNU
NorgesTekniskNaturvitenskapeligeUniversitet
NorwegianUniversityofScienceandTechnology
ThesisfortheDegreeofDoctorofPhilosophy
FacultyofInformationTechnology,Mathematics,andElectricalEngineering
DepartmentofEngineeringCybernetics
c 2017LorenzoFusini
�
ISBN978-82-326-2522-2(printedversion)
ISBN978-82-326-2523-9(electronicversion)
ISSN1503-8181
ITKReport2017-17-W
ThesisatNTNU,2017:225
PrintedinNorwaybySkipnesKommunikasjonAS,Trondheim
Tomymum
Summary
The objective of this thesis is to investigate solutions for a robust autonomous navigation
system for a fixed-wing UAV. Robustness requires the ability to detect faults in the system
and to compensate for the consequent deviations from the nominal behaviour. The specific
aspect studied here is sensor redundancy, which is the inclusion of additional sensors whose
measurements can constitute backup information in case some other measurement becomes
unreliable.
Thefocusisoninertialnavigationofafixed-wingunmannedaerialvehicleusinglow-cost
micro-electro-mechanical-system (MEMS) inertial sensors and machine vision. The infor-
mation coming from the sensors is fused by means of stand-alone non-linear observers and
combinations of non-linear observers and the linearized Kalman filter. The observer designs
are based on a previous work that combined an attitude observer with a translational motion
observer using a feedback interconnection, which was proved to be globally exponentially
stable. Stabilityproofsaregivenforsomeoftheproposedobservers;allobserversaretested
onexperimentaldata.
Machine vision is used to calculate first the optical flow from a series of images, and
second the body-fixedvelocity of the UAV.Two different methods forobtaining the velocity
from the optical flow are investigated. With respect to previous works machine vision can
replace the magnetometer, a sensor that suffers from various disturbances when surrounded
by metal and electric circuits. The estimation of position and linear velocity is aided by a
global navigation satellite system, but when this is not available it is necessary to rely on
dead reckoning. This thesis also includes contributions towards this aspect of navigation,
proving that the velocity calculated through machine vision and non-linear observers helps
reducethegrowthrateofthedeadreckoningpositionerror.
Non-linear observers and the non-linear extensions of the Kalman filter have advantages
anddisadvantages. Arecentstateestimator,referredtoasexogenousKalmanfilter,isformed
as a cascade of a non-linear observer and a linearized Kalman filter, and is able to retain the
advantages of each component while discarding their shortcomings. In this thesis it is tested
for the first time on experimental data and compared with a stand-alone non-linear observer
andanextendedKalmanfilter,providingfurtherinsightintothisnovelstateestimationalgo-
rithm.
The work is organized as a compendium of scientific papers. It includes 3 conference
papers, 2 journal papers, and 1 book chapter. Two additional papers are mentioned, but are
outsidethescopeofthisthesis.
i
Preface
Thisthesisissubmittedinpartialfulfillmentoftherequirementsforthedegreeofphilosophiae
doctor (PhD) at the Norwegian University of Science and Technology (NTNU). The work
presented in this thesis is the result of the research carried out at the Centre for Autonomous
Operations and Systems (NTNU AMOS), Department of Engineering Cybernetics (ITK),
NTNU, under the supervision of Professors Thor I. Fossen and Tor Arne Johansen. The
projectwasfinancedbytheResearchCouncilofNorway,grantsnr. 223254and221666.
Acknowledgements
First of all I would like to thank my supervisor Professor Thor I. Fossen, who gave me the
chance to move to Trondheim, to work with him, and to experience what it feels like to live
in the vibrant environment of ITK. After offering me the PhD position, five years ago, his
next question was ’Do you play football?’... I took this as a good omen. Besides, it was
three days before my 30th birthday, so I also took this offer as a birthday gift. Thank you
for this wonderful gift, Thor! My gratitude goes to my co-supervisor Professor Tor Arne
Johansen too, whose uncanny ability to detect the smallest errors in every paper draft and
idea is a tad depressing but, at the same time, sought after: I know that after following his
precious advices every new paper becomes a gem. I feel like I must also thank the plural
entity composed by both of them, known as Thor & Tor, TIF & TAJ, TNT, or sometimes
simply The Guys: a volcano in perennial activity fed by two chambers of ideas, a constant
eruption of plans, designs, and solutions. It was a true pleasure and honour to be part of this
wondrousengine.
Theexperimentalpayloadwouldnotberealwithouttherestofthenavigationteam,also
known as the Ninja Team, my friends and colleagues Kasper Trolle Borup, Jakob Mahler
Hansen, and Sigur Mørkved Albrektsen: we had laughter, frustration, elation, irritation,
anger, and go figure what else during that long period, but I cherish every single moment
spenttogether,andIthankyouforallthat. Thecodeformachinevision,usedthroughoutmy
PhD,existsonlybecauseofJesperHosenandHa˚konHagenHelgesen: youdidinonemonth
whatIprobablywoulddoinsix,sothankyouforhavingbeensocool.
The department, ITK. I do not know how to begin to extol this group. It feels like it has
its own temperate social microclimate, always pleasant, never extreme. Perhaps I can start
with the administrative staff, without which the pain of bureaucracy would have been excru-
ciating: thank you for your invaluable assistance and for your radiant smiles Tove Kristin
Blomset Johnsen, Eva Amdahl, Unni Johansen, Bente Seem Lindquist, and Janne Karin Ha-
gen. Then the technical staff from the workshop, which helped build the sensor payload:
Torkel Hansen, Stefano Bertelli, Terje Haugen, Per Inge Snildal, and John Olav Horrigmo,
thankyouforyourpatienceandpreciousknowledge. Thecageballgroup,whichisthereason
ii
whyThoraskedmeaboutfootballthatfatefuldayfiveyearsago: thereisnothingbetterthan
ending the working week with 90 minutes of adrenaline, madness, success, failure, exhaus-
tion, and the looming fear of not being able to go home before hitting the hospital first... a
riskybusiness,butonewhichhelpedrelievethestressoftheweek,andIamsogladIcanstill
be part of it. And all my other friends and colleagues from ITK. I am not going to mention
each of you individually, for it would take forever, but you are the best thing I have received
during my PhD. These years would have been catastrophic without the many a happy mo-
ment together: the lunch and coffee breaks, the evenings out, the afternoons in the woods,
the barbecues, the many (yet too few) sessions of board gaming and video gaming, and the
variegateddiscussionsabouteverything. Youarelikefamily.
There are two special characters that have been part of the story of my PhD, permeating
everyaspectofitsplotandstoryline. Theydeserverecognitionfortheirvitalrole,especially
in those moments of hopelessness and lack of concentration all PhD candidates encounter
in their paths. Coffee, you have been a faithful companion, lightening the burden of early
mornings or seemingly endless days in the office. And chocolate, you have had the power
to raise my mood whenever it fell into the woeful pits of despair. To both of you, a heartfelt
thankyou.
Finally, to the pillars of my life. No day goes by without me thinking about my entire
family, my dad Sergio, my sister Costanza, and my brother Raffaele. It was not easy for me
to move away from you, to change country, lifestyle, habits and friends, and it was not easy
foryoutoletmego. Iloveyouall,thankyouforyourunwaveringsupport,andforaccepting
thedistancebetweenus. TomymumGigliola: youpassingleftadeepwoundthatwillnever
ever heal, but I take with me wherever I go all your teachings, principles and memories, and
I shall toil to ensure they will live on, as they moulded who I am today. And to my beloved
wifeIlze,myequilibrium,mysun. Youhavemadeeverythingeasierandmorebearable! You
have always been on my side, supportive, caring, encouraging, believing in me. Returning
home to you, unwinding in your arms after a hard day’s work, is simply the best feeling.
Thankyouforbeingwhoyouare. IloveyouIlze,maybeasmuchaschocolate.
iii
iv
Table of Contents
Summary i
Preface ii
TableofContents vi
ListofFigures vii
Abbreviations viii
1 Introduction 1
1.1 Backgroundandmotivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Stateestimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.2 Machinevisionandopticalflow . . . . . . . . . . . . . . . . . . . . 8
1.2 Listofpublications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3 Maincontributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.1 Magneticfieldreplacedbyvelocityfromopticalflow . . . . . . . . . 12
1.3.2 Noauxiliaryrollandpitchmeasurements . . . . . . . . . . . . . . . 13
1.3.3 Normalizedvelocityfromepipolargeometry . . . . . . . . . . . . . 14
1.3.4 Velocity-aideddeadreckoning . . . . . . . . . . . . . . . . . . . . . 14
1.3.5 TheeXogenousKalmanFilterforUAVnavigation . . . . . . . . . . 15
1.3.6 DeadreckoningwiththeXKF . . . . . . . . . . . . . . . . . . . . . 15
1.4 Thesisoutline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Kinematics,ModelsandNotation 17
2.1 Coordinateframes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Kinematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Sensormodels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
v
3 UAVExperiments 21
3.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Flighttests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4 ConclusionsandFutureWork 27
4.1 Possibleextensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5 Publications 31
5.1 Bookchapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.2 Journalpaper1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.3 Journalpaper2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.4 Conferencepaper1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.5 Conferencepaper2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.6 Conferencepaper3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Bibliography 135
vi
