development of intelligent unmanned aerial vehicles with effective sense and avoid capabilities a

193 Pages·2015·14.01 MB·English

DEVELOPMENT OF INTELLIGENT UNMANNED AERIAL VEHICLES
WITH EFFECTIVE SENSE AND AVOID CAPABILITIES
ANGZONGYAO,KEVIN
(B.Eng.(Hons.),NTU)
A THESIS SUBMITTED
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
ELECTRICAL AND COMPUTER ENGINEERING
NATIONAL UNIVERSITY OF SINGAPORE
2014
Declaration
I hereby declare that the thesis is my original work
and it has been written by me in its entirety. I have
duly acknowledged all the sources of information
which have been used in the thesis.
This thesis has also not been submitted for any
degree in any university previously.
Ang Zong Yao, Kevin
31st October 2014
i
ii
Acknowledgments
Firstandforemost,Iwillliketoexpressmyupmostandsinceregratitudetomysupervisor,Prof.
BenM.Chen,forhisacceptanceandmotivationtowardsmyPh.Dstudies. Heacceptedmeinto
my Ph.D studies even though I had been in the workforce for four years and have not been in
contact with much of academia. His patience and knowledge inspired me to push hard for my
studiesandtoachievethingsIcouldonlydreamabout.
I would also like to express my sincere thanks to my supervisors from Defense Science
Organization (DSO) National Laboratories, Dr Poh Eng Kee, Dr Chen Chang and Dr Rodney
Teo for their rich experience in national defence related projects and the guidance they offered
meduringmystudies.
Special thanks are given to Prof Wang Biao, Prof Luo Delin, Dr Dong Miaobo, Dr Peng
Kemao, Dr Lin Feng, Dr Cai Guowei and Dr Zhao Shiyu who are there whenever I had major
theoreticalproblemsinmyprojectthatneededsolutions. Theirperceptiveviewsareenlighten-
ingandgivemegreatmotivationtoimplementdifferenttechniquesinmyresearch.
DrZhengXiaolianandBaiLimiaowhoaremyofficematesprovidedmewithmuchadvice
and knowledge everyday. They make my day really enjoyable so much so that I have lots of
enthusiasmcomingtoworkeveryday.
I treat everyone in the NUS UAV Research Group as one big family and I will like to tell
themmymostheartfeltthanks. Theyaretheonesthatmademystudieseversoenjoyable. They
are Dr Dong Xiangxu, Dr Wang Fei, Phang Swee King, Cui Jinqiang, Li Kun, Lai Shupeng,
Liu Peidong, Ke Yijie, Wang Kangli, Pang Tao, Bi Yingcai, Li Jiaxin Qing Hailong and Shan
Mo. I will never forget all the international competitions that we have worked so hard for.
The competitions are namely, DARPA UAVForge 2012 Competition in Georgia, USA. AVIC
UAVGP2013inBeijing,China. IMAVCompetition2014inDelft,theNetherlands.
I have two colleagues who have currently left the National University of Singapore for a
brighter future in the USA and Canada. I will like to thank Ali Reza Partovi and Huang Rui
iii
whoIworkedwithinthedevelopmentofmyquadrotor.
Lastly,Ineedtothankmyparents,MrAngTaiePingandMrsTanHongHuay,andgirlfriend
Lin Jing who is always there for me when I needed support. They have worked really hard to
take care of me during these long stretch of time. Without their understanding, kindness and
caretheygaveme,itwouldbeextremelydifficulttofinishmyPh.Dstudies.
iv
Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 LiteratureReviewonNon-activeRangeSensingTechnologies . . . . . . . . . 2
1.2.1 StereoVision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.2 OpticalFlowTechniques . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.3 FeatureDetection,Description&Matching . . . . . . . . . . . . . . . 8
1.3 ContributionoftheThesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 PlatformDevelopment 13
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 PlatformSelection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.1 ESkyBigLamaCo-axialHelicopter . . . . . . . . . . . . . . . . . . . 15
2.2.2 AlignT-Rex450ConventionalHelicopter . . . . . . . . . . . . . . . . 15
2.2.3 XAircraftX650Quadrotorplatform . . . . . . . . . . . . . . . . . . . 16
2.2.4 RelatedWorkonQuadrotors . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 HardwareandSoftwareDevelopment . . . . . . . . . . . . . . . . . . . . . . 18
2.3.1 PlatformDesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.2 AvionicsSystemDesign . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 QuadrotorModeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4.1 QuadrotorFlightMechanism . . . . . . . . . . . . . . . . . . . . . . . 22
2.4.2 QuadrotorDynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.3 ModelParameterIdentification. . . . . . . . . . . . . . . . . . . . . . 28
2.4.4 StaticTests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.5 FlightExperiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5 QuadrotorModelwithGyro-in-the-Loop . . . . . . . . . . . . . . . . . . . . . 39
v
2.5.1 GyroinHoverMode . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.5.2 GyroinCruiseMode . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.5.3 ModelValidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.6 DevelopmentofanUnconventionalUAV. . . . . . . . . . . . . . . . . . . . . 47
2.6.1 DesignMethodology . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.6.2 DesignofU–Lion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.6.3 AdaptiveCenterofGravity . . . . . . . . . . . . . . . . . . . . . . . . 58
2.6.4 MaterialStressAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6.5 ElectronicConfiguration . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.6.6 ControlModes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.6.7 The2nd AVICCup-InternationalUAVInnovationGrandPrix . . . . . 66
2.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3 Vision-basedObstacleDetection 71
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.2 StereoTriangulationErrorModel. . . . . . . . . . . . . . . . . . . . . . . . . 72
3.2.1 StereoCorrelationErrors . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.2.2 StereoEstimationErrors . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.3 StereoVisionDepthMapEstimation . . . . . . . . . . . . . . . . . . . . . . . 79
3.4 MonocularVisionDepthEstimation . . . . . . . . . . . . . . . . . . . . . . . 81
3.4.1 MonocularDepthEstimationAlgorithmDesign . . . . . . . . . . . . . 82
3.4.2 MonocularDepthEstimationSimulationResults . . . . . . . . . . . . 85
3.5 StereoVisionGeneralObstacleDetection . . . . . . . . . . . . . . . . . . . . 88
3.5.1 StereoDepthMapGeneration . . . . . . . . . . . . . . . . . . . . . . 90
3.6 PowerLineDetection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.6.1 ConvolutionFiltering . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.6.2 LineSegmentDetector . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.6.3 LeastSquaresQuadraticCurveFitting . . . . . . . . . . . . . . . . . . 97
3.6.4 PowerlineDetectionResults . . . . . . . . . . . . . . . . . . . . . . . 99
3.7 Vision-basedObstacleTracking . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.7.1 CamShiftAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.7.2 CamShiftFormulation . . . . . . . . . . . . . . . . . . . . . . . . . . 103
vi
3.7.3 SimulationResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4 ActiveStereoVisioninNavigation 109
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.2 ActiveStereoVisionHardwareSetup . . . . . . . . . . . . . . . . . . . . . . 110
4.3 ActiveStereoVisionAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.3.1 FeatureExtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.3.2 FeatureMatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.3.3 LinearStereoTriangulation . . . . . . . . . . . . . . . . . . . . . . . 114
4.3.4 K-meansClustering . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.4 UrbanEnvironmentScenario . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5 Stereo-basedVisualNavigation 119
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.2 FeatureMatching&Calculating3DPoints. . . . . . . . . . . . . . . . . . . . 120
5.3 StereoOdometryMotionEstimation . . . . . . . . . . . . . . . . . . . . . . . 122
5.3.1 RigidMotionComputation . . . . . . . . . . . . . . . . . . . . . . . . 122
5.3.2 Perspective-n-PointsMotionEstimation . . . . . . . . . . . . . . . . . 124
5.4 RANSAC-basedOutlierRejection . . . . . . . . . . . . . . . . . . . . . . . . 126
5.5 Non-linearOptimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.6 PoseEstimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.7 KalmanFilterDesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.8 TransformationofPointsin3DSpace . . . . . . . . . . . . . . . . . . . . . . 132
5.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6 2DMapStitching 141
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.2 Homography-basedMotionModel . . . . . . . . . . . . . . . . . . . . . . . . 142
6.3 OnboardStitchingImplementation . . . . . . . . . . . . . . . . . . . . . . . . 144
6.3.1 FirstImageInitialization . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.3.2 KanadeLucasTomasi(KLT)FeatureDetectionandTracking . . . . . . 146
vii
6.3.3 HomographyCalculation,UpdatingandFailureChecks . . . . . . . . 146
6.3.4 WarpingandCroppingofImages . . . . . . . . . . . . . . . . . . . . 148
6.4 StitchingPerformance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
7 Conclusions&Futurework 155
7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.2 FutureWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
8 ListofAuthor’sPublications 169
viii
Summary
ThisPh.Dthesisdepictsthedevelopmentofunmannedaerialvehicles(UAV)inhardwaredesign
aswellassoftwarealgorithmdevelopment. ThemainUAVdevelopedisaquadrotorandithas
beenthoroughlymodeledandcontrolledthroughtheonboardsoftwaresystemusingourground
controlstation. TheUAVismountedwithanadvancedavionicssystemusedfornavigationand
astereocamerasystemusedforobstaclesensingandnavigationalenhancements.
Obstacledetectioncapabilitiesaremanifestedthroughvisual-basedalgorithmswhichallow
general obstacles and specific obstacles such as power lines to be sensed. The algorithms pro-
posedincludestereo-basedobstaclesensingthatiscapableofdetectingobstaclestoanaccuracy
of 10 cm within a 5 m range based on the camera’s field of view. Visual-based navigation is
also explored using visual odometry where the UAV’s pose estimate is obtained by a fusion of
visual-basedodometryestimatesandinertialmeasurementunit(IMU)readingsusingaKalman
filter. The proposed algorithm relies on the use of the Perspective-n-Points motion estimation
and is shown to be more reliable than the Rigid Motion Computation as it computes relative
motionestimationbasedonimagefeaturepointsandtheir3Dworldcoordinate. Thealgorithm
hasshowntoaccumulateanerroroflessthan5%ofthedistancetraveled.
An active stereo vision system has also been developed to operate in featureless environ-
mentssuchasindoorenvironments. Theactivestereovisionsystemmakesuseofalaseremit-
tertoprojectfeaturesintoanotherwisefeaturelessenvironment. Thestereosystemthentracks
these features and is able to generate a sparse 3D point cloud which could then be used for
obstacledetectionornavigationalpurposes.
Inthisthesis,novelideashavebeenimplementedinbothhardwareandsoftware. Inplatform
development, a hybrid reconfigurable UAV has been designed and built in hopes of having a
moreoptimalplatformtoachievenavigationinurbanenvironments. Itishopedthatthevisual-
based algorithms could be ported to such an unconventional platform. As the platform could
transform from a vertical VTOL form to that of a horizontal cruise form, having the vision
ix