Masters Thesis: High-Level Command Mapping for Multi-Robot Aerial Cinematography

122 Pages·2017·6.07 MB·English

by  Robert Durrant

High-Level Command Mapping
for Multi-Robot Aerial Cine-
matography
Robert Durrant
s
i
s
e
h
T
e
c
n
e
i
c
S
f
o
r
e
t
s
a
M
Cognitive Robotics

High-Level Command Mapping for
Multi-Robot Aerial Cinematography
Master of Science Thesis
For the degree of Master of Science in Systems and Control at Delft
University of Technology
Robert Durrant
November 23, 2018
Faculty of Mechanical, Maritime and Materials Engineering (3mE) · Delft University of
Technology
Copyright (cid:13)c Cognitive Robotics (CoR)
All rights reserved.
Delft University of Technology
Department of
Cognitive Robotics (CoR)
The undersigned hereby certify that they have read and recommend to the
Faculty of Mechanical, Maritime and Materials Engineering (3mE) for
acceptance a thesis entitled
High-Level Command Mapping for Multi-Robot Aerial
Cinematography
by
Robert Durrant
in partial fulfillment of the requirements for the degree of
Master of Science Systems and Control
Dated: November 23, 2018
Supervisor(s):
Dr. J. Alonso-Mora
Reader(s):
Prof.dr.ir D. Abbink
Dr. W. Pan

Abstract
AerialcinematographyhasseenanincreaseduseofUnmannedAerialVehicle(UAV)due
to technological advancements and commercialisation in recent years. The operation of
such a robot can be complex and requires a dedicated person to control it. Automation
of the cinematography allows for the use of multiple robots, which further increases
the complexity of performing cinematography. High-level command interpretation is
required to allow for an intuitive interface suited for an inexperienced user to control
such a system.
Natural Language (NL) is an intuitive interface method which allows a user to specify a
extensive range of commands. A Cinematographic Description Clause (CDC) is defined
to extract information from a processed NL command. A minimum input approach is
considered such that a user has to merely specify the number of robots and the peo-
ple to record, whereby the specification of a behaviour is optional. An environment is
considered in which up to three robots have to frame two people. Taking into account
their orientation, relative global location and the user command, a set of behaviours can
be determined based on cinematographic practices. Camera views and image param-
eters are determined through behaviour specific non-linear optimisations and assigned
to the robots using a Linear-Bottleneck Algorithm (LBA). A collision-free global path
is computed for each robot with an A∗ search algorithm. Finally, a Model Predictive
Control(MPC)determineslow-levelinputssuchthattheusercommandcanbeachieved.
Three situations are considered to validate the performance of the system given the
minimaluserinput. First,trackingofthedynamicorientationsofthepeopleisevaluated
for up to three robots, whereby camera positions are determined autonomously. Next,
dynamic motions of the two people through an environment highlight the limitations
of the system due to collision mitigation, mutual visibility and robot dynamics. An
extensiontomultiplesimultaneouscommandsincreasesthequantityofrobotsandpeople
that can be tracked. This allows for an assessment of the flexibility and scalability of
Master of Science Thesis Robert Durrant
ii
the proposed high-level command interpretation methodology.
Robert Durrant Master of Science Thesis
Table of Contents
Acknowledgements vii
1 Introduction 1
1-1 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1-2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1-3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1-4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Preliminaries 7
2-1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2-2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2-3 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3 Natural Language Processing 13
3-1 Syntax vs. Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3-1-1 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3-1-2 Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3-2 Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3-3 Spatial Description Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3-4 Cinematographic Description Clause . . . . . . . . . . . . . . . . . . . . . . 17
4 Cinematography 19
4-1 Taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4-2 Configuration: Single Person . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4-3 Configuration: Two People . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Master of Science Thesis Robert Durrant
iv Table of Contents
4-3-1 Master Shot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4-3-2 Parallel Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4-3-3 Face to Face Configuration . . . . . . . . . . . . . . . . . . . . . . . 27
4-3-4 Right Angle Configuration . . . . . . . . . . . . . . . . . . . . . . . 28
5 Set-points 31
5-1 Action Line & Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5-2 Target Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5-3 Placement Single Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5-4 Placement Two Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5-5 Placement Three Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5-6 Feasible Setpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6 Assignment & Global Planner 43
6-1 Hungarian Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6-2 Linear-Bottleneck Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6-3 Assignment Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6-4 Global Planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
7 Model Predictive Control 51
7-1 Cost Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
7-2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8 Results 57
8-1 General Set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
8-2 Simulation: Dynamic Target Orientation . . . . . . . . . . . . . . . . . . . . 59
8-3 Simulation: Dynamic Motion . . . . . . . . . . . . . . . . . . . . . . . . . . 71
8-4 Simulation: Multiple Commands . . . . . . . . . . . . . . . . . . . . . . . . 79
9 Conclusion 85
A Cinematographic Elaboration 89
B Simulation Supplement 91
B-1 MPC Weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
B-2 Visualisation tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
B-3 Simulation: Dynamic Target Orientation . . . . . . . . . . . . . . . . . . . . 93
B-4 Simulation: Dynamic Motion . . . . . . . . . . . . . . . . . . . . . . . . . . 93
B-5 Simulation: Multiple Commands . . . . . . . . . . . . . . . . . . . . . . . . 95
Robert Durrant Master of Science Thesis