Smarter objects: programming physical objects with AR technology

122 Pages·2014·20.56 MB·English

Smarter Objects:
Programming Physical Objects with AR Technology
by
Valentin Markus Josef Heun
Diplom Designer MASSACHUSETTS NTrfdtE
OF TECHNOLOGY
Bauhaus Universitat Weimar, Germany, 2009
JUL 14 2014
Submitted to the
Program in Media Arts and Sciences,
LIBRARIES
School of Architecture and Planning,
in partial fulfillment of the requirements of the degree of
Master of Science in Media Arts and Sciences
at the Massachusetts Institute of Technology
September 2013
C 2013 Massachusetts Institute of Technology. All Rights Reserved.
Signature redacted
Signature of Author:
Program in Media Arts and Sciences
August 9th, 2013
Signature redacted
Certified by:
Prof. Pattie Maes
Alex W. Dreyfoos Professor of Media Technology
Program in Media Arts and Sciences
Thesis Supervisor
Signature redacted
Accepted by:
Prof. Pattie Maes
Alex W. Dreyfoos Professor of Media Technology
Associate Academic Head
Program in Media Arts and Sciences
2
Smarter Objects:
Programming Physical Objects with AR Technology
Valentin Heun
Submitted to the Program in Media Arts and Sciences,
School of Architecture and Planning
on August 9th, 2013, in partial fulfillment of the requirements
for the degree of Master of Science in Media Arts and Sciences.
Abstract
This thesis describes a system that associates a virtual object with each physical object.
These objects are connected to support a flexible and easy means of modifying the interface
and behavior of a physical object. It also supports the creation of simple interactions with
other "smarter" objects to create more complex functionality. As a user interacts with a
physical object, an augmented reality interface visualizes the virtual object on top of the
physical object and gives the user an intuitive graphical interface for modifying the object's
interface and behavior or defining how it relates to other objects. As such, the system
provides an innovative interface for programming the behavior of physical objects.
The contribution of the thesis lies in the meaningful separation of physical and virtual
object interactions, a concept of direct mapping and the interprogrammability of Smarter
Objects. The thesis presents the employment of Smarter Objects from all aspects and
discusses a working implementation as well as several usage scenarios demonstrating
potential applications. Finally, the thesis reflects on the implications of this approach for
industrial design.
Thesis Supervisor:
Prof. Pattie Maes
Alex W. Dreyfoos Professor of Media Technology
Program in Media Arts and Sciences
3
4
Smarter Objects:
Programming Physical Objects with AR Technology
by
Valentin Markus Josef Heun
The following people served as readers for this thesis:
Signature redacted
Thesis Reader:
Hiroshi Ishii
Jerome B. Wiesner Professor of Media Arts and Sciences
Program in Media Arts and Sciences
/I
Signature redacted
Thesis Reader:
Neri Oxman
SONY Corporation Career evelopment Professor
Program' Media Arts and Sciences
Signature redacted
Thesis Reader:
David A. Smith
Chief Innovation Officer and LM Senior Fellow
Lockheed Martin Global Training and Logistics (GTL)
5

Acknowledgements
Throughout my life, many people helped me form the mind and character that is able to
write this thesis today. For their friendship, mentorship and community, I want to
acknowledge these people.
Thanks:
To Pattie Maes, my advisor, for seeing both an designer and an engineer in me, supporting
me with full freedom and great leadership to explore important concepts and ideas that
otherwise would not have been explorable.
To my readers Hiroshi Ishii, Neri Oxmen and David Smith for strong thoughts, vision and
discussion that made me dig deeper into my vision and concepts. I am honored to learn
from your remarkable perspectives.
To Shunishi Kasahara for all the creative collaboration, talks, support, great friendship and
for working together to create the foundation that eventually lead to this thesis.
To James Hobin for his assistance writing the code for the prototype. His expertise in
mathematics and object-oriented programming has been crucial for achieving good results.
To Leonard R. Bacich for teaching me about his secrets of happiness and for helping me to
trust my talents.
To Prof. Wolfgang Sattler for opening the doors that gave me opportunities to think and
shift major directions in my life to finally pursue studies in Media Arts and Science.
7
To the MIT Media Lab, Bauhaus University and the Pratt Institute communities for helping
me discover my talents and the Palomar5 community for letting me grow beyond the sky.
To Anissa Talantikite for never ending support, love and all the happiness that one never
could write in such acknowledgements about.
To my family Christof, Jutta, Katharina, Kristina, Franziska, Frank, Jan, Florian, Leo, Kaya,
Luke, my uncle Winfried and everyone in my greater family.
To all my friends.
8
Table of Contents
1. Introduction and Motivation 11
2. Definitions 15
2.1. Object 15
2.2. Smarter Object 16
2.3. Virtual 17
2.4. Physical 17
3. Related Work 18
3.1. Ubiquitous Computing 19
3.2. Internet of Things 20
3.3. Augmented Reality 21
4. Design 26
4.1. How do we use Physical Objects? 26
4.2. Direct Mapping 29
4.3. Interaction with Smarter Objects 31
4.4. Data-Flow Interface 33
4.5. Elements of the Visual Design 35
4.6. Benefits of Smarter Objects 39
5. Technological Implementation 43
5.1 Overview 43
5.2. Interaction Elements 44
5.3. Data Structure 45
5.4. Network 46
5.5. Server 47
5.6. Electronics 48
5.7. Graphical User Interface for the Reality Editor 52
6. Prototypes 55
9
6.1. Markers 55
6.1.1. Natural Materials 61
6.2. Radio and Speaker 68
6.3. Overlapping Interaction 73
6.4. Memory Bar 74
6.5. Lights 77
6.6. Mental Object Model 79
6.7. Data-Flow Prototype 81
6.8. Improving the Data-Flow Prototype 82
7. Use Case Scenarios 85
7.1. Waking Up in My Home 85
7.2. Sunday Morning Lawn Care 88
7.3. Watching the Game 90
7.4. My Surgery in the Hospital 92
8. Evaluation and Discussion 97
8.1. Critique and Evaluation by Professional Designers 97
8.2. Communicating virtual connections 100
8.3. Connected Complexity 103
8.4. Physical Gestures 104
8.5. Markers in Everyday Design 106
8.6. Impact on Industrial Design 108
8.7. What I have Learned 109
9. Conclusions & Future Work 111
9.1 Function 111
9.2. Technology 112
9.3. Design 112
9.4. Gestural Interfaces 113
9.5. Interaction 113
9.6. Vision for Future Work 114
10. References 116
10