Programming interactivity : a designer's guide to processing, Arduino, and openFrameworks

736 Pages·2009·7.07 MB·English

by  Joshua J Noble

Programming Interactivity
A Designer’s Guide to Processing, Arduino, and
openFrameworks
Joshua Noble
Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo
Programming Interactivity
by Joshua Noble
Copyright © 2009 Joshua Noble. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions
are also available for most titles (http://my.safaribooksonline.com). For more information, contact our
corporate/institutional sales department: [phone] or [e-mail].
Editor: Steve Weiss Indexer: Ellen Troutman Zaig
Production Editor: Sumita Mukherji Cover Designer: Karen Montgomery
Copyeditor: Kim Wimpsett Interior Designer: David Futato
Proofreader: Sumita Mukherji Illustrator: Robert Romano
Production Services: Newgen
Printing History:
July 2009: First Edition.
Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of
O’Reilly Media, Inc. Programming Interactivity, the image of guinea fowl, and related trade dress are
trademarks of O’Reilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as
trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a
trademark claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and author assume
no responsibility for errors or omissions, or for damages resulting from the use of the information con-
tained herein.
TM
This book uses RepKover™, a durable and flexible lay-flat binding.
ISBN: 978-0-596-15414-1
[M]
1247251884
Table of Contents
Preface ..................................................................... xv
Part I. Introductions
1. Introducing Interaction Design ............................................ 3
What This Book Is for 3
Programming for Interactivity 4
The Nature of Interaction 5
Messages and Interaction 7
Interfaces and Interaction 8
Languages of Interaction 10
Design and Interaction 12
Art and Interaction 13
Data Exchange and Exploration 15
Working Process 19
2. Programming Basics .................................................... 21
Why You’ll Read This Chapter More Than Once 22
The Nature of Code 22
Variables 23
Simple Types 24
Arrays 29
Casting 33
Operators 33
Control Statements 37
if/then 37
for Loop 38
while Loop 39
continue 40
break 40
Functions 41
iii
Defining a Function 41
Passing Parameters to a Method 42
Some Suggestions on Writing Functions 44
Overloading Functions 44
Objects and Properties 46
Scope 49
Review 50
3. Processing ............................................................ 53
Downloading and Installing Processing 54
Exploring the Processing IDE 54
The Basics of a Processing Application 56
The setup() Method 56
The draw() Method 57
The Basics of Drawing with Processing 60
The rect(), ellipse(), and line() Methods 60
RGB Versus Hexadecimal 62
The fill() Method 63
The background() Method 65
The line() Method 65
The stroke() and strokeWeight() Methods 65
The curve() Method 66
The vertex() and curveVertex() Methods 66
Capturing Simple User Interaction 67
The mouseX and mouseY Variables 68
The mousePressed() Method 69
The mouseReleased() and mouseDragged() Methods 70
The keyPressed and key Variables 73
Importing Libraries 77
Downloading Libraries 77
Loading Things into Processing 79
Loading and Displaying Images 79
Displaying Videos in the Processing Environment 81
Using the Movie Class 81
Reading and Writing Files 83
Running and Debugging Applications 85
Exporting Processing Applications 86
Conclusion 88
Review 89
4. Arduino .............................................................. 91
Starting with Arduino 92
Installing the IDE 93
iv | Table of Contents
Configuring the IDE 96
Touring Two Arduino Boards 97
The Controller 97
Duemilanove Versus Mini 97
Touring the Arduino IDE 102
The Basics of an Arduino Application 105
The setup Statement 106
The loop Method 106
Features of the Arduino Language 108
Constants 109
Methods 110
Arrays 111
Strings 112
How to Connect Things to Your Board 115
Hello World 117
Debugging Your Application 122
Importing Libraries 124
Running Your Code 126
Running Your Board Without a USB Connection 126
Review 127
5. Programming Revisited ................................................ 129
Object-Oriented Programming 129
Classes 130
The Basics of a Class 131
Class Rules 132
Public and Private Properties 133
Inheritance 135
Processing: Classes and Files 137
C++: Classes and Files 139
.cpp and .h 140
A Simple C++ Application 142
Pointers and References 144
Reference 146
Pointer 146
When to Use Pointers 147
Large Data Objects 148
Pointers and Arrays 149
When Are You Going to Use This? 150
Review 151
6. openFrameworks ..................................................... 153
Your IDE and Computer 154
Table of Contents | v
Windows 154
Mac OS X 155
Linux 155
Taking Another Quick Tour of C++ 156
Basic Variable Types 157
Arrays 157
Methods 158
Classes and Objects in C++ 159
Getting Started with oF 160
Touring an oF Application 166
Methods 166
Variables 168
Creating “Hello, World” 168
Drawing in 2D 171
Setting Drawing Modes 172
Drawing Polygons 174
Displaying Video Files and Images 176
Images 176
Video 178
Importing Libraries 180
ofxOpenCv 181
ofxVectorGraphics 181
ofxVectorMath 182
ofxNetwork 182
ofxOsc 182
Compiling an oF Program 183
Compiling in Xcode 183
Compiling in Code::Blocks 184
Debugging an oF Application 184
Using the printf Statement 184
Using the GNU Debugger 185
Using the Debugger in Xcode 186
Using the Debugger in Code::Blocks 188
Review 188
Part II. Themes
7. Sound and Audio ..................................................... 193
Sound As Feedback 194
Sound and Interaction 197
How Sound Works on a Computer 199
Audio in Processing 202
vi | Table of Contents
Instantiating the Minim Library 202
Generating Sounds with Minim 204
Filtering Sounds with Minim 208
Sound in openFrameworks 214
openFrameworks and the FMOD Ex Library 221
The Sound Object Library 228
The Magic of the Fast Fourier Transform 233
Physical Manipulation of Sound with Arduino 238
A Quick Note on PWM 239
Creating Interactions with Sound 242
Further Resources 242
Review 243
8. Physical Input ........................................................ 245
Interacting with Physical Controls 245
Thinking About Kinetics 246
Getting Gear for This Chapter 247
Controlling Controls 248
The Button As an Electrical Object 248
The Button As an Interactive Object 248
The Button As a Value in Code 248
Turning Knobs 249
The Dial As an Interactive Object 249
Potentiometers 249
Using Lights 251
Wiring an LED 252
Detecting Touch and Vibration 253
Reading a Piezo Sensor 254
Getting Piezo Sensors 255
Communicating with Other Applications 259
Sending Messages from the Arduino 262
openFrameworks 263
Detecting Motion 265
PIR Motion Sensor 265
Reading Distance 267
Reading Input from an Infrared Sensor 269
Understanding Binary Numbers 270
Binary Numbers 270
Bits and Bit Operations 271
Why Do You Need to Know Any of This? 273
Detecting Forces and Tilt 273
Introducing I2C 278
What Is a Physical Interface? 283
Table of Contents | vii
What’s Next 284
Review 286
9. Programming Graphics ................................................ 289
The Screen and Graphics 289
Seeing Is Thinking, Looking Is Reading 292
Math, Graphics, and Coordinate Systems 293
Drawing Strategies 296
Use Loops to Draw 296
Use Arrays to Draw 298
Draw Only What You Need 303
Use Sprites 303
Processing and Transformation Matrices 303
Creating Motion 307
Shaping the Gaze 308
Setting the Mood 308
Creating Tweens 310
Using Vectors 315
Using Graphical Controls 325
ControlP5 Library 326
Event Handling 326
Importing and Exporting Graphics 328
Using PostScript in Processing 329
Using PostScript Files in oF 330
What’s Next 333
Review 334
10. Bitmaps and Pixels .................................................... 337
Using Pixels As Data 338
Using Pixels and Bitmaps As Input 340
Providing Feedback with Bitmaps 341
Looping Through Pixels 342
Manipulating Bitmaps 345
Manipulating Color Bytes 347
Using Convolution in Full Color 348
Analyzing Bitmaps in oF 349
Analyzing Color 350
Analyzing Brightness 351
Detecting Motion 353
Using Edge Detection 355
Using Pixel Data 361
Using Textures 368
Textures in oF 369
viii | Table of Contents