Table Of Content9/9/2018 Introduction - WebCOM™ 2.0
UNIVERSITY OF TEXAS - AUSTIN FOUNDATIONS OF ART AND
ENTERTAINMENT TECHNOLOGIES: PIXELS SAMPLES LUMENS
ILLUSION
STAMPS
INTRODUCTION
While it’s true that I have a lot that I can say on the subject of art and entertainment
technologies, the main reason why I am writing this text is simple. I love the subject
matter. I truly do. In the two years I have been researching and teaching these topics,
I have become more and more fascinated by the extent to which the technologies
themselves—each in their own right—can inspire creativity. I have read countless
essays and online discussions by my students who, when presented with this
massive truckload of innovation, think up some rather remarkable new ideas that
incorporate combinations of preexisting technologies or imply the need for a
technology that doesn’t yet exist. This is where the rubber meets the road, so to
speak. This is where innovation happens; in the realm of play and almost childlike
suspension or ignorance of the rules that tether us to the world we know.
I confess—I have a dilemma. I do not like traditional textbooks. I absolutely do. In my
view, the traditional textbook paradigm—replete lots of charts, terms, timelines,
historical �gures, weighty academic jargon like the word, “replete”—is not conducive
to inspiring creativity and relating creative processes. So, this book is written in a
kind of hybrid tone and point of view. I have included lots of nontraditional examples
of process and creative outcomes, some made by me and some by my colleagues.
The purpose of this is simple—to inspire. Consider the writing of this book as a
grand, overarching study in thinking outside of the box. But, “thinking outside of the
box” itself is cliché and dusty. Let’s completely reconsider the nature and purpose of
the box itself.
Of this enormous arts and entertainment technologies world, we can identify three
main areas that will cover most of what we will be covering in this text. At the
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/introduction 1/5
9/9/2018 Introduction - WebCOM™ 2.0
University of Texas (UT) at Austin, in the Department for Arts and Entertainment
Technologies (DAET), these three areas (which, we refer to as emphases at UT) are
Music and Sound, Performance Technology, and Gaming and Mobile Media.
It should be noted that none of these categories are completely discrete or mutually
exclusive. Like an animated and ever-changing Venn diagram, they all rely on cross-
pollination and collaboration across a variety of different disciplines. Pokémon: Go!,
which I talked about in the Preface, relies on all three areas to varying degrees. It’s a
game. It’s a mobile app. It has a big augmented reality (AR) component and of
course, music and sound effects. The balance of all of these disparate areas of
entertainment technologies found in Pokémon: Go!, and the extent to which they
combine effortlessly, I’d argue, account for much if not all of its worldwide success.
In this spirit, I will oftentimes focus on ways in which these areas mix, combine, and
can even create potential new areas. As an example, in just the past eighteen
months, AR and virtual realities (VR) are poised to detach from their present place
in performance technologies and games to become one, discrete category. And why
not? We will consider how a music concert can be enhanced and even transformed
by the increasingly visual technologies that accompany it. Or how photo or video
manipulation can simultaneously entertain and convey a political or philosophical
message. Or how an image can be projected on a moving, three-dimensional (3-D)
object. In many cases, identifying where one technology ends and another begins
within an activity becomes more di�cult as the lines between them blur and run
together.
Of course, it’s possible to include areas beyond the three I mentioned—these are just
broad generalities that will be helpful when thinking about how to organize this tour.
But, any intersection of art or entertainment with technology is a fair game, right?
That is and always should be the essence and spirit of this subject matter. Think of
this text and the supporting materials as a sort of guide or roadmap through what I
hope will be a very fascinating, personal, and perhaps creatively empowering journey
for you.
First, we will be looking at music and sound technologies, tracing the development
of music technology from centuries past to the present. In many ways, music
technology is the oldest identi�able form of entertainment technologies, dating back
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/introduction 2/5
9/9/2018 Introduction - WebCOM™ 2.0
centuries, perhaps even a millennium, depending on how one demarcates it. I have
to confess a certain bias toward music technology, as my own educational
background and professional �eld includes music composition, songwriting, and
production. But I �nd it helpful to start with an examination of the development of
music technology, as many of the trends we will see and lessons we will learn are
ones that will manifest time and time again in other technologies and experiences,
like mobile media.
Performance technology, which, at UT, we refer to as
PLAI, or Projection, Lighting and Interactivity, is
arguably the newest and most open ended of the
three areas covered in this text for a couple of
reasons. One, it offers the most fertile soil for
Image ©
cultivating new ideas and, it allows for the
vectorfusionart/Shutterstock.com
interconnectivity of all of the other areas in never-
before-seen ways. When we reach this unit, you will
see that every case study we look at will be radically different from the rest,
combining existing technologies and inventing new ones in innovative ways to
produce a unique result. From music-making jelly to choreography involving
industrial assembly line machinery to performance utilizing moving screens and 3-D
projection mapping, each piece integrates new technological elements in a way that
will truly challenge your expectations and your sense of perception. But why? What
is it about this area that gives it the greatest capacity for growth and
experimentation? In performance technology, there is an intrinsic emphasis on
innovation—on creating something new and never-before-seen. Additionally, in
performance technology, there is a commitment to incorporating innovation into
every aspect of an artist’s creative endeavors, whether that technology already
exists or needs to be innovated and created on the spot. Finally, all of this revolves
around performance, addressing questions of how new technology may be used to
enhance a given performance from the perspective of the audience. The various
technologies incorporated into a show, concert, or installation may be incredible, yes,
but they all come together to contribute to the overall aesthetic of the performative
arts—a means to an end, rather than an end in itself; incredible but also memorable
and meaningful.
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/introduction 3/5
9/9/2018 Introduction - WebCOM™ 2.0
And that is one of the most crucial keys to unlocking the potential of performance
technology as a promising area. The technology itself and the means used to create
the experience fade into the incidental background of the overall experience. The
artist can create an immersive world where viewers aren’t actively thinking about
how the performance is being created because they are so engrossed by the
experience itself. I keep coming back to this word experience, because, simply put,
performance technology is all about the experience itself. It’s not about how the
experience is being made. It’s about what the experience is.
Video games and mobile media are some of the
fastest-growing and most prevalent forms of
entertainment technology today. Mobile media in
particular—especially with the rise of the smartphone
—epitomizes the values and unique characteristics of
Image © Alexander
the digital revolution. Out of all the major categories
Tolstykh/Shutterstock.com
of arts and entertainment technologies, video games
and mobile media may be the most visible in our
daily lives. Video gaming continues to have an ever-growing impact in society as it
forges distinctive industries, communities, and even professional sporting
organizations. Mobile games and apps continue to reach audiences that have
previously been almost untouched by video gaming, expanding the demographic
in�uence of gaming even further. And I could go on about how mainstream
emerging technologies in VR and AR have become, but I bet there is no need to.
Every day, new developments in VR and AR surface in all kinds of news outlets, not
just on tech websites.
As alluded to, VR and AR technology might very well
be considered a main category of its own. Just in the
past two years, it has become a huge and pervasive
force to be reckoned within technological circles.
What makes VR and AR particularly interesting is
Image © ESB
that it blurs the lines of the traditional gaming
Professional/Shutterstock.com
experience with that of live performance, or perhaps
even an art installation. It seeks to deliver
experiences to the user, but not necessarily in a way that resembles traditional
gameplay characteristics. The idea that an experience could become virtual, much
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/introduction 4/5
9/9/2018 Introduction - WebCOM™ 2.0
as a sound waveform could become digitized, is a powerful idea. The technology is
still very nascent, but growing like a weed. Just a few years ago, the only mass-
marketed VR device was Google Cardboard, commandeering your smartphone and
placing it into a �imsy, cardboard frame. But since then, the Oculus Rift launched to
widespread commercial markets, thus opening the �oodgates to innovation and
currently, dozens of tech companies have been working hard to capture a piece of
the VR market. I can only speculate where VR and AR technology might go in even
the next couple of years, but I am eager to see what happens. And, by the end of this
text, I am sure you will be too.
We will look at the DO-IT-YOURSELF (DIY)
Revolution in electronics, storytelling as examples of
Disruptive Technologies. A Podcast, for example, is
completely free to listen to and has forced the news
and entertainment industries to reevaluate their
priorities and methodologies. Much like the Internet
itself, Podcasting allows for the dissemination of
© monique28/Getty Images
information to every corner of the globe, crossing
barriers of language, culture, and economic status.
Arduino DIY electronics components, like the Podcast, facilitates the rise of the
amateur into a world previously monopolized by professionals. Technology
innovates, and people are empowered in innovative ways. The world is yours to
envision, program, and make.
In conclusion, we will look at subversive potential of arts technologies—from
Photoshop, to politics to activism to the subversion of art itself. And we will close
with a cautionary tale about the perils of overreliance on technology to make art and
an open invitation to keep playing, unencumbered and without rules, in your own
electrostatic sandbox.
WebCOM™ 2.0 is a trademark of Great River Learning. All rights reserved. © 2002-2018.
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/introduction 5/5
9/9/2018 Chapter 1 - WebCOM™ 2.0
UNIVERSITY OF TEXAS - AUSTIN FOUNDATIONS OF ART AND
ENTERTAINMENT TECHNOLOGIES: PIXELS SAMPLES LUMENS
ILLUSION
STAMPS
CHAPTER 1: A BRIEF HISTORY OF MUSIC
TECHNOLOGY: FROM THE HYDRAULIS TO THE
DIGITAL AUDIO WORKSTATION
Of the areas of Art and Entertainment Technology we are going to be examining in
this text, Music and Audio Technology has the longest and most robust history.
Thus, one might argue that the history of arts and entertainment technologies, itself,
begins with music technology. As I hope to demonstrate, the history of music
technology goes back further—way further, in fact—than you might expect. But
before we begin, I want to present a quote from the late twentieth-century American
composer and theorist, Milton Babbitt:
“
“As for the future of electronic music, it seems quite obvious to me that its
unique resources guarantee its use, because it has shifted the boundaries of
music away from the limitations of the acoustical instrument, of the
performer's coordinating capabilities, to the almost in�nite limitations of the
electronic instrument. The new limitations are the human ones of perception”
The last sentence of the quote is especially poignant; “The new limitations are the
human ones of perception.” I don’t want to cast music history as the development of
acoustical instruments towards the invention of electronic music, but one of the
prominent themes we will see recurring throughout this chapter is the trend towards
the arti�cial synthesis of sound and the approximation and re-creation of acoustic
instruments.
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/ch1 1/12
9/9/2018 Chapter 1 - WebCOM™ 2.0
We need a place to start. I propose that the history of hardware-based music
technology begins with the Pipe Organ. Though many of us may think of Bach in the
Baroque period when we think of the pipe organ, its origins can actually be traced all
the way back to the third century BCE and the Hydraulis, meaning “water organ.”
First performance of the reproduction Roman (or Greek) Hydraulis Organ at Bath
Suggesting that the history of arts and entertainment technologies begins with
music technology might be met with criticism, which I recognize and understand.
One might argue, for example, that the Lascaux Caves of France contain wall
paintings that resemble and even suggest animation, or the earliest form of the
animated GIF, which is discussed a little later in the text. A compelling claim, to be
sure. Though the wall paintings of Lascaux are, by some estimates, 17,000 years old,
they do not contain actual technology in their presentation; merely the suggestion of
movement. And that is where I am proposing the line in the sands of time be drawn;
the simultaneous presence of technology and artistic expression.
Wolfgang Amadeus Mozart referred to the pipe
organ as the “King of Instruments,” a title that is well-
deserved when viewed from either a musical or
technological perspective. It’s hard to overstate just
how revolutionary the pipe organ was in its time,
especially when viewed as a pre-cursor of digital Lascaux Caves of France
Image © zatvornik /
music technologies or even the Digital Audio
Shutterstock.com
Workstation (DAW). The pipe organ features the
ability to control and layer hundreds of different
instrument sounds through the use of stops. This concept of the manuals of an
organ being assigned to different sets of sounds that approximated an in�nite
variety of acoustic instruments is the clear ancestor of today’s modern Virtual
Instruments in a DAW. Indeed, the pipe organ is a sort of DAW in and of itself, and
we shall explore the parallels in greater depth in our next chapter.
The pipe organ is thoroughly mechanical in its function. All of the pipes of the pipe
organ operate by the �ow of wind that was initially created manually through the use
of foot bellows, operated by the performer on smaller organs and played by at least
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/ch1 2/12
9/9/2018 Chapter 1 - WebCOM™ 2.0
two, if not several additional players on later, more expansive and elaborate
instruments. In the age before electricity, the pipe organ’s construction was a
mechanical marvel, and they still remain so even today. And of course, today’s
modern organs rely on electric blowers that allow organists to practice more
regularly and without the assistance of bellow operators. Let’s take a look at perhaps
one of the best-known examples of music made possible by the pipe organ. Note in
this example, how the Manuals, or keyboards are coupled, or linked together. This is
one of the signi�cant pipe organ technologies that allow the layering of different
sounds.
Image © James Steidl /
Shutterstock.com
Image © oorka /
Shutterstock.com
J.S. Bach’s Toccata & Fugue in D Minor
You may be asking yourself why there is not more detail on the Hydraulis here and
perhaps, why I consider the origins of music technology beginning with the Hydraulis
but follow it with an immediate argument for the parallels between the pipe organ
and modern music technology. On the one hand, there really isn’t a lot of
documentation and research of the Hydraulis. Remember, its origins date to the third
century BCE. Further, the musical output of the Hydraulis was extremely limited. It
could only play a limited number of notes that were grouped to create an equally
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/ch1 3/12
9/9/2018 Chapter 1 - WebCOM™ 2.0
limited amount of musical possibility. But, the Hydraulis was mechanical and thus,
technological.
The pipe organ, as revolutionary as it was, still required a player. Our next innovation,
from the late nineteenth century, begins to foreshadow the use of music technology
to recreate musical performance and even share music. The Player Piano, also
known as the Pianola is a remarkable step in music technology history. This is how
it worked: the Pianola allowed the user to capture every note from a live
performance, punched and stored on a rotating roll of paper. This allowed for
playback of a captured or recorded performance on any other like-model Pianola.
This means that for the �rst time in history, a person could buy a recording, in the
form of a piano roll, of any famous pianist of the day, like Gershwin or Rachmaninov.
And, the roll (ostensibly, a recording) would contain their performance to be played
back, on your Pianola, in your home or place of business. To this end, the Pianola
was the �rst means of distributing recorded music, as a speci�c performance could
be transported and heard across the world, albeit with the acoustics of a different
space and timbre, or sound quality, of a different model. These instruments were
extremely popular at the turn of the century, and during the 1920s almost every other
piano being made was a Pianola!
Image © Ronald Caswell /
Shutterstock.com
Sidewalks of New York for Player Piano
There is another reason why the Pianola is signi�cant—it provides the foundation
and basic blueprint, to a greater degree than the pipe organ did, for MIDI data. MIDI
is an acronym that stands for Musical Instrument Digital Interface. And, while we
will explore the details of in the DAW chapter, it is su�ce to say here that MIDI is a
language or protocol that is understood by modern musical instruments and devices
that allows for connectivity and networking between them. MIDI allows, for example,
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/ch1 4/12
9/9/2018 Chapter 1 - WebCOM™ 2.0
a MIDI keyboard to convert a musical performance into data that can then be played
back by other instruments that can read the data as musical events. When and what
notes are played and released; how hard the keys are struck, are but a few of the
many variables MIDI allows the user to capture and edit.
Sound familiar? There is a reason why many DAWs call the MIDI Editor Window the
“Piano Roll” after the rolls of paper that could record Pianola performances. Thus,
one could argue that the Pianola is the direct ancestor of the advent of MIDI data,
the life-blood of modern music composition, performance, production, and
arranging.
The technology and the principles found in Thomas Edison’s Phonograph changed
the course of music and music technology forever. Conceptually, the invention of
Phonograph technology made gigantic new strides by successfully converting
sound waves into physical media. Introduced in 1877 by the well-known inventor
Thomas Edison, the Phonograph represented the ful�llment of a long-standing
dream to be able to preserve performances or speeches occurring in a speci�c time
and place for posterity and for future experience.
The Phonograph recorded any sound waves entering its large bell onto tinfoil sheet
cylinders. However, as innovative as Edison’s phonograph was, it was eclipsed in
popularity, by the 1920s, by another similar device known as the Gramophone.
Rather than recording to tinfoil cylinders, the gramophone utilized �at disks that are
known today as vinyls or long plays (LPs). The vinyl LP provided superior audio
�delity to cylinders as well. And, because vinyls were more easily stamped or
pressed, they were better suited to mass production and were thus able to
outperform the phonograph in sales and market share.
If you think the concept of Music Streaming—the
idea of experiencing music remotely from a distant
source—is a more recent innovation, a man named
Thaddeus Cahill had the idea over a century in
advance of Pandora or Spotify. In 1896, Thaddeus
Cahill �led a US Patent for what he called The
Telharmomium. Cahill’s invention embodied the idea
of transmitting music over telephone wires from a
https://utaustin.grtep.com/index.cfm/artentertainmenttech/page/ch1 5/12
