Prestructuring Multilayer Perceptrons based on Information-Theoretic Modeling of a Partido-Alto

545 Pages·2010·5.87 MB·English

Portland State University
PDXScholar
Dissertations and Theses Dissertations and Theses
1-1-2011
Prestructuring Multilayer Perceptrons based on Information-
Theoretic Modeling of a Partido-Alto-based Grammar for Afro-
Brazilian Music: Enhanced Generalization and Principles of
Parsimony, including an Investigation of Statistical Paradigms
Mehmet Vurkaç
Portland State University
Let us know how access to this document benefits you.
Follow this and additional works at:http://pdxscholar.library.pdx.edu/open_access_etds
Recommended Citation
Vurkaç, Mehmet, "Prestructuring Multilayer Perceptrons based on Information-Theoretic Modeling of a Partido-Alto-based Grammar
for Afro-Brazilian Music: Enhanced Generalization and Principles of Parsimony, including an Investigation of Statistical Paradigms"
(2011).Dissertations and Theses.Paper 384.
10.15760/etd.384
This Dissertation is brought to you for free and open access. It has been accepted for inclusion in Dissertations and Theses by an authorized
administrator of PDXScholar. For more information, please [email protected].
Prestructuring Multilayer Perceptrons based on Information-Theoretic Modeling of a
Partido-Alto-based Grammar for Afro-Brazilian Music: Enhanced Generalization and
Principles of Parsimony, including an Investigation of Statistical Paradigms

by
Mehmet Vurkaç

A dissertation submitted in partial fulfillment of the
requirements for the degree of

Doctor of Philosophy
in
Electrical and Computer Engineering

Dissertation Committee:
George G. Lendaris, Chair
Douglas V. Hall
Dan Hammerstrom
Marek Perkowski
Brad Hansen

Portland State University
©2011
ABSTRACT
The present study shows that prestructuring based on domain knowledge leads
to statistically significant generalization-performance improvement in artificial neural
networks (NNs) of the multilayer perceptron (MLP) type, specifically in the case of a
noisy real-world problem with numerous interacting variables.
The prestructuring of MLPs based on knowledge of the structure of a problem
domain has previously been shown to improve generalization performance. However,
the problem domains for those demonstrations suffered from significant shortcomings:
1) They were purely logical problems, and 2) they contained small numbers of variables
in comparison to most data-mining applications today. Two implications of the former
were a) the underlying structure of the problem was completely known to the network
designer by virtue of having been conceived for the problem at hand, and b) noise was
not a significant concern in contrast with real-world conditions. As for the size of the
problem, neither computational resources nor mathematical modeling techniques were
advanced enough to handle complex relationships among more than a few variables
until recently, so such problems were left out of the mainstream of prestructuring
investigations.
In the present work, domain knowledge is built into the solution through
Reconstructability Analysis, a form of information-theoretic modeling, which is used to
identify mathematical models that can be transformed into a graphic representation of
the problem domain’s underlying structure. Employing the latter as a pattern allows the
researcher to prestructure the MLP, for instance, by disallowing certain connections in
i
the network. Prestructuring reduces the set of all possible maps (SAPM) that are
realizable by the NN. The reduced SAPM—according to the Lendaris–Stanley
conjecture, conditional probability, and Occam’s razor—enables better generalization
performance than with a fully connected MLP that has learned the same I/O mapping
to the same extent.
In addition to showing statistically significant improvement over the
generalization performance of fully connected networks, the prestructured networks in
the present study also compared favorably to both the performance of qualified human
agents and the generalization rates in classification through Reconstructability Analysis
alone, which serves as the alternative algorithm for comparison.
ii
Dedicated to my mother, Sabiha Tuğcu Vurkaç

iii
ACKNOWLEDGMENTS
I am grateful to many people for their help while I worked on this dissertation.
First, I would like to thank my mother, Sabiha Tuğcu Vurkaç, for more than twenty
years of emotional and financial sacrifice and support as I pursued my higher education
away from home, and for a lifetime of love, friendship, teaching, guidance, and music.
This dissertation would not have been possible without the insights,
encouragement, mentoring, wisdom, inspiration, confidence and optimism provided by
(and the support and open-mindedness of) my adviser, teacher, mentor, and ally Dr.
George G. Lendaris. Dr. Lendaris has shown a level of care and engagement in both my
general academic, scientific, and intellectual growth, and specifically my research and
dissertation well beyond what anyone would expect. He is a true teacher and mentor.
This dissertation also would not have been possible without generous and timely
gifts of housing and computing resources from George Karagatchliev and Lisa Brandt
Heckman (respectively) that made my continued work possible at the most difficult
times of my student career. I’m similarly indebted to Eric Egalite and Joe Reid of OIT
for automation and analysis assistance, respectively, and the valuable insights gathered
through years of discussions and music-making with Derek Reith. I am also grateful to
Woods Stricklin, Patryk Lech, Travis Henderson, Anita Rodgers, Hank Failing, Gary
Beaver, Michelle Thayer, Cory Troup, Dr. Bruce Barnes, Andrew Zvibleman, Dr. Xin
“Ryan” Wang, and my aunt Semra Bastıyalı for critical assistance at crucial points in the
PhD process.
iv
In addition, I would like to specially mention the vital role played by Christina
Luther, friend and SEVIS adviser, in keeping me legal and on track.
Also as a primary contributor, I’d like to thank the emergent complexity that
resulted from the billions of years of evolution that enabled my wet neural network to
engage in such a pursuit as this dissertation.
The presentation of this dissertation and associated work was greatly improved
thanks to input from Woods Stricklin, Sabiha Vurkaç, Dr. Eve Klopf, Dan Craver, Dr.
Ahmet Müfit Ferman, and the PSU Writing Center, and due to the technical assistance
of Dr. Serap Emil (literature review), Tamara Turner and Đhsan Tunç Çakır (notation),
and Buğra Giritlioğlu (music computing).
I would also like to thank Dr. Melanie Mitchell for inspiring me to go into
Computational Intelligence; Brian Davis, Andrew Hartzell and Derek Reith of Lions of
Batucada for providing the environment for samba to become an integral part of my life;
and once again, Dr. Lendaris for bringing the two realms together in my research.
I would also like to thank Dr. Brad Hansen (for his all-around knowledge of the
topics, and the efficiency and legitimacy he has brought to my work), Dr. Douglas V.
Hall (for being a true teacher and role model), Dr. Dan Hammerstrom (for guidance at
various stages, and for introducing me to Jeff Hawkins’ work), Dr. Marek Perkowski
(for his excitement, encouragement, and many research sources), Dr. Fu Li (for
peppering his lectures with invaluable insights), Mestre Jorge Alabê (for all his
encouragement, musicality, private lessons, workshops, brain dumps, and repique solos),
v
“long-lost brother” Mark Lamson (for the fourth clave-direction category and for
verifying my conceptualization of rhythm), Michael Spiro (for asking the tough
questions, and for the IMD factor), Dr. Rob Daasch (for directing me to Dr. Lendaris),
Dr. David Glenn of Whitman College (for believing in and supporting me, and giving
me time and resources with which to learn to play), Dean/Professor/Cellist Robert
Sylvester (for helping me recognize the breadth of my musical experience), Dr. James
Morris (for recognizing my teaching ability, which in turn helped finance my studies and
prepared me for a teaching career), Dr. Malgorzata Chrzanowska-Jeske (for support and
guidance), Professor and Chair Harold Gray (for being a resource and friend), Dr.
Martha Balshem (for helping me begin to learn how to conduct research in
Ethnomusicology), Derek Reith (first tamborim teacher), Brian Davis (first pandeiro
teacher), Andrew Hartzell (first surdo and repique teacher), Chris Perry (first reviewer of
my clave tutorial), Tobias Manthey (for clave vigilance), Andy Sterling, David Huerta &
Jesse Brooke (for clues to 12/8 directionality), Dr. Michael Cummings & Dr. Bill
Becker (for early thesis-writing advice), Đhsan Tunç Çakır (for the proper way to write
standard European music notation), Emily Brown, Steve White and Matthew Stanbro
(for “getting it”), Sue Firpo (for substantial assistance with the paperwork and logistics
of financing the early years of my PhD), and the following for their work on the golden-
ear and benchmark sessions: Tobi Lehman, Ron Scroggin, Lisa Brandt Heckman, Krasi
Nikolov, Chaz Mortimer, Tobias Manthey, Rafael Otto, Jake Pegg, John Jenness, Tofer
Towe, Gary Beaver, Rachel Sandy, Cory Troup, Michelle Becka and Renata Secco.
vi
For the information-theoretic aspect of this work, I would like to thank Dr.
Martin Zwick and Joe Fusion, along with Better World roadside service, and Duane at
Farwell Towing.
Furthermore, since this dissertation is truly my life’s integrated work, I would
like to express my gratitude to Professors David Weber, Don Moor, and Peter Nicholls
(for changing my brain for the better); Michelle Thayer; Cingöz Canavar Charlie
Monster Vurkaç (my sole companion throughout my candidacy); Torimaru Yumi
Octopus (for the subtleties of dentou geinou and examples of Japanese notation systems),
Cheryl Ramette (for taking the step that made progress possible when everything was
about to fall apart), Dr. Candyce Reynolds, Michelle Gano (for helping me through the
financial bureaucracy of PSU), students of my Saturday samba class (for all their
questions, enthusiasm, and friendship), Danny Norton (clave-conscious Drum Buddy
virtuoso), Devon Halloran and Emily Philips (for emergency academic/clerical
assistance), The PSU University Studies 2007–2008 Mentor Community (especially
Prof. Teresa Taylor, Kim Heidenreich, Nancy D’Inzillo, Jason Baidenmann, Nathania
Cha, Christina Overturf, Chris Howell, Cara Glennon, Christie Toth, and Kate Gentry),
session musician and childhood friend Yavuz Çetin (for opening the door to so much
music), Đzi Eli (first music teacher), Faik Tuğcu (my grandfather and first music
mentor), my aunt Nemika Tuğcu (role-model for strength and positivity), my aunt
Solmaz (Vurkaç) Başpınar, Dr. James McNames (for scientific vigilance); Nikki and
Shiva Thayer; Lions of Batucada, Mais Que Samba, California Brazil Camp, toyboat
toyboat toyboat, Obsessive Percussive Disorder, Whatsoever Creepeth, Kafana Klub,
vii
Takohachi, Mirah Yom Tov Zeitlyn and Emily Kingan; multi-talented
drummer/composer/role-models Roger Taylor, Memo Acevedo, Airto Moreira, Stevie
Wonder, Phil Collins, Vanderlei Pereira, Mauricio Zottarelli, and Yoshiki Hayashi; Igor
Stravinsky, Anton Webern, Karlheinz Stockhausen and John Cage (for advancing tonal
and timbral democracy); Scott Wardinsky, Engin Gürkey and Jozi Levi; Paul R.
Bergmann, Tony Williams and Dick Schalk; Okay Temiz, Trilok Gurtu, Giovanni
Hidalgo, Doudou N’Diaye Rose, and Gregg Bendian (five inspiring drummers who
took the time to talk to me and encourage me); Robert College (my middle and high
school in Turkey, for instilling American-style skepticism), Whitman College, Portland
State University (for giving me the chance many other institutions may not have),
Oregon Institute of Technology (Anna Bailey, Professors Jamie Zipay, Paul Dingman,
Valerie Ball, and Matt Gildehaus, Casey Coulson, Dr. Bruce Barnes, Dr. Claudia Torres-
Garibay, Dr. Xin “Ryan” Wang, and Dr. Mateo Aboy, who supported me and my work
in tangible ways while I completed the research for this dissertation); the FSM and the
IPU; and most importantly, Mustafa Kemal Atatürk (for creating modern, secular
Republic of Turkey, making my-life-as-I-know-it possible).
In closing this rather long acknowledgement, I’d like to also thank the people
who shaped my musical or academic life the most, respectively: John Deacon, Roger
Taylor, Freddie Mercury and Dr. Brian May (of Queen), and Yılmaz Vurkaç (my late
father, for the drum sticks, the love of books, obsessiveness, and who knows what else).

viii