Neural Networks in Bioprocessing and Chemical Engineering

493 Pages·1995·24.971 MB·English

by  D. R. Baughman and Y. A. Liu (Auth.)

Neural Networks
in Bioprocessing and
Chemical Engineering
D. R. Baughman
Y. A. Liu
Department of Chemical Engineering
Virginia Polytechnic Institute and State University
Blacksburg, VA
Academic Press
San Diego New York Boston London Sydney Tokyo Toronto
This book is printed on acid-free paper. ©
Copyright © 1995 by ACADEMIC PRESS, INC.
All Rights Reserved.
No part of this publication may be reproduced or transmitted in any form or by any
means, electronic or mechanical, including photocopy, recording, or any information
storage and retrieval system, without permission in writing from the publisher.
Academic Press, Inc.
A Division of Harcourt Brace & Company
525 B Street, Suite 1900, San Diego, California 92101-4495
United Kingdom Edition published by
Academic Press Limited
24-28 Oval Road, London NW1 7DX
Library of Congress Cataloging-in-Publication Data
Liu, Y. A. (Yin An)
Neural networks in bioprocessing and chemical engineering / by
Y. A. Liu, D. Richard Baughman.
p. cm.
Includes bibliographical references and index.
ISBN 0-12-083030-2 (alk. paper)
1. Biotechnological process control. 2. Chemical process control.
3. Neural computers. I. Baughman, D. Richard. II. Title.
TP248.25.M54L54 1995
660\6'028563~dc20 95-34841
CIP
PRINTED IN THE UNITED STATES OF AMERICA
96 97 98 99 00 01 QW 9 8 7 6 5 4 3 2 1
Preface
Over the past decade, neural networks have received a great deal of attention
among scientists and engineers, and they are being touted as one of the greatest
computational tools ever developed. Much of this excitement is due to the appar-
ent ability of neural networks to emulate the brain's ability to learn by examples,
which in turn enables the networks to make decisions and draw conclusions when
presented with complex, noisy, and/or incomplete information. Moreover, at some
primitive level, neural networks seem able to imitate brain's creative processes to
adapt to novel situations.
In 1988, Professor David M. Himmelblau of the University of Texas at Austin,
and his student, Josiah C. Hoskins, published the first article on neural networks in
chemical engineering, applying them to the fault diagnosis of a chemical reactor.
Since then, the number of publications on neural network applications in chemical
engineering has risen significantly. It is hard, especially for a person unfamiliar with
the subject, to separate the substance from the hype. Our purpose in writing this
textbook is to help our reader understand both the strengths and limitations of
neural networks.
This book introduces the fundamental principles and practical aspects of neural
networks, focusing on their applications in bioprocessing and chemical engineer-
ing. In the classroom, the book can serve as the textbook for a one-semester course
on neural networks for seniors or graduate students. It is also appropriate as a self-
study guide for practicing biotechnologists and chemical engineers who wish to
understand more about neural networks. Finally, computer scientists interested in
biological and engineering applications of neural networks should find this book
informative, and rich in references.
This book is unique in that it focuses on neural network applications and imple-
mentations using commercially available, user-friendly PC-based software tools,
rather than predominantly on neural network theory as essentially all of the existing
books and monographs do. We introduce neural networks and provide an overview
of their structures, strengths, and limitations, together with a survey of their poten-
tial and commercial applications (Chapter 1). In addition to covering both the fun-
damental and practical aspects of neural computing (Chapter 2), we demonstrate,
by numerous illustrative examples, practice problems, and detailed case studies,
how to develop, train and apply neural networks in bioprocessing and chemical
engineering. We divide the neural network applications of interest to the biotechnol-
ogists and chemical engineers into four main groups: (1) classification—fault classi-
fication and feature categorization (Chapter 3); (2) prediction and optimization
xiii
XIV Neural Networks in Bioprocessing and Chemical Engineering
(Chapter 4); (3) process forecasting, modeling, and control of time-dependent sys-
tems (Chapter 5); and (4) preliminary design of complex processes using a hybrid
combination of expert systems and neural networks (Chapter 6).
This book is also unique in that it includes the following ten detailed case stud-
ies of neural network applications in bioprocessing and chemical engineering:
• Process fault-diagnosis of a chemical reactor.
• Leonard-Kramer fault-classification problem.
• Process fault-diagnosis for an unsteady-state continuous stirred-tank reactor
system.
• Classification of protein secondary-structure categories.
• Quantitative prediction and regression analysis of complex chemical kinetics.
• Software-based sensors for quantitative predictions of product compositions
from fluorescent spectra in bioprocessing.
• Quality control and optimization of an autoclave curing process for manufac-
turing composite materials.
• Predictive modeling of an experimental batch fermentation process.
• Supervisory control of the Tennessee Eastman plantwide control problem
• Predictive modeling and optimal design of extractive bioseparation in aqueous
two-phase systems
We provide the reader with the detailed specifications of the structures and
training procedures for all neural network examples discussed in the book. In addi-
tion, this book includes a diskette of the input data files for all illustrative exam-
ples, case studies, and practice problems that are essential to providing the reader
with hands-on experience in developing, training, and applying neural networks.
To our knowledge, no other textbook on neural networks gives its readers such
detailed network specifications and data files as ours. We also include a glossary,
which explains the terminology used in neural network applications in science and
engineering.
Overall, this textbook serves as an introduction to neural networks in science
and engineering, particularly in bioprocessing and chemical engineering. It includes
both a broad coverage and an in-depth discussion of neural networks. We hope
that this book will benefit engineering students, practicing biotechnologists, chemi-
cal engineers, and computer scientists in neural network applications in science and
engineering.
Software Selection and References
Three of the many companies advertising PC-based neural network software tools
are: (1) AI WARE Incorporated, 11000 Cedar Avenue, Cleveland, OH 44106.
Phone [phone]. FAX [phone]; (2) HNC Corporation, 5501 Oberlin
Drive, San Diego, CA 92121. Phone [phone]. FAX [phone]; and
(3) NeuralWare, Inc., 202 Park West Drive, Pittsburgh, PA 15275. Phone (412)
787-8222. FAX [phone]. Throughout this text, we have used NeuralWare's
beginner software, NeuralWorks Explorer, and advanced software, Professional
II/PLUS. We recommend that the reader practice developing neural networks by
using a beginner software available from these and other companies. Also, most
companies offer university prices and group discounts for their software.
In addition to the software packages, NeuralWare offers four useful manuals for
buyers of their beginner or advanced software:
• Neural Computing: A Technology Handbook for Professional II/PLUS and
NeuralWorks Explorer (1993).
• Reference Guide: Software Reference for Professional II/PLUS and Neural-
Works Explorer (1993).
• Using NeuralWorks: A Tutorial for NeuralWorks Professional II/PLUS and
NeuralWorks Explorer (1993).
• Advanced Reference Guide, Software Reference for Professional II/PLUS and
NeuralWorks Explorer (1993).
xv
Acknowledgments
It is a pleasure to thank a number of very special persons and organizations who
contributed to the preparation of this book.
The idea for this book originated from the doctoral work of the junior author,
Rick Baughman. The junior author would like to thank the members of his advisory
committee, in particular: Professor Y. A. Liu, who developed the original idea and
the details of the book and was the major advisor; Professor Peter Rony, whose
thorough review and comments proved invaluable and who assisted us with the
Tennessee Eastman plantwide control problem; Professor Richey Davis for his sup-
port and encouragement; Professor William Conger, the Department Head who pro-
vided both technical and logistic support; and finally, Professor Henry McGee, who
served on the committee even while on leave at the National Science Foundation as
Director of the Division of Chemical and Thermal Systems, and after his move to
the Virginia Commonwealth University as Associate Provost for Engineering.
We wish to thank NeuralWare, Inc., Pittsburgh, PA for providing us with a free
copy of their PC-based software, Professional II/PLUS, for use in our case studies.
In addition, we wish to express our sincere gratitude to Mr. Lee R. Partin of
Eastman Chemical Company, Professor James F. Davis of Ohio State University,
Professor Jay H. Lee of Auburn University, Professor David M. Himmelblau of the
University of Texas at Austin, Professor Michael Mavrovouniotis of Northwestern
University, Professor Thomas McAvoy of the University of Maryland, Professor
George Stephanopoulos of the Massachusetts Institute of Technology, Professor
Lyle Ungar of the University of Pennsylvania, Professor Venkat Venkatasubraman-
ian of Purdue University, and Professor Nam-Sun Wang of the University of Mary-
land, who have kindly provided us with course notes, student dissertations, data
files, preprints, and reprints on neural networks. The senior author continues to be
grateful to Professors Stephanopoulos, Venkatasubramanian, and Ungar and Davis
for their excellent short course on artificial intelligence taught at MIT during the
Summer of 1987, which inspired his entry into the challenging field of AI.
We should like to thank Dr. Gulam (Sam) Samdani, Associate Editor, Chemical
Engineering Magazine, for his continued support and encouragement, particularly
in providing us with examples of commercial and emerging applications of neural
networks.
To Ms. Marie Paretti, Department of English, University of Wisconsin, we owe
a deep gratitude for her tireless and skillful copy-editing of the entire manuscript
for this book.
We would like to express our appreciation to the Eastman Chemical Company,
for providing a fellowship award to the junior author. We wish also to thank
XVII
xviii Neural Networks in Bioprocessing and Chemical Engineering
Messrs. Robert Inge, Craig Lane, Richard Rodgers, and Ms. Ashley Wright, chemi-
cal engineering seniors at Virginia Tech (1993-94), for their undergraduate research
projects that contributed to the experimental fermentation modeling and the Ten-
nessee Eastman plantwide control problem. In addition, we are grateful to a grant
from the National Science Foundation for establishing a SUCEED Center of Excel-
lence in Undergraduate Engineering Design at Virginia Tech, which provides finan-
cial support to these undergraduate research projects.
The original manuscripts of this text were taught by the senior author in train-
ing courses to university faculty and practicing engineers held in Taipei, Taiwan in
November, 1994 and in Beijing and Qingdao, China in April, 1995. The feedback
from course participants greatly improves the manuscript contents. He should like
to thank the financial sponsors of these training courses: China Technical Consul-
tants, Inc., Taipei, under the leadership of Mr. Chiu-Chin Lin, Secretary General,
and Dr. Dennis W. B. Wang, Director, Energy Technical Service Center; United
Nations Development Program, Resident Representative in China, Beijing, Mr.
Arthur N. Holcombe; and Ministry of Foreign Trade and Economic Cooperation,
China International Center for Economic and Technical Exchanges, Beijing, partic-
ularly Mr. Gong Tingrong, and Ms. Yin Weijing. He wishes to express his gratitude
to Professor Cheng Siwei, Vice Minister, Ministry of Chemical Industry, China and
President, Process System Engineering (PSE) Society of China, and to Mr. Shen
Tonghe, Secretary-General of the PSE Society of China, for their kind invitation
and continued support.
The senior author would like to thank the Virginia Tech administration for
granting him a sabbatical leave in Spring 1995 for completing the book manu-
script, and also the enduring support of his wife, Hing-Har Liu, through the labori-
ous process of this book writing and revision.
Lastly, we wish to thank Mrs. Diane Cannaday, Department of Chemical Engi-
neering, Virginia Tech, who patiently and graciously helped in all aspects of the
preparation of this writing project.
About the Authors
D. Richard Baughman received his B.S. and Ph.D. in the Department of Chemical
Engineering from Virginia Polytechnic Institute and State University ("Virginia
Tech") in 1986 and 1995, respectively. He completed his doctoral dissertation on
neural networks in bioprocessing and chemical engineering in March 1995, which
was the basis for this textbook.
Dr. Baughman worked for Haven Corporation, Baltimore, Maryland, a manu-
facturer of specialty coatings for the printed circuit board industry, as material and
process control engineer from 1986 to 1991. He returned to Virginia Tech in Fall
1991 for graduate studies. For his outstanding academic progress at Virginia Tech,
he has received the Tennessee Eastman Graduate Fellowship.
Y.A. Liu is the Frank C. Vilbrandt Professor of Chemical Engineering at Virginia
Tech. He received his B.S. (1967), M.S. (1970), and Ph.D. (1974) degrees from
National Taiwan University, Tufts University and Princeton University, respectively.
Professor Liu taught at Auburn University from 1974 to 1981, where his last
position was Alumni Associate Professor endowed by the Auburn Alumni Associa-
tion. He joined Virginia Tech as a Professor of Chemical Engineering in 1982. In
1983, he was appointed the Vilbrandt Professor.
He has two patents and over 100 publications in the fields of process design,
magnetochemical engineering, coal desulfurization and fluidization processes. He
has also published four books in these fields. In 1991, his Ph.D. student, Thomas
E. Quantrille, and he coauthored Artificial Intelligence in Chemical Engineerings
Academic Press, San Diego, California, the first AI text in chemical engineering.
Professor Liu's contributions to chemical engineering teaching and research have
been recognized by university and national awards. He has received three awards
from the American Society of Engineering Education (ASEE): the Fred Merryfield
Design Award (1993) for creativity and excellence in teaching and research of engi-
neering design; the George Westinghouse Award, ASEE's highest honor for an engi-
neering educator under age 45 for outstanding early achievements in both teaching
and scholarship; and the Western Electric Award (1984) for excellence in instruc-
tion of engineering students. In 1986, he received the National Catalyst Award for
excellence in chemical education from the Chemical Manufacturers Association. In
1990, he received the Distinguished Alumni Award from Tufts University.
Professor Liu serves as a Senior Technical Advisor, United Nations Development
Program, helping developing countries train university faculty and practicing engi-
neers on artificial intelligence and engineering design.
xix
1
Introduction to Neural Networks
This text introduces the fundamental concepts and practice of neural networks,
emphasizing their applications in bioprocessing and chemical engineering. In this
chapter, we introduce the basic concepts, beginning with simple definitions, and
then present a brief overview of the structure of neural networks and describe their
development. Next, we explain their properties, compare them to empirical model-
ing techniques, and list many of their strengths and limitations. Finally, we describe
the potential applications of neural networks (both in general and as applied to
bioprocessing and chemical engineering), and cite some recent reports of commer-
cial and emerging uses. We note that parts of Sections 1.1 and 1.2 have been
adopted and updated from Quantrille and Liu (1991; pp. 440-445; 481-483).
1.1 Introduction
A. Artificial Intelligence
The term "neural network" resulted from artificial intelligence (AI) research, which
attempts to understand and model brain behavior. According to Barr and Feigen-
baum (1981):
Artificial intelligence is the part of computer science concerned with designing intelli-
gent computer systems, that is, systems that exhibit characteristics we associate with
intelligence in human behavior.
This definition simply states that the goal of AI is to make computers "think," to
make them solve problems requiring human intelligence.
Focusing on the means of achieving this goal, Buchanan and Shortliffe (1983)
offer another definition of AI:
Artificial intelligence is the branch of computer science dealing with symbolic, non-
algorithmic methods of problem solving.
This second definition of AI emphasizes two aspects of AI-based methods for prob-
lem solving. First, AI does not use an algorithm, that is, a formal procedure speci-
fying a step-by-step execution path that guarantees a correct or optimal solution at
some point. Second, AI involves symbolic processing, a branch of computer science
1
2 1. Introduction to Neural Networks
that deals with non-numerical symbols and names. In contrast, the more classical
numerical processing deals with numerical calculations and processes.
Three major AI-based technologies with growing technical and commercial sig-
nificance are expert systems, neural networks, and fuzzy-logic systems (VerDuin,
1995; Crowe and Vassiliadis, 1995). Quantrille and Liu's previous text, Artificial
Intelligence in Chemical Engineering (Quanttrille and Liu, 1991), focuses on expert
systems, while this book concentrates on neural network applications. We also
briefly discuss fuzzy-logic systems in Sections 2.6.D and 5.6.E.
B. Expert Systems, Neural Networks, and Subsymbolic Processing
An expert system (see also Section 6.1.B), also known as a knowledge-based
system, is a computer program that uses high-quality, in-depth, knowledge to solve
complex and advanced problems typically requiring human experts. Expert systems
operate symbolically, on a macroscopic scale, processing non-numerical symbols
and names. They require knowledge of relationships and do not care how these
relationships develop.
Neural networks, on the other hand, use subsymbolic processing, characterized
by microscopic interactions that eventually manifest themselves as macroscopic,
symbolic, intelligent behavior. Thus, Robert Hecht-Nielson (1990) defines neural
networks as follows:
A neural network is a computing system made up of a number of simple, highly inter-
connected nodes or processing elements, which process information by its dynamic
state response to external inputs
The goal of a neural network is to map a set of input patterns onto a correspond-
ing set of output patterns. The network accomplishes this mapping by first learning
from a series of past examples defining sets of input and output correspondences
for the given system. The network then applies what it has learned to a new input
pattern to predict the appropriate output.
To understand how neural networks work more clearly, let us consider how the
human brain functions. Most neurologists believe that true intelligence goes
beyond the kind of symbolic processing used in expert systems. In the human
brain, neurons within the nervous system interact in a complex fashion. The
human senses detect stimuli, and send this "input" information (via neurons) to the
brain. Within the brain, other neurons are excited, and they interact with each
other. Based on the input, the brain reaches a "conclusion", and sends an "output"
in the form of an answer or a response. The microscopic interactions between neu-
rons is invisible, but they manifest themselves as identifiable behavior. For example,
assume that you touch the side of a pot of boiling water. Your sense of touch sends
its input to your brain, which causes neurons there to interact—in ways invisible to
you. As a result of these interactions, though, your brain sends an output, causing
you to withdraw your hand. The input—touching the hot surface—and the out-
put—withdrawing your hand—are clear, but the processing in your brain that led
from one to the other remains hidden.
Neural networks use the same type of structure for computer modeling of intelli-
gent behavior. Neurologists and artificial intelligence researchers have proposed a