Artificial Neural Networks for Engineers and Scientists: Solving Ordinary Differential Equations

169 Pages·2017·9.06 MB·English

by  S. Chakraverty

Artificial Neural
Networks for Engineers
and Scientists
Solving Ordinary Differential
Equations

Artificial Neural
Networks for Engineers
and Scientists
Solving Ordinary Differential
Equations
Snehashish Chakraverty and Susmita Mall
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2017 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
International Standard Book Number-13: 978-1-4987-8138-1 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors
and publishers have attempted to trace the copyright holders of all material reproduced in this
publication and apologize to copyright holders if permission to publish in this form has not been
obtained. If any copyright material has not been acknowledged please write and let us know so we
may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known
or hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.
copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC),
222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that
provides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Names: Chakraverty, Snehashish. | Mall, Susmita.
Title: Artificial neural networks for engineers and scientists : solving
ordinary differential equations / Snehashish Chakraverty & Susmita Mall.
Description: Boca Raton : CRC Press, 2017. | Includes bibliographical references.
Identifiers: LCCN 2017004411| ISBN 9781498781381 (hardback) | ISBN
9781498781404 (ebook)
Subjects: LCSH: Differential equations--Data processing. | Engineering
mathematics--Data processing. | Artificial intelligence.
Classification: LCC QA372 .C42527 2017 | DDC 515/.3520285632--dc23
LC record available at https://lccn.loc.gov/2017004411
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface ......................................................................................................................ix
Acknowledgments ..............................................................................................xiii
Authors ...................................................................................................................xv
Reviewers ............................................................................................................xvii
1. Preliminaries of Artificial Neural Network ...............................................1
1.1 Introduction ...........................................................................................1
1.2 Architecture of ANN ............................................................................2
1.2.1 Feed-Forward Neural Network .............................................3
1.2.2 Feedback Neural Network .....................................................3
1.3 Paradigms of Learning .........................................................................4
1.3.1 Supervised Learning or Associative Learning .....................4
1.3.2 Unsupervised or Self-Organization Learning .....................4
1.4 Learning Rules or Learning Processes ...............................................5
1.4.1 Error Back-Propagation Learning Algorithm or Delta
Learning Rule ...........................................................................5
1.5 Activation Functions.............................................................................8
1.5.1 Sigmoid Function .....................................................................8
1.5.1.1 Unipolar Sigmoid Function ....................................8
1.5.1.2 Bipolar Sigmoid Function .......................................9
1.5.2 Tangent Hyperbolic Function ................................................9
References .........................................................................................................9
2. Preliminaries of Ordinary Differential Equations .................................11
2.1 Definitions ............................................................................................12
2.1.1 Order and Degree of DEs ......................................................12
2.1.2 Ordinary Differential Equation ............................................12
2.1.3 Partial Differential Equation ................................................12
2.1.4 Linear and Nonlinear Differential Equations ....................13
2.1.5 Initial Value Problem .............................................................13
2.1.6 Boundary Value Problem ......................................................14
References .......................................................................................................15
3. Multilayer Artificial Neural Network .......................................................17
3.1 Structure of Multilayer ANN Model ................................................18
3.2 Formulations and Learning Algorithm of Multilayer
ANN Model .........................................................................................18
3.2.1 General Formulation of ODEs Based on ANN Model .....18
v
vi Contents
3.2.2 Formulation of nth-Order IVPs ............................................20
3.2.2.1 Formulation of First-Order IVPs ..........................21
3.2.2.2 Formulation of Second-Order IVPs .....................21
3.2.3 Formulation of BVPs .............................................................22
3.2.3.1 Formulation of Second-Order BVPs ....................22
3.2.3.2 Formulation of Fourth-Order BVPs .....................23
3.2.4 Formulation of a System of First-Order ODEs ..................24
3.2.5 Computation of Gradient of ODEs for Multilayer
ANN Model ............................................................................25
3.3 First-Order Linear ODEs ....................................................................27
3.4 Higher-Order ODEs ............................................................................32
3.5 System of ODEs ...................................................................................34
References .......................................................................................................36
4. Regression-Based ANN ...............................................................................37
4.1 Algorithm of RBNN Model ..............................................................37
4.2 Structure of RBNN Model .................................................................39
4.3 Formulation and Learning Algorithm of RBNN Model ...............39
4.4 Computation of Gradient for RBNN Model ...................................40
4.5 First-Order Linear ODEs ....................................................................40
4.6 Higher-Order Linear ODEs ...............................................................50
References .......................................................................................................56
5. Single-Layer Functional Link Artificial Neural Network .....................57
5.1 Single-Layer FLANN Models............................................................58
5.1.1 ChNN Model ..........................................................................58
5.1.1.1 Structure of the ChNN Model ..............................58
5.1.1.2 Formulation of the ChNN Model ........................59
5.1.1.3 Gradient Computation of the ChNN Model ......60
5.1.2 LeNN Model ...........................................................................62
5.1.2.1 Structure of the LeNN Model ...............................62
5.1.2.2 Formulation of the LeNN Model .........................63
5.1.2.3 Gradient Computation of the LeNN Model .......63
5.1.3 HeNN Model ..........................................................................64
5.1.3.1 Architecture of the HeNN Model ........................64
5.1.3.2 Formulation of the HeNN Model ........................65
5.1.4 Simple Orthogonal Polynomial–Based Neural
Network (SOPNN) Model ....................................................66
5.1.4.1 Structure of the SOPNN Model ............................66
5.1.4.2 Formulation of the SOPNN Model ......................67
5.1.4.3 Gradient Computation of the SOPNN Model .....68
5.2 First-Order Linear ODEs ....................................................................68
5.3 Higher-Order ODEs ............................................................................69
5.4 System of ODEs ...................................................................................71
References .......................................................................................................74
Contents vii
6. Single-Layer Functional Link Artificial Neural Network
with Regression-Based Weights .................................................................77
6.1 ChNN Model with Regression-Based Weights ...............................78
6.1.1 Structure of the ChNN Model ..............................................78
6.1.2 Formulation and Gradient Computation
of the ChNN Model ...............................................................79
6.2 First-Order Linear ODEs ....................................................................79
6.3 Higher-Order ODEs ............................................................................83
References .......................................................................................................85
7. Lane–Emden Equations ...............................................................................87
7.1 Multilayer ANN-Based Solution of Lane–Emden Equations .......89
7.2 FLANN-Based Solution of Lane–Emden Equations ......................93
7.2.1 Homogeneous Lane–Emden Equations .............................94
7.2.2 Nonhomogeneous Lane–Emden Equation ......................101
References .....................................................................................................102
8. Emden–Fowler Equations .........................................................................105
8.1 Multilayer ANN-Based Solution of Emden–Fowler
Equations ............................................................................................106
8.2 FLANN-Based Solution of Emden–Fowler Equations ................110
References .....................................................................................................113
9. Duffing Oscillator Equations ...................................................................117
9.1 Governing Equation .........................................................................117
9.2 Unforced Duffing Oscillator Equations .........................................118
9.3 Forced Duffing Oscillator Equations ..............................................123
References .....................................................................................................131
10. Van der Pol–Duffing Oscillator Equation ..............................................133
10.1 Model Equation .................................................................................134
10.2 Unforced Van der Pol–Duffing Oscillator Equation ....................135
10.3 Forced Van der Pol–Duffing Oscillator Equation .........................135
References .....................................................................................................144
Index .....................................................................................................................147

Preface
Differential equations play a vital role in various fields of science and engi-
neering. Many real-world problems in engineering, mathematics, physics,
chemistry, economics, psychology, defense, etc., may be modeled by ordi-
nary or partial differential equations. In most of the cases, an analytical/
exact solution of differential equations may not be obtained easily. Therefore,
various types of numerical techniques have been developed by researchers
to solve such equations, such as Euler, Runge–Kutta, predictor-corrector,
finite difference, finite element, and finite volume techniques. Although
these methods provide good approximations to the solution, they require
the discretization of the domain into a number of finite points/elements, in
general. These methods provide solution values at predefined points, and
computational complexity increases with the number of sampling points.
In recent decades, among the various machine intelligence procedures,
artificial neural network (ANN) methods have been established as power-
ful techniques to solve a variety of real-world problems because of ANN’s
excellent learning capacity. ANN is a computational model or an information
processing paradigm inspired by the biological nervous system. Recently,
a lot of attention has been devoted to the study of ANN for solving dif-
ferential equations. The approximate solution of differential equations by
ANN is found to be advantageous but it depends upon the ANN model that
one considers. Here, our target is to handle ordinary differential equations
(ODEs) using ANN. The traditional numerical methods are usually iterative
in nature, where we fix the step size before the start of the computation. After
the solution is obtained, if we want to know the solution between steps, the
procedure needs to be repeated from the initial stage. ANN may be one of the
ways to overcome this repetition of iterations.
In solving differential equations by ANN, no desired values are known and
the output of the model can be generated by training only. As per the existing
training algorithm, the architecture of a neural model is problem dependent
and the number of nodes, etc., is taken by trial-and-error method where the
training depends upon the weights of the connecting nodes. In general, these
weights are taken as random numbers that dictate the training. This book
includes recently developed new ANN models to handle various types of
ODES. A brief outline of each chapter is given next.
The preliminaries of ANN are presented in Chapter 1. Definitions of ANN
architecture, paradigm of learning, activation functions, leaning rules and
learning processes, etc., are reviewed here. Chapter 2 describes preliminaries
of ODEs. We recall the definitions that are relevant to the present book such
ix