Table Of ContentSPRINGER BRIEFS IN STATISTICS
Christos P. Kitsos
Optimal
Experimental
Design for
Non-Linear Models
Theory and
Applications
123
SpringerBriefs in Statistics
For furthervolumes:
http://www.springer.com/series/8921
Christos P. Kitsos
Optimal Experimental
Design for Non-Linear
Models
Theory and Applications
123
Christos P.Kitsos
Department of Informatics
Technological Educational Institute
of Athens
Athens
Greece
ISSN 2191-544X ISSN 2191-5458 (electronic)
ISBN 978-3-642-45286-4 ISBN 978-3-642-45287-1 (eBook)
DOI 10.1007/978-3-642-45287-1
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2013956330
(cid:2)TheAuthor(s)2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe
work. Duplication of this publication or parts thereof is permitted only under the provisions of
theCopyright Law of the Publisher’s location, in its current version, and permission for use must
always be obtained from Springer. Permissions for use may be obtained through RightsLink at the
CopyrightClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Acknowledgments
I would like to thank Assistant Prof. V. Zarikas (Lamia), and part-time Assistant
Profs. T. Toulias, Y. Hatzikian, and K. Kolovos (Athens), for useful discussions
during the preparation of this monograph. I would like to thank the reviewers for
their time working with the text. They made very sharp and useful comments,
which are very much appreciated. The encouragement of Springer, BioMathe-
matics/Statistics, is very much appreciated.
v
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Motivation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Notation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Assumptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 On the Existence of Estimators. . . . . . . . . . . . . . . . . . . . . . . . 8
2.5 Fisher’s Information Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Linearization of the Model. . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.6.1 Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Locally Optimal Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 Special Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.5 Canonical Form of a Design (c(h)-Optimality) . . . . . . . . . . . . . 21
3.6 Designs for Subsets of Parameters. . . . . . . . . . . . . . . . . . . . . . 23
3.7 Geometrical Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.8 Partially Nonlinear Models. . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.9 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 Static Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 Locally Optimal Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.3 Lauter’s Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.4 Stone and Morris Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.5 Maxi–Min Criterion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.6 Constant Information Designs. . . . . . . . . . . . . . . . . . . . . . . . . 34
4.7 Applications from Chemical Kinetics. . . . . . . . . . . . . . . . . . . . 35
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
vii
viii Contents
5 Sequential Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.3 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.4 Binary Response Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.5 Stochastic Approximation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5.6 Application to Cancer Problems . . . . . . . . . . . . . . . . . . . . . . . 49
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6 Approximate Confidence Regions. . . . . . . . . . . . . . . . . . . . . . . . . 51
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.3 Confidence Regions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6.4 Simulation Study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6.5 The Michaelis–Menten Model of Pharmacokinetics. . . . . . . . . . 55
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7 Simulation Studies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.2 The Dilution Series Assessment . . . . . . . . . . . . . . . . . . . . . . . 59
7.3 The Strategy of Simulation I. . . . . . . . . . . . . . . . . . . . . . . . . . 62
7.4 Simulation Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7.5 Discussion I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
7.6 The First Order Growth Law . . . . . . . . . . . . . . . . . . . . . . . . . 67
7.7 Strategy and Procedures of Simulation II . . . . . . . . . . . . . . . . . 68
7.8 Discussion II. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
8 Optimal Design in Rythmometry . . . . . . . . . . . . . . . . . . . . . . . . . 73
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
8.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
8.3 D-Optimal Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
8.4 c-Optimal Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
8.5 Restricted Design Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
8.6 Synopsis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
8.7 Engineering Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
9 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Appendix 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Appendix 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Contents ix
Appendix 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Appendix 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Appendix 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Chapter 1
Introduction
Abstract This chapter defines the Nonlinear Experimental Design problem and
traces its evolution. It is emphasized that the (continuous) Design Theory started
with a Non-linear problem.
In a brief look back at the origin of the methods discussed in this monograph, it
seems that the first real-life problems discussed were nonlinear rather than linear.
The Least Square Method originated from a nonlinear problem, thanks to the
Mozart of Mathematics: Carl Friedrich Gauss (1777–1855). Experimental Design
theory, so called by its founder Sir Ronald Aylmer Fisher (1890–1962), also
originated from a nonlinear problem. I shall briefly trace these problems.
Oneofthemostfamousdatasetsistheonecollectedin1793fordefininganew
measurementunit:themeter,seethecontributionofStigler(1981)tothehistoryof
Statistics. What is also very important is that for this particular data set:
• In recent statistical terms, a nonlinear function had to be finally estimated,
between the modulus of arc length (s), latitude (d) and meridian quadrant (L).
• A linear approximation was used in 1755 for this problem and a second order
expansion was applied in 1832.
The meter was eventually defined as equal to one 10,000,000th part of the
meridian quadrant, i.e. the distance from the North Pole to the Equator along a
paralleloflongitudepassingthroughParis,andthelinearapproximationwasinthe
form:
s
gðLÞ¼ ¼h þh (cid:2)sin2L
d 1 2
With h being the ‘‘length of a degree at the equator’’ and h ‘‘the excess of a
1 2
degree at the pole over one at the equator’’. Then the ellipticity (e) wasestimated
through h and h by the following nonlinear relationship:
1 2
1 h 3
¼3(cid:2) 1þ
e h 2
2
C.P.Kitsos,OptimalExperimentalDesignforNon-LinearModels, 1
SpringerBriefsinStatistics,DOI:10.1007/978-3-642-45287-1_1,
(cid:2)TheAuthor(s)2013
2 1 Introduction
ItisclearthatGauss,seePlackett(1972)fordetails,treatedthedatainhisown
remarkable mathematical way, considering a nonlinear function.
The first statistical treatment of a nonlinear function came from the pioneer of
modern Statistics, R. A. Fisher. He started work at Rothamsted Experimental
Station in 1919, and around 1922 he came across what is known as the dilution
seriesproblem.Abriefdescriptionisasfollows.Forasmallvolumeutakenfrom
the volume V of a liquid containing n tiny organisms (such as bacteria), the
probability p that u contains no organisms can be evaluated as
(cid:2) u(cid:3)n
p¼ 1(cid:3) ffiexpð(cid:3)nu=VÞ¼expð(cid:3)huÞ:
V
Theparameterh,thedensityperunitvolume,hastobeestimated(seedetailsin
Sect.7.2).Thequestionishowoneshouldperformtheexperimenttogetthe best
possible estimate. The probability p in the above relation is expressed as a non-
linear function. Fisher solved this nonlinear problem in 1922, using a concept of
his own: his information.
Since Fisher’s pioneering work in experimental design, Statistics has become
involved in all experimental sciences: Chemistry, Biology, Pharmacology,
Industry,Psychology,Toxicologyandsoon.Ofcoursestatisticiansdonotprovide
methodsfordesigningexperimentsinisolation.However,incooperationwiththe
experimenter, for whom the objective of an experiment is clear, the statistician
provides the most informative pattern of the experiment so that the required
objective can be achieved.
The objectives of the experimenter can be:
1. Toobtainanestimateforaresponseyinsomeparticularregionusingtheinput
variable vector u¼ðu ;u ;...;u Þ. This is the response surface problem
1 2 k
introduced by Box and Draper (1959).
2. To determine the best mathematical model describing most precisely the
investigated phenomenon. This is the discrimination problem between rival
models and was reviewed by Hill (1976).
3. In some sense, all or a subset of the parameters are to be estimated as well as
possible. This is the optimal experimental design problem originated by Smith
(1918).
The above-mentioned objectives are common to linear and nonlinear experi-
mental designs (LED and NLED), i.e. when the assumed suitable (and correct)
model, describing the underlying phenomenon, is linear or nonlinear with respect
to its parameters.
There is no such volume of review work in NLED, although work on experi-
mentaldesignsstartedwithanonlinearproblem,thankstoGaussandFisher.Some
work has been performed by Kitsos (1986), Ford et al. (1989) reviewed the prob-
lem, Kitsos (1989) worked with the sequential procedures and Abdelbasit and
