Table Of ContentAdvances in Numerical Mathematics
Edited by
Prof. Dr. Dres. h. c. Hans Georg Bock
Prof. Dr. Dr. h. c. Wolfgang Hackbusch
Prof. Mitchell Luskin
Prof. Dr. Dr. h. c. Rolf Rannacher
Andreas Potschka
A Direct Method for
Parabolic PDE Con-
strained Optimization
Problems
Andreas Potschka
Heidelberg, Germany
Dissertation Heidelberg University, Germany, 2011
ISSN 1616-2994
ISBN 978-3-658-04475-6 ISBN 978-3-658-04476-3 (eBook)
DOI 10.1007/978-3-658-04476-3
Th e Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografi e;
detailed bibliographic data are available in the Internet at http://dnb.d-nb.de.
Library of Congress Control Number: 2013955622
Springer Spektrum
© Springer Fachmedien Wiesbaden 2014
Th is work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifi cally the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfi lms or in any other physical
way, and transmission or information storage and retrieval, electronic adaptation, compu-
ter soft ware, or by similar or dissimilar methodology now known or hereaft er developed.
Exempted from this legal reservation are brief excerpts in connection with reviews or schol-
arly analysis or material supplied specifi cally for the purpose of being entered and executed
on a computer system, for exclusive use by the purchaser of the work. Duplication of this
publication or parts thereof is permitted only under the provisions of the Copyright Law of
the Publisher’s location, in its current version, and permission for use must always be obtained
from Springer. Permissions for use may be obtained through RightsLink at the Copyright
Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
Th e use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specifi c statement, that such names are
exempt from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date
of publication, neither the authors nor the editors nor the publisher can accept any legal re-
sponsibility for any errors or omissions that may be made. Th e publisher makes no warranty,
express or implied, with respect to the material contained herein.
Printed on acid-free paper
Springer Spektrum is a brand of Springer DE.
Springer DE is part of Springer Science+Business Media.
www.springer-spektrum.de
Acknowledgments
IsubmittedthismathematicaldoctoralthesisinAugust2011tothefacultyofnat-
ural sciences and mathematics of Heidelberg University and successfully passed
itsdisputationonDecember19,2011.
IacknowledgethesupportbyDeutscheForschungsgemeinschaft(DFG)within
theInternationalesGraduiertenkolleg710Complexprocesses:Modeling,Simula-
tionandOptimization,undergrantBO864/12-1withintheSchwerpunktprogramm
1253OptimizationwithPartialDifferentialEquations,andwithintheHeidelberg
Graduate School for Mathematical and Computational Methods in the Sciences
(HGS). I am also grateful for support by Bundesministerium fu¨r Bildung und
Forschung (BMBF) under grant 03BONCHD. Furthermore I would like to ac-
knowledge support by the European Commission within the project Embedded
OptimizationforResourceConstrainedPlatforms(EMBOCON).
Iwouldespeciallyliketothank: myadvisersHansGeorgBockandRolfRan-
nacher; my mentors Sebastian Sager and Johannes Schlo¨der; my collaborators
JanVanImpe,SebastianEngell,MoritzDiehl,StefanKo¨rkel,EkaterinaKostina,
Achim Ku¨pper, and Filip Logist; my research stay hosts Moritz Diehl, Andreas
Griewank, Matthias Heinkenschloss, Nick Trefethen, and Fredi Tro¨ltzsch; my
colleagues Jan Albersmeyer, Do¨rte Beigel, Kathrin Hatz, Christian Hoffmann,
Christian Kirches, Simon Lenz, Andreas Sommer, Andreas Schmidt, and Leo
Wirsching; my rocking friends Tommy Binford, Eddie Castillo, Rick Castillo,
RogerFletcher, Sean Hardesty, DenisRidzal; my veryclosefriends TomKraus,
Anita and Micha Strauß, Evelyn and Tobias Stuwe; my siblings Susanne and
Manuel;andmyparentsMariaandMichael.
I am indebted to Falk Hante, Christian Kirches, Mario Mommer, Sebastian
Sager,andJohannesSchlo¨derforvaluablecommentsonafirstdraftofthisthesis.
AndreasPotschka
Abstract
InthisthesiswedevelopanumericalmethodbasedonDirectMultipleShooting
forOptimalControlProblems(OCPs)constrainedbytime-periodicPartialDiffer-
ential Equations (PDEs). The proposed method features asymptotically optimal
scale-up of the numerical effort with the number of spatial discretization points.
ItconsistsofaLinearIterativeSplittingApproach(LISA)withinaNewton-type
iterationwithglobalizationonthebasisofnaturallevelfunctions. Weinvestigate
theLISA-NewtonmethodintheframeworkofBock’sκ-theoryanddevelopreli-
ablea-posterioriκ-estimators.MoreoverweextendtheinexactNewtonmethodto
aninexactSequentialQuadraticProgramming(SQP)methodforinequalitycon-
strainedproblemsandprovidelocalconvergencetheory. Inadditionwedevelop
aclassicalandatwo-gridNewton-PicardpreconditionerforLISAandprovegrid-
independentconvergenceoftheclassicalvariantforamodelproblem. Basedon
numerical results we can claim that the two-grid version is even more efficient
thantheclassicalversionfortypicalapplicationproblems. Moreoverwedevelop
atwo-gridapproximationfortheLagrangianHessianwhichfitswellinthetwo-
grid Newton-Picard framework and yields a reduction of 68% in runtime for a
nonlinearbenchmarkproblemcomparedtotheuseoftheexactLagrangianHes-
sian.Weshowthatthequalityofthefinegridcontrolstheaccuracyofthesolution
whilethequalityofthecoarsegriddeterminestheasymptoticlinearconvergence
rate,i.e.,Bock’sκ. Basedonreliableκ-estimatorswefacilitateautomaticcoarse
grid refinement to guarantee fast convergence. For the solution of the occurring
large-scale Quadratic Programming Problems (QPs) we develop a structure ex-
ploiting two-stage approach. In the first stage we exploit the Multiple Shooting
and Newton-Picard structure to reduce the large-scale QP to an equivalent QP
whosesizeisindependentofthenumberofspatialdiscretizationpoints. Forthe
secondstagewedevelopextensionsforaParametricActiveSetMethod(PASM)
toachieveareliableandefficientsolverfortheresulting,possiblynonconvexQP.
Furthermoreweconstructthreeillustrative,counter-intuitivetoyexampleswhich
showthatconvergenceofaone-shotone-stepoptimizationmethodisneithernec-
essarynorsufficientfortheconvergenceoftheforwardproblemmethod.Forthree
regularizationapproachestorecoverconvergenceouranalysisshowsthatde-facto
loss of convergence cannot be avoided with these approaches. We have further
implementedtheproposedmethodswithinacodecalledMUSCOPwhichfeatures
VIII Abstract
automaticderivativegenerationforthemodelfunctionsanddynamicsystemsolu-
tionsoffirstandsecondorder,parallelizationontheMultipleShootingstructure,
andahybridlanguageprogrammingparadigmtominimizesetupandsolutiontime
for new application problems. We demonstrate the applicability, reliability, and
efficiencyofMUSCOPandthustheproposednumericalmethodsandtechniques
onasequenceofPDEOCPsofgrowingdifficultyrangingfromlinearacademic
problems, over highly nonlinear academic problems of mathematical biology to
ahighlynonlinearreal-worldchemicalengineeringprobleminpreparativechro-
matography:TheSimulatedMovingBed(SMB)process.
Contents
Acknowledgments V
Abstract VII
1 Introduction 1
1.1 Resultsofthisthesis . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Thesisoverview . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
I Theoreticalfoundations 9
2 Problemformulation 11
2.1 DynamicalmodelsdescribedbyPartialDifferentialEquations . . 11
2.2 CoupledODEsandPDEs . . . . . . . . . . . . . . . . . . . . . . 15
2.3 TheOptimalControlProblem . . . . . . . . . . . . . . . . . . . 15
3 DirectOptimization:Problemdiscretization 19
3.1 Discretize-then-optimizeapproach . . . . . . . . . . . . . . . . . 19
3.2 MethodOfLines:Discretizationinspace . . . . . . . . . . . . . 20
3.3 DirectMultipleShooting:Discretizationintime . . . . . . . . . . 23
3.4 Discretizationofpathconstraints . . . . . . . . . . . . . . . . . . 25
3.5 TheresultingNonlinearProgrammingProblem . . . . . . . . . . 25
4 Elementsofoptimizationtheory 27
4.1 Basicdefinitions. . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2 Necessaryoptimalityconditions . . . . . . . . . . . . . . . . . . 28
II Numericalmethods 31
5 InexactSequentialQuadraticProgramming 33
5.1 Newton-typemethods . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2 Localconvergence . . . . . . . . . . . . . . . . . . . . . . . . . 36
X Contents
5.3 Globalizationofconvergence . . . . . . . . . . . . . . . . . . . . 38
5.3.1 Generalizedlevelfunctions. . . . . . . . . . . . . . . . . 39
5.3.2 TheNewtonpath . . . . . . . . . . . . . . . . . . . . . . 40
5.3.3 ThenaturallevelfunctionandtheNaturalMonotonicityTest 41
5.4 ARosenbrock-typeexample . . . . . . . . . . . . . . . . . . . . 45
5.5 TheRestrictiveMonotonicityTest . . . . . . . . . . . . . . . . . 46
5.6 NaturalMonotonicityforLISA-Newtonmethods . . . . . . . . . 46
5.6.1 Estimatesfornaturalmonotonicity . . . . . . . . . . . . . 47
5.6.2 ALinearIterativeSplittingApproach . . . . . . . . . . . 54
5.6.3 GINKOAlgorithm . . . . . . . . . . . . . . . . . . . . . 67
5.7 Inequalityconstrainedoptimizationproblems . . . . . . . . . . . 68
5.7.1 SQPwithexactderivatives . . . . . . . . . . . . . . . . . 68
5.7.2 InexactSQP . . . . . . . . . . . . . . . . . . . . . . . . 72
6 Newton-Picardpreconditioners 75
6.1 TheNewton-Picardmethodforfindingperiodicsteadystates . . . 76
6.2 Discretizationofthemodelproblem . . . . . . . . . . . . . . . . 77
6.3 Newton-Picardforoptimalcontrolproblems . . . . . . . . . . . . 80
6.3.1 Generalconsiderations . . . . . . . . . . . . . . . . . . . 81
6.3.2 SimultaneousNewton-Picarditeration . . . . . . . . . . . 81
6.3.3 ConvergenceforclassicalNewton-Picard . . . . . . . . . 85
6.3.4 Numericalsolutionoftheapproximatedlinearsystem . . 90
6.3.5 Pseudocode . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.3.6 Algorithmiccomplexity . . . . . . . . . . . . . . . . . . 95
6.4 ExtensiontononlinearproblemsandMultipleShooting . . . . . . 97
7 One-shotone-stepmethodsandtheirlimitations 99
7.1 Illustrative,counter-intuitiveexamplesinlowdimensions . . . . . 101
7.1.1 Fastforwardconvergence,optimizationdivergence . . . . 102
7.1.2 Forwarddivergence,fastoptimizationconvergence . . . . 102
7.2 Subproblemregularization . . . . . . . . . . . . . . . . . . . . . 103
7.3 Analysisoftheregularizedsubproblems . . . . . . . . . . . . . . 104
7.3.1 Primalregularization . . . . . . . . . . . . . . . . . . . . 105
7.3.2 Primal-dualregularization . . . . . . . . . . . . . . . . . 106
7.3.3 Hemi-primalregularization. . . . . . . . . . . . . . . . . 107
7.3.4 Divergenceanddefactolossofconvergenceforsubprob-
lemregularizations . . . . . . . . . . . . . . . . . . . . . 109
8 Condensing 111
Contents XI
8.1 Multipleshootingstructure . . . . . . . . . . . . . . . . . . . . . 111
8.2 Newton-Picardstructure . . . . . . . . . . . . . . . . . . . . . . 115
8.3 EliminationofdiscretizedPDEstates . . . . . . . . . . . . . . . 118
8.4 Newton-PicardHessianapproximation . . . . . . . . . . . . . . . 120
8.5 ScalinginvarianceoftheNewton-PicardLISA-Newtonmethod . . 121
9 AParametricActiveSetmethodforQPsolution 123
9.1 GeneralremarksonQuadraticProgrammingProblems . . . . . . 123
9.1.1 Optimalityconditions . . . . . . . . . . . . . . . . . . . 124
9.1.2 Existingmethods . . . . . . . . . . . . . . . . . . . . . . 124
9.1.3 Existingsoftware . . . . . . . . . . . . . . . . . . . . . . 125
9.2 ParametricActiveSetmethods . . . . . . . . . . . . . . . . . . . 127
9.2.1 TheParametricQP . . . . . . . . . . . . . . . . . . . . . 127
9.2.2 TheParametricQuadraticProgrammingalgorithm . . . . 127
9.2.3 Farbounds . . . . . . . . . . . . . . . . . . . . . . . . . 132
9.3 Fundamentalnumericalchallenges . . . . . . . . . . . . . . . . . 133
9.4 Strategiestomeetnumericalchallenges . . . . . . . . . . . . . . 134
9.4.1 Roundingerrorsandill-conditioning. . . . . . . . . . . . 134
9.4.2 Comparisonwithzero . . . . . . . . . . . . . . . . . . . 136
9.4.3 Cyclingandties . . . . . . . . . . . . . . . . . . . . . . 137
9.5 Thecoderpasm:APASMinMatlab® . . . . . . . . . . . . . . . 139
9.6 Comparisonwithexistingsoftware . . . . . . . . . . . . . . . . . 139
9.6.1 Criteriaforcomparison . . . . . . . . . . . . . . . . . . . 139
9.6.2 Discussionofnumericalresults . . . . . . . . . . . . . . 140
9.7 DrawbacksoftheproposedPASM . . . . . . . . . . . . . . . . . 142
9.8 NonconvexQuadraticPrograms . . . . . . . . . . . . . . . . . . 143
10 Automaticderivativegeneration 145
10.1 AlgorithmicDifferentiation . . . . . . . . . . . . . . . . . . . . . 146
10.2 TheprincipleofIND . . . . . . . . . . . . . . . . . . . . . . . . 147
10.3 INDforimplicittime-steppingwithmonitorstrategy . . . . . . . 148
10.4 NumericaleffortofIND . . . . . . . . . . . . . . . . . . . . . . 151
11 ThesoftwarepackageMUSCOP 153
11.1 Programmingparadigms . . . . . . . . . . . . . . . . . . . . . . 153
11.1.1 Hybridlanguageprogramming . . . . . . . . . . . . . . . 153
11.1.2 Nodataencapsulation . . . . . . . . . . . . . . . . . . . 154
11.1.3 Algorithmcenterednotmodelcentered . . . . . . . . . . 155
11.1.4 ReverseCommunicationinterface . . . . . . . . . . . . . 155
