Forecasting with Maximum Entropy: The Interface Between Physics, Biology, Economics and Information Theory

221 Pages·2022·17.007 MB·English

by  Hugo Fort

Forecasting with
Maximum Entropy
The interface between physics, biology, economics and information theory
Online at: https://doi.org/10.1088/978-0-7503-3931-5

Forecasting with
Maximum Entropy
The interface between physics, biology, economics and information theory
Hugo Fort
Department of Physics, Republic University, Montevideo, Uruguay
IOP Publishing, Bristol, UK
ªIOPPublishingLtd2022
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem
ortransmittedinanyformorbyanymeans,electronic,mechanical,photocopying,recording
orotherwise,withoutthepriorpermissionofthepublisher,orasexpresslypermittedbylawor
undertermsagreedwiththeappropriaterightsorganization.Multiplecopyingispermittedin
accordancewiththetermsoflicencesissuedbytheCopyrightLicensingAgency,theCopyright
ClearanceCentreandotherreproductionrightsorganizations.
CertainimagesinthispublicationhavebeenobtainedbytheauthorfromtheWikipedia/
Wikimediawebsite,wheretheyweremadeavailableunderaCreativeCommonslicenceorstated
tobeinthepublicdomain.Pleaseseeindividualfigurecaptionsinthispublicationfordetails.To
theextentthatthelawallows,IOPPublishingdisclaimanyliabilitythatanypersonmaysufferasa
resultofaccessing,usingorforwardingtheimages.Anyreuserightsshouldbecheckedand
permissionshouldbesoughtifnecessaryfromWikipedia/Wikimediaand/orthecopyrightowner
(asappropriate)beforeusingorforwardingtheimages.
PermissiontomakeuseofIOPPublishingcontentotherthanassetoutabovemaybesought
[email protected].
HugoForthasassertedhisrighttobeidentifiedastheauthorofthisworkinaccordancewith
sections77and78oftheCopyright,DesignsandPatentsAct1988.
ISBN 978-0-7503-3931-5(ebook)
ISBN 978-0-7503-3929-2(print)
ISBN 978-0-7503-3932-2(myPrint)
ISBN 978-0-7503-3930-8(mobi)
DOI 10.1088/978-0-7503-3931-5
Version:20221101
IOPebooks
BritishLibraryCataloguing-in-PublicationData:Acataloguerecordforthisbookisavailable
fromtheBritishLibrary.
PublishedbyIOPPublishing,whollyownedbyTheInstituteofPhysics,London
IOPPublishing,No.2TheDistillery,Glassfields,AvonStreet,Bristol,BS20GR,UK
USOffice:IOPPublishing,Inc.,190NorthIndependenceMallWest,Suite601,Philadelphia,
PA19106,USA
To the memory of my parents.

Contents
Preface xi
Acknowledgements xvii
Author biography xviii
List of acronyms xix
1 Entropy as missing information: from Shannon’s information 1-1
theory to Jaynes’ maximum entropy principle
1.1 Information and its processing in biology, economics and physics 1-2
1.2 Uncertainty in communication systems: Shannon entropy 1-4
1.2.1 First attempts to quantify information content 1-4
1.2.2 Shannon’s information entropy 1-7
1.3 Entropy as missing information 1-13
1.4 Working with incomplete information: the principle of maximum 1-17
entropy to find minimally prejudiced distributions
1.4.1 Frequentist or physical probability versus subjective or 1-18
Bayesian probability
1.4.2 MaxEnt as a method of making predictions from limited 1-19
data by assuming maximal ignorance
References 1-25
2 The synthesis of information theory and thermodynamics: 2-1
Shannon entropy and Boltzmann entropy are the same thing
2.1 Basics of statistical physics 2-2
2.1.1 The program of statistical physics: from microphysics to 2-2
macrophysics
2.1.2 Entropy and the Second Law of Thermodynamics: from 2-4
the efficiency of heat engines and refrigerators to
entropy as disorder
2.1.3 Boltzmann and the statistical character of the Second Law of 2-8
Thermodynamics
2.1.4 Boltzmann–Gibbs maximum entropy approach to statistical 2-11
mechanics
2.1.5 All is in the partition function 2-16
2.2 MaxEnt derivation of statistical mechanics 2-20
2.2.1 The equivalence between Boltzmann’s and Shannon’s entropy 2-21
vii
ForecastingwithMaximumEntropy
2.2.2 Inferring the canonical probability distribution by the 2-22
MaxEnt recipe
2.3 Converting information into energy: from Maxwell’s demon to 2-23
Landauer’s eraser
2.4 Conclusion 2-30
References 2-31
3 Elements of physical biology: the Lotka–Volterra equations 3-1
3.1 The kinetic formulation of population dynamics 3-2
3.1.1 One isolated species 3-2
3.1.2 Many interacting species 3-4
3.2 The Lotka–Volterra linear model for single-trophic communities 3-8
3.2.1 Obtaining the model parameters from monoculture and 3-9
biculture experiments
3.2.2 Quantifying the accuracy of the linear model for predicting 3-11
species yields in single-trophic communities
3.3 The statistical mechanics of populations 3-17
3.3.1 Rationale and first attempts 3-17
3.3.2 MaxEnt formulations 3-18
3.3.3 Uses of statistical mechanics-inspired lattice models I: 3-19
overgrazing of semi-arid lands
3.3.4 Uses of statistical mechanics-inspired lattice models II: 3-25
modelling the dynamics of biodiversity of communities
of trees in tropical forests
3.4 Conclusion 3-32
Appendix A: Equilibrium stability in population ecology 3-33
References 3-45
4 Economics as physics, economics as biology 4-1
4.1 Economics as social physics, physics as Nature’s economics 4-2
4.2 Neoclassical economics 4-4
4.2.1 The homo economicus of rational choice theory 4-5
4.2.2 The marginalist revolution and the general 4-5
equilibrium theory
4.3 Economics as biology, or evolutionary economics 4-7
4.3.1 The irrationality of rational decision theory 4-7
4.3.2 Evolutionary economics 4-9
viii
ForecastingwithMaximumEntropy
4.4 Selection dynamics 4-11
4.4.1 The simplest selection model 4-11
4.4.2 Frequency dependent selection: the replicator 4-13
dynamics equation
4.5 Linking selection dynamics with ecology and physics 4-15
4.5.1 The equivalence of the replicator dynamics with the 4-18
generalized Lotka–Volterra equations
4.5.2 The selection equations regarded as master equations 4-20
4.6 Innovation through mutations 4-21
4.6.1 Evolution as a two-step mutation–selection process 4-21
4.6.2 Mutation–selection equations: from the Crow–Kimura 4-24
equation to the replicator–mutator equation
4.7 Implementing evolution in economics 4-25
4.8 The ‘Marshall problem’ or a transdisciplinary synthetic 4-27
perspective of economics
References 4-28
5 Inferring effective interaction matrices through MaxEnt 5-1
5.1 Working with imperfect information 5-2
5.2 The Lotka–Volterra maximum entropy interaction matrix 5-2
5.2.1 Choosing the right constraints 5-2
5.2.2 The MaxEnt interaction matrix and its properties 5-4
5.2.3 A non-symmetrical MaxEnt interaction matrix 5-6
5.3 How good is the pairwise approximation? 5-7
References 5-8
6 Early warning indications of species crashes from effective 6-1
intraspecific interactions in tropical forests
6.1 Background: diversity loss and early warning signals 6-2
6.1.1 On fluctuations of biodiversity and what drives these changes 6-2
6.1.2 Early warning signals 6-3
6.2 Goal 6-3
6.3 Data 6-4
6.4 Estimating the interaction matrix through MaxEnt 6-5
6.5 Intraspecific competition interactions are enough to predict the 6-7
trajectories of tree species
6.6 A new early warning signal 6-8
ix