Table Of ContentAdvanced Probability and Statistics:
Applications to Physics and
Engineering
Advanced Probability and Statistics:
Applications to Physics and
Engineering
Harish Parthasarathy
Professor
Electronics & Communication Engineering
Netaji Subhas Institute of Technology (NSIT)
New Delhi, Delhi-110078
Firstpublished2023
byCRCPress
4ParkSquare,MiltonPark,Abingdon,Oxon,OX144RN
andbyCRCPress
6000BrokenSoundParkwayNW,Suite300,BocaRaton,FL33487-2742
©2023HarishParthasarathyandManakinPress
CRCPressisanimprintofInformaUKLimited
TherightofHarishParthasarathytobeidentifiedasauthorofthisworkhasbeenassertedin
accordancewithsections77and78oftheCopyright,DesignsandPatentsAct1988.
Allrightsreserved.Nopartofthisbookmaybereprintedorreproducedorutilisedinany
formorbyanyelectronic,mechanical,orothermeans,nowknownorhereafterinvented,
includingphotocopyingandrecording,orinanyinformationstorageorretrievalsystem,
withoutpermissioninwritingfromthepublishers.
Forpermissiontophotocopyorusematerialelectronicallyfromthiswork,accesswww.
copyright.comorcontacttheCopyrightClearanceCenter,Inc.(CCC),222RosewoodDrive,
Danvers,MA01923,978-750-8400.ForworksthatarenotavailableonCCCpleasecontact
[email protected]
Trademarknotice:Productorcorporatenamesmaybetrademarksorregisteredtrademarks,
andareusedonlyforidentificationandexplanationwithoutintenttoinfringe.
PrinteditionnotforsaleinSouthAsia(India,SriLanka,Nepal,Bangladesh,Pakistanor
Bhutan).
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
LibraryofCongressCataloging-in-PublicationData
Acatalogrecordhasbeenrequested
ISBN:9781032384375(hbk)
ISBN:9781032384382(pbk)
ISBN:9781003345060(ebk)
DOI:10.1201/9781003345060
TypesetinArial,MinionPro,Symbol,CalisMTBol,TimesNewRoman,RupeeForadian,
Wingdings,ZapDingbats,Euclid,MT-Extra
byManakinPress,Delhi
Preface
This book is primarily a book on advanced probability and statistics that
could be useful for undergraduate and postgraduate students of physics, engi-
neering and applied mathematics who desire to learn about the applications of
classical and quantum probability to problems of classical physics, signal pro-
cessing and quantum physics and quantum field theory. The prerequisites for
reading this book are basic measure theoretic probability, linear algebra, differ-
ential equations, stochastic differential equations, group representation theory
andquantummechanics. Thebookdealswithclassicalandquantumprobabili-
tiesincludingadecentdiscussionofBrownianmotion,Poissonprocessandtheir
quantum non-commutative analogues. The basic results of measure theoretic
integrationwhichareimportantinconstructingtheexpectationofrandomvari-
ables are discussed. The Kolmogorov consistency theorem for the existence of
stochastic processes having given consistent finite dimensional probability dis-
tributionsisalsooutlined. FordoingquantumprobabilityinBosonFockspace,
we require the construction of the tensor product between Hilbert spaces. This
construction based on the GNS principle, Schur’s theorem of positive definite
matricesandKolmogorov’sconsistencytheoremhasbeenoutlined. Thelawsof
largenumbersforsumsofindependentrandomvariablesareintroducedhereand
we state the fundamental inequalities and properties of Martingales originally
due to J.L.Doob culminating finally in the proof of the Martingale convergence
theorembasedonthedowncrossing/upcrossinginequalities. Doob’sMartingale
inequality can be used to give an easy proof of the strong law of large numbers
andwementionithere. Doob’soptionalstoppingtheoremforsubmartingalesis
proved and applied to calculating the distribution of the hitting times of Brow-
nian motion. We give another proof of this distribution based on the reflection
principleofDesire’AndrewhichonceagainrestsonthestrongMarkovproperty
of Brownian motion.
Applications of the theory of stochastic processes especially higher order
statistics and spectra to the characterizing of diseased brain EEG and speech
signals has also been discussed in here. Several different models of diseased
brain data have been proposed here as an application of higher order spectra.
These include (a) modeling the brain signals as the output of Volterra filters to
Gaussian signals and estimating the Volterra coefficients from the higher order
spectra of the output measured EEG, speech and MRI data, and (b) construct-
ingdynamicalmodelsforthespeech,EEGandMRIsignalfielddataassolutions
topartialdifferentialequationsinspace-timewithGaussiannoiseinputandthen
estimatingthe parametersof thesepde’sfrom noisy sparsemeasurementsusing
the extended Kalman filter. We propose this method as a training process for
determiningthediseasedbrainparametersforeachdiseaseandthengivenfresh
data, we estimate the same parameters using the EKF and match it to those
obtained during the training process.
Here, we assume that the reader is familiar with non-linear filtering theory
based on the Kushner-Kallianpur approach. Of course, the derivation of this
filterrequiresknowledgeofconditionalexpectationsandIto’sformulaforBrow-
v
nian motion, and of course also basic facts about Markov processes. We also
in this section discuss group theoretic signal and image processing and apply
these methods also to EEG data analysis in the sense that when the dynam-
ics of the spatio-temporal dynamics of the EEG field data is invariant under a
Lie group of transformations acting on the spatial variables and also when the
noise has G-invariant statistics, how the dynamics can be simplified by taking
group theoretic Fourier transforms based on the irreducible representations of
thegroup. Withasimplifieddynamics,theparametersofthedynamicalsystem
are more easily estimated. We also discuss statistical aspects of 3-dimensional
robot links with random torque using the techniques of Lie-group, Lie algebra
theory applied to the 3-dimensional rotation group. The theory of quantum
neural networks has also been outlined here. It involves estimating dynami-
cally the probability density of a random vector using the modulus square of a
wave function following Schrodinger’s equation and driven by a potential that
represents the difference between the true pde and the Schrodinger pdf. This
is a natural approach since the Schrodinger equation by virtue of its unitary
evolutionhasthepropertyofnaturallygeneratingawholefamilyofprobability
densitiesindexedbytime,nomatterwhatthedrivingpotentialisaslongasitis
real, so that the Hamiltonian is a self-adjoint operator. In this section, we also
introduce the reader to how probabilities of events are computed in quantum
scattering theory based on the wave operators associated with two Hamiltoni-
ans. The ultimate aim of scattering theory is to get to know the asymptotic
probabilitydistributionofscatteredparticleswhentheparticlesarefreeprojec-
tileswhichgettointeractwithascatteringcentre. Oneimportantprobabilistic
feature of scattering theory is to determine the relative average time spent by
the scattered particle within a Borel set in position space and we discuss this
problem here. We also discuss in this section some work carried out by the
author’s doctoral student along with the author on quantum image processing.
This involves encoding classical image fields into quantum states and designing
quantum unitary processors that will denoise this state and then decoding the
processed state back into a classical image field. The other parts of the work
carriedoutbythetheauthor’sdoctoralstudentalongwiththeauthordealswith
estimatingtimevaryingclassicalparametersofaquantumelectromagneticfield
(ie a quantum image field) by allowing the quantum field to interact with an
electron of an atom and measuring the transition probabilities of the electron
between two of its stationary states as a function of time. The EKF has been
appliedtothisproblemtoobtainrealtimeestimatesoftheclassicalparameters
thataregovernedbyaclassicalstochasticdifferentialequation. Applicationsof
V.P.Belavkin’s theory of quantum filtering with non-demolition measurements
taken on the Hudson-Parthasarathy noisy Schrodinger equation are discussed.
These include problems like obtaining real time estimates of the cavity electro-
magnetic field, the cavity gravitational field etc when the bath surrounding the
cavityhasanoisyelectromagneticfield. Thisproblemcanbeviewedasanexer-
ciseinnon-commutativeprobabilitytheory. Wealsopresentsomeworkdealing
with linearization of the Einstein-Maxwell field equations since this work fits
into the framework of linear partial differential equations driven by stochastic
vi
fieldsandhenceisaprobleminadvancedstochasticfieldtheory. Wealsodiscuss
some applications of advanced probability to general electromagnetics and ele-
mentary quantum mechanics like what is the statistics of the far field radiation
pattern produced by a random current source and how when this random elec-
tromagneticfieldisincidentuponanatommodeledbytheSchrodingerorDirac
equation, the stochastically averaged transition probability can be computed in
terms of the classical current correlations. Many other problems in statistical
signal processing like prediction and filtering of stationary time series, Kalman,
Extended Kalman and Unscented Kalman filters, the MUSIC and ESPRIT al-
gorithms for estimating the directions of random signal emitting sources, the
recursive least squares lattice algorithm for order and time recursive prediction
and filtering arediscussed. We have alsoincluded some material on superstring
theory since it is closely connected with the theory of operators in Boson and
Fermion Fock spaces which is now an integral component of non-commutative
probability theory. Some aspects of supersymmetry have also been discussed
in this book with the hope that supersymmetric quantum systems can be used
to design quantum gates of very large size. Various aspects and applications
of large deviation theory have also been included in this book as it forms an
integral part of modern probability theory which is used to calculate the prob-
ability of rare events like the probability of a stochastic dynamical system with
weak noise exiting the stability zone. The computation of such probabilities
enables us to design controllers that will minimize this deviation probability.
The chapter on applied differential equations focuses on problems in robotics
andotherengineeringorphysicsproblemswhereinstochasticprocessesandfield
inevitably enter into the description of the dynamical system. The chapter on
circuit theory and device physics has also been included since it tells us how to
obtain the governing equations for diodes and transistors from the band struc-
tureofsemiconductors. Whenacircuitisbuiltusingsuchelementsandthermal
noise is present in the resistances, then the noise gets distorted and even am-
plified by the nonlinearity of the device and the mathematical description of
such circuits can be calculated by perturbatively solving associated nonlinear
stochastic differential equations. Quantum scattering theory has also been in-
cludedsinceittellsushowquantumeffectsmaketheprobabilitydistributionof
scatteredparticlesdifferentfromthatobtainedusingclassicalscatteringtheory.
Thus, quantum scattering theory is an integral part of quantum probability.
Many discussions on the Boltzmann kinetic transport equation in a plasma are
included in this book since the Boltzmann distribution function at each time
t can be viewed as an evolving probability density function of a particle in
phase space. In fact, the Bolzmann equation is so fundamental that it can be
used to derive not only more precise forms of the fluid dynamical equations but
also describe the motion of conducting fluids in the presence of electromagnetic
fields. Any book on applications of advanced probability theory must therefore
necessarily include a discussion of the Boltzmann equation. It can be used to
derive the Fokker-Planck equation for diffusion processes after making approx-
imations and by including the nonlinear collision terms, it can also be used to
prove the H-theorem ie the second law of thermodynamics. The section on the
vii
Atiyah-Singerindextheoremhasbeenincludedbecauseitformsanintegralpart
of calculating anomalies in quantum field theory which is in turn a branch of
non-commutative probability theory.
At this juncture, it must be mentioned that this book in the course of dis-
cussing applied probability and statistics, also surveys some of the research
work carried out by eminent scientists in the field of pure and applied prob-
ability, quantum probability, quantum scattering theory, group representation
theoryandgeneralrelativity. Insomecases,wealsoindicatethetrainofthought
processes by which these eminent scientists arrived at their fundamental con-
tributions. To start with, we review the axiomatic foundations of probability
theory due to A.N.Kolmogorov and how the Indian school of probabilists and
statisticians used this theory effectively to study a host of applied probability
and statistics problems like parameter estimation, convergence of a sequence
of probability distributions, martingale characterization of diffusions enabling
one to extend the scope of the Ito stochastic differential equations to situations
whenthedriftanddiffusioncoefficientsdonotsatisfyLipschitzconditions,gen-
eralization of the large deviation principle and apply it to problems involving
randomenvironment,interactingparticlesystemsetc. Wethendiscussthework
of R.L.Hudson along with K.R.Parthasarathy on developing a coherent theory
of quantum noise and apply it to study in a rigorous mathematical way the
Schrodinger equation with quantum noise. This gives us a better understand-
ing of open quantum systems, ie systems in which the system gets coupled to
a bath with the joint system-bath universe following a unitary evolution and
after carrying a out a partial trace of the environment, how one ends up with
thestandardGorini-Kossokowski-Sudarshan-Lindblad(GKSL)equationforthe
system state alone–this is a non-unitary evolution. The name of George Su-
darshan stands out here as not only one of the creators of open quantum sys-
tem theory but also as the physicist involved in developing the non-orthogonal
resolution of the identity operator in Boson Fock space which enables one to
effectively solve the GKSL equation. We then discuss the work of K.B.Sinha
along with W.O.Amrein in quantum scattering theory especially in the devel-
opment of the time delay principle which computes the average time spent by
thescatteredparticleinascatteringstaterelativetothetimespentbyitinthe
free state. We discuss the heavy contributions of the Indian school of general
relativitstsliketheNobellaureateSubramaniyamChandraskeharondeveloping
perturbative tools for solving the Einstein-Maxwell equations in a fluid (post-
Newtonian hydrodynamics) and also the work of Abhay Ashtekar and Ashoke
Sen on canonical quantization of the gravitational field and superstring the-
ory. We discuss the the train of thought that led the famous Indian probabilist
S.R.S.VaradhantodevelopalongwithDanielW.Stroockthemartingalecharac-
terization of diffusions and along with M.D.Donsker to develop the variational
formulation of the large deviation principle which plays a fundamental role in
assessing the role of weak noise on a system to cause it to exit a stability zone
by computing the probability of this rare event. We then discuss the work of
the legendary Inidian mathematician Harish-Chandra on group representation
viii
theory especially his creation of the discrete series of representations for groups
having both non-compact and compact Cartan subgroups to finally obtain the
Plancherel formula for such semisimple Lie groups. We discuss the impact of
Harish-Chandra’s work on modern group theoretical image processing, for ex-
ample in estimating the element of the Lorentz group that transforms a given
image field into a moving and rotating image field. We then discuss the con-
tributions of the famous Indian probabilist Gopinath Kallianpur to developing
non-linearfilteringtheoryinitsmodernformalongwiththeworkofsomeother
probabilists like Kushner and Striebel. Kallianpur’s theory of nonlinear filter-
ing is the most general one as we know today since it is applicable to situations
when the process and measurement noises are correlated. Kallianpur’s mar-
tingale approach to this problem has in fact directly led to the development
of the quantum filter of V.P.Belavkin as a non-commutative generalization of
theclassicalversion. Nonlinearfilteringtheoryhasbeenappliedinitslinearized
approximateform-theExtendedKalmanfiltertoproblemsinroboticsandEEG-
MRI analysis of medical data. It is applicable in fact to all problems where the
system dynamics is described by a noisy differential or difference equation and
one desires to estimate both the state and parameters of this dynamical system
on a real time basis using partial noisy measurement data. We conclude this
work with brief discussions of some of the contributions of the Indian school
of robotics and quantum signal processing to image denoising, robot control
via teleoperation and to artificial intelligence/machine learning algorithms for
estimating the nature of brain diseases from slurred speech data. This review
also includes the work of K.R.Parthasarathy on the noiseless and noisy Shan-
noncodingtheoremsininformationtheoryespeciallytoproblemsinvolvingthe
transmission of information in the form of stationary ergodic processes through
finite memory channels and the computation the Shannon capacity for such
problems. It also includes the pedagogical work of K.R.Parthasarathy in sim-
plfyingtheproofofAndreasWinterandA.S.Holevooncomputingthecapacity
of iid classical-quantum channels wherein classical alphabets are encoded into
quantumstatesanddecodingpositiveoperatorvaluedmeasuresareusedinthe
decodingprocess. WealsoincludeheretheworkofK.R.Parthasarathyonrealiz-
ingviaaquantumcircuit,therecoveryoperatorsintheKnill-Laflammetheorem
for recovering the input quantum state after it has been transmitted through a
noisyquantumchanneldescribedintheformofChoi-Krauss-Stinespringopera-
tors. SomegeneralizationsofthesinglequbiterrordetectionalgorithmofPeter
Shor based on group theory due to K.R.Parthasarathy are also discussed here.
ThisbookalsocontainssomeoftherecentworkoftheIndianschoolofrobotics
involving modeling the motion of rigid 3-D links in a robot using Lie group-Lie
algebra theory and differential equations on such Lie groups. After setting up
these kinematic differential equation in the Lie algebra domain of SO(3)⊗n, we
include weak noise terms coming from the torque and develop a large deviation
principle for computing the approximate probability of exit of the robot from
thestabilityzone. Weincludefeedbacktermstothisrobotsystemandoptimize
this feedback controller so that the probability of stability zone exit computed
using the large deviation rate function is as small as possible.
ix
