Fat-Tailed Distributions: Data, Diagnostics and Dependence

139 Pages·2014·1.313 MB·English

by  Roger M. Cooke, Daan Nieboer, Jolanta Misiewicz

W792-Cooke.qxp_Layout 1 20/10/2014 15:26 Page 1
1
MATHEMATICS AND STATISTICS SERIES
MATHEMATICAL MODELS AND METHODS IN RELIABILITY SET
This book has been written for numerate non-specialists and serves J D R
o a o
three purposes. la a g
n n e
Firstly, it gathers together mathematical material from diverse but ta N r M
i
related fields of order statistics, records, extreme value theory, M e .
b
majorization, regular variation and subexponentiality. Secondly, it isi oe Co
presents a new measure of obesity. Thirdly, and most importantly, the ew r ok
authors hope to convince readers that fat-tail phenomena pose real ic e
z
problems; they are really out there and they seriously challenge our
usual ways of thinking about historical averages, outliers, trends,
regression coefficients and confidence bounds among many other
things. Data on flood insurance claims, crop loss claims, hospital
discharge bills, precipitation, and damages and fatalities from natural
catastrophes drive this point home.
While most fat-tailed distributions are "bad", research into fat tails is one
distribution whose tail will hopefully get fatter.
F
a
t
-
T Volume 1
a
i
l
e
Fat-Tailed
Roger M. Cooke works at Resources for the Future Washington, USA d
and the University of Strathclyde, Dept. of Management Science in the
D
UK. He is retired from the Delft University of Technology, Department of
i
Mathematics, the Netherlands. His research interests include s
uncertainty analysis, risk analysis, dependence modeling, climate t Distributions
r
change and expert judgment. i
b
Daan Nieboer works at the Dept. of Public Health, Erasmus Medical u
Centre − University Medical Centre Rotterdam, the Netherlands. His t
i
research interests include the development, validation and updating of o
clinical prediction models. n
s
Data, Diagnostics and Dependence
Jolanta K. Misiewiczworks at the Dept. of Mathematics and Information
Science, Warsaw University of Technology, Poland. Her research
interests include random vectors and stochastic processes with stable,
sub-stable, weakly stable and pseudo-isotropic (thus mostly heavy tail)
distributions and connections with the geometry of Banach spaces.
Roger M. Cooke
Daan Nieboer and Jolanta Misiewicz
Z(7ib8e8-CBHJCI(
www.iste.co.uk

Fat-tailed Distributions

Mathematical Models and Methods in Reliability Set
coordinated by
Nikolaos Limnios and Bo Henry Lindqvist
Volume 1
Fat-tailed Distributions

Data, Diagnostics and Dependence

Roger M. Cooke
Daan Nieboer
Jolanta Misiewicz
First published 2014in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced,
stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers,
or in the case of reprographic reproduction in accordance with the terms and licenses issued by the
CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the
undermentioned address:
ISTE Ltd John Wiley & Sons, Inc.
27-37 St George’s Road 111 River Street
London SW19 4EU Hoboken, NJ 07030
UK USA
www.iste.co.uk www.wiley.com

© ISTE Ltd 2014
The rights ofRoger M. Cooke, Daan Nieboer and Jolanta Misiewicz to be identified as the authors of this
work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.
Library of Congress Control Number: 2014953028

British Library Cataloguing-in-Publication Data
A CIP record for this book is available from the British Library
ISBN 978-1-84821-792-8
Contents
INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . ix
CHAPTER 1. FATNESS OF TAIL . . . . . . . . . . . . . . 1
1.1.Fat tail heuristics . . . . . . . . . . . . . . . . . . . 1
1.2.History and data . . . . . . . . . . . . . . . . . . . . 4
1.2.1. US flood insurance claims . . . . . . . . . . . . 4
1.2.2. US crop loss . . . . . . . . . . . . . . . . . . . . . 5
1.2.3. US damages and fatalities from natural
disasters . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.4. US hospital discharge bills . . . . . . . . . . . 6
1.2.5. G-Econ data . . . . . . . . . . . . . . . . . . . . . 6
1.3.Diagnostics for heavy-tailed phenomena . . . . . 6
1.3.1. Historical averages . . . . . . . . . . . . . . . . 7
1.3.2. Records. . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.3. Mean excess . . . . . . . . . . . . . . . . . . . . . 11
1.3.4. Sum convergence: self-similar or normal . . . 12
1.3.5. Estimating the tail index. . . . . . . . . . . . . 15
1.3.6. The obesity index . . . . . . . . . . . . . . . . . 20
1.4.Relation to reliability theory . . . . . . . . . . . . . 24
1.5.Conclusion and overview of the technical
chapters . . . . . . . . . . . . . . . . . . . . . . . . . 25
vi Fat-tailedDistributions
CHAPTER 2. ORDER STATISTICS . . . . . . . . . . . . . 27
2.1.Distribution of order statistics . . . . . . . . . . . 27
2.2.Conditional distribution . . . . . . . . . . . . . . . 32
2.3.Representations for order statistics . . . . . . . . 33
2.4.Functions of order statistics . . . . . . . . . . . . . 36
2.4.1. Partial sums . . . . . . . . . . . . . . . . . . . . 36
2.4.2. Ratio between order statistics. . . . . . . . . . 37
CHAPTER 3. RECORDS . . . . . . . . . . . . . . . . . . . 41
3.1.Standard record value processes . . . . . . . . . . 41
3.2.Distribution of record values. . . . . . . . . . . . . 42
3.3.Record times and related statistics . . . . . . . . . 44
3.4.k-records . . . . . . . . . . . . . . . . . . . . . . . . . 46
CHAPTER 4. REGULARLY VARYING AND
SUBEXPONENTIAL DISTRIBUTIONS . . . . . . . . . . . 49
4.1.Classes of heavy-tailed distributions . . . . . . . . 50
4.1.1. Regularly varying distribution functions . . . 50
4.1.2. Subexponential distribution functions . . . . 55
4.1.3. Related classes of heavy-tailed
distributions . . . . . . . . . . . . . . . . . . . . 58
4.2.Mean excess function . . . . . . . . . . . . . . . . . 59
4.2.1. Properties of the mean excess function . . . . 60
CHAPTER 5. INDICES AND DIAGNOSTICS OF
TAIL HEAVINESS . . . . . . . . . . . . . . . . . . . . . . . 65
5.1.Self-similarity . . . . . . . . . . . . . . . . . . . . . . 66
5.1.1. Distribution of the ratio between order
statistics . . . . . . . . . . . . . . . . . . . . . . . 69
5.2.The ratio as index . . . . . . . . . . . . . . . . . . . 76
5.3.The obesity index . . . . . . . . . . . . . . . . . . . . 80
5.3.1. Theory of majorization . . . . . . . . . . . . . . 85
5.3.2. The obesity index of selected data sets . . . . 91
Contents vii
CHAPTER 6. DEPENDENCE . . . . . . . . . . . . . . . . . 95
6.1.Definition and main properties . . . . . . . . . . . 95
6.2.Isotropic distributions . . . . . . . . . . . . . . . . . 96
6.3.Pseudo-isotropic distributions . . . . . . . . . . . . 100
6.3.1. Covariation as a measure of dependence for
essentially heavy-tail jointly pseudo-isotropic
variables . . . . . . . . . . . . . . . . . . . . . . . 104
6.3.2. Codifference . . . . . . . . . . . . . . . . . . . . . 109
6.3.3. The linear regression model for essentially
heavy-tail distribution . . . . . . . . . . . . . . 110
CONCLUSIONS AND PERSPECTIVES . . . . . . . . . . . 115
BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . 119
INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123