Probability. A brief Introduction

154 Pages·2017·1.224 MB·english

by  Erio Castagnoli, Margherita Cigola, Lorenzo Peccati

Probability
Erio Castagnoli
Margherita Cigola
Lorenzo Peccati
Copyright © 2017 Bocconi University Press
EGEA S.p.A.
EGEA S.p.A.
Via Salasco, 5 - 20136 Milan, Italy
Phone + 39 02 5836.5751 - Fax [phone]
[email protected] - www.egeaeditore.it
First Edition: January 2017
ISBN International Edition 978-88-85486-09-6
ISBN Domestic Edition 978-88-99902-12-4
ISBN Pdf Edition 978-88-85486-10-2
Print: Digital Print Service, Segrate (Milan)
Contents
1 General questions 5
2 The various approaches to probability theory 7
2.1 Classical probability . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Frequency-based (or empirical) approach . . . . . . . . . . . . . . . . 9
2.3 Thesubjectiveapproach: itsrelevancetoEconomicsandManagement 11
3 The axiomatic approach, or the maths of probability 17
3.1 Sample space and events . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 The axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 Conditional probability, correlation between events, stochastic inde-
pendence and Bayes Theorem . . . . . . . . . . . . . . . . . . . . . . 30
4 Random numbers 38
4.1 What a random number is . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2 The probability distribution of a random number . . . . . . . . . . . 42
4.3 Computer simulation of random numbers . . . . . . . . . . . . . . . 50
5 Expected value of (a function of) a random number 53
5.1 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.2 Moment generating function . . . . . . . . . . . . . . . . . . . . . . . 60
5.3 Conditional random numbers and conditional expectations. . . . . . 67
5.4 A brief summary of standard distributions . . . . . . . . . . . . . . . 68
5.4.1 The Poisson distribution . . . . . . . . . . . . . . . . . . . . . 69
5.4.2 The binomial distribution . . . . . . . . . . . . . . . . . . . . 69
5.4.3 The exponential distribution . . . . . . . . . . . . . . . . . . 71
5.4.4 The normal distribution . . . . . . . . . . . . . . . . . . . . . 73
5.4.5 The uniform distribution . . . . . . . . . . . . . . . . . . . . 75
6 Expected utility and certainty equivalent 77
6.1 The problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.2 The answer to the problem . . . . . . . . . . . . . . . . . . . . . . . 82
6.3 The estimation of u . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.4 The notion of risk-aversion . . . . . . . . . . . . . . . . . . . . . . . 85
6.5 Some popular utility functions . . . . . . . . . . . . . . . . . . . . . 87
6.5.1 The linear utility . . . . . . . . . . . . . . . . . . . . . . . . . 87
6.5.2 The exponential utility. . . . . . . . . . . . . . . . . . . . . . 87
6.5.3 The logarithmic utility . . . . . . . . . . . . . . . . . . . . . . 88
6.5.4 The isoelastic utility . . . . . . . . . . . . . . . . . . . . . . . 89
6.5.5 The quadratic utility . . . . . . . . . . . . . . . . . . . . . . . 89
7 Random vectors: first notions 90
7.1 Notion of random vector . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.2 The probability distribution of a random vector . . . . . . . . . . . . 93
7.3 The notion of stochastic independence between two random numbers 98
7.4 The expectation of a random vector . . . . . . . . . . . . . . . . . . 101
7.5 The expectation of a function of a random vector . . . . . . . . . . . 101
7.6 Second order moments for random vectors . . . . . . . . . . . . . . . 102
7.7 The variance of a linear function of random numbers . . . . . . . . . 109
8 Exercises 122
1 General questions
Probability is the proper existing tool for managing uncertainty. The fact that
almost all the fields of Economics and of Management Science have to take uncer-
tainty into account makes this tool necessary beyond any reasonable doubt. For
instance,itsrelevanceforFinanceisascertainedandneedsherenofurtherremark.
However a correct use of probability is far from being generally diffused.
There are many reasons that can explain such a scarce diffusion, but, in our
opinion,themainoneisthatmostofthepeoplefacinguncertaintyhavefrequently
twooppositeandextremebehaviors: someofthemareterriblyworriedandanxious
(andthereforetakeirrationaldecisions)whereassomeothersacttoooptimistically
with a great self-confidence to be able to predict the future (and therefore take
irrational decisions). Only few face uncertainty in a rational way.
One more reason making probability unfamiliar to many people is of logical
nature. That there should be a number of different viewpoints about probability
is a matter of course. In point of fact , the issue rapidly appears to be far more
subtle than it looks like at first sight.
Take also into account the frequently observed confusion among:
what probability is;
•
how it can be assessed within special frameworks;
•
how it can be manipulated, that is, how the arithmetic of probability works.
•
We shall deal with complete theories which will provide us with an answer to
all the questions as well as an incomplete theory, which, popular though it might
be, provides an answer only to the latter.
The historical sketch of the essence of probability we start out with is clearly
meant to help the reader avoid the aforementioned confusion and keep in mind in
which sense probability should be dealt with.
Consider two classical problems:
Example 1 [Oughtred rings problem] – A box contains two rings. Maybe that
both are golden (composition I) or only one is, the second being a silver one (com-
position II). You attribute the same probability (1) to both compositions. From the
2
box you randomly extract a ring which turns out to be a golden one. The question
is about the probability that the other ring should be golden too. Many people argue
that the information conveyed by the first ring is far from decisive: the outcome
“silver ring” would provide us with a certainty solution to the problem, whereas the
“golden ring” outcome appears to leave things unchanged. Since the second ring
can be a silver or a golden one, the majority of people think that the two composi-
tions (I or II) continue to deserve the same probability. Well, we will see that this
opinion is incorrect.
5
Example 2 [Ten heads problem] – A guy tosses a “perfect” coin. The outcome
of every toss can be, with equal probability, H(ead) or T(ail). Ten tosses are made
and the outcome is always H. The question is about the probability p to get H
in the next toss. Some people say that p = 1, because of the independence of the
2
outcomes. Some people say, because of the law of large numbers and since so many
heads occurred, that p should be smaller that 1 (this party includes all the rather
2
foolish people betting on the delays on Lotto results). Some people think that the
coin might be not perfect as it should and, learning from experience, think, because
of the observed sequence, that p> 1. Clearly there are (apparent) justifications for
2
all the possibilities, that is to believe that p= 1, p< 1, p> 1!
2 2 2
These examples show how questionable the field we are entering is. Our intu-
ition, which maycomeinhandyin manycircumstances, can fail miserablywhenit
comes to probability problems. Further examples will be provided to support our
common inadequacy to cope with uncertainty contexts.
Human beings have tried to define probability depending on the way you ap-
proach every single problem. This fact constitutes a further source of difficulty, as
the logical value of a statement is closely linked to the approach chosen.
The best way to cope with this problem is to review all of these possibilities
(the so called “approaches to probability theory”), in order to understand their
relevant characteristics which turn out to be:
the practical context in which every approach was born and its range of
•
applicability;
the possible logical mistakes or confusions underlying each attempt to for-
•
malize a given notion of probability;
the reasons for rejecting some of these conceptions stems from the fact that
•
you may take into account the difference between two distinct questions,
which are frequently intertwined and give thus rise to confusion: (1) what
probability is; (2) how probability can be evaluated in specific problems.
Wewouldverymuchlikeourreadertobeconfrontedwithatheorywhichshould
be both logically well-founded and in a position to help her/him solve problems of
practical relevance1.
Reviewing the various approaches we will realize that:
the demand for a probability notion in a position to cope with a larger and
•
larger set of problems has actually shown fallacies in the previously existing
approaches, or, at least, strong practical limitations;
the distinction between complete and incomplete theories does help reach a
•
great clarity and a correct understanding of the various positions.
1Wewilltrytokeeptheformalaspectsattheirlowestpossibletechnicallevel,inordertomake
the presentation easy alsoforreaders withoutspecific mathematicaltraining.
6
2 The various approaches to probability theory
All of us should have an intuitive idea of probability. If we expect that something
is likely to be true, in common language, we can use sentences like “This is true
with a very high probability, say, 99%”. If something is almost impossible we
can use sentences like “This is true with a very low probability, say, 3%”, etc.
As percentages are usual tricks to deal with fractions with denominator 100, we
could replace the two numbers stated above with 0.99 = 99/100 or 0.03 = 3/100,
respectively. This way every probability is a number in [0,1].
2.1 Classical probability
The field where probability was firstly systematically used is perhaps gambling
(choosing cards, tossing dice, flipping coins,...) and we can go back to the 16th
centurytofindtheearliestexamples. Probablythefirstsystematicpresentationof
this approach is due to the French mathematician Pierre Simon de Laplace [9].
Gamblingproblemsarecharacterizedbyarandomexperiment(forinstance,we
pick at random one card from a deck). Such random experiment has n possible
outcomes,whichareinsomesymmetricalposition,inthesensethatnooneofthem
is more likely than the others to turn up (no card in our deck has any privilege in
being picked with respect to (w.r.t.) the others).
An event is a collection of k outcomes out of the n possible ones (of course:
05k 5n). The classical probability is defined to be the ratio k/n.
Let us now deal with such a pseudo-idea2. Precisely:
Definition 1 For a random experiment with n equally likely outcomes, an event
consisting in k outcomes out of them has probability:
k
n
Inmanycasesthecomputationofsucharatiocanbechallenging,whileinsome
other cases it is simply a matter of course. Let us start from this latter case.
Example 3 We flip a perfect coin. Its sides are perfectly symmetrical and corre-
spond to head and tail. The number of the possible outcomes is n = 2. For the
event “head” the number of outcomes is k =1. The classical probability of “head”
is thus k/n=1/2=0.5.
The classical probability to get “head or tail”, being in this case k = 2, is k/n =
2/2=1.
The classical probability to get “tail”, being in this case k = 1, is again k/n =
1/2=0.5.
The classical probability to flip coin and get as outcome a “living rabbit” is k/n=
0/2=0.
2The reason forthe“pseudo”willbe cleared shortly (see nextpage).
7
Example 4 A popular game in Italy is Lotto. An urn contains 90 balls, each
denoted with an integer number: 1,2,3,...,90. A candid kid is asked to randomly
pick five balls from the urn. Call C the set of five winning numbers on which we
gamble. What about the classical probability to divine exactly the outcome C of the
random experiment? Combinatorics teach us that the number of different sets of
five numbers we can obtain with the experiment (called combinations of 90 objects
of class 5) is3:
90 90!
n= =
5 5! 85!
µ ¶ ×
The number of favourable outcomes is k =1. Therefore, the classical probability to
win is:
k 1 5! 85!
n = 90! = ×90! ≈2.27535×10−8
5! 85!
×
which is rather small, actually.
Example 5 Now it is time to use this notion to foster one of the possible answers
to the “Run of ten heads problem” we saw above on p. 6. The sequence of eleven
consecutive heads is only one:
HHHHHHHHHHH
11tosses
The sequences starting with te|n heads ar{eztwo: }
HHHHHHHHHHH and HHHHHHHHHHT
11tosses 11tosses
No sequence is|privileged{zand therefo}re the p|robability o{zf the eleve}nth consecutive
head is k = 1 =0.5 as most people correctly declare in answering the question.
n 2
Themaincriticismswhichcanbelevelledagainstsuchadefinitionofprobability
should be rather evident:
Whataboutprobabilityintheabsenceofsymmetryamongthevariouspossi-
•
ble outcomes? To understand the relevance of the question, just think of the
problemofevaluatingtheprobabilitypthat“Youwillbestillalivetomorrow
at12:00o’clock”. Acomputationofthetypep=k/nwouldbevoidofsense
(and rather worrying).
3Thesymbol m representsthenumberofcombinationsofmdistinctobjectsofclassk,that
k
is, the number of sets of k objects out of the given mwhich can be constructed. Such a number
(cid:3) (cid:4)
turns outtobe:
m m!
=
k k!(m k)!
(cid:19) (cid:20) −
and itis usually called binomial coefficient.
8
This way to approach probability is clearly based on the confusion between
•
what probability is and how we can try to evaluate it in appropriate frame-
works (for instance when dealing with fair games).
Think of a coin which is not perfect. If we are in the know, we would never
• accept to evaluate the probability of getting head as 1. This fact tells us
2
that theevaluation of the probabilityof an event should significantly depend
on the information available to the evaluator. In the naïve version of the
classical approach there is no way for the available information to play its
significant role.
The classical approach defines probability by resorting the the very notion
•
of probability itself: thus “equally likely” means “with the same probabil-
ity”. In actual fact, from a strictly logical viewpoint, this definition is quite
inconsistent.
2.2 Frequency-based (or empirical) approach
Thesymmetryproperty,typicalofmanygames,wheretheclassicalapproachstems
from, fails immediately when we need to use probability in the wider world of
science.
The frequency-based approach to probability was formally introduced in the
most popular natural sciences (like physics or biology). It is based on the possibil-
itytorepeatanexperimentunderconditionswhichareviewedasequivalent. Thus
the evaluation of probability and with the way it works, are essentially based on
the observation of the results of such experiments. The probabilist-cum-economist
R. von Mises, back in 1919, tried to provide this approach with a sounder theo-
retical background. This was also strongly supported by the great Soviet school of
probability very much active in the same period. Statistical induction4, perhaps
the most important branch of Statistics, is traditionally based on the notion of
probability corresponding to this approach.
Let us try to outline the issue properly.
A sequence of random experiments , , ,..., ,... is considered. They
1 2 3 n
E E E E
are supposed to be performed under the same conditions. One of the possible final
events, say A, is labelled “success”. The ratio:
number of successes in the first n experiments
f (A)=
n n
is called frequency of success over these n experiments. Of course f (A) will fluc-
n
tuate when n changes.
4How to infer from the outcomes of a random experiment something about random self-
generating mechanisms? For instance, we wonder which could be the composition of an urn
containingballsofvarious colours,on the basisofsomeinformation aboutthecoloursofanum-
ber of balls picked out of the urn. The logical procedure is exactly the same we resort to when
wepurchasesomemerchandiseand,inordertocontrolitsquality,welimitourselvestochecking
asample.
9
The so-called “Empirical Law of Chance” asserts that, when n diverges, the
frequency f (A) becomes more and more stable. This behavior suggests that
n
lim f (A)exists. Well,theprobabilityofA,denotedwithP(A),isprecisely
n + n
the l→imi∞t of the frequency sequence. In symbols:
P(A):= lim f (A) (1)
n
n +
→ ∞
It is obvious that, in practice, formula (1) cannot possibly be used to compute
probabilities. The suggestion of the empirical craft is: “Don’t worry! You can use
f (A),withnsufficientlylarge,asagoodapproximationofP(A). Andin practice
n
this will be sufficient”.
Example 6 A coin is tossed 1000 times and the outcomes obtained are 505 heads
and 495 tails. Call A the event: “Tossing that coin the outcome is head”. The
frequency of A turns out to be:
505
f (A)=
1000 1000
Therefore, an empiricist would assert that:
505
P (A) f (A)= =0.505
≈ 1000 1000
Different empirical evidence on the same event bring to different probability
evaluations, but, given a body of evidence, especially in case it should be signifi-
cantlylarge,theprobabilityevaluationofferedby(1)isexpectedtobesubstantially
the same and that it can be thought of as more or less objective.
Also the frequency-based viewpoint can be dismissed rather easily. A list of
criticisms follows.
The definition is based on a main pillar, which is nothing but an act of faith
•
(the convergence property of f (A)).
n
Some people, willing to reject the previous assertion, surprisingly invoke the
•
so-called J. Bernoulli Theorem5. Such a result is correct and highly re-
spectable and, at least at first sight, a superficial reader could think of being
able to replace the aforementioned act of faith. A more accurate analysis
of this objection should point out that there is an evident logical mistake:
J. Bernoulli’s result uses the notion of probability, which should be given in
advance and therefore cannot be used in... defining probability.
5Inthecaseofindefiniteandindependentrepetitionsofanexperimentunderthesamecondi-
tions,ifpis the probability to observe the event A,then forevery ε>0:
lim P(fn(A) p <ε)=1
n + | − |
→ ∞
10