Table Of ContentSchaum's Outline of
Theory and Problems of
Probability, Random Variables, and Random
Processes
Hwei P. Hsu, Ph.D.
Professor of Electrical Engineering
Fairleigh Dickinson University
Start of Citation[PU]McGraw-Hill Professional[/PU][DP]1997[/DP]End of Citation
HWEI P. HSU is Professor of Electrical Engineering at Fairleigh Dickinson
University. He received his B.S. from National Taiwan University and M.S. and
Ph.D. from Case Institute of Technology. He has published several books which
include Schaum's Outline of Analog and Digital Communications and Schaum's
Outline of Signals and Systems.
Schaum's Outline of Theory and Problems of
PROBABILITY, RANDOM VARIABLES, AND RANDOM PROCESSES
Copyright © 1997 by The McGraw-Hill Companies, Inc. All rights reserved. Printed
in the United States of America. Except as permitted under the Copyright Act of
1976, no part of this publication may be reproduced or distributed in any form or by
any means, or stored in a data base or retrieval system, without the prior written
permission of the publisher.
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 PRS PRS 9 0 1 0 9 8 7
ISBN 0-07-030644-3
Sponsoring Editor: Arthur Biderman
Production Supervisor: Donald F. Schmidt
Editing Supervisor: Maureen Walker
Library of Congress Cataloging-in-Publication Data
Hsu, Hwei P. (Hwei Piao), date
Schaum's outline of theory and problems of probability, random
variables, and random processes / Hwei P. Hsu.
p. cm. — (Schaum's outline series)
Includes index.
ISBN 0-07-030644-3
1. Probabilities—Problems, exercises, etc. 2. Probabilities-
Outlines, syllabi, etc. 3. Stochastic processes—Problems, exercises, etc. 4. Stochastic
processes—Outlines, syllabi, etc.
I. Title.
QA273.25.H78 1996
519.2'076—dc20 96-
18245
CIP
Start of Citation[PU]McGraw-Hill Professional[/PU][DP]1997[/DP]End of Citation
Preface
The purpose of this book is to provide an introduction to principles of
probability, random variables, and random processes and their applications.
The book is designed for students in various disciplines of engineering,
science, mathematics, and management. It may be used as a textbook and/or as
a supplement to all current comparable texts. It should also be useful to those
interested in the field for self-study. The book combines the advantages of both
the textbook and the so-called review book. It provides the textual explanations
of the textbook, and in the direct way characteristic of the review book, it gives
hundreds of completely solved problems that use essential theory and
techniques. Moreover, the solved problems are an integral part of the text. The
background required to study the book is one year of calculus, elementary
differential equations, matrix analysis, and some signal and system theory,
including Fourier transforms.
I wish to thank Dr. Gordon Silverman for his invaluable suggestions and
critical review of the manuscript. I also wish to express my appreciation to the
editorial staff of the McGraw-Hill Schaum Series for their care, cooperation,
and attention devoted to the preparation of the book. Finally, I thank my wife,
Daisy, for her patience and encouragement.
HWEI P. HSU
MONTVILLE, NEW JERSEY
Start of Citation[PU]McGraw-Hill Professional[/PU][DP]1997[/DP]End of Citation
Contents
Chapter 1. Probability 1
1.1 Introduction 1
1.2 Sample Space and Events 1
1.3 Algebra of Sets 2
1.4 The Notion and Axioms of Probability 5
1.5 Equally Likely Events 7
1.6 Conditional Probability 7
1.7 Total Probability 8
1.8 Independent Events 8
Solved Problems 9
Chapter 2. Random Variables 38
2.1 Introduction 38
2.2 Random Variables 38
2.3 Distribution Functions 39
2.4 Discrete Random Variables and Probability Mass Functions 41
2.5 Continuous Random Variables and Probability Density Functions 41
2.6 Mean and Variance 42
2.7 Some Special Distributions 43
2.8 Conditional Distributions 48
Solved Problems 48
Chapter 3. Multiple Random Variables 79
3.1 Introduction 79
3.2 Bivariate Random Variables 79
3.3 Joint Distribution Functions 80
3.4 Discrete Random Variables - Joint Probability Mass Functions 81
3.5 Continuous Random Variables - Joint Probability Density Functions 82
3.6 Conditional Distributions 83
3.7 Covariance and Correlation Coefficient 84
3.8 Conditional Means and Conditional Variances 85
3.9 N-Variate Random Variables 86
3.10 Special Distributions 88
Solved Problems 89
v
vi
Chapter 4. Functions of Random Variables, Expectation, Limit Theorems 122
4.1 Introduction 122
4.2 Functions of One Random Variable 122
4.3 Functions of Two Random Variables 123
4.4 Functions of n Random Variables 124
4.5 Expectation 125
4.6 Moment Generating Functions 126
4.7 Characteristic Functions 127
4.8 The Laws of Large Numbers and the Central Limit Theorem 128
Solved Problems 129
Chapter 5. Random Processes 161
5.1 Introduction 161
5.2 Random Processes 161
5.3 Characterization of Random Processes 161
5.4 Classification of Random Processes 162
5.5 Discrete-Parameter Markov Chains 165
5.6 Poisson Processes 169
5.7 Wiener Processes 172
Solved Problems 172
Chapter 6. Analysis and Processing of Random Processes 209
6.1 Introduction 209
6.2 Continuity, Differentiation, Integration 209
6.3 Power Spectral Densities 210
6.4 White Noise 213
6.5 Response of Linear Systems to Random Inputs 213
6.6 Fourier Series and Karhunen-Loéve Expansions 216
6.7 Fourier Transform of Random Processes 218
Solved Problems 219
Chapter 7. Estimation Theory 247
7.1 Introduction 247
7.2 Parameter Estimation 247
7.3 Properties of Point Estimators 247
7.4 Maximum-Likelihood Estimation 248
7.5 Bayes' Estimation 248
7.6 Mean Square Estimation 249
7.7 Linear Mean Square Estimation 249
Solved Problems 250
vii
Chapter 8. Decision Theory 264
8.1 Introduction 264
8.2 Hypothesis Testing 264
8.3 Decision Tests 265
Solved Problems 268
Chapter 9. Queueing Theory 281
9.1 Introduction 281
9.2 Queueing Systems 281
9.3 Birth-Death Process 282
9.4 The M/M/1 Queueing System 283
9.5 The M/M/s Queueing System 284
9.6 The M/M/1/K Queueing System 285
9.7 The M/M/s/K Queueing System 285
Solved Problems 286
Appendix A. Normal Distribution 297
Appendix B. Fourier Transform 299
B.1 Continuous-Time Fourier Transform 299
B.2 Discrete-Time Fourier Transform 300
Index 303
Chapter 1
Probability
1.1 INTRODUCTION
The study of probability stems from the analysis of certain games of chance, and it has found
applications in most branches of science and engineering. In this chapter the basic concepts of prob-
ability theory are presented.
1.2 SAMPLE SPACE AND EVENTS
A. Random Experiments:
In the study of probability, any process of observation is referred to as an experiment. The results
of an observation are called the outcomes of the experiment. An experiment is called a random experi-
ment if its outcome cannot be predicted. Typical examples of a random experiment are the roll of a
die, the toss of a coin, drawing a card from a deck, or selecting a message signal for transmission from
several messages.
B. Sample Space:
The set of all possible outcomes of a random experiment is called the sample space (or universal
set), and it is denoted by S. An element in S is called a sample point. Each outcome of a random
experiment corresponds to a sample point.
EXAMPLE 1.1 Find the sample space for the experiment of tossing a coin (a) once and (b) twice.
(a) There are two possible outcomes, heads or tails. Thus
S = {H, T)
where H and T represent head and tail, respectively.
(b) There are four possible outcomes. They are pairs of heads and tails. Thus
S = (HH, HT, TH, TT)
EXAMPLE 1.2 Find the sample space for the experiment of tossing a coin repeatedly and of counting the number
of tosses required until the first head appears.
Clearly all possible outcomes for this experiment are the terms of the sequence 1,2,3, . . . . Thus
s = (1, 2, 3, .. .)
Note that there are an infinite number of outcomes.
EXAMPLE 1.3 Find the sample space for the experiment of measuring (in hours) the lifetime of a transistor.
Clearly all possible outcomes are all nonnegative real numbers. That is,
S=(z:O<z<oo}
where z represents the life of a transistor in hours.
Note that any particular experiment can often have many different sample spaces depending on the observ-
ation of interest (Probs. 1.1 and 1.2). A sample space S is said to be discrete if it consists of a finite number of
PROBABILITY [CHAP 1
sample points (as in Example 1.1) or countably infinite sample points (as in Example 1.2). A set is called countable
if its elements can be placed in a one-to-one correspondence with the positive integers. A sample space S is said
to be continuous if the sample points constitute a continuum (as in Example 1.3).
C. Events:
Since we have identified a sample space S as the set of all possible outcomes of a random experi-
ment, we will review some set notations in the following.
C
If is an element of S (or belongs to S), then we write
If S is not an element of S (or does not belong to S), then we write
u s
A set A is called a subset of B, denoted by
A c B
if every element of A is also an element of B. Any subset of the sample space S is called an event. A
sample point of S is often referred to as an elementary event. Note that the sample space S is the
subset of itself, that is, S c S. Since S is the set of all possible outcomes, it is often called the certain
event.
EXAMPLE 1.4 Consider the experiment of Example 1.2. Let A be the event that the number of tosses required
until the first head appears is even. Let B be the event that the number of tosses required until the first head
appears is odd. Let C be the event that the number of tosses required until the first head appears is less than 5.
Express events A, B, and C.
1.3 ALGEBRA OF SETS
A. Set Operations:
I. Equality:
Two sets A and B are equal, denoted A = B, if and only if A c B and B c A.
2. Complementation:
Suppose A c S. The complement of set A, denoted A, is the set containing all elements in S but
not in A.
A= {C: C: E Sand $ A)
3. Union:
The union of sets A and B, denoted A u B, is the set containing all elements in either A or B or
both.
4. Intersection:
The intersection of sets A and B, denoted A n B, is the set containing all elements in both A
and B.
