Table Of ContentEntropy and Information Optics
Connecting Information and Time, Second Edition
OPTICAL SCIENCE AND ENGINEERING
Founding Editor
Brian J. Thompson
University of Rochester
Rochester, New York
RECENTLY PUBLISHED
Entropy and Information Optics: Connecting Information and Time, Second Edition,
Francis T. S. Yu
Laser Beam Shaping Applications, Second Edition, Fred M. Dickey and Todd E. Lizotte
Fundamentals and Basic Optical Instruments, edited by Daniel Malacara Hernández
Tunable Laser Applications, Third Edition, edited by F. J. Duarte
Laser Safety: Tools and Training, Second Edition, edited by Ken Barat
Optical Materials and Applications, edited by Moriaki Wakaki
Lightwave Engineering, Yasuo Kokubun
Handbook of Optical and Laser Scanning, Second Edition, Gerald F. Marshall
and Glenn E. Stutz
Computational Methods for Electromagnetic and Optical Systems, Second Edition,
John M. Jarem and Partha P. Banerjee
Optical Methods of Measurement: Wholefield Techniques, Second Edition, Rajpal S. Sirohi
Optoelectronics: Infrared-Visible-Ultraviolet Devices and Applications, Second Edition,
edited by Dave Birtalan and William Nunley
Photoacoustic Imaging and Spectroscopy, edited by Lihong V. Wang
Polarimetric Radar Imaging: From Basics to Applications, Jong-Sen Lee and Eric Pottier
Near-Earth Laser Communications, edited by Hamid Hemmati
Slow Light: Science and Applications, edited by Jacob B. Khurgin and Rodney S. Tucker
Dynamic Laser Speckle and Applications, edited by Hector J. Rabal and Roberto A. Braga Jr.
Biochemical Applications of Nonlinear Optical Spectroscopy, edited by Vladislav Yakovlev
Optical and Photonic MEMS Devices: Design, Fabrication and Control, edited by Ai-Qun Liu
The Nature of Light: What Is a Photon?, edited by Chandrasekhar Roychoudhuri,
A. F. Kracklauer, and Katherine Creath
Introduction to Nonimaging Optics, Julio Chaves
Introduction to Organic Electronic and Optoelectronic Materials and Devices,
edited by Sam-Shajing Sun and Larry R. Dalton
Fiber Optic Sensors, Second Edition, edited by Shizhuo Yin, Paul B. Ruffin, and Francis T. S. Yu
Terahertz Spectroscopy: Principles and Applications, edited by Susan L. Dexheimer
Photonic Signal Processing: Techniques and Applications, Le Nguyen Binh
Please visit our website www.crcpress.com for a full list of titles
Entropy and Information Optics
Connecting Information and Time, Second Edition
Francis T. S. Yu
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2017 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
International Standard Book Number-13: 978-1-138-55549-5 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and informa-
tion, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have
attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form
has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic,
mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval
system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright .com (http://www.copyright.com/) or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses
and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been
arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without
intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Names: Yu, Francis T. S., 1932- author.
Title: Entropy and information optics : connecting information and time / Francis T.S. Yu.
Description: Second edition. | Boca Raton : Taylor & Francis, CRC Press,2017. | Includes bibliographical references and index.
Identifiers: LCCN 2017032627| ISBN 9781138555495 (hardback : alk. paper) | ISBN 9781315149561 (ebook)
Subjects: LCSH: Optical communications. | Entropy (Information theory) | Optics. | Optical data processing.
Classification: LCC TK5103.59 .Y8 2017 | DDC 621.382/7--dc23
LC record available at https://lccn.loc.gov/2017032627
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
From the Series Editor .................................................................................................................................................................ix
Preface..........................................................................................................................................................................................xi
Author ........................................................................................................................................................................................xiii
Chapter 1 Introduction to Information Transmission ..............................................................................................................1
1.1 Information Measure ....................................................................................................................................2
1.2 Entropy Information .....................................................................................................................................4
1.3 Communication Channels ............................................................................................................................7
1.4 Memoryless Discrete Channels ....................................................................................................................8
1.5 Continuous Channels with Additive Noise .................................................................................................10
1.6 Summary and Remarks ..............................................................................................................................15
References .............................................................................................................................................................16
Chapter 2 Diffraction and Signal Analysis ............................................................................................................................17
2.1 Introduction to Diffraction .........................................................................................................................17
2.2 Fresnel–Kirchhoff Theory ..........................................................................................................................18
2.3 Linear Systems and Fourier Analysis .........................................................................................................19
2.4 Finite Bandwidth Analysis .........................................................................................................................21
2.5 Degrees of Freedom of a Signal .................................................................................................................23
2.6 Gabor’s Information Cell ............................................................................................................................25
2.7 Signal Detection .........................................................................................................................................26
2.8 Statistical Signal Detection ........................................................................................................................27
2.9 Signal Recovering .......................................................................................................................................29
2.10 Signal Ambiguity........................................................................................................................................30
2.11 Wigner Signal Representation ....................................................................................................................31
2.12 Fourier Transform Properties of Lenses .....................................................................................................33
References .............................................................................................................................................................34
Chapter 3 Optical Spatial Channel and Encoding Principles ................................................................................................35
3.1 Optical Spatial Communication Channel ...................................................................................................35
3.2 Optical Message in Spatial Coding ............................................................................................................37
3.3 Spatial Channel with Resolution Cells of Different Sizes ..........................................................................39
3.4 Matching a Code with a Spatial Channel ...................................................................................................41
References .............................................................................................................................................................43
Chapter 4 Entropy and Information .......................................................................................................................................45
4.1 Fundamental Laws of Thermodynamics ....................................................................................................45
4.2 Physical Entropy and Information ..............................................................................................................46
4.3 Trading Entropy with Information .............................................................................................................47
4.4 Typical Examples .......................................................................................................................................48
4.5 Remarks ......................................................................................................................................................50
References .............................................................................................................................................................50
Chapter 5 Demon Exorcist and Cost of Entropy ....................................................................................................................53
5.1 Perpetual Motion Machine .........................................................................................................................53
5.2 Maxwell’s Demon .......................................................................................................................................54
5.3 Information and Demon Exorcist ...............................................................................................................54
5.4 Demon Exorcist, a Revisit ..........................................................................................................................57
5.5 Szilard’s Demon..........................................................................................................................................58
v
vi Contents
5.6 Diffraction-Limited Demon .......................................................................................................................60
5.7 Minimum Cost of Entropy .........................................................................................................................60
5.8 Gabor’s Perpetuum Mobile of the Second Kind ........................................................................................62
References .............................................................................................................................................................63
Chapter 6 Observation and Information ................................................................................................................................65
6.1 Observation with Radiation ........................................................................................................................65
6.2 Simultaneous Observations ........................................................................................................................68
6.3 Observation and Information .....................................................................................................................69
6.4 Accuracy and Reliability in Observations ..................................................................................................71
6.5 Observation by Interference and by Microscope .......................................................................................74
6.6 Uncertainty and Observation ......................................................................................................................78
6.7 Remarks ......................................................................................................................................................79
References .............................................................................................................................................................79
Chapter 7 Image Restoration and Information ......................................................................................................................81
7.1 Image Restoration .......................................................................................................................................81
7.2 Uncertainty and Image Restoration ............................................................................................................84
7.3 Resolving Power and Information ..............................................................................................................85
7.4 Coherent and Digital Image Enhancement.................................................................................................87
7.5 Information Leakage Through a Passive Channel .....................................................................................88
7.6 Restoration of Blurred Images ....................................................................................................................90
References .............................................................................................................................................................93
Chapter 8 Quantum Effect on Information Transmission .....................................................................................................95
8.1 Problem Formulation and Entropy Consideration ......................................................................................95
8.2 Capacity of a Photon Channel ....................................................................................................................96
8.3 Informational Theoristic Approach ............................................................................................................98
8.4 Narrowband Photon Channel ...................................................................................................................100
8.5 Optimum Signal Power Distribution: A Special Case .............................................................................103
References ...........................................................................................................................................................104
Chapter 9 Coherence Theory of Optics ...............................................................................................................................105
9.1 Aspects of Coherence ...............................................................................................................................105
9.2 Spatial and Temporal Coherence ..............................................................................................................106
9.2.1 Spatial Coherence ........................................................................................................................106
9.2.2 Temporal Coherence ...................................................................................................................107
9.3 Coherent and Incoherent Processing ........................................................................................................108
9.4 Exploitation of Coherence ........................................................................................................................109
9.4.1 Exploitation of Spatial Coherence ...............................................................................................109
9.4.2 Exploitation of Temporal Coherence ............................................................................................110
9.4.3 Complex Amplitude Processing ...................................................................................................110
9.4.4 Incoherent Superposition ..............................................................................................................112
9.5 Remarks .....................................................................................................................................................112
References ............................................................................................................................................................112
Chapter 10 Wavelet Transforms with Optics ..........................................................................................................................113
10.1 Aspects of Wavelet Transform ...................................................................................................................113
10.2 Fourier Domain Processing (FDP) ............................................................................................................113
10.3 Wavelet Transform .....................................................................................................................................116
10.4 Optical Implementations ...........................................................................................................................118
10.5 Simulations ................................................................................................................................................118
Contents vii
10.6 Remarks .....................................................................................................................................................119
References ............................................................................................................................................................119
Chapter 11 Pattern Recognition with Optics .........................................................................................................................121
11.1 Optical Correlators ...................................................................................................................................121
11.2 Optical Disk–Based Correlator ................................................................................................................123
11.3 Photorefractive-Based Correlator .............................................................................................................123
11.4 Optical Neural Networks ..........................................................................................................................124
11.5 Composite Filters ......................................................................................................................................126
11.6 Remarks ....................................................................................................................................................127
References ...........................................................................................................................................................128
Chapter 12 Computing with Optics .......................................................................................................................................129
12.1 Logic-Based Computing ...........................................................................................................................129
12.2 Optical Interconnects and Shuffling .........................................................................................................129
12.3 Matrix–Vector Multiplication ....................................................................................................................131
12.4 Systolic Processor .....................................................................................................................................132
12.5 Matrix–Matrix Processing .......................................................................................................................133
12.6 Expert System and Artificial Intelligence ................................................................................................135
12.7 Remarks ....................................................................................................................................................138
References ...........................................................................................................................................................138
Chapter 13 Communication with Fiber Optics ......................................................................................................................139
13.1 Aspects of Fiber-Optic Communication ...................................................................................................139
13.2 Optical Fiber Structures ............................................................................................................................141
13.3 Fiber-Optic Transmission .........................................................................................................................142
13.4 Types of Optical Fibers ............................................................................................................................145
13.5 Fiber-Optic Communications ...................................................................................................................146
13.6 Remarks ....................................................................................................................................................148
References ...........................................................................................................................................................149
Chapter 14 Synthetic Radar Imaging with Optics .................................................................................................................151
14.1 Synthetic Aperture Radar .........................................................................................................................151
14.2 Radar Data Format Synthesis ...................................................................................................................151
14.3 Radar Imaging with Optics ......................................................................................................................153
14.4 Remarks ....................................................................................................................................................154
References ...........................................................................................................................................................154
Chapter 15 Wideband Signal Analysis with Optics ...............................................................................................................155
15.1 Time Signal to Two-Dimensional Spatial Conversion .............................................................................155
15.2 Spectrum Analysis with Optics ................................................................................................................155
15.3 Processing with Area Modulation ............................................................................................................157
15.4 Broadband Area Modulation Processing ..................................................................................................158
15.5 Remarks ....................................................................................................................................................159
References ...........................................................................................................................................................159
Chapter 16 Information in Myth of Science and Creation......................................................................................................161
16.1 Space and Information ..............................................................................................................................161
16.2 Time and Information ................................................................................................................................161
16.3 Entropy and Information ..........................................................................................................................163
16.4 Substance and Information .......................................................................................................................163
16.5 Uncertainty and Information ....................................................................................................................163
viii Contents
16.6 Certainty and Information ........................................................................................................................164
16.7 Relativity and Information .......................................................................................................................165
16.8 Creation and Information .........................................................................................................................167
16.9 Price Tag and Information ........................................................................................................................167
16.10 Remarks ....................................................................................................................................................168
References ...........................................................................................................................................................168
Chapter 17 Time: The Enigma of Space .................................................................................................................................171
17.1 Time and Energy .......................................................................................................................................171
17.2 Einstein’s Energy Equation .......................................................................................................................171
17.3 Energy Conversion and Reservoir .............................................................................................................171
17.4 Trading Mass and Energy ..........................................................................................................................171
17.5 Physical Substance and Space ..................................................................................................................172
17.6 Absolute and Physical Subspaces .............................................................................................................172
17.7 Time and Physical Space ...........................................................................................................................173
17.8 Electromagnetics and Laws of Physics ......................................................................................................173
17.9 Trading Time for Space .............................................................................................................................174
17.10 Relativistic Time and Temporal Space ......................................................................................................175
17.11 Time and Physical Space ...........................................................................................................................175
17.12 We Are Not Alone .....................................................................................................................................176
17.13 Remarks .....................................................................................................................................................176
References ............................................................................................................................................................176
Chapter 18 Time–Space Quantum Entanglement .................................................................................................................177
18.1 Science and Mathematics .........................................................................................................................177
18.2 Time and Temporal Space ........................................................................................................................177
18.3 Quantum Entanglement .............................................................................................................................178
18.4 Relativistic Quantum Entanglement ..........................................................................................................178
18.5 The Art of Quantum Limited-Subspace ...................................................................................................179
18.6 Remarks .....................................................................................................................................................181
References ............................................................................................................................................................181
Appendix A: Linear Difference Equation with Constant Coefficients ..............................................................................183
Appendix B: Solution of the A Priori Probabilities of Eqs. (5.37) and (5.38) ....................................................................185
Appendix C: Probability Energy Distribution .....................................................................................................................187
Appendix D: Einstein’s Energy Equation .............................................................................................................................189
Appendix E: Gravitation and EM Field Interaction ............................................................................................................191
Index .........................................................................................................................................................................................193
From the Series Editor
Fundamentally optical beams and optical systems transmit defect, relative structural content, and correlation quantity
and analyze information. The information can be analog or were concepts introduced by E. H. Linfoot in 1964. Formal
digital. It can be three-dimensional, two-dimensional, or one- definitions of entropy and information content were to follow,
dimensional. It can be in the traditional form of an image or and the field continues to expand, driven by the explosion of
information that is coded and/or compressed. The light beam high-speed, high-data-rate, and high-capacity communica-
carrying the information can be incoherent, coherent, or even tion systems.
partially coherent. This volume discusses the fundamentals and the applica-
In the early days of this important field, the concepts of tions of entropy and information optics by means of a sampling
communication theory had a major impact on our understand- of topics in this field, including image restoration, wavelet
ing and our descriptions of optical systems. The initial impe- transforms, pattern recognition, computing, and fiber-optic
tus was to deal with images and image quality. Concepts of communication.
impulse responses and transfer functions caused considerable
rethinking about the design and evaluation of optical systems. Brian J. Thompson
Resolution criteria were only the beginning; fidelity, fidelity
ix
