IPSec the new security standart for the internet, intranets and virtual private networks

280 Pages·2003·1.899 MB·English

by  Doraswamy, Naganand, Harkins, Dan

IPSec
The New Security
Standard for the Internet,
Intranets, and Virtual
Private Networks
Second Edition
ISBN 013046189-X
94499
9 780130 461896
This page intentionally left blank
IPSec
The New Security Standard
for the Internet, Intranets, and
Virtual Private Networks
Second Edition
Naganand Doraswamy
Dan Harkins
Prentice Hall PTR, Upper Saddle River, NJ 07458
www.phptr.com
Library of Congress Cataloging-in-Publication Date
Doraswamy, Naganand.
IPSec: the new security standard for the Internet, intranets,
and virtual private networks, Second Edition / Naganand Doraswamy, Dan Harkins.
p. cm. -- (Prentice-Hall PTR Web infrastructure series)
Includes bibliographical references and index.
ISBN 0-13-046189-X
1. IPSec (Computer network protocol) 2. Internet (Computer
networks) -- Security measures. 3. Intranets (Computer networks) --
Security measures. 4. Extranets (Computer networks) -- Security measures.
I. Harkins, Dan. II. Title. III. Series.
TK5105.567 .D67 2002
005.8 -- dc21 02-23833
CIP
Editorial/Production Supervision: Mary Sudul
Page Layout: FASTpages
Acquisitions Editor: Mary Franz
Editorial Assistant: Noreen Regina
Manufacturing manager: Alexis Heydt-Long
Art Director: Gail Cocker-Bogusz
Series Design: Meg VanArsdale
Cover Design: Anthony Gemmellaro
Cover Design Direction: Jerry Votta
© 2003 by Prentice Hall PTR
Prentice-Hall, Inc.
Upper Saddle River, NJ 07458
Prentice Hall books are widely used by corporations and government agencies for training,
marketing, and resale.
The publisher offers discounts on this book when ordered in bulk quantities. For more information,
contact Corporate Sales Department, phone: [phone]; fax: [phone]; email: [e-mail]
Or write Corporate Sales Department, Prentice Hall PTR, One Lake Street, Upper Saddle River, NJ 07458.
Product and company names mentioned herein are the trademarks or registered trademarks
of their respective owners.
All rights reserved. No part of this book may be
reproduced, in any form or by any means, without
permission in writing from the publisher.
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
ISBN 0-13-046189-X
Pearson Education LTD.
Pearson Education Australia PTY, Limited
Pearson Education Singapore, Pte. Ltd.
Pearson Education North Asia Ltd.
Pearson Education Canada, Ltd.
Pearson Educación de Mexico, S.A. de C.V.
Pearson Education — Japan
Pearson Education Malaysia, Pte. Ltd.
To Amma, Appa, Roopa, Ananth, and Anu.
Naganand
To Marina, the giant on whose shoulders I stand.
Dan
This page intentionally left blank
Chapter
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xi
Part One: Overview
1 Cryptographic History and Techniques . . . . . . . . 1
Secrets in History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Rise of the Internet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Internet Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Cryptographic Building Blocks . . . . . . . . . . . . . . . . . . . . . . 6
Crypto Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
More Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
vii
viii IPSec
2 TCP/IP Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Introduction to TCP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Addressing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Domain Name System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Security—at What Level?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3 IP Security Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
The Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Encapsulating Security Payload (ESP). . . . . . . . . . . . . . . . . . . . . . . . . . 50
Authentication Header (AH) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Internet Key Exchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Part Two: Detailed Analysis
4 IPSec Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59
The IPSec Roadmap. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
IPSec Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
IPSec Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Security Associations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
IPSec Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Fragmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
ICMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5 The Encapsulating Security Payload (ESP) . . . . . . . . . . . . . .83
The ESP Header. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
ESP Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
ESP Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Table of Contents ix
6 The Authentication Header (AH) . . . . . . . . . . . . . . . . . . . . . 93
The AH Header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
AH Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
AH Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7 The Internet Key Exchange . . . . . . . . . . . . . . . . . . . . . . . . . 101
ISAKMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
IKE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
The IPSec DOI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Part Three: Deployment Issues
8 Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Policy Definition Requirement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Policy Representation and Distribution . . . . . . . . . . . . . . . . . . . . . . . 135
Policy Management System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Setting Up the Policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
9 IPSec Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Implementation Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
IPSec Protocol Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Fragmentation and PMTU. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
ICMP Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
10 IP Security in Action. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
End-to-End Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168