Table Of ContentDedication
To Karen, for her constant love and support.
To Calvin and Kate, for patiently waiting for Daddy to play.
To Uncle Frank, who would have loved to read it.
To Mom, my teaching model.
Preface
? This course is intended primarily for upper-level undergraduate mathematics majors, although
math education, economics, computer science, and other majors regularly frequent my course,
including some graduate students (in fact, many of the challenges and projects are aimed at
graduatestudents). Butthefocus,asyouwillreadabout,iscenteredmoreontheunderstanding
of the mathematics itself (and mathematical applications) than on the industrial applications
and use of relevant software to solve large problems. That doesn’t mean one can’t use this text
for such other purposes, and I hope you’ll see that you can, but instead that we want to do
mathematics first and foremost. There are plenty of books that follow the applied model for
teaching this subject, and too few that follow the path we outline below. We do assume that
students have passed the usual introductory courses in linearalgebra, computer algorithms,and
proof writing — typical requirements of all math majors.
The Subject
A little explanation is in order for our choice of the title Linear Optimization1 (and corresponding
terminology) for what has traditionally been called Linear Programming. The word programming in this
context can be confusing and/or misleading to students. Linear programming problems are referred to as
optimization problems but the general term linear programming remains. This can cause people unfamiliar
with the subject to think that it is aboutprogrammingin the sense of writing computer code. It isn’t. This
workbook is about the beautiful mathematics underlying the ideas of optimizing linear functions subject
to linear constraints and the algorithms to solve such problems. In particular, much of what we discuss is
the mathematics of Simplex Algorithm for solving such problems, developed by George Dantzig in the late
1940s.
The word program in linear programming is a historical artifact. When Dantzig first developed the
Simplex Algorithm to solve what are now called linear programming problems, his initial model was a class
ofresourceallocationproblemsto be solvedforthe U.S.Air Force. The decisionsaboutthe allocationswere
called‘Programs’by the Air Force,andhence the term. Dantzig’sarticle2 is a fascinatingdescriptionof the
origins of this subject written by the person who originated many of the ideas. Included is a description of
how Tjalling Koopmans (who won a Nobel Prize in economics for his work in decision science) suggested
shortening Dantzig’s description ‘programming in a linear structure’ to ‘linear programming’during a walk
onthe beachwith Dantzig. Also includedis anote that, at thetime, ‘code’ wasthe wordusedforcomputer
instructions and not ‘program’.
To be clear to potential and current students that this is a mathematics course requiring a background
in writing proofs and not a computer coding class, we prefer the terminology linear optimization. We do
look atcomputer algorithmsbut focus onthe underlying mathematics. A smallamountofcomputer coding
(forexample,simple MAPLE)programswillbeveryuseful, butwritingcodeisnotthe centralpurposeofthe
course. The shorthand LP has been used to refer to both the generalsubject of linear programming as well
as specific instances of linear programs. To distinguish, in our notation, LO (linear optimization) refers to
1I’mnotthatoriginal—thereareatleastadozenbooksthatusethisterminology.
2G.Dantzig,LinearProgramming,Operations Research50(2002), 42–47.
viii Preface
the general class of optimizing linear functions subject to linear constraints while LOP (linear optimization
problem) refers to specific instances of such problems. Furthermore, using the optimization term brings
the subject in line with other, closely related fields that are increasingly called Optimization (Nonlinear,
Quadratic, Convex, Integer, Combinatorial).
? Of course, if you are more inclined to maintain the status quo, feel free to continue verbalizing
“ell-pee” in place of “lop”; beware, however, that the extra syllable spoken repeatedly over the
course of an entire semester can take its toll on your energy.
The Simplex Algorithm is the focus of study in this book. In particular, we do not discuss Karmarkar’s
AlgorithmorKhachian’sEllipsoidAlgorithmormoregeneralinterior-pointapproachestosolvingLOPs. The
main reason for this is that, as I hope you’ll experience, the Simplex Algorithm leads to richer connections
with linear algebra, geometry, combinatorics, game theory, probability, and graph theory. Furthermore, in
the post-optimality analysis that occurs in economic modeling and in Integer Optimization, the Simplex
Algorithm plays a central role.
? For a superb and extensive treatment of interior methods, let me suggest [[14]].
Why Teach This?
This is a reasonable question, and there are better answers than, “Why not?” and, “Because it’s
cool!”
One of the areas in which Mathematics departments lag behind those of, say, Physics and, most
notably these days, Biology, is in the delivery of up-to-date discoveries in the subject (galaxies, big
bang,genomics,medicine,etc.). Today’scollegemathematicscurriculumis virtuallyunchangedfrom
50 years ago — go into your library and look at the degree requirements from 1960, or 1940, for
that matter—and, in orderto competefor majors,departmentsneed to be perceived bystudents as
far more relevant to the modern world. One of the main recommendations of the CUPM Guide,3 in
fact, is awareness of connections to other subjects and introducing contemporary topics to enhance
student perceptions of the vitality and importance of mathematics in the modern world.
Forexample,theSimplexAlgorithmiswidelyrecognized4 asoneofthetoptenalgorithmsofthe20th
century for its widespread use, its powerful applications to industry and to mathematics, it’s elegant
simplicity, and it’s fascinating mathematical underpinnings, and yet, the percentage of departments
offering LO in 2000–01 was only 13%.5 Given the revolution that the subject has brought to the
business world alone, it is difficult to argue against having the subject available to math majors at
virtually every school in the country.
The case for LO increases when we realize that most of the students in mathematics programs do
not go on to graduate school, but instead to jobs in a wide range of industries. In fact, 92% of
studentsearningbachelor’sdegreesinthe mathematicalsciencesfrom 1994–96wentdirectlyintothe
workforce,many of whom ended up in management.6 How nice it would be for them to have such a
tool as Simplex, or at least to know of the benefits of optimization to their enterprise. In addition, a
significant portion (26% in 20007) of students taking upper division courses are preservice secondary
mathematics teachers. This is relevant because Discrete Mathematics is now in state curriculum
3Undergraduate Programs and Courses in the Mathematical Sciences: CUPM Guide 2004, www.maa.org/cupm.
4J. Dongarra and F. Sullivan, Top ten algorithms of the century, Computing Science and Engineering, January/February
2000.
5D.J.Lutzer,J.W.MaxwellandS.R.Rodi,StatisticalAbstractof Undergraduate Programs intheMathematicalSciences
inthe UnitedStates: Fall 2000 CBMS Survey,AMS,2002.
6National Survey of Recent College Graduates, NationalScienceFoundation, 1997,www.nsf.gov/sbe/srs/nsf01337.
7Statistical Abstract,op. cit.
Preface ix
standards throughout the country, and LO is listed in that category in many states.8 How nice it
would be for our high school teachers to understand more of the applicability of the algebra they
teach.
Moreover, our undergraduate curricula compartmentalizes topics. Consequently, most math majors
graduatewitha sensethat manyoftheircourses havelittle ornothingtodowith eachother. Aclass
in LO can do much to counter this misimpression,blending linearalgebra, geometry, modeling,prob-
ability,gametheory,algorithms,combinatorics,graphtheory,computerprogramming,andtheoretical
computer science into one big, tasty LOG.9
Finally, why not teach it? It’s cool!
Terminology
Besides the usage of LOP and ILOP, we introduce other quirks into the language, mostly for handiness
and consistency and occasionally for fun. For example, we discuss four kinds of linear combinations, based
on whether or not the extra affine and conic conditions hold, so it makes sense to use the similar notations
lspan, aspan, nspan, and vspan for linear, affine, conic, and convex (both affine and conic) combinations,
respectively. In particular, lspan makes more sense in this scheme than does span, the more common term
found in linear algebra texts. Geometric hulls get the same treatment, with lhull, ahull, nhull, and vhull,
respectively. For fun we use the term FLOP (Fractional LOP) when we need to distinguish a LOP from
being an ILOP. Indeed, the first step in solving an ILOP is to relax the integer constraints to allow for
rationals and find the resulting floptimal solution, which is used as a first approximation to the iloptimal
solution. The term BLOP refers to an ILOP whose variables are binary (either 0 or 1).10 Also, when we
discuss game theory, we talk about the GLOP (Game LOP) derived from a game. We don’t go too much
farther down the self-parody road — hopefully there is no SLOP in the book.
I think we’re also the first LO book to use the term parameter in place of nonbasic variable. That must
be worthsomekindofaward,right? Actually,I stoleitfromvirtuallyeverylinearalgebratexteverwritten.
Chapter Flow
Outlandish as it may seem, someone studying from this text will need to start with Chapter 1, followed
by Chapter 2. After that, there are many directions of travel.
? Each chapter includes an initial commentary that elaborates further on the following — which
parts of the chapter need what prerequisites and which are prerequisites for others.
If and when you wish to study geometry, you’ll want to learn Chapter 3 before Chapter 8, but you can
pretty much learn them whenever you want. No other chapter uses Chapter 3 explicitly, although it does
offer very beneficial intuition that permeates almost every other chapter. This is why it is placed so early.
Chapter8,however,isn’tnecessaryformuch(unlessonecontinuesontostudy graduateleveloptimization),
but does use material from Chapter 7 (and Section 8.4 needs Chapter 6) — and is kind of fun.
? Chapter3canbe replacedbyChapter0forthosewholiketodevelopstudents’theorem-proving
abilities right off the bat. I sometimes enjoy using Chapter 0 as a theoretical, rather than
industrial motivation for developing the Fundamental Theorem of Linear Optimization 2.9.1.
8In fact, this course covers 8 of the 10 NCTM standards (Algebra, Geometry, Data Analysis and Probability, Problem
Solving,ReasoningandProof,Communication,Connections, andRepresentation)—onesummerItaughtaspecialsectionof
thecoursetohighschoolteachers whobenefitted greatlyfromsuchbreadth.
9LinearOptimizationGoulash.
10ThusitisquitenaturalformanyacademicstobeinterestedinBO.
x Preface
Chapter 4 is really the heart of the course. Everything feeds off of duality. This is why we derive the
dual immediately in Section 1.1. Spending extra time here pays dividends later, as everything thereafter
depends critically upon it.
The material of Chapter 5 is useful for learning the tip of the how-this-is-done-in-the-real-worldiceberg.
Theabilitytostateandworkwitheverythinginmatrixformisaveryusefulskillingeneral,andinparticular
comes in handy in Chapter 12. Thus it does not need to be studied before any other chapter (if at all, as it
is not essential material otherwise — in fact, Chapter 5 is only crucial to proving Theorem 12.1.4).
Chapter 6 puts Chapter 4 in general context, and is required for all subsequent chapters.
? Itispossible,however,tocoverthematerialofChapter6moreinformally,asneeded. Infact,as
with most chapters, one can decide to present onlyits first two sections(maybe plus its Section
3 Practice) in an effort to cover a broader spectrum of material. This text is designed for such
a degree of flexibility.
Chapter 7 contains material that is necessary for Chapter 8. Chapter 9 is not needed by anyone who
doesn’twanttohaveagoodtime. Chapter10isrequiredbyChapter11,andSection12.4iskeyforChapter
13. Chapters 7–13 offer the greatest flexibility for studying your favorite topics within a semester’s time.
Of course, you could slow down, spend extra time on the exercises, and complete the whole book in two
semesters. Have at it!
A visual description of the above dependencies is given below.
Book Format and Usage
It is not difficult to notice that this text is different from most, and not just because I can’t resist even
the lamest of jokes,11 so it may be worth some discussion on why this is so, what benefits this may have for
you, and how best to take advantage of the new format.
First and foremost, a great deal of information is missing, information that typically is included in
mathematics texts.
? Mosttextsprohibitdiscoverystylelearningbyexplainingeverythinginsightonthepage,leaving
little room for instructor influence, whereas the premise of the discovery method is to withhold
information! The commonly reported benefits of the approach (increased understanding, reten-
tion, enjoyment, motivation, confidence,etc.) motivate this format, from which you can decide
how much more you wish to share with your students.
Having spent most of your life reading from such texts, you may be used to being fed facts and algorithms,
andwell usedto memorizingthem in orderto reproducethem on exams. But I believe that youare capable
ofmuchmore: ofderivingresults,infact,discoveringthemthroughexperimentation,ofmakingconjectures,
of proving theorems, of solving problems and checking them yourself, and of asking creative questions.
11Mostoftheserequiremyagetounderstandanyway.
