satisfiability-based program

198 Pages·2010·1.4 MB·English

SATISFIABILITY-BASED PROGRAM
REASONING AND PROGRAM SYNTHESIS 1
by
Saurabh Srivastava
1Revisedwithminorcorrections,andaddendumstotheoriginal[249].
ABSTRACT
Title of dissertation: SATISFIABILITY-BASED PROGRAM
REASONING AND PROGRAM SYNTHESIS
Saurabh Srivastava, Doctor of Philosophy, 2010
Dissertation directed by: Professor Jeffrey S. Foster
Department of Computer Science
Programreasoningconsistsofthetasksofautomaticallyandstaticallyverifyingcorrectness
and inferring properties of programs. Program synthesis is the task of automatically generating
programs. Both program reasoning and synthesis are theoretically undecidable, but the results in
this dissertation show that they are practically tractable. We show that there is enough structure
in programs written by human developers to make program reasoning feasible, and additionally
we can leverage program reasoning technology for automatic program synthesis.
This dissertation describes expressive and efficient techniques for program reasoning and
program synthesis. Our techniques work by encoding the underlying inference tasks as solutions
tosatisfiabilityinstances. Acoreingredientinthereductionoftheseproblemstofinitesatisfiability
instances is the assumption of templates. Templates are user-provided hints about the structural
formofthedesiredartifact, e.g., invariant, pre-andpostconditiontemplatesforreasoning; orpro-
gram templates for synthesis. We propose novel algorithms, parameterized by suitable templates,
that reduce the inference of these artifacts to satisfiability.
Weshowthatfixed-pointcomputation—thekeytechnicalchallengeinprogramreasoning—
is encodable as SAT instances. We also show that program synthesis can be viewed as generalized
verification, facilitating the use of program reasoning tools as synthesizers. Lastly, we show that
program reasoning tools augmented with symbolic testing can be used to build powerful synthe-
sizers with approximate guarantees.
We implemented the techniques developed in this dissertation in the form of the VS3—
Verification and Synthesis using SMT Solvers—suite of tools. Using the VS3 tools, we were able
to verify and infer expressive properties of programs, and synthesize difficult benchmarks from
specifications. These prototype tools demonstrate that we can exploit the engineering advances
in current SAT/SMT solvers to do automatic program reasoning and synthesis. We propose
building future automatic program reasoning and synthesis tools based on the ideas presented in
this dissertation.
SATISFIABILITY-BASED PROGRAM
REASONING AND PROGRAM SYNTHESIS
by
Saurabh Srivastava
Dissertation submitted to the Faculty of the Graduate School of the
University of Maryland, College Park in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
2010
Advisory Committee:
Professor Jeffrey S. Foster, Chair/Advisor
Professor Michael W. Hicks
Dr. Sumit Gulwani
Professor Jonathan Katz
Professor Mike Boyle
(cid:13)c Copyright by
Saurabh Srivastava
2010
To my parents,
Preeti and Prakash,
and my grandparents.
Acknowledgments
I am greatly indebted to my advisor, Jeff Foster, whose willingness to accept, encourage,
and fund, every experimental and contentious research thought I decided to pursue, amazes me in
hindsight. It would be fair to say that he introduced me to research in programming languages.
My interest in the foundations of programming languages started with a course he taught. To say
thattheknowledgeIgotfromhimwasjusttechnicalwouldbeagrossunderstatement. Researchis
half about knowing what problems to solve. Jeff has given me the perspective to pursue the right
research ideas, and also the right attitude to overcome the hurdles a research project invariably
presents. Not only that, his constant advising on matter not just technical, has helped hone my
skills in writing, presentation, articulation, and maybe even helped me become a more capable
social being.
Talkingoftheresearchandmentoring“process,”thereisprobablynobetterpersontorefer
to than Mike Hicks, my second advisor. Mike has the uncanny ability to recognize a student’s
aptitude, interests, and potential, and then to encourage them in exactly the right way. I started
workinginprogramminglanguagesashisstudentandheintroducedmetotherigorinthefounda-
tions of programming languages—which he knew would attract me to the field. His own research
diversity and time management skills have been a constant source of inspiration and I hope I can
one day be as efficient a researcher as he is. His advising and presence ensured that my graduate
career was a breeze and enjoyable every step of the way.
“Thisappearstobeanimpossibleproblem. Letusseeifwecansolveit.” Ifthiswasevera
researchphilosophy,SumitGulwani’swouldbethis. FromSumit,mythirdadvisor,Ihavelearned
that a research agenda is not useful if it is not adventurous, ambitious, and borderline crazy. In
myinternshipunderhisguidance,Sumitintroducedmetoprogramverification,andwedecidedto
ignore traditional approaches in favor of an experimental satisfiability-based approach to program
reasoning. Ourexperimentalprogramreasoningapproachservedasasegueintoautomaticprogram
synthesis, a problem typically considered intractable. Also, under his mentorship, I have learnt
that research is a social, collaborative, activity. Only through constant conversation do guesses
and intuition develop into concrete solutions. Lastly, from his work ethic I have learned that a
task worth pursuing is a task worth doing well; even if that means spending 16 to 18 hours a day
on it.
I have gained immensely from my interactions with all three of my advisors. I persevere to
constantly change myself to imbibe the qualities they each possess, and I admire. I am grateful to
them for having invested as much time as they did, and for showing me the way forward.
Mygraduateresearchcareerhasbeenmeandering,andIwouldnothavebeeninmycurrent
position,ifithadnotbeenfortheeffortsofmyadvisors“ontheway”. Aspecialthanksgoesoutto
Bobby Bhattacharjee, whose prodding is partly the reason why I stuck around in graduate school.
He has mentored me in all things non-technical, and indoctrinated in me the perspective that the
onlylivableplaceisacademia! Hismotivationandencouragementhashelpedmeimmensely. Also,
a thanks to my undergraduate research advisors, Dheeraj Sanghi and Ajit K. Chaturvedi, at IIT
Kanpur, who got me started on research early.
I would also like to thank other professors in the department and on my committee. I am
grateful to Samir Khuller, Aravind Srinivasan, Jonathan Katz, and Mike Boyle for having taken
the time out to give me very useful feedback on my research.
A shout-out is in order to the graduate students, present and past, that make up the
Programming Languages group at the University of Maryland (PLUM). Nik, Polyvios, Iulian,
Chris, Yit, Elnatan, Mark, David An, Mike Furr, Martin, Evan, Avik, Stephen, Ted, and David
Greenfieldboyce were all a constant source of enlightening conversations (research or otherwise).
Also, to all my other friends in the department, Srinivas Kashyap, Shiv, Bhargav, Aswin, Gaurav,
Narayanan, Akhil, and Abheek; who made my graduate school days worth every moment.
Only half jokingly, a nod to all the local coffee shops, without their presence there would
be no caffeine induced writing rampages and this dissertation would have taken twice as long.
A special thanks to my family, immediate and extended. I am humbled by the constant
support and encouragement that my parents have provided throughout this period. My father, a
professorandresearcherinInorganicChemistry,hasbeenaconstantrolemodelformyprofessional
iii
life; while my mother, with her open-mindedness and enthusiasm for absorbing ideas has been a
constant role model for my personal life. I am also thankful to them for providing the most
intellectuallynurturingenvironmentthatachildcouldaskfor,andforsettingtherightexpectations
for me as an adult.
Also,abigthankstomyyoungerbrothers,AbhishekandPrashast. Theyhavebeenasource
of constant pride and perspective. They are my connection to the non-academic world. Lastly,
to the person whose constant presence made the last four years incredibly enjoyable, and who I
cannot thank enough, Lauren.
iv
Table of Contents
List of Tables viii
List of Figures ix
List of Abbreviations x
1 Introduction 1
1.1 Satisfiability- and Template-based Program Reasoning and Synthesis . . . . . . . 2
1.1.1 Satisfiability for Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Satisfiability for Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.3 Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.4 Maximally Best Solutions using Satisfiability . . . . . . . . . . . . . . . . . 6
1.2 Program reasoning with linear arithmetic . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Program reasoning with predicate abstraction . . . . . . . . . . . . . . . . . . . . . 8
1.4 Verification-inspired synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Testing-inspired synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6 Engineering Verifiers and Synthesizers . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.7 Key Contributions and Organization . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 Program Reasoning over Linear Arithmetic 14
2.1 Using SAT Solvers for Invariant Inference . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Program Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.1 Verification Conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.2 Template specification T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.3 Constraint solving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.4 Choice of cut-set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3 Interprocedural Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 Maximally weak Precondition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4.1 Locally pointwise-weakest strategy . . . . . . . . . . . . . . . . . . . . . . . 27
2.5 Maximally strong Postcondition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6 Specification Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.7 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.7.1 Termination and Bounds Analysis . . . . . . . . . . . . . . . . . . . . . . . 34
2.7.2 Counterexamples for Safety Properties . . . . . . . . . . . . . . . . . . . . . 35
2.7.3 Counterexamples for Termination Properties . . . . . . . . . . . . . . . . . 37
2.8 Experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.10 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3 Program Reasoning over Predicate Abstraction 41
3.1 Using SMT Solvers for Program Reasoning . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Motivating Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3.1 Templates for Predicate Abstraction . . . . . . . . . . . . . . . . . . . . . . 45
3.3.2 Program Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3.3 Invariant Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.4 Optimal Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.5 Iterative Propagation Based Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5.1 Least Fixed-point. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.5.2 Greatest Fixed-point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.6 Satisfiability-based Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
v
3.6.1 SAT Encoding for Simple Templates . . . . . . . . . . . . . . . . . . . . . . 57
3.6.1.1 Encoding VCs as SAT for Simple Templates . . . . . . . . . . . . 59
3.6.2 SAT Encoding for General Templates . . . . . . . . . . . . . . . . . . . . . 61
3.6.2.1 Encoding VCs as SAT using OptimalNegativeSolutions . . . . . 62
3.7 Specification Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.7.1 Maximally Weak Pre- and Maximally Strong Postconditions. . . . . . . . . 63
3.8 Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.8.1 Templates and Predicates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.8.2 Verifying standard benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.8.3 Proving ∀∃, worst-case bounds, functional correctness . . . . . . . . . . . . 69
3.8.4 Properties of our algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.8.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.10 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4 Proof-theoretic Synthesis: Verification-inspired Program Synthesis 76
4.1 Program Synthesis as GeneralizedVerification . . . . . . . . . . . . . . . . . . . . . 76
4.1.1 Motivating Example: Bresenham’s Line Drawing . . . . . . . . . . . . . . . 77
4.2 The Synthesis Scaffold and Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.2.1 Picking a proof domain and a solver for the domain . . . . . . . . . . . . . 81
4.2.2 Synthesis Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.3 Synthesis Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.3.1 Using Transition Systems to Represent Acyclic Code . . . . . . . . . . . . . 82
4.3.2 Expanding a flowgraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.3.3 Encoding Partial Correctness: Safety Conditions . . . . . . . . . . . . . . . 85
4.3.4 Encoding Valid Control: Well-formedness Conditions . . . . . . . . . . . . . 86
4.3.5 Encoding Progress: Ranking functions . . . . . . . . . . . . . . . . . . . . . 89
4.3.6 Entire Synthesis Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.4 Solving Synthesis Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.4.1 Basic Requirement for Solver(sc) . . . . . . . . . . . . . . . . . . . . . . . 94
4.4.2 Satisfiability-based Verifiers as Solver(sc) . . . . . . . . . . . . . . . . . . . 95
4.4.3 Iterative Verifiers as Solver(sc). . . . . . . . . . . . . . . . . . . . . . . . . 95
4.5 Experimental Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.5.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.5.2 Algorithms that use arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.5.3 Sorting Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.5.4 Dynamic Programming Algorithms . . . . . . . . . . . . . . . . . . . . . . . 100
4.5.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.5.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.7 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5 Path-based Inductive Synthesis: Testing-inspired Program Synthesis 106
5.1 Using Symbolic Testing to Synthesize Programs . . . . . . . . . . . . . . . . . . . . 107
5.2 Motivating Example and Technical Overview . . . . . . . . . . . . . . . . . . . . . 108
5.3 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.4 PINS: Synthesizing programs using symbolic testing . . . . . . . . . . . . . . . . . 113
5.4.1 Safety and Termination Constraints . . . . . . . . . . . . . . . . . . . . . . 113
5.4.2 Satisfiability-based Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.4.3 Directed path exploration using solution maps . . . . . . . . . . . . . . . . 117
5.4.4 PINS: Path-based Inductive Synthesis . . . . . . . . . . . . . . . . . . . . . 118
5.5 Synthesizing inverses using PINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.5.1 Mining the flowgraph, predicates and expressions . . . . . . . . . . . . . . . 120
5.5.2 Axiomatization for handling Abstract Data Types . . . . . . . . . . . . . . 121
vi
5.5.3 Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.5.4 Sequential composition: Synthesizing Inverses . . . . . . . . . . . . . . . . . 123
5.5.5 Parallel composition: Synthesizing Network Programs . . . . . . . . . . . . 124
5.6 Experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.6.1 Case Study: Inverting the LZ77 Compressor. . . . . . . . . . . . . . . . . . 126
5.6.2 Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.6.2.1 Program Inversion: Sequential Composition . . . . . . . . . . . . . 128
5.6.2.2 Client-Server: Parallel Composition . . . . . . . . . . . . . . . . . 129
5.6.3 Experience and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.8 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
6 Engineering Satisfiability-based Program Reasoning/Synthesis Tools 133
6.1 Using off-the-shelf SAT/SMT solvers . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2 Tool Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2.1 Tool Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.2.2 Solver Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.2.2.1 Compensating for limitations of SMT solvers . . . . . . . . . . . . 135
6.2.2.2 Axiomatic support for additional theories . . . . . . . . . . . . . . 136
6.3 Concurrent reduction for super-linear speedup . . . . . . . . . . . . . . . . . . . . . 137
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
7 Extensions and Future Work 139
7.1 Expressiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.2 Applications to non-(sequential, imperative) models . . . . . . . . . . . . . . . . . 140
7.3 Synthesis as augmenting compilation . . . . . . . . . . . . . . . . . . . . . . . . . . 141
8 Related Work 142
8.1 Program Reasoning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.1.1 Program Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.1.1.1 Invariant validation using SMT solvers . . . . . . . . . . . . . . . 142
8.1.1.2 Invariant Inference over Linear Arithmetic . . . . . . . . . . . . . 143
8.1.1.3 Invariant Inference over Predicate Abstraction . . . . . . . . . . . 145
8.1.1.4 Verification without invariant inference . . . . . . . . . . . . . . . 146
8.1.2 Specification Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.2 Program Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.2.1 Deductive Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
8.2.2 Inductive Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
8.2.3 A Liberal View of Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
9 Conclusion 152
A Correctness of Satisfiability-based Algorithms 153
A.1 Linear Arithmetic: Correctness of Precondition Inference . . . . . . . . . . . . . . 153
A.2 Linear Arithmetic: Refined neighborhood structure N . . . . . . . . . . . . . . . 155
c,π
A.3 Predicate Abstraction: Correctness of Optimal Solution Computation . . . . . . . 155
A.4 Predicate Abstraction: Correctness of the Reduction to SAT . . . . . . . . . . . . 162
B Code Listings 164
B.1 Linear Arithmetic Invariant Inference . . . . . . . . . . . . . . . . . . . . . . . . . 164
B.2 Predicate Abstraction Invariant Inference . . . . . . . . . . . . . . . . . . . . . . . 164
B.3 Proof-theoretic Synthesis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
B.4 Path-based Inductive Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Bibliography 169
vii