Approximating Context-Free Grammars for Parsing and Verification

222 Pages·2008·2.8 MB·English

Approximating Context-Free Grammars
for Parsing and Verification
Thethornbushofambiguity.
Sylvain Schmitz
Laboratoire I3S, Universit´e de Nice - Sophia Antipolis & CNRS, France

Ph.D. Thesis
Universit´e de Nice - Sophia Antipolis, E´cole Doctorale (( Sciences
et Technologies de l’Information et de la Communication ))
Informatique Subject
Jacques Farr´e, Universit´e de Nice - Sophia Antipolis Promotor
Referees
Eberhard Bertsch, Ruhr-Universit¨at Bochum
Pierre Boullier, INRIA Rocquencourt
Defense
September 24th, 2007 Date
Sophia Antipolis, France Place
Jury
Pierre Boullier, INRIA Rocquencourt
Jacques Farr´e, Universit´e de Nice - Sophia Antipolis
Jos´e Fortes G´alvez, Universidad de Las Palmas de Gran Canaria
Olivier Lecarme, Universit´e de Nice - Sophia Antipolis
Igor Litovsky, Universit´e de Nice - Sophia Antipolis
Mark-Jan Nederhof, University of St Andrews
G´eraud S´enizergues, Universit´e de Bordeaux I

Approximating Context-Free Grammars
for Parsing and Verification
Abstract
Programming language developers are blessed with the availability
of efficient, powerful tools for parser generation. But even with au-
tomated help, the implementation of a parser remains often overly
complex.
Although programs should convey an unambiguous meaning, the
parsersproducedfortheirsyntax,specifiedbyacontext-freegram-
mar, are most often nondeterministic, and even ambiguous. Fac-
ing the limitations of traditional deterministic parsers, developers
have embraced general parsing techniques, capable of exploring ev-
ery nondeterministic choice, but offering no unambiguity warranty.
The real challenge in parser development lies then in the proper
identification and treatment of ambiguities—but these issues are
undecidable.
Thegrammarapproximationtechniquediscussedinthethesisisap-
pliedtonondeterminismandambiguityissuesintwodifferentways.
The first application is the generation of noncanonical parsers, less
prone to nondeterminism, mostly thanks to their ability to exploit
an unbounded context-free language as right context to guide their
decision. Suchparsersenforcetheunambiguityofthegrammar,and
furthermore warrant a linear time parsing complexity. The second
application is ambiguity detection in itself, with the insurance that
a grammar reported as unambiguous is actually so, whatever level
of approximation we might choose.
Key Words
Ambiguity, context-free grammar, noncanonical parser, position
automaton, grammar engineering, verification of infinite systems
ACM Categories
D.2.4 [Software Engineering]: Software/Program Verification;
D.3.1 [Programming Languages]: Formal Definitions and Theory—
Syntax ; D.3.4 [Programming Languages]: Processors—Parsing ;
F.1.1 [Computation by Abstract Devices]: Models of Computation;
F.3.1 [Logics and Meanings of Programs]: Specifying and Verify-
ingandReasoningaboutPrograms; F.4.2[MathematicalLogicand
Formal Languages]: Grammars and Other Rewriting Systems

Approximation de grammaires alg´ebriques
pour l’analyse syntaxique et la v´erification
R´esum´e
La th`ese s’int´eresse au probl`eme de l’analyse syntaxique pour les
langages de programmation. Si ce sujet a d´ej`a ´et´e trait´e `a maintes
reprises, et bien que des outils performants pour la g´en´eration
d’analyseurs syntaxiques existent et soient largement employ´es,
l’impl´ementation de la partie frontale d’un compilateur reste en-
core extrˆemement complexe.
Ainsi, si le texte d’un programme informatique se doit de n’avoir
qu’une seule interpr´etation possible, l’analyse des langages de pro-
grammation, fond´ee sur une grammaire alg´ebrique, est, pour sa
part, le plus souvent non d´eterministe, voire ambigu¨e. Confront´es
aux insuffisances des analyseurs d´eterministes traditionnels, les
d´eveloppeurs de parties frontales se sont tourn´es massivement vers
destechniquesd’analyseg´en´erale,`amˆemed’explorerdeschoixnon
d´eterministes, mais aussi au prix de la garantie d’avoir bien trait´e
toutes les ambigu¨ıt´es grammaticales. Une difficult´e majeure dans
l’impl´ementationd’uncompilateurr´esidealorsdansl’identification
(non d´ecidable en g´en´eral) et le traitement de ces ambigu¨ıt´es.
Les techniques d´ecrites dans la th`ese s’articulent autour d’appro-
ximations des grammaires `a deux fins. L’une est la g´en´eration d’a-
nalyseurssyntaxiquesnoncanoniques,quisontmoinssensiblesaux
difficult´esgrammaticales,enparticulierparcequ’ilspeuventexploi-
ter un langage alg´ebrique non fini en guise de contexte droit pour
r´esoudre un choix non d´eterministe. Ces analyseurs r´etablissent la
garantie de non ambigu¨ıt´e de la grammaire, et en sus assurent un
traitement en temps lin´eaire du texte `a analyser. L’autre est la
d´etection d’ambigu¨ıt´e en tant que telle, avec l’assurance qu’une
grammaire accept´ee est bien non ambigu¨e quel que soit le degr´e
d’approximation employ´e.
Mots clefs
Ambigu¨ıt´e,grammairealg´ebrique,analyseursyntaxiquenoncanon-
ique,automatedepositions,ing´enieriedesgrammaires,v´erification
de syst`emes infinis

Foreword
This thesis compiles and corrects the results published in several articles
(Schmitz, 2006; Fortes G´alvez et al., 2006; Schmitz, 2007a,b) written over
thelastfouryearsattheLaboratoired’InformatiqueSignauxetSyst`emesde
Sophia Antipolis (I3S). I am indebted for the excellent working environment
I have been offered during these years, and further grateful for the advice
and encouragements provided by countless colleagues, friends, and family
members.
Iamindebtedtomanyfriendsandmembersofthelaboratory,notablyto
Ana Almeida Matos, Jan Cederquist, Philippe Lahire, Igor Litovsky, Bruno
MartinandS´ebastienVerel,andtothemembersofmyresearchteam,Carine
F´ed`ele, Michel Gautero, Vincent Granet, Franck Guingne, Olivier Lecarme,
and Lionel Nicolas, who have been extremely helpful and supportive beyond
any possible repay. I especially thank Igor Litovsky and Olivier Lecarme for
accepting to examine my work and attend to my defense. I am immensely
thankful to Jacques Farr´e for his insightful supervision, excellent guidance,
and extreme kindness, that greatly exceed everything I could have wished
for.
I thank Jos´e Fortes G´alvez for his fruitful collaboration, his warm wel-
come on the occasion of my visit in Las Palmas, and his acceptance to work
on my thesis jury. I gratefully acknowledge several insightful email discus-
sions with Dick Grune, who taught me more on the literature on parsing
technologiesthananyonecouldwishfor, andwhofirstsuggestedthatstudy-
ing the construction of noncanonical parsers from the NFA to DFA point
of view could bring interesting results. I am also thankful to Terence Parr
for our discussions on LL(*), to Bas Basten for sharing his work on the
comparison of ambiguity checking algorithms, and to Claus Brabrand and
Anders Møller for our discussions on ambiguity detection and for granting
me access to their tool.
I am obliged to the many anonymous referees of my articles, whose
remarks helped considerably improving my work, and I am much obliged to
Eberhard Bertsch and Pierre Boullier for their reviews of the present thesis,
and the numerous corrections they offered. I am honored by their interest
x Foreword
in my work, as well as by the interest shown by the remaining members of
my jury, Mark-Jan Nederhof and G´eraud S´enizergues.
Finally, I thank all my friends and my family for their support, and my
beloved Adeline for bearing with me through these years.
Sophia Antipolis, October 16, 2007