LPeg – Parsing Expression Grammars for Lua

LPeg - Parsing Expression Grammars For Lua

LPeg

Parsing Expression Grammars For Lua, version 1.1

LPeg

Home

Introduction

Functions

Basic Constructions

Grammars

Captures

Some Examples

The re Module

Download

License

Introduction

LPeg is a pattern-matching library for Lua, based on

Parsing Expression Grammars (PEGs). This text is a reference manual for the library. For those starting with LPeg,

Mastering LPeg presents a good tutorial. For a more formal treatment of LPeg, as well as some discussion about its implementation, see

A Text Pattern-Matching Tool based on Parsing Expression Grammars. You may also be interested in my talk about LPeg given at the III Lua Workshop.

Following the Snobol tradition, LPeg defines patterns as first-class objects. That is, patterns are regular Lua values (represented by userdata). The library offers several functions to create and compose patterns. With the use of metamethods, several of these functions are provided as infix or prefix operators. On the one hand, the result is usually much more verbose than the typical encoding of patterns using the so called regular expressions (which typically are not regular expressions in the formal sense). On the other hand, first-class patterns allow much better documentation (as it is easy to comment the code, to break complex definitions in smaller parts, etc.) and are extensible, as we can define new functions to create and compose patterns.

For a quick glance of the library, the following table summarizes its basic operations for creating patterns:

Operator Description lpeg.P(string) Matches string literally lpeg.P(n) Matches exactly n characters lpeg.S(string) Matches any character in string (Set) lpeg.R("xy") Matches any character between x and y (Range) lpeg.utfR(cp1, cp2) Matches an UTF-8 code point between cp1 and cp2 patt^n Matches at least n repetitions of patt patt^-n Matches at most n repetitions of patt patt1 * patt2 Matches patt1 followed by patt2 patt1 + patt2 Matches patt1 or patt2 (ordered choice) patt1 - patt2 Matches patt1 if patt2 does not match -patt Equivalent to ("" - patt) #patt Matches patt but consumes no input lpeg.B(patt) Matches patt behind the current position, consuming no input

As a very simple example, lpeg.R("09")^1 creates a pattern that matches a non-empty sequence of digits. As a not so simple example, -lpeg.P(1) (which can be written as lpeg.P(-1), or simply -1 for operations expecting a pattern) matches an empty string only if it cannot match a single character; so, it succeeds only at the end of the subject.

LPeg also offers the re module, which implements patterns following a regular-expression style (e.g., [09]+). (This module is 270 lines of Lua code, and of course it uses LPeg to parse regular expressions and translate them to regular LPeg patterns.)

Functions

lpeg.match (pattern, subject [, init])

The matching function. It attempts to match the given pattern against the subject string. If the match succeeds, returns the index in the subject of the first character after the match, or the captured values (if the pattern captured any value).

An optional numeric argument init makes the match start at that position in the subject string. As in the Lua standard libraries, a negative value counts from the end.

Unlike typical pattern-matching functions, match works only in anchored mode; that is, it tries to match the pattern with a prefix of the given subject string (at position init), not with an arbitrary substring of the subject. So, if we want to find a pattern anywhere in a string, we must either write a loop in Lua or write a pattern that matches anywhere. This second approach is easy and quite efficient; see examples.

lpeg.type (value)

If the given value is a pattern, returns the string "pattern". Otherwise returns nil.

lpeg.version

A string (not a function) with the running version of LPeg.

lpeg.setmaxstack (max)

Sets a limit for the size of the backtrack stack used by LPeg to track calls and choices. (The default limit is 400.) Most well-written patterns need little backtrack levels and therefore you seldom need to change this limit; before changing it you should try to rewrite your pattern to avoid the need for extra space. Nevertheless, a few useful patterns may overflow. Also, with recursive grammars, subjects with deep recursion may also need larger limits.

Basic Constructions

The following operations build patterns. All operations that expect a pattern as an argument may receive also strings, tables, numbers, booleans, or functions, which are translated to patterns according to the rules of function lpeg.P.

lpeg.P (value)

Converts the given value into a proper pattern, according to the following rules:

If the argument is a pattern, it is returned unmodified.

If the argument is a string, it is translated to a pattern that matches the string literally.

If the argument is a non-negative number n, the result is a pattern that matches exactly n...

LPeg – Parsing Expression Grammars for Lua

Related Articles

The Newest Instagram "Exploit" Is the Goofiest I've Seen

It's Not Just X. It's Y

Amazon, Facebook, FBI have access to a private intelligence-sharing network

Show HN: GoPeek – open links in live mini browser windows without new tabs

Agent Memory: An Anatomy