Introduction

When you program computers, you will sooner or later need to parse a text into something that your program can directly manipulate. Even though you will probably not write a compiler for a full programming language, there are countless other situations in which a parser would help you: extracting options specified in command-line arguments, reading configuration files, building a contact database from mails, or extracting information from XML documents. For common tasks there are often specialized solutions (there are ways to parse XML documents, for example, which are covered in Chapter 4, and there is the optparse extension for handling command-line arguments, which can be downloaded from RAA). However, when specialized tools do not address your problem, you will need to produce a parser for your needs. In this chapter we'll take a look at your options.

In general, to parse is to reconstruct the structure of some data in a linear form according to a grammar. The linear form is often ASCII text but can be any linear data such as binary data or Unicode text. The grammar is a description of all the valid forms the linear data can take and, at the same time, a description of the structure of the data.

You have two main options in producing a parser. You can write manually or you can use a parser generator that will generate a parser from the grammar. The benefits from writing your parser manually are that it can be very fast, you...