Higher-Order Parsing

Lexing

First, our program must identify things like NUMBER
Idea: preprocess the input
- Turn it from a character string into a list of tokens
- Each token is an atomic piece of input
- Examples: sin, x, +, 12345
Humans do this when they read
- First, turn the sequence of characters into a sequence of words
- Then, try to understand the structure of the sentence based on meanings of words
This is called lexing