Regular Expression Mastery

Tokenizing

A different version of the same thing:

        my $s;
        sub set_string {
            $s = shift;
        }

        sub next_token {
          return POWER        if $s =~ /\G\*\*/gc;
          return ASSIGNMENT   if $s =~ /\G:=/gc;
          return "OP $1"      if $s =~ /\G([-+*/^()=])/gc;
          return IDENT        if $s =~ /\G[A-Za-z]\w*/gc;
          return FLOAT        if $s =~ /\G\d*\.\d+(?:[Ee]\d+)?/gc;
          return INT          if $s =~ /\G\d+/gc;
          return next_token() if $s =~ /\G\s+/gc;
          return BAD_CHAR     if $s =~ /\G./gc;
        }

This uses the /gc modifier with \G
\G anchors the match to occur at the current pos()
- Rather than somewhere to the right of it as usual
Normally, the pos() is discarded if the match fails
- /c disables this misfeature