This saves some effort and probably some performance for things like integers, but I'm mainly doing this in anticipation of string literals, because it would just be ridiculous to read code that lexes string literals twice.
Now tests compare the literal parser against the output of the
compiler. The little example contracts we are compiling for the
AACI already had the FATE value in them, in the form of the
instruction
{'RETURNR', {immediate, FateValue}}
so we just extract that and use it for the tests.
We tokenize, and then do the simplest possible recursive descent.
We don't want to evaluate anything, so infix operators are out,
meaning no shunting yard or tree rearranging or LR(1) shenanigans
are necessary, just write the code.
If we want to 'peek', just take the next token, and pass it around
from that point on, until it can actually be consumed.
