Parser – STL Preparations

The changes required to make the parser routines use the STL are going to be extensive, but an attempt will be made to break the changes into smaller incremental changes.  Since the parser routines make use of various table functions, it will be necessary to modify the table entries and its functions to use STL.  Some preparatory changes were made.

The table entries are divided into several groups for searching, which includes plain word, parentheses words, data type words and symbols.  The parser utilizes these groups when searching if strings have a code.  The data type words section was empty and upon consideration it was concluded that this group is not needed.  This group may have been originally conceived for internal functions that don't have arguments (for example, a DATE$ function).  However, these internal functions can go into the plain word group (the RND no argument function is already in this group).  This group type along with its bracketing entries were removed.

[branch parser commit 0043105154]

The issue of the token being left allocated when the parser throws an exception could be resolved by not creating a token until a valid token is found.  Used of the token member before an exception is thrown were examined and the only use was in the creation of the error exception.  The only token members used were its column and length.  The column was always the same as the current input position (except one instance) and the length was always 1.

All of the throw statements were modified to not use the token instead using the current input position with a length of 1.  One case used a length of 2 (2 was previously used).  For the "floating point constant is out of range" error, the statement to set the new input position was moved to after an error is thrown.

For the "expected sign or digits for exponent in floating point constant" error, two columns were reported, the column at the beginning of the number (for operator state) and the alternate column at the beginning of the error (for operand state).  A number token is no longer accepted when invalid, so only the alternate column was being used.  This mechanism was not needed, and for this error, the position of the error only is reported.  This mechanism was also removed from the tester print error function.

[branch parser commit e92da57ef1]