Token – Type As Enumeration Class

A series of changes were made to change the token type enumeration to an enumeration class because its size of enumerator was used to dimension several arrays.  The changes were put into several individual commits on a work branch, which were later combined (squashed using the interactive git rebase command) into a single commit and merged back to the cpp11 branch before pushing to GitHub.  Details of these changes follow.

1) Unrelated to the token type enumeration, the Table class convert code function was moved to the Token class.  This function contained arguments for a token with a data type and the desired data type to convert to.  It returned a conversion code.  However, it did not actually access the table.  Since codes are not part of the table (the table is only indexed by codes) and this function took a token pointer, it made more sense for this to be a Token member function.

2) Also unrelated to the token type enumeration, the auto-generated data type to string map was changed to a brute force static function with a switch statement in the test source file as described in the previous post.  (The compiler in-lined this function when set to release build.)  The generate map function was removed from the test names awk script and the dependency on the main header file was removed from the auto-generated test names source file from the CMake build file.

3) Two of the uses of the token type enumerator was the static has parentheses flag and precedence arrays used to determine if the token type has a parentheses and the precedence of the token type.  These arrays were initialized by the static initialize function.  These arrays were changed to unordered maps with initializer lists for their values.  Note that the maps do not contain values for all the token types.  When the map is accessed for one of these other token types, a new element will be added with the correct default values (false for the flag, zero for the precedence).  Since these maps are initialized, the initialize function was no longer needed and was removed.

4) The token type enumeration was changed to an enumeration class and was moved into the Token class where it belongs.  Outside of the Token class, the token type enumerators need a double scope as in the Constant_TokenType enumerator becomes Token::Type::Constant.

5) The auto-generated token type to string array was changed to a brute force function also.  (The compiler also in-lined this function.)  The find enumeration function in the test names awk script was now no longer used and so was removed and the dependency on the token header file was removed from the auto-generated test names source file from the CMake build file.

[branch cpp11 commit e7dc6a964a]