Sub-Code Enumeration Refactoring

When the Double sub-code enumerator value was changed to support double identifiers with the double data type character (#), it was noticed that the Parentheses and Colon sub-codes would never be used with the same codes.  The Parentheses sub-code will only be set for operators and operands (variable, constants, etc.), and the Colon sub-code will only be set for commands (with some exceptions).

The available sub-code bits are limited - there are only six.  Several sub-codes were already combined with the Option sub-code ('LET' for assignment codes, 'Question' for the Input Begin String code, and 'Keep' for the INPUT and INPUT PROMPT command codes).  The same thing was done for the Parentheses and Colon sub-codes, but instead of using the same sub-code enumerator for both (creating an appropriate name was impossible), both current enumerators were simply given the same value.

There was one problem with this scheme.  Two codes that could have the Colon sub-code are the Comma and Semicolon codes, which are only present in a PRINT statement and will be commands at run-time.  These codes are defined as operators.  The assignment codes are defined as no types, but are commands and could have the Colon sub-code.  To indicate that these are commands, a new Command table flag was added and their table entries were given this flag.

Therefore, a code has the Colon sub-code if it is a command type or has the Command table flag, otherwise the code has the Parentheses sub-code.  Checks for the Command table flag were added to the recreator function operator, the stream insert operator for program words, and the test stream insert operator for tokens.  A few other minor sub-code related changes were made, click Continue... for details.

The None sub-code (no bits) was removed - the default sub-code enumerator is used .  Three additional Unused enumerators were added for the remaining unused sub-code bits.  The sub-code arguments to token sub-code access functions and the program code instruction has sub code function were changed to the Sub-Code enumeration type.  The sub-code token member was changed from an integer to an unsigned 16-bit integer.  Changing to the Sub-Code enumeration would have prevented bit operations without using complicated and undesirable type-casting.

When the Command table flag was added to the assignment codes, it was noticed that two MID$ entries still contained the Multiple table flag.  This flag is now set automatically, so this flag was removed from these entries.

When the None sub-code was replaced in the token constructors, the constructor for integer constants was changed to use an ordinary initializer for the double value member instead of assigning its value in the constructor body.  The assignment was used to convert the integer value to a double since this cannot be done with a C++11 initializer, but than can also be done with an ordinary initializer.

[branch table commit e95ad4ff4f]