Double Identifier Problem

An existing problem was discovered when the parser was modified to not store the data type character of identifiers.   The issue was with double identifiers when using the optional # data type character.  The identifiers Variable and Variable# were incorrectly added to the dictionary as separate entries when they should have been the same entry.

The parser get identifier function was modified to not store the data type character in the token.  This caused a problem when recreating double identifiers where the # character entered would disappear.  Recreating all double identifiers with a # character was also not desirable.  This was corrected by adding the Double sub-code to the token.  A sub-code argument was added to the token constructor for identifiers.  This sub-code is encoded into the program code so that the # character is recreated when it is entered.

The Double sub-code was only being used for constants.  When the value of a constant is within the integer range, its data type is set to integer, and if a decimal point is present, the Double sub-code is set.  The translator uses this sub-code to determine if a constant can be used as a double even though the data type is integer (see post from October 28 for details).  This sub-code does not survive past the translator (not put into the program code).

A new string with data type access function was added to the token to add the data type character (#, % for integers, and $ for strings) to the token string returned.  A # character is only added if the Double sub-code is set.  This function replaced the string access function in the test token stream insert operator, tester print token, and several recreate functions.

A sub-code argument was added to the table entry operand text functions.  The variable operand text functions were modified to add the data type character to the variable name.  For double variables, the character is only added if the Double sub-code is set.  An Ignore sub-code enumerator was added, and when passed to the operand text function, no data type character is added to the variable name.  This option was needed for the program model decode function that uses the operand text function to set the string of the token (since tokens no longer store the data type character).

The value of the Double sub-code was changed so that its bit value was within the range of the sub-code bits (not necessary before since this sub-code was not used in the program code).  The return type of program code instruction sub-code access function was changed to the Sub-Code enumeration type (from an integer).  The expected encoder test results were updated, specifically the dictionaries output since the data type characters are no longer present in the entries.

[branch table commit e97057efca]