Pre-Encoder Issues – Constants

While thinking about the design of the first step of the encoder (assigning codes to all tokens), I realized that there is no reason to add hidden conversion codes after numeric constants - the constants can be converted during translation, not every time the constant is executed at run-time.  However, not all constants can be converted.  All integer constants can be converted to double but not from double to integer.

The union for the constant in a token for either the double or integer value was removed so that both values are available and the one needed is used.  A constant token needs to be able to hold there different data types, one for an integer, one for a double that can be used as a integer, and one for a double that can't be used as integer (outside the range of an integer).

To accomplish this, if a double constant (one that has a decimal point or exponent) that is within the integer range, its data type is set to integer and a new Double sub-code is set.  A double constant that can't be converted is set to the double data type.  When the constant is parsed and the integer data type is set, both the integer and double values in the token are set (in other words, the parser does the conversion).

For integer constants, the translator changes the data type of a constant token to the data type needed in the expression and the Double sub-code is removed.  A hidden conversion code no longer needs to be added after constants.  However, for double constants (those outside the range of a integer), if an integer is called for, the new "expected valid integer constant" error is returned.

For debugging, a "%" character is output after a constant if its data type was set to integer by the translator (even though it may be a floating point value).  A constant that was set to the double data type lacks this indicator character.  The expected results files were updated for the new indicator character and lack of convert codes after constants.  New translator test #17 was added to test many aspects of each constant data type (integer, integer with double sub-code, and double).  See the commit log for the full details of the changes.  A commit was also made renaming the token sub-code access functions for better readability (and functions not being used were removed).

[commit f104bdf37e] [commit 0d3151f5f9]