Pre-Encoder Issues – Array References

Before creating the new encoder class, some issues need to be taken care of in preparation for encoding.  The first was the index member variable that was added to the token class for the token caching feature.  There also needs to be an index member for use by the encoder.  Also, the index token caching index does not have the "m_" prefix that member variables should have.  Both of these issues were handled by giving this member variable the "m_id" name.

The second issue was that array references were not being translated correctly.  When the get operand routine was called to get a reference, the process parentheses token routine called for the identifier with parentheses token was only asking for numeric expressions and if the expression only contained an identifier with or without parentheses and did not have the parentheses sub-code set, the reference flag of the token was set.  Consider this sample statement and its incorrect translation:

A(B,C)=D
B<ref> C<Ref> A(<ref>[B<ref>,C<ref> D Assign

The reference flags on the sub-scripts should not have been set, there should be integer conversion code inserted after the B and C variables, and there is no reason to attach the sub-scripts to the A( token since this token is known to be an array as functions can not be used in this way.  This also applies to INPUT statements with array elements.  Here is the correct translation:

B CvtInt C CvtInt A(<ref> D Assign

In the process parentheses token routine, if the token being processed has its reference flag set, then the expected data type of the expressions is set to integer for array subscripts.  After getting each expression, for identifier with parentheses tokens being processed (this routine also handles defined functions with parentheses), if its reference flag is set, the resulting item on top of the done stack is dropped (it has already been checked for an integer).  Otherwise, the done stack top items reference flag is set conditionally as before.  This is for the possibility that the token being processed is a function call, which the encoder will handle.

At the end of the process parentheses token routine when a close parentheses is obtained, a check was added if the token being processed has its reference flag set, then no tokens are attached since it is an array reference and its sub-scripts have already been checked for integers.

The expected results for several translator tests (#1, #4, #7, #8 and #12) were updated for the proper handling of array reference sub-scripts.

[commit 34a796984a] [commit 5d3b918e67e]