Table Class Hierarchy – LET and Assign Entries

The LET and assign table entries were implemented into the new table class hierarchy.  The sub-string assign entries were not done at this time because they are alternates of the sub-string function entries and these will need to be done at the same time.  The diagram below shows the table sub-class hierarchy for these table entries:

Table
  ├── Command
  │      └── Let [let]
  └── Internal
          └── Assign
                 ├── AssignDbl [assignDbl]
                 ├── AssignInt [assignInt]
                 ├── AssignListDbl [assignListDbl]
                 ├── AssignListInt [assignListInt]
                 ├── AssignListStr [assignListStr]
                 └── AssignStrBase
                            ├── AssignStr [assignStr]
                            └── AssignKeepStr [assignKeepStr]

An AssignStrBase intermediate class was needed because two of the string entry classes have a different recreate function.  The sub-string assign entries will also need this recreate function.  The assign string recreate function was left in place since it is still used by the sub-string assign entries.  For now the recreate function of the AssignStrBase calls this function.

The let recreate function that adds the LET keyword if the Option sub-code is set was made a static function of the Assign intermediate class and renamed to recreate let.  It was made public static to be accessible by the assign string recreate function.  This will be cleaned up when the sub-string assign entries are implemented into the new table class hierarchy.

New Table, Operand and Internal constructors were added with additional arguments needed to support these new classes.  The base virtual table translate function was modified to call the LET translate virtual function directly.  The LET translate function determines the validity of the token.  This was the last use of the translate function pointer member, so it was removed along with its access function.

[branch table commit c575b478ac]

Recreator – Sub-String Assignments

Translated sub-string assignments (previously described on July 14) consist of the string reference, followed by the expressions of the arguments of the sub-string function with the assign sub-string code.  This is the same form as when these functions appear in expressions, so sub-strings can be recreated the same way.  This is not the case with multiple sub-string assignments where the sub-string assign codes are all at the end of the translated statement with special assign keep codes except for the final code.

At run time, after popping the value to assign and assignment the value, the assign keep codes push the value back to the stack for the next assignment.  The final non-keep assignment just pops the value off of the stack.  The recreation of multiple sub-string assignments works similarly.  The assign keep code will pop the recreated string of the value expression to assign.  A sub-string assign keep code recreates the sub-string as a function and push the result back to the stack.  For all assign keep codes, the reference string is popped, the current separator is added followed by the value string.  This string is pushed back to the stack.

For the final non-keep assign code, the LET keyword is appended to the output string if the assign code has the option sub-code.  The value string is popped from the stack, which will also contain the other references in a multiple assignment statement.  The string of the final reference is popped and appended to the output string.  Finally the value string is appended to the output string.

Implementation

A separator character member variable was added to the recreator class with access functions for getting the separator character, checking if it is set (not the null character), clearing it (setting it to the null character), and setting it.

The new assign string recreate function contains a local string variable and starts by checking if the separator character is not set.  An unset separator indicates the first and maybe only reference. For the first reference, the string is set to the assignment operator (an equal) surrounded by spaces and the separator is set to a comma.  If the separator is set, then the string is set to it (which is a comma) plus a space.  The string on top of the holding stack is popped and appended to the string.

If the assignment token has the sub-string flag set, then the code of the original sub-string function is obtained.  This was accomplished by making the original sub-string function code the second associated code in all the table entries of the assign keep and non-keep sub-string codes.  The name and the operand count is obtained for the original sub-string function code and the push with operands routine is called to process the arguments of the sub-string function.  The resulting string is left on top of the holding stack.

If the assignment code is a keep code (determined if the second associated code index is zero, which is only the case for the keep codes), the value string with the separator and other references prefixed is appended to the string on top of the holding stack.  Otherwise the assignment code is the end of the statement.  The let recreate function is called to append the LET keyword if the option sub-code is set.  The last reference is popped from the holding stack, appended to the output string, and the string with the rest of the references, separators and value is appended to the output string.  The separator is cleared for the next command.

All of the translator assignment tests (#1 through #5) are all now recreated corrected except for a single PRINT statement in test #5 (since the PRINT recreation has not been implemented yet).

[commit b1c7dc5bce]

Recreator – Assignments

Translated assignment statements consists of one (single assignment) or more (list assignment) variables or arrays (references) followed by the expression of the value to assign with an assignment code at the end, which may have the option sub-code indicating the optional LET keyword was entered at the beginning of the statement.  When these statement are being recreated upon reaching the assignment code at the end, the strings of the references will be on the string holding stack (the first or only variable at the bottom) along with the string of the value expression (at the top of the stack).

The new assign recreate function handles the assign double, assign integer, assign double list, assign integer list and assign string list codes.  The assign string code could be at the end of a mixed sub-string assignment statement and these will be handled differently (see July 14).  The strings on the holding stack need to be appended to the output string in reverse order with the necessary separators between them (a comma between the references when there are more than one, and an equal between the last or only reference and the value expression).

To reverse the strings, the assign recreate function contains a local string stack and starts by popping the value expression string from the holding stack and pushing it to this local stack.  A local separator string is set to the assignment operator (an equal) with surrounding spaces.  The routine loops until the holding stack is empty.  For each reference, the separator string is appended and the popped string is pushed to the local stack.  The separator string is set to the comma with a space for the next reference.

A new let recreator support function was also added that checks if the assignment token has the option sub-code set and appends the LET keyword with a space to the output string if it does.  This function will also be used for multiple mixed sub-string assignment statements.  The assign recreate function calls this function and then loops until the local stack is empty appending each string popped to the output string.

A new recreator stack is empty access function was needed to determine if the holding stack is empty.  This was implemented in the recreator class header file since it was only one line of code.  Since the recreator top, top append, and append access functions are also only one line of code, their definitions were also moved to the header file.  Finally, the pointer to the assign recreate function was added to the table entries of the above mentioned assignment codes.

The results for translator tests #1 to #5 (various assignment tests) were changed to the expected results (inputs with spaces added and upper case keywords).  The statements in tests #1 (assignment tests) and #3 (data type assignments) are recreated correctly.  Many of the statements in the other tests are also recreated correctly, but the ones that are not contain string and sub-string list assignments yet to be implemented.

[commit 97c666a9d3]

New Translator – LET Statements (Tagged)

The implementation of LET statements including multiple assignments and sub-string assignments in the new translator is now complete and version v0.4.2 has been tagged.  All tests pass with the old translator routines (after some corrections were made).  All expressions tests and all assignment statements in the translator tests pass in the new translator routines.  Some minor cleanup was preformed with the latest commit along with updating the files for v0.4.2 (see the commit log for details).  Implementation of PRINT statements can now commence in the new translator.

[commit a5d3434fbe]

Memory Testing / Minor Memory Leak

Since all of the tests are now working with the new translator (excluding the commands not yet implemented), it seemed appropriate to change the memtestn script to run all of the tests.  After changing this script, two memory leaks were discovered in translator tests #7 (Errors) and #9 (Semicolon Errors).  The memory leak determined to be occurring with sub-string assignment statements that contained an error.

The memory leak occurred because an RPN item was allocated for the sub-string assignment token, which is not appended immediately to the RPN output list.  The RPN item is left on the done stack, which the LET translate routine pops and pushes it's token to the LET stack and then deletes the RPN item.  However, if the next token that should be a comma or equal token is not or a parser error occurred, then this does not occur.  The error clean up code assumes that all RPN items on the done stack have been added to the RPN output list, so only the items in the output list are deleted.

This problem was corrected by slightly rearranging the code in the LET translate routine where if there is an error with the comma or equal token, and the top of the done stack contains a sub-string assignment token (that has not been added to the RPN output list), then the done item on top of the done stack is popped and deleted, which deletes the RPN item and its token(s).  With this change, all of the tests with the new translator have no memory errors.

[commit 229af22a78]

New Translator – Minor Code Improvements

During the implementation of the LET command translation using the new translator routines (which is now complete), some areas of improvement were seen in the code that could be made.  This changes were kept separate from the latest sub-string assignment implementation commit.

The first was really a correction in the LET translate routine that would be needed once the multiple statements per line ability, separated by colons, was implemented.  The issue was when an error is detected at the very beginning of a LET statement.  To detect if the error was at the beginning, the column of the token was checked to see if it was zero.  This was used to determine which error to return.  However, this only works for a LET statement at the very beginning of a line.  To correct this, before entering the get references loop, the column of the first token (the command token if there was one, or the first token) is saved.  This saved column is then used to detect if the token with an error is at the beginning of the statement.

An improvement was made in how a flag is accessed from a table entry.  There were flags() functions (taking either a code or a token pointer) that returned the flags for the table entry (if the code has an entry, otherwise the null flag was returned).  The returned value was then anded to the desired flag to see if the result was non-zero (flag set) or zero (flag not set)  These were changed to the hasFlag() functions that take a second argument for the desired flag, and return non-zero (flag is set) or zero (flag is not set).

While testing the LET translation, a lot of time was spent chasing down token memory leaks.  The solution was to set the UnUsed sub-code in the token if it was not used.  Care was needed to not set this sub-code if the token was used (for instance, the token was in the RPN output list or on the hold stack).  There were quite a few of these set statements.  As an alternative solution, this sub-code is now set once when a token is obtained from the parser by the get token routine.  When the token is added to the output list, this sub-code is cleared.  A simple output append routine was implemented to do this for all locations appending to the output list.

[commit a67b759432] [commit 6193ef5550] [commit 887cfc06e1]

New Translator – Sub-String Assignments

Sub-string assignments were implemented as described on the two posts from Sunday.  In the LET translate routine, a flag was added, which is set if any of the assignments are a sub-string assignment.  Before converting the comma or equal token to an assign code, the token on top of the done stack is checked to see if its code's table entry has the sub-string flag.  If it does, the comma or equal token is deleted since it will not be used and the sub-string function is popped from the done stack, converted to the appropriate assign sub-string code and pushed to the local token stack.

After the equal token is received, and the tokens on the local token stack are processed, if the stack is not empty after popping the last token (indicating a multiple assignment) and there is a sub-string assignment (the new flag is set), then starting with the first token, each assign token is converted to an AssignKeep token (included a regular AssignStr code) and appended to the RPN output list.  This continues until the last token is popped, which is appended to the output as a regular assign code by the process final operand routine.

Translator tests #4 (sub-string assignments) and #5 (LET commands) now work with the new translator routines except for a single PRINT command at the end of test #5 (which for now reports a "not yet implemented" bug error).  Because of the change in sub-string assignment translations, the expected results were updated.  As a result, the old translator will fail with these tests.  The old results files for these tests were temporarily saved for reference.  The temporary memory test script was updated to include these tests.  See the commit log for more details of the changes made to implement sub-string assignments.

[commit e56db8f188]

LET Command – Single/Multiple Assignments

The design of each command will be reconsidered with the new translator design.  Several designs for the LET command were considered, but in the end, the current design seems to the most efficient at run time with the minor exception of multiple sub-string assignments.  Excluding sub-string assignments, LET statements are translated as follows:

A = 5.0              A<ref> 5.0 Assign
A,B,C = 5.0          A<ref> B<ref> C<ref> 5.0 AssignList

For multiple assignments, all variables being assigned must be the same data type.  The data type of the value being assigned must match the variables being assigned, however, for numeric types, an appropriate hidden conversion code will be added as needed.  If the optional LET keyword was specified, the hidden LET sub-code is set in the final assignment token.

Click Continue... For details of the implementation of the LET translation.  See the commit log for other minor changes made.  Translator tests #1 through #3 (various assignment tests) now pass with the new translator routines.

[commit f965e0f649]

New Translator – LET Translation (Begin)

Before beginning the implementation of the LET command translation routine, so thought was given on how the project should be organized.  The old token centric translator design had the translation of commands embedded throughout the translator, specifically in the token handling functions.  With the new translator design being command centric, the various routines (translate, recreate and execute) for each command can be organized into their own files.

A decision was also made not to clutter up the main project directory with all the various command source files, so these files will be put into a sub-directory.  The name "basic" was chosen for this sub-directory since all the sources files will be related to the BASIC language.  This sub-directory will also contain the execute routines for all of the operators and internal functions of the BASIC language.  The various command function prototypes (and any other command related definitions needed) will be put into the commands.h header file in this sub-directory.

The let.cpp source file was created in the basic sub-directory to hold LET command routines.  An initial translate routine was implemented for the LET command.  For now, this routine just checks if the LET keyword was specified and returns two different BUG Debug statuses to distinguish between the two forms.  Finally, a function prototype for this routine was added to the commands.h header file.

Some changes were also needed to the CMake build configuration file starting with added the let.cpp source file to the list of source files.  So that the various files in the basic sub-directory can access the various header files, the main project source directory was added to the list of include directories.  It turns out that it is not necessary to make the executable dependent on the list of project header files as CMake automatically figures out all the dependent header files for each source file, so this list was removed.

The initial LET function was tested with translator test #5 (LET command tests) to make sure the temporary BUG Debug errors were reported correctly.

[commit 3330b28c9e]

Translator – Assignment Table Entries

 The assignment operator table entries previously only had one operand, which was used for both the variable(s) being assigned and for the expression being assigned, since there both the same data type. This did not work for temporary strings because the variable operand needed to a string and operand being assigned needed to be a temporary string. So, the assignment operators were given two operands, the first for the variable(s) being assigned and the second for the operand of the assignment.

This change caused a problem with the main assign and assign list operators, which contained the list of all the related associated assignment operator codes (one for each data type). When find code is processing the assignment operand (the second operand) for double assignment, it does not need search for associated codes. To stop this, the index to the second set of associated codes, needed to be set to the number of associated codes, but this caused the Table initialization check to fail. So this initialization was modified to accept this condition.

The Table initialization was also modified to not count the first string operand for assignment operators (the entry's reference flag is set) because the string variable(s) being assigned are not attached to the assign string operator.

One strange problem remained with assignments, which involved the INSTR table entries...

Translator – Debugging - Sub-String Assignments (III)

The next issue is the output of the correct error messages. Consider these expressions and the current error messages (underlined tokens indicate where the error is pointing):

RIGHT$(LEFT$(A$,1),2) = B$  item cannot be assigned
RIGHT$(A,2) = B$  expected string expression
RIGHT$(A%,2) = B$  expected string expression
RIGHT$(A$+B$,2) = C$  item cannot be assigned

The desired error message is “expected string variable” for the first three statement and “expected comma” for the fourth statement. The section that receives an internal function token and checks for a sub-string function in an assignment was previously corrected to accept sub-string functions and set their reference flag (so that the reference flag of the first operand is checked).

An additional check was needed to catch a second sub-string function. Checking if there is a token on top of the hold stack with its reference flag set is sufficient. This condition only occurs when there is a sub-string assignment. However, this check is only be performed when on the first operand.

When the find code routine found an operand with an incorrect data type, it was returning “expected <datatype> expression” errors. This was modified that if the reference flag of the token is set, then an expected variable error is returned. However, for strings, the “expected string item for assignment” error was returned to indicate sub-string assignments are allowed. But if already inside the sub-string function, this needed to be the “expected string variable” error.

For the last statement, a check was added to the operator section in the add token routine, the same if the token on top of the hold stack has its reference flag set on the first operand check, if the token is not a comma, then an “expected comma” error is returned.

Translator – Debugging - Sub-String Assignments (II)

The next issue was more difficult to resolve and was a strange problem because statements that appeared to cause the program to crash, worked by themselves. Using the process of elimination, the statement below was finally found to be causing the crash:

RIGHT$(A$+B$,2) = C$

When the comma token is received, the +$ token is added to the output list and is pushed on the done stack. The RIGHT$ token is on top the hold stack with its reference flag set because it is a sub-string assignment statement. Being a sub-string assignment, the first operand must have the reference flag set, which for the +$ token was not, hence an error.

This error is detected in the find code routine where it is checking for a reference of the first operand when the token passed in has its reference flag set. If the reference flag of the operand is not set, an error is returned. The passed in token is first deleted and the error token is set to the token of the first operand.

The problem in this case was that the passed in token, the sub-string function, that gets deleted, is still on top of the hold stack. When an error occurs on a line, one of the things that the cleanup routine does is delete all the tokens on the hold stack. The sub-string function token gets deleted twice, which causes a crash later on – allocated memory must not be freed more than one.

The solution upon these reference errors is to delete the token in the find code routine only if the token is not an internal (sub-string) function. The find code is only called from two places where the token will have its reference flag set: internal sub-string functions in sub-string assignment statements (which are on the hold stack and should not be deleted) and assignment tokens (which should be deleted).

The next problem is that currently, the error reported against the + in the statement above is “item can not be assigned” but should be “expected comma” instead. This problem affects several other statement all related to the fact that a string variable is expected and the error message should reflect this...

Translator – Debugging - Sub-String Assignments

There were no issues with the seventh (Data Type Assignment) Translator tests other than the expected text file needed to be updated for all the new operator codes. The eighth (Sub-String Assignment) tests were all failing with a bug message indicating the done stack was empty when it was expecting operands and one of the tests were crashing the program.

The done stack empty errors were caused because the string within the sub-string that was being assigned was already popped off of the done stack by the assignment (variables being assigned do not need to be attached to the assignment token). But when the sub-string function was processed, it expected to find the first operand, which is a string, on the done stack. A check needed to be added to the process final operand function to not count string operands for functions that have the reference flag set (which is only set for sub-string functions on the left side of an assignment statement).

The next issue for the sub-string assignment tests, was that the Translator was incorrectly adding an AssignStr code instead of the correct AssignSubStr code. This problem was caused because the find code routine was using the wrong condition to check for an exact data type match when looking at the operands of the main and associated codes to find the proper code for the operand. The code obtained from the conversion code table was used – the exact match check was if the conversion code was the Null code. This allowed the Sub-String data type to match the AssignStr that was expecting a String data type instead of continuing to check the AssignSubStr, which expects the Sub-string data type.

To correct this problem, the exact match check was changed to compare the actual data type to the code's operand's data type. This is the check the old find code routine used. Also, the section that inserts the conversion code was modified to only add the conversion code from the conversion table if it is not the Null code. In other words, the Null code indicates that the data type can be converted, but no actual conversion code is needed.

Translator – Debugging - String Assignments

Debugging has continued; problems with the third (Array/Function Parentheses Expressions), fourth (Internal Functions), and fifth (Assignments) Translator tests were corrected. There were some done stack not empty errors from the sixth (Data Type) test with the string assignment tests.

The problem turned out to be that the string variable being assigned was not popped from the done stack because it was a string. (Strings that are not determined to be temporary strings are left on the done stack so that they can be popped later and attached to the item that uses them – like an operator, function or an array.) However, these string variables were not being popped and attached to the assignment command/operator.

This was by design, the thinking was that items being assigned are definitely variables or arrays and not function calls. (Functions can also be assigned, but for these assignments, the function names will not have arguments and are only permitted within the function body.)  In any case, variables being assigned do not need to be attached to the assignment command, including string variables.

To solve this issue, in the find code routine where non-string values are popped from the done stack – an additional check was added that if the token needed to be a reference, then it will be popped from the done stack regardless of its data type. The reference flag is checked when the calling token has its reference flag set (which is set only for assignment operators).

Translator – New Find Code (Multiple Assignment)

The last issue for the new find code implementation is the most tricky and requires the elimination of the C-like multiple equal assignment (for example, A=B=C=0). The problem occurs when the items being assigned are array elements, which can be shown in this example statement:

A(I) = B(I) = 5

In this example, the A can be assumed to be an array because a function (with arguments) cannot be assigned. The subscripts must be numeric expressions. However, after the first equal, this statement can be interpreted two ways depending on whether B is an array or a function call:

I CvtInt A(<ref> I CvtInt B(<ref> 5 AssignList
I CvtInt A(<ref> I B([I] 5 = CvtDbl Assign

The first interpretation is a multiple list assignment of two array elements and the second interpretation is a single assignment to the result of an equality comparison between the result of a function call and a constant. The two resulting translations are radically difference. The Encoder can't be expected to change one translation into the other once it determines whether B is an array or a function.

A similar problem can occur with multiple sub-string assignments. These problems occur because an equal can be one of two different operators, assignment and equality. C resolves this issue by having difference operators for assignment (=) and equality (==). If this C-like multiple equal assignment is eliminated, then the rules are greatly simplified.

Therefore, in an expression, any equals will be equality operators. In a statement, only the first equal is the assignment operator. After this equal, an expression follows so any equals will be equality operators. Multiple assignments can still be performed by using the comma to separate the items being assigned.

Translator – New Find Code (Reference Flag)

here has been a struggle with a number of issues that have turned up with the new find code implementation, including reference checking, internal functions with multiple forms (different number of arguments), and multiple assignments (specifically multiple equal assignments, which may need to be removed from the language).

The current find code routine checks if the first argument is a reference if the table entry for the token has the Reference Flag. This was used for the assignment operators when called from the set assign command routine, which is called from the comma and equal token handlers (assignment operators have the Reference Flag). If this would be the only location that needs to check for a reference flag, the checking could be moved from the find code routine to the set assign command routine.

However, there is another situation that needs to check for a reference flag - the first argument to a sub-string assignment. This is currently handled by an additional check in the find code routine. With the new find code, this will occur at the first comma for the sub-string function (from the comma token handler). So the reference check should still be done in the new find code routine.

The reference flag can't be added to the sub-string table entries since not all instances of the sub-string functions are assignments. The token being checked (internal function, operator, assignment code or print code) is passed to the find code routine. A simply method would be to set the reference flag in this token argument if it's operand needs to be a reference. The find code would check for a reference if the token's reference flag is set.

Therefore, in the set assign command, the reference flag of the assignment token will be set. Also, when internal functions are first pushed to the hold stack, a check is made to see if the mode is currently one of the assignment modes and only a sub-string function is permitted. At this time, the reference flag of the sub-string function can be set.

There will be no problem leaving the reference flag set in the internal function or operator token since nothing downstream will be checking for a reference flag on these type of tokens.

Translator – Expression Types and Parentheses

Normally during the translation process, parentheses tokens are removed. However, for reporting errors with the data type of an expression, if an expression starts with an parentheses and is the wrong data type, then the error needs to point to the opening parentheses, not to a token inside the parentheses.

When a closing parentheses is received and processed, after emptying the hold stack of all operators, the opening parentheses will be popped from the hold stack. Previously, both the open and closing parentheses tokens were deleted. If the parentheses were unnecessary, the parentheses sub-code was set in the last operator appended to the output. The last operator will also be on top of the done stack with it's first operand. Consider this invalid statement:

Z$ = A$ + B$ + (C + D * E)

When the closing parentheses is processed, there will be a * token on top of the stack and its first operator will be set to the C token. The “expected string expression” should point to the open parentheses. Therefore, the closing parentheses needs to change the first operand token to the open parentheses token. This also means that the open parentheses token can't be deleted.

Since the open parentheses token can't be deleted, it must be marked as a temporary token (using a Temporary sub-code). When an operator token is popped from the done stack (either by another operator or at the end of the expression when a command is processed), if it contains a temporary first operand token, the token must be deleted.

When an error occurs, or an expression is prematurely ended (also an error), the data type of the first operand will be checked to determine which error will be reported (based on what type of expression is expected). Open parentheses tokens do not have a data type. Therefore, when an operator's first operand is set to an open parentheses token, it must inherit the data type of the first operand that it is replacing.

Parentheses may also be nested, so another open parentheses token may replace a first operand that is already set to an open parentheses. No extra checking is necessary since the previous open parentheses will have a data type – the new open parentheses token will inherit the same data type.

Translator – Expression Type (Procedure)

To show how keeping the first operand will aid it reporting errors at the correct token, consider these examples again:

Z% = A$ + B$ + C$
Z% = A$ + B$ > C$

The first statement is processed in this sequence:

1. The Z% token is appended to the output and pushed to the done stack.
2. Since the mode is Command, the = token is interpreted as an assignment, so an AssignInt command is pushed to the command stack, the Z% token is popped from the done stack and the mode is set to EqualAssigment.
3. The A$ token is appended to the output and pushed to the done stack.
4. The first + is pushed to the hold stack, and being an operator, the mode is changed to Expression (further equal tokens will be interpreted as an equality operator).
5. The B$ is appended to the output next and pushed to the done stack.
6. When the second + is received, it empties the first + from the hold stack (being the same or higher precedence).
7. The first + pops the A$ and B$ from the done stack, and a +$ is appended to the output. The first operand, A$, does not contain a first operand (it's not an operator), so the +$ is pushed to the done stack with A$ as it's first operand.
8. The second + is pushed to the hold stack.
9. The C$ is appended to the output.
10. The end of statement empties the second + from the hold stack.
11. The second + pops the +$(A$) and the C$ from the done stack, and a +$ is appended to the output. The first operand, the +$(A$) has a first operand, A$, so the second +$ is pushed to the done stack with the A$ as it's first operand.
12. The assign command handler will be called since there is an assignment on the command stack, which will pop the value being assigned, the second +$, and will see that it is the wrong data type (an integer is expected), so an “expected numeric expression” is reported. But instead of pointing to the second +$, it's first operand, the A$ token, is returned.

For the second example, the >$ that would be on top of the done stack when the assign command handler is called. It's data type is an integer, which is correct, so no error occurs. However, sub-expressions in parentheses need additional handling...

Translator – Expression Type (New Design)

The new design for the rest of the data type error detection consists of remembering the token of the first operand of each sub-expression within an expression, so that an error can be reported against this token when a data type error is detected. Consider this invalid assignment statement:

Z% = A$ + B$ + C$

Currently this reports an “expected numeric value” at the second + operator. It should report “expected numeric expression” at the A$ token. However, the detection can't occur at the A$ token before the entire expression is processed. Consider this valid assignment statement:

Z% = A$ + B$ > C$

The expression becomes an integer at the > operator, therefore an “expected numeric expression” can't be reported at the A$ token.

Each binary operator token appended to the output is also pushed onto the done stack, replacing it's operands. The token of the first operand of the operator will be attached to the operator when it is pushed to the done stack. If the first operand is another operator, then this operator's first operand is attached (in other words, the operator will inherit the first operand's first operand if there is one).

Translator – Assignments (Development)

The last several days was spent implementing the new design for handling assignments. The assignment operators are no longer handled by the operator routines, and now by the comma and equal token handlers and the assign command handler (which previously didn't do anything). Two support functions were also implemented, one to put the appropriate assignment command on the command stack based on the first (perhaps only) item being assigned, and the other to check each assignment item for the correct data type (allowing for mixed string and sub-strings).

At the end of the statement, the value (expression) being assigned is checked for the correct type in the assign command handler, adding a hidden conversion as needed for the numeric data types. Because the assignment operators are no longer handled as binary operators, the table entries for the assignment operators were modified where each only has one operand (for the value being assigned).

Also, tokens with parentheses being assigned can be assumed to be arrays since a function with arguments cannot be assigned (only the function name alone, without parentheses, can be assigned). Therefore, unlike tokens with parentheses in expressions that can be either an array or a function call, the values in parentheses of an array being assigned can be assumed to be subscripts, which must be integers (or doubles with conversion). If it turns out that the name is not an array when encoded, the Encoder will report the error.

Several other data type reporting error issue were also corrected but without adding any special expression type handling as was planned in the failed design concept. A new concept was developed, which probably won't require the Translator to keep track of the expression type while translating. More details to follow...