Expressions are now fully supported by the recreator. All the expressions test recreate the original source correctly. The next step is to add support for the first round of commands (LET, PRINT, INPUT and REM). The first tag of the 0.6 development series was added, v0.6.0.
[commit 0dc1f0ebf5]
Saturday, November 9, 2013
Recreator – Arrays and Functions
Supporting arrays and functions in the recreator is only necessary for the translator tests. So that arrays and functions are recreated properly for these tests, preliminary recreate functions were created for arrays, defined functions and user functions.
The array and function recreate functions get the name from the string of the token. An open parentheses is added to the name since it is not stored in the token. The name and the attached count in the RPN item is passed to the push with operands routine. The define function recreate function handles both define function with and without tokens, so an open parentheses is only added to the name if the attached count is not zero. Pointers to these functions were added to the preliminary table entries for the codes of these tokens.
All the expressions in expression test #3 (parenthetical tokens) now recreate the correct output.
[commit b6e8973352]
The array and function recreate functions get the name from the string of the token. An open parentheses is added to the name since it is not stored in the token. The name and the attached count in the RPN item is passed to the push with operands routine. The define function recreate function handles both define function with and without tokens, so an open parentheses is only added to the name if the attached count is not zero. Pointers to these functions were added to the preliminary table entries for the codes of these tokens.
All the expressions in expression test #3 (parenthetical tokens) now recreate the correct output.
[commit b6e8973352]
Translator – Arrays and Functions
A problem with recreating arrays and functions (user and define) is that the translator was not setting a code in these tokens. Preliminary table entries were added for the codes needed. These codes include an array, a define function with no parentheses, a define function with parentheses and a function. There will eventually be codes for each data type, but these four codes are sufficient for now.
In the get operand routine, for the define function with and with no parentheses token types, the appropriate token code is assigned. Since there is only a single code for each of these, the data type of the token is preserved because the table set token routine sets the data type of the token to the one in the table entry for the code.
The process parentheses token routine was modified to set the code of the token to the array code or the function code if the identifier starts with an 'F' character. This routine also processes define functions with parentheses tokens, so the code is not set for these tokens. Later in the routine when the RPN item is created, the attached count is no longer set to zero for array tokens.
The attached count of the RPN item can no longer be used to detect if there are attached tokens. The attached array pointer will instead be used for this detection by checking if the pointer is set to null. The RPN item text routine was updated to use the attached array pointer to detect attached tokens instead of the attached count. This routine was also changed to use member variables directly instead of using the access functions.
[commit e1919952fa]
In the get operand routine, for the define function with and with no parentheses token types, the appropriate token code is assigned. Since there is only a single code for each of these, the data type of the token is preserved because the table set token routine sets the data type of the token to the one in the table entry for the code.
The process parentheses token routine was modified to set the code of the token to the array code or the function code if the identifier starts with an 'F' character. This routine also processes define functions with parentheses tokens, so the code is not set for these tokens. Later in the routine when the RPN item is created, the attached count is no longer set to zero for array tokens.
The attached count of the RPN item can no longer be used to detect if there are attached tokens. The attached array pointer will instead be used for this detection by checking if the pointer is set to null. The RPN item text routine was updated to use the attached array pointer to detect attached tokens instead of the attached count. This routine was also changed to use member variables directly instead of using the access functions.
[commit e1919952fa]
Recreator – Internal Functions
The arguments (operands) of internal functions precede the code of the internal functions, so the strings of the operands will be on the string holding stack in reverse order when the internal function token is processed. The arguments of define and user functions work the same way as do the subscripts of arrays.
A generic push with operands routine was implemented taking the name string of the function or array and the count of operands, and contains a local string stack and a separator string initialized to a closing parentheses (the string on top of the stack will be the last operand). Looping to the count, the string of an operand is popped from the holding stack, the separator string is appended, this string is pushed to the local stack and the separator string is set to a comma and a space for the next operand (if there is one). The strings are popped from the local stack until empty and appended to the name. The name is pushed to the holding stack.
The push with operands routine also works with functions with no arguments (nothing is pushed to the local stack when the count is zero, so only the name is pushed to the holding stack). An internal function recreate function was added that gets the name and the operand count of the internal function code from the table and calls the push with operands routine. A pointer to this function was added to all of the internal function code table entries.
While trying expression test #3 (parenthetical tokens), there were blank invalid tokens. These occurred from the hidden conversion codes that were present in the first test expression. A blank recreate function was added that does nothing. A pointer to this function was added to the hidden integer and double conversion codes. Eventually, all codes will have a recreate function.
The expected results for expression test #3 and test #4 (internal functions) were set to the correct expected results (with appropriate spacing). All of the expressions in test #4 produce the correct output. Some of the expressions in test #3 do not produce the correct output because they contain arrays, define functions or user functions, which are not being handled yet.
[commit ac9ab4774d]
A generic push with operands routine was implemented taking the name string of the function or array and the count of operands, and contains a local string stack and a separator string initialized to a closing parentheses (the string on top of the stack will be the last operand). Looping to the count, the string of an operand is popped from the holding stack, the separator string is appended, this string is pushed to the local stack and the separator string is set to a comma and a space for the next operand (if there is one). The strings are popped from the local stack until empty and appended to the name. The name is pushed to the holding stack.
The push with operands routine also works with functions with no arguments (nothing is pushed to the local stack when the count is zero, so only the name is pushed to the holding stack). An internal function recreate function was added that gets the name and the operand count of the internal function code from the table and calls the push with operands routine. A pointer to this function was added to all of the internal function code table entries.
While trying expression test #3 (parenthetical tokens), there were blank invalid tokens. These occurred from the hidden conversion codes that were present in the first test expression. A blank recreate function was added that does nothing. A pointer to this function was added to the hidden integer and double conversion codes. Eventually, all codes will have a recreate function.
The expected results for expression test #3 and test #4 (internal functions) were set to the correct expected results (with appropriate spacing). All of the expressions in test #4 produce the correct output. Some of the expressions in test #3 do not produce the correct output because they contain arrays, define functions or user functions, which are not being handled yet.
[commit ac9ab4774d]
Recreator – Operand Recreate Function
The method of using an unset (null) recreate function pointer to indicate an operand is problematic during development. It is also possible that the recreate function for code is not yet implemented. Further, using a blank token string to indicate an unimplemented code is also a problem since this also could mean that the code does not produce any output (take for example the hidden conversion codes).
Therefore, an operand recreate function was implemented, which simply pushes the string of the token to the string holding stack. A pointer to this function was added to the constant and variable code table entries. The main recreate routine was modified to surround the string of a token code with no recreate function with question marks. A check was also needed for whether the code in the token is valid before retrieving its recreate function.
[commit 9734165b8b]
Therefore, an operand recreate function was implemented, which simply pushes the string of the token to the string holding stack. A pointer to this function was added to the constant and variable code table entries. The main recreate routine was modified to surround the string of a token code with no recreate function with question marks. A check was also needed for whether the code in the token is valid before retrieving its recreate function.
[commit 9734165b8b]
Recreator – Error Checks / Expression Mode
Some error checks were added to the recreator along with an expression test mode. The first check is to make sure the string holding stack is empty upon returning from the main recreate routine. Any items left on the stack are popped and appended to the output string prefixed by "NotEmpty" to indicate an error before returning.
However, the expression test mode left the resulting string on the holding stack. Therefore, a check was required for the expression test mode. An expression mode flag argument was added to the recreate routine. When set, the string on top of the holding stack is popped and appended to the output string. This is followed by the check for an empty stack.
An error check was also added to the pop routine to make sure the holding stack is not empty. If the stack is empty, the "<Empty>" string is returned to indicate an error.
The tester translate input routine was modified to pass the expression mode flag to the recreate routine and to just use the output string returned (it is no longer necessary to pop from the holding stack of the recreator for expression mode).
[commit 2980b89a24]
However, the expression test mode left the resulting string on the holding stack. Therefore, a check was required for the expression test mode. An expression mode flag argument was added to the recreate routine. When set, the string on top of the holding stack is popped and appended to the output string. This is followed by the check for an empty stack.
An error check was also added to the pop routine to make sure the holding stack is not empty. If the stack is empty, the "<Empty>" string is returned to indicate an error.
The tester translate input routine was modified to pass the expression mode flag to the recreate routine and to just use the output string returned (it is no longer necessary to pop from the holding stack of the recreator for expression mode).
[commit 2980b89a24]
Subscribe to:
Posts (Atom)