Saturday, November 16, 2013

Recreator – Unary Operator Problems

Many of the other translator tests were being recreated correctly including tests #7 (errors), #8 (more errors), #9 (semicolon errors), #10 (expression errors), #11 (temporary errors), and #14 (parser errors) once the expected results were updated as these did not have the not yet implemented INPUT and REM statements or colons.  However, tests #13 (negative constants) and #17 (constants) were not recreated correctly due to problems involving unary operators.

A problem occurred when a negate unary operator preceded a numeric constant.  When created, there was no space between the negate operator and the number.  If this statement is translated again, the negate operator and the number become a negative constant, which is not the same, through technically equivalent.  This will cause the line change detection in the program model to incorrectly detect a change.  The unary operator recreate function was modified to also add a space after the unary operator if the operand begins with a digit or decimal point.

A problem occurred with the negate integer operator.  The precedence of the negate integer was incorrectly set to 40 causing parentheses to be added incorrectly during recreation.  The precedence should have been 48, the same as the negate double operator, so the table entry for was corrected.

A problem occurred when a unary operator followed a power (exponential) operator, a higher precedence operator.  The operand was incorrectly surrounded with parentheses.  This binary operator recreate function was modified to also check if the second operand is a unary operator then the operand is not surrounded by parentheses.

These corrections allowed tests #13 and #17 to be recreated correctly.  The regression test script was also modified to not ignore white space when comparing to the expected results.  Without this change, the first problem above was not detected.  The memory test was already not ignoring white space.  I'm not sure what the reason was for making the regression test ignore white space.

[commit cf67d09f36]

Recreator – PRINT Statements

There are several codes that make up a PRINT statement including print item (double, integer or string), comma, print function (TAB and SPC), semicolon (only at the end of the statement) and the print command.  A recreate function was implemented for each of these codes.  Because the PRINT statement is composed of several codes, the separator member variable of the recreator instance is used between the processing of these codes to keep track of separators between the print items.

As the codes of the PRINT statement are processed, the resulting PRINT statement is built up by adding to the string on top of the holding stack.  Generally, the string for the current item is popped from the stack, a separator is added to the string on top of the stack, which contains previous items and the string of the current item is added to the string of the previous items that is on top of the stack.  At the end of the statement, the PRINT keyword is added to the output string along with the built up string of the print items and separators that is popped from the stack.

Implementation

The print item recreate function contains a local string variable.  If the separator is set from a previous print code, the string is set to it.  If this separator is not a space, then a space is added after the separator.  The separator is a space if the last print code was a comma (see below).  The string on top of the stack is popped and added to the string.  If the stack is now empty, then the string is pushed to the stack, otherwise the string is added to the string on top of the stack.  The separator is set to a semicolon for the next item if there is one.

Spaces are normally added after a comma like a semicolon, but spaces are not added between multiple commas.  The print comma recreate function contains a local string variable.  If the holding stack is not empty, the string on top of the stack is popped into the local string.  A comma is added to the string (which is empty if the stack was empty, like when there is a comma directly after the PRINT keyword).  The local string is pushed to the stack.  The separator is set to a space.  A space is only added after the last consecutive comma.

The print function recreate function first calls the internal function recreate function (since print functions are translated the same as other internal functions), which will process the print function and its operand and leave the result on top of the stack.  The print item recreate function is called to process the print function like any other item.

The semicolon code is only found at the end of a PRINT statement and replaces the print command code.  The print semicolon recreate function pops the string from the holding stack, adds a semicolon, and pushes it back to the stack.  The print recreate function is called to complete the PRINT statement.

The print recreate function adds the PRINT keyword to the output string.  If the holding stack is not empty, a space is added to the output string, and the string is popped from the stack and added to the output string.

A new separator is set access function was added to the recreator to a specific character, which is used by the print item recreate function.  Pointers to the new print recreate functions were added to the table entries of the various codes.  The expected recreated outputs for translator test #6 (PRINT statements) were updated and now recreated correctly.

[commit 1d3a05f610]

Recreator – Interactive Testing

Up to now, the only way to test the recreator was by using the expression and translator test files with the batch test mode.  An interactive recreator mode was not implemented since the recreator only supported expressions and creating separate modes for both expressions and commands was unnecessary.  With support for commands (just assignments at the moment), an interactive mode for the recreator could be added.

Before implementing the interactive recreator mode, the translator, program unit and recreator instances (which were local variable in the tester class run routine), were changed to the member variables of the tester class.  As local variables, it was necessary to pass references to them between the various tester routines, which defeated the purpose of having a class.  The output stream is now given to the tester constructor, which is stored in a member variable so that it can be shared by all the class routines, instead of being an argument to the run routine and passed to the other routines.

The tester class was modified to support the new interactive recreator test mode, which is activated with the new "-tr" command line option.

The translate input routine was modified to accept a header string for the list of translated token output.  If this header string is not used, then "Output:" is used as before.  This routine was also modified to return the pointer to the RPN list if the header string is used.  Otherwise, the RPN list is deleted as before and a null pointer is returned, which is also returned when the input line has an error.

The new recreate input routine was added, which starts be calling the translate input routine with the header set to the "Token:" string.  If an RPN list is returned (no error detected), the RPN list is recreated and deleted.  The recreated output is prefixed with the "Output:" string as its header.

[commit 2dc4b17e97] [commit 0ccfb105e7]