REM Operator (For BASIC Comments)

Implementing the REM operator (single quote) turned out to be simpler than anticipated.  The REM operator type of REM always occurs at the end of a line, therefore, it can be treated as the end of the line except that the REM operator token (with comment string) will be added to the end of the RPN output list.

A new REM operator token handler was implemented, which first checks if the command stack is not empty or if the current token mode is not command (occurs for assignment statements without the LET keyword).  Otherwise the REM operator being processed occurred at the beginning of the line and no current command needs to be processed.

To process the current command, the end of line token handler is called since it contains all the needed functionality for processing the command when the end of a line is reached.  A new end of line token is created and passed to the end of line token handler.  A new token was needed in case the command changes the end of line token into another token to add to the RPN output list (as the INPUT command does).  If the command does not use this token, the end of line token handler will delete it.  If an error is returned, then the end of line token is deleted and the error is returned.  Otherwise, the REM operator token is appended to the end of the RPN output list.

Since the REM operator token acts as the end of the line, the table entry for the REM operator code entry was changed to include the end expression and end statement flags.  The pointer to the new REM operator token handler was also added.  Several new REM tests were added to translator test #15 for the REM operator on various types of commands.  Some error tests were also added.

[commit da69b3ba07]

REM Command (For BASIC Comments)

Before continuing with highlighting errors in the edit box, I noticed (when implementing the routine that converts the token contents to text for display in the program view) that the Remark token type was not actually being used.  The parser is handling two types of remarks, the REM command and the single quote comment method, which is treated as an operator since it can appear at the end of any line and does not need to be preceded by the colon statement separator.

However, these two tokens (REM command and REM operator) were not being handled in the translator.  The REM command token was returning a "Not Yet Implemented" error message.  For the REM command token to be handled and not return this error message, the REM code in the table needed to be assigned a token mode.  It turned out that the specific token mode assigned was not important as long as it wasn't the default NULL token type, which triggers the error message, because the REM command will always be the last command on a line.

To process the REM command token, a new REM command handler was implemented that simply adds the REM command token (which contains the actual comment text in the string of the token) to the RPN output list.  A new translator test (#15) was added for various REM command tests.  The REM operator token is a little trickier to handle, which will be implemented next.

[commit 258f3a9df0]