Table – Entry Virtual Functions

The implemented commands (LET, REM, PRINT, INPUT, and INPUT PROMPT) each have different initialization values (a code for LET and REM, an option name for the INPUTs, a translate function pointer for all except REM, a recreate function pointer for all but LET, and an encode, operand text and remove functions pointers for only REM).  Creating a single command sub-class constructor to cover these would be messy, and so would creating a different constructor each.

The intention was to replace the functions pointers with virtual pointers, so it was not worth the effort to try and handle these.  Virtual functions were added to the base table class, which temporarily call the function pointer if set (translate and recreate), or just call the function pointer (encode, operand text, and remove).  For the command sub-class, which is only used for commands not yet implemented, the translate virtual function overrides the base class function and throws a "Not Yet Implemented" error.

The is code with operand access function checked if the operand text function pointer was set.  Since the function pointers will be removed, another method for checking if the code has an operand was needed.  A new Operand table flag was added to the REM, REM operator, three constant, and six variable table entries.  This access function was modified to check this flag.

The users of the function pointers (translator and program model) were modified to call the virtual functions.  For the encode, operand text, and remove functions, the program model needs to call the is code with operand function before calling the function.  This resulting code a bit messy and needs to be improved.

[branch table commit c24cf75c90]

Table – Command Sub-Class (Initial)

An initial command table entry derived class was implemented consisting of a single constructor taking name and second name string arguments with the second name defaulting to a blank string.  This constructor can be used with both one-word and two-word commands.  It calls the base table constructor passing the two names along with values needed by most commands (Command table flag, precedence of 4 and a pointer to the null expression info instance).

This command constructor is only sufficient for the not yet implemented commands.  For now it also sets the default code (no code), blank option string, and null function pointers.  This class definition was put into a new header file.  In the corresponding source file, instances were created for all of the not yet implemented commands, for example:

static Command Dim("DIM");

The entries for these commands were removed from the static table entry array and the code index enumerators for these commands were removed from the code index enumeration.

The type of the string arguments to the table constructor (name, second name, and option) was changed from a standard string to a C-style string (constant character array pointer) because otherwise, passing a blank string to this constructor required a standard string std::string("") instance, instead of just a blank "" string.

The first time these changes were one, a crash occurred during initialization when trying to add the first of the new command instances because the static table members had not been initialized yet.  Because the new command source file appeared before the table source file in the source file list in the CMake build file, so command instances were initialized before the static table members, but these are needed to initialize the command instances.  To resolve this, the basic sub-directory source files (which will contain the entry instances) were moved to the end of the source file list so that the table source file with the static table members are initialized first.

[branch table commit 97ff351060]

Table – Automatic Two Flag

The table is no longer dependent on the static table entry array so the implementation of the table class hierarchy can begin.  The first sub-class will be for commands.  While investigating the requirements for this sub-class, it was noticed that the Two table flag could be set automatically.  The Two table flag indicates if a command has a two-word form (for example the INPUT and INPUT PROMPT commands).  It is also used for symbol operators that has a two-character form (for example <, <= and <> operators).

The erector class functions were modified to set the Two table flag automatically for these commands and operators.  After a two word command is added to name to entry map, if the first word of the two-word command has an entry, then the Two flag of the one-word entry is set.  Similarly, after a new primary entry is added to the name to entry map, if the entry is a two-character operator, the Two flag is set on the entry for the first character is there is one.  Some additional table entry access functions were added.

The Two table flag was removed from all of the entries.  The Two flag was set on the <> operator, which was not necessary (only the one-character form required this flag).  Not all of the command entries with the flag were covered by parser tests, so additional tests were added to parser test #2 (identifiers).  Not all forms of all operators were covered by the tests.  Translator test #11 (temporary strings) contained an extra set of <= tests, which should have been >= tests.  New translator test #18 was added to cover all operators with all possible data types.

[branch table commit 937c5e3a57]

Table – Entry Indexes

Up to now, the index for a code that is stored in the program is the index of the table entry in the static table entry array defined in the table source file.  The new table model will not have an array of table entries, but instead will be separate table instances spread among several source files.  The common connection will be through the constructor of the base table class.  This constructor will put pointers to the entries into a standard vector.  As each entry is put into this vector, its index within the vector will be put into an index member of the table entry.

A static index to entry vector was added to the table class along with the index instance member.  The set index and add entry access function was added to set the index of the an entry and add the pointer to the entry to this vector.  A call to this function was added to the erector function.  The index access function was changed to return the value of the index member.  The entry access function was changed to access the index to entry vector instead of the static table entries array.  Both functions were made inline.

The comments were removed from the instance members since they were only stating the obvious, plus the comment for the expression info pointer member stated that a null pointer indicates no expression info even though this no longer applies because all entries are now assigned an expression info instance even it it is the null instance.  This is an example of comments going stale and lying - a good reason for eliminating comments with code explains itself.

[branch table commit 9d74aacf42]

Table – Static Member Access

While refactoring the add to table function and sub-functions into the Erector class functions, one of the desired changes to improve readability was to provide access functions to the static table members.  All of these statements contained the table class scope (Table::) to get to the static member and a table entry pointer.  This meant that table entry access functions could be added that would then access the static table member.

Ideally these access functions would be inline for efficiency.  The reason these changes were not made was that in order to be inline, the table class needs to be defined before the table entry class, but the table entry class needs to be defined before the table class.  Using forward declarations may have solved this issue.  This would not be necessary if the two classes were merged and since they were going to be merged, no effort was used to solve this issue.

With the two classes merged, these access functions could be implemented.  The necessary access functions for the expected data type, alternate, name to entry, and code to entry maps were added and callers were updated accordingly.  The table class scope was eliminated from the erector class functions.  The erector add to code map and two add to expected data type functions were moved back to the table class since both were passed entry pointers.

Several of the existing access functions were renamed for improved readability.  One of the access functions, the alternate count function, was only called from one location in the translator and was used to determine if the unary operator in the token has an alternate binary operator.  This section of code looked like this (note the comments and magic numbers):

// check if code has a binary operator
if (m_token->alternateCount(1) > 0) {
    m_token->setFirstAlternate(1);  // change to binary operator
}

This function was renamed to the has binary operator function.  Several constant expressions (integers) were added for predefined operand indexes for use with the various alternate access functions so magic numbers are not used.  With the renamed access function and constant expressions, the resulting code is much more readable (note the absence of comments):

if (m_token->hasBinaryOperator()) {
    m_token->setToFirstAlternate(BinaryOperator);
}

[branch table commit 03140858b0]

Table – Single Class

The table class, with static members, and the table entry class, with entry (instance) members, were combined into a single class, named the Table class.  The former table entry constructor (which had no body) was moved to the table source file and a call to an Erector instance was added to add the needed info to the static table table members.

The iteration of the entries in the static table entry array that called an Erector instance for each entry was removed from the default constructor since the table entry constructor now does this.  The only statements remaining in this constructor are the initialization of the other alternates.  This constructor along with the single table instance will remain to initialize these other alternates until this can be reimplemented in the new class hierarchy.

With these constructors in place, the class hierarchy can be implemented one derived class at a time.  As each is implemented, the entries will be removed from the static array along with the enumerator from the entry.

The method previously used to disable the copy constructor and copy assignment was to make them private so they are not accessible, at least from outside the class.  C++11 has a better method of accomplish this using the new delete feature.  C++11 also has move constructors and assignments, so these also needed to be deleted.  For example, the copy constructor and assignments are disabled like this:

Table &operator=(const Table &) = delete;
Table(const Table &) = delete;

Some other minor changes were made.  All instances of variables named operand index (mostly arguments of access functions) were renamed operand. This was already done in the Erector class.  Several redundant comments were also removed.

[branch table commit 9153b17e5e]

Table – Add To Table Refactoring (Part 3)

The add to table function and all the functions it calls share several variables, which required them to be passed between the functions.  Enter the new Erector class to contain these functions and shared variables.  This class is a function operator class with a constructor that takes a table entry pointer, and the function operator function for adding the element to the table (static member variables).  This class was made a friend class of the table and table entry classes to make private member functions accessible.

The shared members of the Erector class include the table entry pointer being added, table entry pointer to the primary entry, table entry pointer to an alternate entry, and an operand index used for iterating and used by multiple functions.  Most of the member functions were declared in the header file after the class definition to not clutter up the class definition.  Each of these were declared inline since each is called once.  Two functions called multiple times were put into the source file.

Many of the functions were renamed again to better explain what they do.  There was addition refactoring moving statements up and done the functions during this renaming.  Not widely known (at least is wasn't to me) is that C++ contains alternate representations for the logical operators, for example and for &&, or for ||, and not for !.  To make the code a little more readable, the not operator was used since the ! operator can be head to see.

The main function was restructured to eliminate the need of the Done structure exception.  This unusual mechanism was used as sort of a goto to cause an exit from anywhere.  The elimination of this exception-exit mechanism was now possible since the sub-functions no longer returned table entry pointers (which are now member variables and set directly) opening up the use of boolean return values.

[branch table commit f199936e70]

Table – Add To Table Refactoring (Part 2)

Refactoring the remaining large function called from the add to table function was painful, but was finally achieved, though the result is far from ideal.  The problem is that there are several variables that need to be shared among the functions.  Some functions take a variable and then return the same variable possibly modified.  Another needs to modify two arguments, so couldn't return both so reference arguments were used.

Having arguments used as both input and output can be confusing when reading code (arguments should be used for inputs, return values should be used for outputs).  Sharing several variables between functions implies that the functions should be wrapped up into another class.  This will be the subject of the next refactoring change.

[branch table commit a9cd8e9a1a]

Table – Add To Table Refactoring

The add to table function is large and hard to read so it will be refactored.  Refactoring this function will require several passes of changes.  The first round was to break the function into seven smaller functions.  The add expected data type function was already separate since it is called from three places.  The result was a simpler main function, but is still rather involved.  The separated functions are also simpler, but less than readable (and one is still quite large).

I won't detail the results since this is not the final form (and if successful, no explanation should be necessary as the code should be easy to read).

[branch table commit 23515be87b]

For the next round, I wanted to clean up the main add to table function.  There were a number of return statements that were cluttering up the function.  The first change was to restructure two of the if statements to eliminate two of the return statements.  For the others, there was a pattern with three of the separated functions returning a null pointer indicating that no further action was necessary.  The main function then checked for this null pointer and returned.

To clean this up, an empty Done structure was added with no members to be used as an exception.  Instead of these three functions returning a null, they instead throw an instance of this empty structure.  The main function catches this exception and does nothing (just returns).  This is not really an exception condition, but it cleaned up the main function quite a bit.  This mechanism allows lower-level functions to cause a return of the add to table function.

[branch table commit 69bd606994]

For the third round, there was some more attempt to make the main add to table function and some of the separated function more readable.  This involved renaming functions (again) and adding access functions to make what is being tested in if statements more self explanatory.

One of the new access functions was named has operands, which was too similar to the has operand access functions.  These were used to determine if the code has an operand (variable, constant, array, etc.) by checking if the code has an operand text function.  These functions were renamed to is code with operand.  The last operands access functions were corrected to return an integer instead of a boolean (though this didn't appear to affect its functionality).

[branch table commit 8b9b3f9a4b]

Table – Single Base Class

The Table and Table Entry classes are going to be combined into a single class, which will be the base class in the new table model class hierarchy.  The only members left in the Table class are static members and functions except for the constructor and entry member pointer, which is just pointing to the static array of initialized table entries.

The entry pointer member was removed and the static array is accessed directly.  The entry pointer and size arguments were removed from the constructor.  The constructor loops through each entry in the array calling the add function to add the necessary info to the static table members.  The alternate info initializer list is then iterated to add other alternates not automatically set up by the table entries in the add function.

Previously when the add function threw an error, the constructor caught the error and added it to a list.  At the end of the constructor, if this list is not empty, then the errors are output and the application is aborted.  With the new table model, it will not be possible to catch all errors in a list since errors thrown by static constructors abort the application immediately.  The add function was modified to catch its own error, output it and abort.  This is only a diagnostic tool during development, so trying to report all errors would be nice but not a necessity.

Both the add and add expected data type functions operated on a single table entry and therefore were moved to the Table Entry class.  These functions will be called by the constructors of the new table class hierarchy.  The function was renamed to add to table, but the entire function is too large and needs to be cleaned up (refactored, in other words, broken up into smaller functions; and this is certainly not the only function with this problem).

[branch table commit 07064dbde4]

Table – Code Type Enumeration

The code enumeration where enumerators were used as table indexes was replaced with a code type C++11 enumeration class.  Code types are only needed for codes that are referenced, for example, special operators (open parentheses, close parentheses, equal, comma, semicolon, and colon), some commands (LET and REM), hidden codes (convert to double or integer), and codes with operands (constant, variable, array, etc.).  Several codes are assigned to the same code type enumerator (for example double, integer, string, non-reference and reference variables).

The code type enumeration was named the same as the old Code enumeration because this makes the code read better where they are used.  The old Code enumeration was moved to the table source file for the temporary alternate code initializer lists and was renamed to Code Index to conflict with the new Code enumeration class.  These enumerators will eventually be removed.  In the rest of the classes, the old enumerators were changed to the new enumerators.

The table entry type member was changed to the code (new code type enumeration).  Table entry initializers were updated with new enumerators.  For most of the entries, this was just the default code type enumerator.  The type access functions were removed from the table entry and token classes, and the code access functions were updated to use the new code member.

A new code to entry map static member was added to the table class.  In the add function, if the code is not the default code type and the code type of the entry is not already in the map, then the pointer to the entry is added to the map for the code type.  The argument of the entry static access functions were changed from an index to a code type enumerator.  An entry access function taking an index is still needed for converting program code back to tokens.

The switch statements on the token type in the translator and tester class routines were updated to switch on code type.  This included the test token type name function, where the names output were also changed to match the code type.  This affected the parser tests output, which were updated accordingly.

[branch table commit f19b061fe3]

Table – Internal Function Type

The next type replaced with a table flag was the Internal Function type.  There were no complications here either.  A new Function table flag was added (the "internal" nomenclature will no longer be used).  An is function table entry access function was added that just calls the has flag function with the Function flag.  A token is function table pass-through access function was also added.  The Function table flag was added to all table entries with the Internal Function type, which were then changed to the default type.

[branch table commit eaa6e6d7f5]

Table – Operator Type

The next type replaced with a table flag was the Operator type.  There were no complications.  A new Operator table flag was added.  An is operator table entry access function was added that just calls the has flag function with the Operator flag.  A token is operator table pass-through access function was also added.  The Operator table flag was added to all table entries with the Operator type, which were then changed to the default type.

[branch table commit e064047030]

Table – Command Type

The only uses of code enumerators remaining are those that will become code type enumerators and for the temporary initializers of additional alternate codes.  The replacement of the type enumerators can now begin.  As mentioned previously (see January 3), table flags will be used to identify commands, operators and functions since some of these codes will need their own code type enumerator (for example, LET, Equal, Comma and Semicolon).

The command type was first to be replaced with use of a table flag.  A Command table flag already exists and was being used for the Comma, Semicolon, and the assignment codes.  The purpose this flag is to identify that these codes can have the Colon sub-code like command codes and not the Parentheses sub-code that operators can have.  Both of these sub-codes share the same sub-code bit.

For the purposes of checking for a command type code, these other codes needed to be excluded.  An is command table entry access function was added to do this check, which first checks if the Command table flag is set.  The assignment codes all have the Reference table flag, so these are excluded by checking if this flag is not set.  For Comma and Semicolon (used as commands for PRINT statements) are treated as special operators allowing them in expressions where an operator is expected.  These are excluded by checking if the code is not an operator, which for now checks for an Operator type but will be changed to checking the new Operator flag.

A token is command table pass-through access function was also added.  When checking for the Colon and Parentheses sub-codes (in the program word stream insert operator and recreator operator functions), just the Command table flag is checked, otherwise the is command function is used (throughout the translator and tester classes).

For all of the command code table entries where the Command table flag was added, the current type initializer was changed to the default type (null).  This was necessary since the Command type was removed.  The type initializers will be replaced with code type enumerators.

[branch table commit 098009de24]

Table – Assign Code Recreation

A small part of a change made a few commits ago (see January 10) modified the assign string recreate function where the name of the Equal code is used instead of the name of the assign code.  This was done to remove the use of the Assign code enumerator (the Equal code enumerator is used in many places and will be one of the enumerators in the new code type enumeration).

The assign recreate function was modified to also get the name from the equal code table entry.  Since both assign recreate functions use the name from the equal code table entry, their table entries no longer need a name, so their names were set to a blank string.  The expected test results were updated accordingly.

[branch table commit 0c107e08f9]

Table – Code Indexes

Indexes of table entries are used in the program code to represent a table entry of a code.  When the program code is read, then these indexes should be converted back to a table entry point, which is then used to access information about the code.  These code indexes should only be used in the program code.  However, the code indexes have been used throughout mainly within tokens.

As of the last change, they are no longer used within tokens.  The only remaining uses are a few code enumerators to get a table entry and some uses within the tester class.  The few code enumerators still used but will become code type enumerators when these code enumerators are combined with the type enumerators.

The tester class was using code indexes to determine if define functions contain parentheses or not.  When define functions are implemented, their dictionary entries will contain the number of arguments.  No arguments will mean that there are no parentheses.  Since this is not yet implemented, the code indexes were used temporarily to determine if parentheses were present.  The tester class should not be using code indexes.

A new Define Function No Arguments type was added for the three defined function codes without parentheses.  The tester routines were modified to use this new type instead of checking for code indexes.  There will be an equivalent temporary code type enumerator for this same purpose once the type enumeration and code index enumeration is removed.  A case for this new type was added to the translator get operand function at the current Define Function case.  With a new type, the expected results for parser test #2 (identifiers) needed to be updated.

[branch table commit c9c4ddeb44]