Translator – Token Handlers

The Translator's add token function is becoming quite large and at the current rate will become much larger as the different commands are implemented. It's never a good idea to have giant functions. Therefore, this routine needs to be broken up into separate functions. The code that contains the special token code handling (for equal, comma, closing parentheses, and end-of-line) will be separate functions. These functions will be called token handlers. Most of the commands will also have token handlers.

Having a switch statement on the token code where each case calls a token handler function is not  the most efficient implementation. The concern is not execution time, but the amount of code lines required, in other words, a large switch statement. A better implementation is to store a function pointer in each table entry where a code requires a token handler. The Translator will check if there is a function pointer and then call the function to process the token.

And here is where it gets complicated. Pointers to member functions are not allowed. So the token handlers can't be Translator member function. But they need access to all of the Translator data members. The solution was to make them C++ friend functions of the Translator class with an reference argument to the Translator instance (a Translator function will pass *this).

The token handlers functions will also require a reference to the current token pointer (a reference so that it can be changed for an error) and will return the status of the operation. The Status enumeration needs to be moved out of the Translator because in order to define the return value for the token handler function pointer, it needs to be defined before TableEntry (and Table). But Table needs to be defined before TableEntry. Catch 22.

A forward reference for the Translator class can be added before TableEntry (needed for the Translator reference argument, which is simply class followed by the class name and a semicolon), but not for enumerations inside the Translator. Therefore the status enumeration will be moved out of Translator and renamed to TokenStatus (appropriate for a token handler, and the word Token is shorter than Translator).