Token – With Shared Pointers

Fortunately there were no major problems this time around, but quite some time was required to go through each translator function to make sure token pointers were copied or transferred (moved) appropriately.  The token pointer alias temporarily set to a plain pointer was changed to a shared pointer.

By using shared pointers, it is no longer necessary to track the use of tokens, keeping them when they are still used, and deleting them when they are no longer needed.  When a shared token pointer goes out of scope or is in a container that gets deleted, it will be automatically deleted if it is no longer used.  This also eliminates the need to mark closing parentheses tokens as used or unused and doing special handling to delete parentheses tokens that are in the first and last token of items on the done stack.

Only two minor problems occurred with the changes.  The first was in the LET translate routine where the hidden option sub-code (indicating the LET keyword is not present) was set in the assignment token after the token had been moved when appended to the RPN output list.  This was resolved by setting the sub-code before appending.  The other was that two PRINT statement errors were reversed due to a check for null token pointer being changed incorrectly.  Click Continue... for additional details of the changes made.

[branch cpp11 commit fe37432066]

Shared Pointer Reseting

By using std::shared_ptr, it is not necessary to initialize a pointer to null as the default constructor does this.  There are times when a token is no longer needed.  Previously it was deleted (and the pointer was set to null).  With shared pointers, the same thing is accomplished in a number of ways.  The first is to simply set the shared pointer to a null by using one of these methods:

token = nullptr;
token = TokenPtr{};   → set to default shared pointer (a null pointer)
token.reset();

For the most part, the third method was used, however, there is another way depending on the logic that came before the pointer needs to be reset.  If the token pointer is copied to another variable, or put into a container (like pushed to a stack), then the contents of the pointer can be moved to the receiver instead of copied using the standard move function:

m_pendingParen = std::move(token);
-or-
m_holdStack.emplace(std::move(token));

The standard move function moves the pointer out of the shared pointer variable.  In first statement, the pointer is moved from the token variable, and the destination pointer get ownership of the pointer.  In the second statement, the pointer is moved from the token variable into the hold item that is constructed on top of the stack, which received ownership.  In both statements, the token variable is reset to a null pointer.  This eliminates having to increment the use count for a copy, then a decrement when the pointer is reset.

Shared Pointer Checking

Comparing a token shared pointer directly with a null pointer is not necessary as the boolean type case (as used in if, while, for, and the tertiary condition operator) is defined for the std::shared_ptr class to do this (this also works for plain pointers but was not used):

if (token == NULL)    →    if (!token)
if (token != NULL)    →    if (token)

More Changes

Like the Translator class that has a member variable for the RPN output list that is returned from the translate routine, which transfers ownership of the list to the caller, the Parser class has a member token pointer shared among the parser member functions.  Similarly, the parse routine transfers ownership of this token pointer to the owner using the standard move function.

The code in the Token class added to detect variable memory issues (tokens never deleted and tokens deleted more than once) was removed, which included the overloaded new and delete operators.  Use of shared pointers insures that these situations never occur as tokens get deleted and only are deleted when they are no longer used.

The Done Stack source file only contained functions for handling parentheses tokens.  With these removed, this source file was empty and was removed.  The Done Item class was also significantly reduced, so the Done Stack header file was also removed and the Done Item class, was moved into the Translator class and changed to a structure.

Other minor changes were also made (see the commit for details).