Extra Token Delete Detection

The other type of token allocation error that can occur is when a token that has already been deleted is deleted again.  This is a less common problem, but can still could occur none the less.  Extra token delete detection was also previously implemented, but removed.  The original implementation contained a list of tokens deleted extra times.  A copy of the token was appended to the list.

This time around, the text of the token (by using the text access function) along with its index (as part of the string) is saved in a list of strings of deleted tokens.  When it comes to to report these errors, each string in the list is output.  In the reimplemented delete operation function, before the token is marked a unused in the used token vector (its pointer set to null), if the token is already marked as unused, its index and text is appended to the deleted list instead and the token is not pushed onto the free token stack (it is already there).

A new private DeletedList class was  implemented inside the Token class.  The destructor is called automatically upon termination of the application and any extra token deletes are reported. This class also contains a report errors function that does the work of outputting the deleted list to the standard error output stream and clears the list afterward.  The destructor calls this function.

Again, a separate function was implemented so that it can be called at the end of each line translated in test mode and extra token deletes for that specific line are reported with the line.  For non-test (GUI) mode, any extra token deletes are reported when the application terminates.