Translator – Correcting Token Leaks

The crash on Translator test 12 was caused because the token leaks were only being output after a successful translation but not when an error was reported. After a number of errors, the token leaks built up and when they were finally output, their information didn't line up with the current input line causing a crash when accessing outside of a buffer. Once that problem was fixed, the token memory leaks were fixed one by one.

For one of the leaks, an incorrect fix was applied causing multiple token deletes. For some reason these tokens were not output. A segmentation fault occurred when attempting to debug this problem before it even got to the output code when it was attempting to delete the string in the token a second time. Setting the string pointer to NULL didn't help – apparently the memory was already being used for something else.

Apparently gdb (under NetBeans) catches these segmentation faults where the program running by itself does not. In any case, at least the code was at least detecting the extra delete condition.  Details of the memory leaks found and corrected so far after the continue. A common theme is that the token leaks occur when something is popped from the stack and not put anywhere because any tokens in any of the stacks, the pending parentheses token or the output list get deleted by the Translator's cleanup routine.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
In the comma token handler, the comma token was not deleted when an argument to an internal function was the wrong expression type.

In the closing parentheses token handler, the top token was popped from the done stack, but if error occurs, the error is pointing to the close parentheses token and the top token was forgotten. The code was changed to only pop the top token when no error occurs.

In the closing parentheses token handler, the closing parentheses token was not deleted when an argument to an internal function was the wrong expression type.

In the assignment command handler, if the operand being assigned was the wrong expression type, the first and last operands from the item on top of the done stack are used to report the error. However, it didn't call the delete open and close parentheses routines for the first and last operands if there was no expression type error – the first and last operands were no longer needed.

When checking the item on top of the done stack in an assignment list, if the item was an array or sub-string, it had a closing parentheses set as its last operand, which was not getting deleted.

Before a new operator is processed, any higher precedence operators on the hold stack are popped first to added to the output list. If an error occurs (like the operand for the higher precedence operator is the wrong expression type), both the operator from the hold stack and the new operator tokens were not deleted - the token returned was the operand with the error.

When a range of tokens is reported for an error, the first operand token is returned with its length modified to include all the tokens through the last operand token. However, the last operand was not deleted if it was a closing parentheses when the operand from the done stack was popped.

When processing the first operand of an operator, if the first operand was the wrong expression type, then the first operand of the operator is returned for the error, but the original operator token was not deleted.

When an error occurs, the caller of the Translator (the test code) was only deleting the token passed to the Translator if it was the original token passed, otherwise the token was assumed to be a stack or output list and would be deleted by the Translator clean up routine. An open parentheses token was an exception because these are temporarily used as first operand tokens, so if this was the token with the error, the open parentheses token was not deleted. The delete open parentheses routine was used since it has all the proper checks to delete this token (this function was made public to be accessible).

In the end-of-line token handler, the command token was popped from the command stack before the check if the command handler called returned an error. When an error occurred, the command token was not deleted. The command token pop was moved to after the error check so that the command token will still be on the command when the clean up routine is called.

In the end-of-line token handler, when the command handler returned an error, if the token returned is different from the end-of-line token, the end-of-line token is not deleted. A check was added that if the token returned is a different toke, the end-of-line token is deleted.