Colon Precedence Correction

The segmentation fault mentioned in the last post that was occurring with some files (like accidentally loaded expected translator test file) was caused by the presence of a colon.  The colon will be a statement separator or will indicate a label at the beginning of a line.  In the translator, the colon is considered an operator.  Eventually there will be a special colon token handler, but this has not yet been implemented.

The problem occurred when the colon operator token was processed as a regular operator.  The precedence of the colon was set to zero, which is also the same as the NULL operator that is put on top of the of hold stack as a blocker to prevent it from being popped (because all operators are suppose to have higher precedences).  However, since the colon also had a zero precedence, the translator popped the NULL operator from the hold stack.  The segmentation fault occurred because the NULL operator had no expression information (a NULL pointer).

To correct this problem, the precedence of the colon operator was changed to a four, which is the same precedence as the End-of-Line operator since a colon is also indicate the end of a statement.

[commit d971d6d436]

Program Model – Storing RPN Lists

The program model receives program line changes from the edit box, which up to now just stored the text of the program lines in a string list.  Eventually the program model will store the internal code of the BASIC program after the line is translated and encoded.  Since the encoder is not yet implemented, the output RPN lists from the translator will be stored so that more components of the GUI can be developed.

In order for the program model to store RPN lists, the program lines need to be translated, which means that the program model class needed a translator instance.  A translator instance member pointer was added, which is created by the constructor and deleted in the destructor.  To hold the RPN lists, a member was added for the list of RPN list pointers.

The update slot that receives the program lines from the edit box was modified to translate new lines and insert the resulting RPN list (which may contain an error); delete the RPN list for a deleted line; and delete the RPN list, translate a changed line and replace the RPN list pointer for the changed line.

The data function was modified to return the text of the RPN list for the requested line (if it doesn't have an error), or the column, length and message as a single string for a line with an error.  Eventually the error will need to be highlighted some how in the edit box.

All uses of the program line string list were replaced with the new translated line RPN list.  The string list member was left in and is still updated by the update slot, and it is also still used to detect line changes.  Shortly this list will be removed when the comparison is made using the RPN lists.

But first a small problem needs to be investigated.  During initial testing, an expected translator test file was loaded instead of the input data file and a segmentation fault occurred.  Loading any text file should just produce a bunch of errors (which is did when one of the other text files used for debugging the edit box was loaded).  Loading the desired translator test file and removing the comment lines produced the translated RPN lists in the program view as expected including lines with errors, which produced the expected error string.

[commit c77bdb9274]

RPN Output List – Including Errors

When a program line is sent from the edit box to the program model, it will be translated to an RPN list.  This RPN list will then be encoded into program code and stored.  For now, the RPN list will be stored.  However, a program line may contain an error.  Program line errors will need to be displayed in the edit box and temporarily in the program view widget.

Previously, the translator held the error token (that points to the error on the line) and the error message.  If there was a translation error, the RPN output list was cleared and the RPN list instance was deleted.  The caller would then obtain the error token and message instead of the retrieving the RPN list.  Since a program consists of many lines, several lines could contain errors.  There errors need to be kept with the program line and not in the translator instance (which only holds one error; the most recent).

Since the program model will be storing the RPN lists of the program lines, the error token and error message was moved from the Translator class to the RPN List class along with their accessor functions.  The RPN List instance is no longer deleted upon an error since it will be holding the error token and message, though the actual list is still cleared.

The translator's functions set input (given an input line, which was translated, and returned translation status) and output (returned the RPN output list upon successful translation) were replaced with a new translate function.  This function is essentially the set input function except it now returns the RPN output list (which may contain an error).  For safety, this function resets the RPN list output member pointer before returning - the caller takes possession of the RPN list instance.  A new has error function was added to the RPN List used by the caller to check the translation status.

[commit 27cd073e43]

Clear RPN List Issue Resolved

The next small incremental change implemented the RpnItem::text() and RpnList::text() member functions that return string representations of the RPN item (token with any operand tokens in square brackets) and of the RPN list respectively.  Previously the Tester::printOutput() function performed these tasks directly to the standard output stream.  A string is now necessary so that it can be output to the program view widget.

It was hoped that having these functions would resolve the segmentation faults that was occurring at the conclusion of the foreach loop that processed the RPN list.  However. it did not.  Therefore, to eliminate this problem, the foreach was simply replaced with a regular for loop for the count of the items in the list using the at() function to access the elements in the list.

This eliminated the unwanted call to the destructor and a call to the clear() function was added to the destructor so it is no longer necessary to call the clear() function from the users of the RPN list instance when they are deleting the instance.

[commit e634d6f196] [commit 1438e1e987]

Translator RPN List Output

After two failed attempts to create an RPN list class to hold the translator output, smaller incremental changes were made and tested after each change, including the memory test, and then the small changes was committed.

The first small change implemented the new Token::text() member function that returns a string representation of the token.  Previously the Tester::printSmallToken() function performed this task directly to the standard output stream.   It is now necessary to have a string so that it can be output to the program view widget.

The second small change created a new RpnList class based on QList<RpnItem*>, which has all the same functionality as the original list.  This class was defined in a new header file and the RpnItem class definition was moved to this header file from the translator header file.  This class is necessary so that the instance of this class can also hold the error token and message.

The third small change implemented a clear memory function to the new RpnList class to delete the memory used by the RPN item instances created in the translator along with the token instance each item holds.  It is a good idea to have the class perform this task and not every user of the class.

This was where at least one of the problems occurred.  It made sense that the destructor for the RpnList class call this clear function so that the caller would not have clear the list before deleting the instance, but a segmentation fault occurred in the Tester::PrintOutput() function that outputs the text of the RPN List.  The destructor of the instance was called, which deleted all the tokens (it appears the foreach macro is doing this).  When the RPN List instance is then finally deleted, the fault occurred because the token had already been deleted.

This bad change was put on a failedClearAttempt branch [commit a16abf424a] to show what apparently cannot be done.  This will be investigated next.

[commit 9d3d8bc834] [commit f49560c08f] [commit 9522d2a0ad]

Expected Batch Test Results Change

The translator currently has several outputs depending on the input, either there is an RPN output list (if translation succeeds), or there is an error token and error message (if translation fails).  For now, the program model will be changed to hold the RPN output list for each program line instead of the text of the line.  However, if there is an error, details of the error (location and message) will need to be stored instead.

A new RpnList class was created to contain the list of RpnItem instances, and any error token (with location) and error message.  An instance of this class will be stored in the program model.  After making these changes and some other changes (though there is no point in describing these here), because as a result, all of the parser tests failed.  Worst, all the expression and translator tests caused the application to crash.

The parser tests failed simply because the copyright year had been updated from 2012 to 2013.  The test results could be updated for this, but this will need to be done every year.  There is no point for these expected batch test output files to contain the copyright message along with the warranty and table initialization messages anyway.

Therefore, the copyright, warranty and table initialization messages were removed from the output, but these are still appropriately output in console input test mode.  Since these messages are no longer being put in the expected results files, the translate tr() function was added to these strings.  All of the tests results files were updated, and should not need to be updated again unless the test inputs change or for some reason the outputs change.  Now back to the RPN list changes...

[commit ff17139321]

Initial Program Model Complete

There were some minor problems with the Save As action causing it to not work as desired, namely the file name was incorrectly defaulting to "." and the directory of the file saved was not being remembered.  The default file name was set to "." in the attempt to use the current directory, but the get save file name dialog saw this as the file name with no directory and used the directory the program was started in.  The string "./" should have been used.

The way it should be working is that the file name should start with the current file name giving the user the option to replace it or modify it.  If the user doesn't change the file name, the dialog will issue a warning asking if the file should be overwritten.  If the current file is untitled, then the default file name will be set to the current directory, and the dialog will contain a blank file name.  After the file is saved successfully, the current directory (that gets saved in the program settings upon exit) is set to the directory of the saved file.

Now it is time to start hooking up the parser and translator between the edit box and program model.  But first, this is a good point to make a development release.  The release related files were updated for a new release and the repository was given the tag v0.3.3.

[commit 143f449330] [commit f95a3d8d55]

Another Undo Issue

Testing has now been completed and I think most of the possible change scenarios have been tested.  There was some more problems with undo operations when at a new line that was not yet reported as inserted.  For some undo operations, a line was not being reported as changed when it should have been.  This occurred when the line of the beginning of the undo change was more than one line away from the current line that was new.

To correct this problem, a check was added if the line of the beginning of the change is more than one line away from the modified (new) line, then the number of lines modified is set to one from zero and since this is the next line, the changed line number is incremented.

Getting into this situation was rather complex, which required an insert of a new line (Control+Enter) in the middle of a line, moving to the end of this new line, deleting the end of this line to combine it with the next line, and then one undo.  This worked so far, but doing undo again is when the problem occurred.  I'm sure there could be more complex sequences that will not be detected correctly, but it's time to move on.  Any new problems that occur will be dealt with as they appear.

There is also a screen update problem that sometime occurs during an undo where a entire line is not redrawn correctly on the screen.  However, this was proven to be a bug in the QPlainTextEdit base class, not in the EditBox class because a simple program using just the QPlainTextEdit also exhibits the same bug (though the QTextEdit class does not).

[commit 2fc26b865a]

Paste Over Selection Update Screen Issue

Another unrelated issue was found while testing, this time with paste.  The problem occurs when there is a selection, and some text is pasted over the selection, replacing the selected text.  Sometimes, the screen does not update correctly where part of the selection remains.

This can be seen by first copying a single line (some characters followed by a new line) into the clipboard.  Then selecting from the middle of a line to the end of the next non-empty line.  Pasting at this point places the cursor at the beginning of the second, the first character on the second line is removed from the screen, but the rest of the line remains and appears selected.  The document is properly updated as can be seen in the program view.

This problem was corrected by reimplementing the paste function (again).  This time it was given an argument for the clipboard mode with a default of clipboard.  The current text cursor is obtained.  If the cursor has a selection, it is copied into a temporary cursor.  With this temporary cursor, the selection is cleared and the edit box text cursor is set to the temporary cursor to applied the cleared selection.  This may not be the best way to solve this issue, but several other attempts did not work and this solution did.

The selection remains in the origin cursor.  The text is then obtained from the clipboard for the clipboard mode selected and inserted using the original cursor (replacing the selection if there was one).  To also correct the middle-click paste, the insert text call with the current text cursor was replaced with a call to the reimplemented paste function with the clipboard selection mode.

[commit 0ef619bc5c]

Handling Shift+Enter Correctly

Testing continues and no new problems have be found so far.  However, an unrelated problem was discovered with the Shift+Enter key sequence.  When there was no selection, a Shift+Enter behaved liked a regular Enter, either moving to the next line or entering a new line if the cursor was at the end of the line.  But if there was a selection, a new line character was inserted in the middle of the line.  This was evidenced because no line number was displayed including in the program view, which is the same as for a long line.

Normally, the plain text edit widget allows the insertion of a new line with Shift+Enter (like word processors).   The reimplemented edit box key press event handler partially changed this behavior, but the old behavior remained when there was a selection.

I decided to make a Shift+Enter behave as extending the selection (or starting a selection if none is present) to the beginning of the next line.  This is the same as when holding the Shift with other movement keys (arrows, Home, End, etc.).  There is no reason to allow new lines to be inserted in the middle of program lines.

This was corrected by incepting the Shift+Enter key sequence first.  The same moveCursor()  function is called with the same NextBlock move operation, but instead of using the default MoveAnchor move mode, the KeepAnchor move mode was added, which extends (or starts) a selection.

[commit 32149da677]

New Line Detection Correction

The next problem discovered occurred when a multiple line selection was replaced with less lines than the selection and the cursor was at the end of the line after the change.  The lines removed were not being reported as deleted, in fact nothing was reported.  This was in the same section of code as the first problem, but this time the problem was caused because the modified line is new flag was being set when it should not have been.

As described on March 6 (second paragraph), an earlier fix added a check if characters were added when the cursor is at the end of the line to set this flag.  However, for this case, both of these conditions were true, so that flag was wrongly set.

Instead of checking if characters were added (which catches some of the wrong conditions), the code was changed to check if lines were added to the document, which was accomplish by checking if the local net line count variable is greater than zero.  If lines were deleted, or no lines were deleted or added, then this flag is not set.

[commit dea0ef4951]

Multiple Line Undo Correction

The first problem identified while testing the many possible change scenarios was with an undo for a multiple line insert, where the last line is still marked as new (the cursor hasn't been moved away from the line yet when it would be reported as inserted).  The previously inserted lines removed by the undo were not reported as being deleted.

The problem occurred in the section that handles when lines are deleted and the cursor line was marked as new.  This was thought to only occur when the operation was either a backspace at the beginning of a new line or a delete at the end of a new line.  This situation was handled by resetting the modified line is new flag and not reporting an lines as being deleted.

However, the undo of a multiple line insert also causes this condition.  This was corrected decrementing the number of lines deleted instead of reporting no lines deleted.  For the case of a backspace or delete (and also an undo of a single line insert), the one line deleted becomes zero and no lines are reported as deleted.  For the undo of a multiple line insert, the correct number of lines are reported as deleted - one less for the new line that hasn't been inserted yet.

[commit faa2467463]

Program – Custom Data Model

Up to now only the simple predefined string list model has been used to hold the program.  The next task was to start the implementation of a custom data model that will eventually hold the program code that will be run.  To get he program model started, it will just hold a list of strings representing the lines of the program.

The new ProgramModel class is based on the QAbstractListModel as it only needs to hold a list of program lines.  This will be changed as needed since the program model will eventually need to be able to hold multiple lists of program lines, one for the main routine and several for the subroutines and functions of the program.

Now that the initial program model has been implemented (click Continue... for details), using the program view, all the various edit change operations and scenarios will now be tested to make sure all the possible changes are covered and working properly.  This is necessary to make sure the edit box keeps the program model properly informed of program changes.  Then the process of connecting to the parser and translator will begin.

[commit b543f74b6d]

Edit Box Line Numbers – New Lines

When a new line is being entered into the edit box, a line that has not been inserted into the program yet, the line numbers don't match between the edit box and the program view from the new line to the end.  Therefore, the drawing of line numbers in the edit box was updated to display no number fora new line (just the '+' character).  In the line number paint routine, each line after the new line needed to have one subtracted from its real line number in the edit box document to match the line numbers in the program view.

This lead to a problem after the new line was inserted into the program, the line numbers for all the lines after the new line were not updated to reflect their real line numbers.  This was corrected in the capture modified line routine.  When the line reported was marked as a new line, the update routine of the line number widget (the line number area) was called with the coordinates of the line number area rectangle.  Technically, only the rectangle from the current line down to the bottom is needed, but I couldn't figure out how to determine the Y-coordinate of the current line.  Redrawing the whole line number area shouldn't be a problem.

[commit c75d67d509]

Program View Line Numbers

A number of changes were made to display the line numbers in the program view similar to the edit box.  The paint function in the program line delegate was updated to display the line number and the program line in separate rectangles.  The background color of the line number rectangle is displayed in light gray like the edit box.  To space the characters in these rectangles nicely, a half character space was added on both sides of the line number and in front of the program line text.

Before the line number can be drawn, the width of the line number rectangle needs to be determined, which is based on the number of lines present in the program.  A new update width function was added that is called with a new line count each time it changes.  This function calculates the number of digits the new line count will take plus one character to account for the two half spaces on either side of the line number.  The number of pixels required is calculated and if different from the previous value calculated, the new value is stored into a new member variable and a signal is emitted to indicate that the program view needs to be updated on the screen.

Two other members were added to the class, the pixel width of a digit and the base line number (the number of the first line).  The constructor was modified with two new arguments, one for the base line number, which is stored, and the font metrics of the font used for the program view, which is used to get the pixel widget of one digit and is stored for the new width update function.

The lines to set up the program line delegate and program view in the main window constructor needed to be moved to before the initial program is loaded so that the correct line number count gets to the program line delegate.  The program changed slot was modified to check when the program line count has changed by checking a new program line count member variable, which is then stored and the update width function is called.

Finally, the program view update signal is connected to a new main window slot that updates the program view rectangle.  This slot and the program line count member variable are temporary until a more advanced program model is implemented.  Currently the program model is a simple string list model that is unaware of the program.  The other commit made was to have the program view use the same fixed width font of the edit box.

[commit 3b68b11d11] [commit a7f38c99e4]

Saving The Window State Settings

 Saving the window geometry alone is not sufficient to save the state of the Program View dock widget. The state of the main window also needed to be saved to preserve the settings of the tool bars and dock widgets, including where they are docked or floating (undocked), their sizes, and whether they are turned on or off. This window state was added to the save and restore settings routines of the main window, which were given the settings name windowState.

A dock widget can also be turned off by clicking its close button on its upper-right corner. To turn it back on, there is a context (right-click) menu that can be accessed by clicking on the unused part of the menu bar or tool bar area. A dock widget can be undocked (to make it floating) by dragging the dock widget title bar away from the main windows. Tool bars can also be undocked.

When I discovered this context menu when noticing that there was an empty tool bar beside the main tool bar that should not have been there. Besides the Program View dock widget, this context menu also showed a blank tool bar and toolBar. The blank tool bar was actually the main tool bar and it was blank because the main tool bar was never given a title (the windowTitle property).

These issues were corrected by assigned the main tool the title Main Tool Bar and the extra tool bar was removed (which must have been added by default when the Main Window GUI class was created and a new tool bar was inadvertently created to hold the tool bar actions).

[commit 9d3c4a9404]