Tester Class – Single Option

As the design of the command line class was being developed, I realized that the Tester class should not be looping through all of the command line arguments.  This lead to the problem mentioned at the end of the last post when other options are specified.  The current version and test options are mutually exclusive - only one should be specified.

The version option was already checking to be make it is the only option specified.  The Tester class was modified to also make sure only one of the test options are specified - there is no reason to loop through the command line arguments since the number of arguments must match the form used (either one of two arguments).  This greatly simplified the Tester code.

All the access read only Tester functions were made constant (previously missed).  A new access function was also added to return a list of valid test options, which is used by the caller to construct the usage message.

[commit 710414e4ba]

Tester Class

A Tester class was created to contain all the test routines - all the functions in the test_ibcp.cpp source file were put into this new class.  The class definition was put into the new test_ibcp.h header file.  The main test function was split into two functions: the constructor for the Tester class, which will be given the list of command line arguments and will be parsed for test options, and a run function that will run the testing.

Once a Tester instance is created, the caller can check the status.  There will be a function for checking if errors occurred with any command line test arguments, a function to get an error message if an error occurred, and a function to check if there were any test arguments specified.  If there are test options, the run function can then be called.

The code to initialize the static Token data, create a table instance and output any errors, and create a translator instance was moved from the main function to the tester run function along with the output of the GPL header (which is now only output if no error occurs during startup).

The main() function now only handles the command line arguments - outputting the version information if the version option was specified (if this was the only option specified), otherwise creating a tester instance, checking for test argument errors, checking if there are any tests arguments, outputting the usage message if not or running the tester if there are (outputting a message if an error occurs during the test).  If the run function returns false, an error occurred during testing (like it could not open the test file specified) and the error message access function is used to get the message to output.

The tester argument parser in the constructor loops over all arguments looking for valid test options.  It verifies that only one test argument is specified, but ignores non-test arguments.  Right now, if valid test arguments are specified, the test will be run and the program will exit.  If other options are also specified (say version or an invalid option), they are currently ignored.  This will be taken care of next with a new command line class.

[commit f70cf1fc73]

CMake Issues

The patch release number has not been updated for recent development tags.  There needed to be way to automatically keep the release numbers in the CMake build file up to date with to release number in the git repository (which is determined from the most recent tag).

To prevent this from happening in the future, the CMake file was modified to check the release numbers to the current tag in the Git repository.  Previously, if the Git program was found, the release string was obtained from Git using the git describe command, otherwise the release string was set to the release numbers specified in the CMake file.

The CMake file was modified to instead first set the release string to the release numbers.  If the Git program is found, then its release number is obtained and put into a temporary variable.  If the Git release number was obtained (it wouldn't be if no repository is present), then a check is made to made sure the release string matches the first part of the Git release number (using the string command with the REGEX MATCH operation).  If it doesn't match, then a fatal error is produced.  This will catch a mismatch after a new tag is added if the release numbers are not also  updated before changes are pushed to GitHub.

One other minor change was made to the CMake file.  If the build type contains an empty string, it is now set to "Release" so that it is not empty.  Though technically this is the same as an empty string, at least now when CMake is run, the "Build type:" message does not show nothing.

[commit c0c027c07b]

Qt Application – Main Function

Another convention of Qt applications is that the main() function goes in the main.cpp source file, so the ibcp.cpp source file was renamed (and the CMake build file was updated).

While reviewing the CMake documentation, I discovered that there is a specific FindGit module, so it is not necessary to use the more generic find_program command.  The CMake build file was updated to use the find_package with the FindGit module (by using the Git argument).  There is no specific module for finding the Awk program, so this will remain using the generic find_program command.

Note: For now on instead of saying the changes have been pushed to GitHub, the specific commit ID (short form) will be put at the bottom of the post with a direct link to the commit on GitHub that is associated with the post (instead of a generic link to the Git repository, which is on the right under Downloads).  The post will also be given the GitHub label.  Recent posts are being updated to this convention.

I also noticed that the patch version number in the CMake file hasn't been updated for recent development tags.  This is not an issue when building from a source directory with the Git repository, but is when building from downloaded archives from GitHub.  So to set things straight going forward, tag v0.2-5 was added with the correct patch number in the CMake file.

[commit 381d30da1a]

Qt Application – Memory Errors

A simply program was created that containing a single main() function with a QApplication instance, a single shot timer to force the program to quit and a call to the Qt event processing executive.  This simply program also had the same memory issues, so this is some sort of issue with Qt, and probably explains why the External Errors are disabled by default in the Analyzer.

The valgrind utility has an option to disable (suppress) errors from being reported.  The ‑‑gen‑suppressions=all option can be used to generate a list of errors to suppress.  These error suppressions (extracted from the output - between the sets of braces) were put into the ibcp.supp file in the test sub-directory.  The memory test script was modified with the ‑‑suppression=$dir/ibcp.supp option where $dir is set to the test sub-directory in the source directory within the script.

This error suppression file can also be added to the Analyzer in QtCreator by going to Options... in the Tools menu and going to the Analyzer options page.  In the Memory Analysis Options section, the Add... button is used to select the ibcp.supp file.  (Note: this is only for running on Linux.)

The commented static linking commands in the CMake file were removed.  Since the executable is now going to require two Qt library files, there is no longer any reason to link the MinGW libraries required by the executable statically.  All the required dynamic link libraries will be included in future releases of the binary zip file for Windows.  If I read the licensing correctly, this is permitted if the libraries were not built from custom source.

[commit  af6faac469]

Qt Application – Building

Now that the program is using the QApplication class, the QtGui component needed to be added to the find package for Qt4 command in addition to the QtCore component in the CMake file. 

Upon testing these changes, there were no differences, however, the memory test reported a number of lost memory blocks.  It was not obvious where the problem was and was just reported on the line with the QApplication app instance.  After much experimentation, the problem was caused by statically linking libgcc and libstdc++.  Without the static linking, there were no more errors - at least when using Analyzer in QtCreator.

When running the memory test script, many more memory errors were reported, again against the QApplication app instance.  These were actually being reported in Analyzer, but were not listed because they were disabled (External Errors on the funnel looking icon on the Analyzer tool bar).

A commit was made with the changes for the QApplication instance and how the command line arguments are handled.  The static linking part of the CMake build file was temporarily commented (this will be dealt with next).

[commit  01a79dda20]

Qt Application – Event Loop

The Qt event loop is started with a call to the exec() member function of QApplication.  Normally when the users requests the program to exit from the GUI, the exec() function returns.  This occurs when the quit() function is called from one of the GUI elements (for example, the close on the application windows of Exit from the File menu).

Since no GUI elements have been implemented yet including any window, somehow the quit() function needs to be called.  This is accomplished with a single shot timer initiated with the call:

QTimer::singleShot(0, &app, SLOT(quit()));

This timer times out immediately and calls the quit() slot of the QApplication instance, but fortunately this does not occur until the exec() function is called.  Once exec() is called, it immediately returns, exiting the application.

Qt Application – Command Line Arguments

Up to now, the source code has been modified to use Qt support classes, but the program is not yet a Qt application.  A Qt application has a GUI (usually though it is possible to have a command line Qt application).  A GUI application has an event processing loop, which basically means that instead of processing sequentially from start to end, it processes events when they occur such as a keyboard press or a mouse click and calling the appropriate function to carry out the selected action.

A Qt application starts by creating an instance of a QApplication (or a QCoreApplication for a command line application).  Since this project will have a GUI, a QApplication instance will be created, but no GUI will be started if either the version or a test mode options are selected on the command line.

The command line arguments, via the argc and argv arguments of main(), are passed to the QApplication constructor because there are some Qt options that can be specified on the command line.  The arguments() member function of QApplication is used to obtain a QStringList of the remaining options (with the Qt options removed).  The first string in this list is still the name of the program.  The argc and argv arguments to the various functions were changed to QStringList &args, which is set to the arguments.

Internationalization (Qt)

Applications developed with Qt can support internationalization meaning support for multiple languages.  While I have no intention in implementing any language other than English, the door can be left open to add additional language translations later.

This is accomplished by adding the QObject::tr() function around all constant strings that would need to be translated.  The Qt linguist utilities use this as one way to identify strings that need a translation.  The Qt widget classes inherit this function from QObject and therefore just tr() is used without the scope.  This function can also be added to classes that don't inherit from QObject by adding a macro to the beginning of the class definition:

class MyClass
{
    Q_DECLARE_TR_FUNCTIONS(MyClass)
    ...
};

So, this was added to the Token class (so tr() could be used on the token status messages) and the Parser class (so tr() could be used on the parser error messages).

The scoped form QObject::tr() was added for the version, usage, error and test strings.  However, the GPL and test output strings were not changed so that the output of the regression tests would not change (otherwise the results will not match the expected result files).  The table initialization error messages were also not changed since these are development errors and will never occur in an official release.  There are alternatives to using scoped QObject::tr() form, but these need to wait until the program is turned into a full fledge Qt application (next up).

One other minor change was made to all class, struct and enum definitions where the opening brace was moved to a separate line instead of the end of the line, which is the same format used for if, for, while, etc. statements.  The opening brace on array initializers remain at the end of the line (for now).  And two unnamed enumerations were given names.

[commit  58b55f0b51]

Constant Access Getter Functions

Class member functions that do not modify the class instance should be defined to indicate this, which is coded as a trailing const on the function inside the class definition:

class MyClass {
    int value;
    ...
public:
    int value(void) const
    {
        return m_value;
    }
    int valueSquared(void) const;
}

If the function body is defined in class source file instead in the class definition in the header file, the trailing const is also necessary:

int MyClass::valueSquared(void) const
{
    return m_value * m_value;
}

The functions in the Table, Token and Translator classes that don't modify the instances were made constant functions.  The Parser class doesn't have any non-modifying access functions.  The getToken() function was also renamed to the more consistent token().  This is a good place to create another development tag: v0.2-4.

[commit  af69957e69]

Parser Errors – Resolved

It turned that many more places needed to be tested for parser errors.  The number of tests ballooned from 112 to 301 (43 places times seven parser errors).  A commit was made with all these tests.  While working on correcting the errors, there were only really two types of parser errors, an unrecognizable character and a numerical error.  So the tests were reduced to 86 with the six possible numerical errors spread across the 43 places.

The numerical parser errors were rephrased to the "expected such-and-such" format except for the "floating point constant is out of range" error ("expected valid floating point constant" didn't seem quite appropriate).  To determine which type of parser error occurs, the data type of the token is set to Double for numeric errors (previously the data type was set to None for all errors).

Numerical parser errors should only be used only when a numeric expression is expected.  In other words, when a number constant is expected, but there is something wrong with the number, then the appropriate numeric parser error should be reported.  In this case, the error should point where in the error is detected on the constant.  However, when a non-numeric expression is expected, the error should point to the beginning of the constant.  Consider the following two numerical parser errors (missing sign or digits in the exponent):

A = B + 1.2e
A$ = B$ + 1.2.e

In the first statement, the error should point to the character following the "e" indicating that a sign or digit(s) were expected in the exponent.  However, in the second statement, the same error does not make sense (pointing to the character after the "e").  The correct error should point to the "1" saying that a string expression was expected.

The remaining parser error (unrecognizable character) was treated as a normal bad token with the proper translator error being reported so the "unrecognizable character" error should never occur.  All the parser error reporting has been corrected.  More work is expected once more commands are implemented.

[commit 885389f640] [commit 6de8df90b] [commit  82ebc0beab]

Parser Errors – New Test

There are seven parser errors that can occur, but only two major types - an unrecognizable character error and some type of error in a numerical constant.  To test each of these seven errors, a new translator test (#14) was created for all the possible places during translation each of these errors can occur.  If no instances were missed, this is 16 places for each of the seven errors for a total of 112 test inputs.

The translator was temporarily modified to output the string "PARSER:" in front of the parser errors so they can easily be seen.  This does affect any of the current expected test results since none of the current translator tests have parser errors (an obvious oversight).

For now, the expected results for this new test contains the current output, but will be updated as these are corrected to the desired "expected such-and-such" error messages.  The numeric errors are appropriate if the translator was expecting a numerical constant (in other words, an operand), so the existing parser errors will be changed to the "expected such-and-such" format with the possible exception of the "constant is out of range" error.

Note that because of the recent changes to the regression test scripts (which now automatically detect tests), no modifications were needed to add this new test.

[commit  aa9271b8c9]

Translator Class Usage

Just like the Parser class, a single instance of the Translator class is created and a reference to it is passed around the program.  Unlike the Parser class, the Translator class has many more member variables, many of which are complex types (mainly stacks).  So that all these members do not need to be initialized for each new instance, the single instance design will remain for now.

There is a loop that performs the translation of an input line - the test translate input routine - which also prints the results of the translation.  This translation loop should really be handled within the Translator class, which should either return the translated output or an error.  The test routine should call this and then handle printing the results.

Therefore, a new setInput() function was implemented in the Translator class (the function name chosen to mirror the Parser class).  The functionality of the start() and getOperateState() functions was moved to this new function, which also instances its own parser.  A boolean success/fail flag is returned.

For the caller, upon success, it can proceed to obtain the output using the Translator output() function, renamed from getResults().  If an error is found, the Translator will clean up its internal variables and save the token at which the error was found with an error message.  The caller can access these through the new errorToken() and errorMessage() access functions.

The Translator will handle releasing the memory used by the error token.  There is a new internal function to set the error token.  If the error token pointer is already set, the old error token is freed.  The error token is also freed in the Translator's destructor if the pointer is set.  There is one problem with this scheme - Parser errors are returned directly, and these are not in the form "expected such-and-such" so errors could be confusing to the user.  This will be tackled next.

One other minor change was made, there was a setDefaultDataType() function in the Translator for setting the default data type of a token.  This was the remaining in-line access function still in the Translator header file and worked by examining the various members of the token.  No Translator variables were used, so it is more appropriate for this to be a Token class function, so was moved to the Token class as a new setDataType() function (taking no arguments, overloading the current function that takes a data type argument).

[commit 0dc86d1a4] [commit a88ed2f2bb]

Parser Class Usage

Currently, an instance of the Parser class is created for the program and a reference to it is passed around between the various routines.  The Parser class only has a few member variables and except for the reference to the table instance, these members are initialized for each new line that is parsed.  Therefore, it is not necessary to have a single parser instance for the program - a parser instance can be created as needed and destroyed when no longer needed.

The Parser class can be though of as a function call, though a complex one.  The parser is given an input line and returns one token at a time until either the end of the line is reached or an error is found.  A new function could be implemented to return a list of tokens for the line being parsed.  There are two locations where the parser is currently being used: the test parse input and test translate input routines.

Both of these callers work slightly different.  The test parse input routine just gets tokens until the line end or an error is found.  However, the test translate routine sets the Parser operands state from its own operand state (whether looking for an operand or not) before getting each token.  This is done so that the Parser can determine when it should be looking for a negative sign on a constant (operand state) or a minus operator (not operand state).

This implies a intimate use of the parser while a line is being translated.  So a function was not implemented to return a list of tokens.  The code was modified to instance a Parser when needed - in the two test routines - instead of passing a single instance reference of the Parser around.  These leads to the usage of the Translator class...

New Change Comment Philosophy

Going through the code and for the most part removing all the change comments (with a date), I've had a change in philosophy one how code should be commented.  Perhaps for a released program, it makes sense to add a comment with a date when a bug is fixed, but during development and debugging it's expected that there will be a lot of changes.  Making a dated change comment for every single change is just a bit excessive - and this certainly could be seen here.

The comments at the beginning of the files probably should be removed also or at least cleaned up, but these were left alone for the time being.  Perhaps some of the major highlights will be left in.  In any case, going forward, the git commit comments will be relied on to comment the changes made to the source file.  Functionality comments will still be added as needed, but without all the change comments, the code should be easier to read.

The latest commit contains more of this comment clean up.  All of the project is now using Qt functions; all the standard C library functions (and associated include statements) have been removed.  Most if not all the variables and functions have been renamed to the new naming scheme.  This is probably a good place to add another tag, and so the current code was tagged as v0.2-3.

[commit ebe943379d]

Qt Transition – Translator Class

The variables and functions of the Translator class were renamed to the Qt style naming.  Nothing much to report here except that the RpnItem structure used by the translator was changed to a class with private members with access functions added.  Through the translator source file along with the new token and command handler source files, all of the dated change comments were mostly removed (some were appropriate comments so only the date was removed).  These were just cluttering up the code (for now the header comments were left alone).

[commit  85daa1fc7e]