Command Line Class

To create the new CommandLine class, the new file wizard in QtCreator was used.  The wizard is started by selecting New File or Project... on the File menu.  The C++ type was selected under Files and Classes, and C++ Class was selected on the right upper box, which will create both a C++ header and a source file for the new class.

On the next dialog, the class name is entered and the wizard automatically creates the header and source file names along with the path, any of which can be changed.  A Qt base class can also be selected, but wasn't for this class.  The final dialog allow the new source files to be added to version control.  If QMake was being used, the file would be added to the QMake build file, but for CMake, this has to be done manually.

This new class now handles the version option along with using the Tester class for the test options and running the tests if specified on the command line.  The command line arguments are processed in the constructor, which sets a processed flag if the arguments result in using the command line only, or if an error occurs.  There are only command lines at the moment, so this flag is always set.  The intention is that the caller then checks this flag (via a constant access function) and if not set, the GUI will be started.

Besides the constructor, this class contains a private function for handling the version option and a function to get an output stream to the standard output device (which the output stream on the first call).  The destructor deletes this output device if it was created.  The class contains members for the program base name (no access function since it is currently only used internally), the processed flag, the output stream and a list of strings for the GPL statement.

Previously there was a function to output the GPL statement.  As part of the constructor, a list of strings is created for this statement.  The QT_TR_NOOP() macro is used (verses the tr() function) so that the strings will be found by the translation utilities.  Only if the tr() function is used later on the strings will translations be used.  This list is passed to the Tester class for output, but the tr() function is not used on the strings so they will not be translated in the test output (to make sure test results match the expected results).

The main function now creates a CommandLine instance passing the command line arguments to the constructor.  If the processed flag is set (for now it will be), a single slot timer is connected to the application quit function to force the program to quit upon entered the event processing loop.  The Tester class run function was modified to accept the GPL statement string list, which is output when appropriate.

[commit ea8e56dd68]

String Comparisons

I realized that the various string comparisons in the code could be performed with the equality and inequality operators, which the QString class supports, instead of using the specific compare function, at least when case insensitive compares are not needed.  With C character arrays, the string compare function was needed, but that is not the case with the higher level QString class.

[commit 11ae68634b]

Tester Class – Single Option

As the design of the command line class was being developed, I realized that the Tester class should not be looping through all of the command line arguments.  This lead to the problem mentioned at the end of the last post when other options are specified.  The current version and test options are mutually exclusive - only one should be specified.

The version option was already checking to be make it is the only option specified.  The Tester class was modified to also make sure only one of the test options are specified - there is no reason to loop through the command line arguments since the number of arguments must match the form used (either one of two arguments).  This greatly simplified the Tester code.

All the access read only Tester functions were made constant (previously missed).  A new access function was also added to return a list of valid test options, which is used by the caller to construct the usage message.

[commit 710414e4ba]

Tester Class

A Tester class was created to contain all the test routines - all the functions in the test_ibcp.cpp source file were put into this new class.  The class definition was put into the new test_ibcp.h header file.  The main test function was split into two functions: the constructor for the Tester class, which will be given the list of command line arguments and will be parsed for test options, and a run function that will run the testing.

Once a Tester instance is created, the caller can check the status.  There will be a function for checking if errors occurred with any command line test arguments, a function to get an error message if an error occurred, and a function to check if there were any test arguments specified.  If there are test options, the run function can then be called.

The code to initialize the static Token data, create a table instance and output any errors, and create a translator instance was moved from the main function to the tester run function along with the output of the GPL header (which is now only output if no error occurs during startup).

The main() function now only handles the command line arguments - outputting the version information if the version option was specified (if this was the only option specified), otherwise creating a tester instance, checking for test argument errors, checking if there are any tests arguments, outputting the usage message if not or running the tester if there are (outputting a message if an error occurs during the test).  If the run function returns false, an error occurred during testing (like it could not open the test file specified) and the error message access function is used to get the message to output.

The tester argument parser in the constructor loops over all arguments looking for valid test options.  It verifies that only one test argument is specified, but ignores non-test arguments.  Right now, if valid test arguments are specified, the test will be run and the program will exit.  If other options are also specified (say version or an invalid option), they are currently ignored.  This will be taken care of next with a new command line class.

[commit f70cf1fc73]

CMake Issues

The patch release number has not been updated for recent development tags.  There needed to be way to automatically keep the release numbers in the CMake build file up to date with to release number in the git repository (which is determined from the most recent tag).

To prevent this from happening in the future, the CMake file was modified to check the release numbers to the current tag in the Git repository.  Previously, if the Git program was found, the release string was obtained from Git using the git describe command, otherwise the release string was set to the release numbers specified in the CMake file.

The CMake file was modified to instead first set the release string to the release numbers.  If the Git program is found, then its release number is obtained and put into a temporary variable.  If the Git release number was obtained (it wouldn't be if no repository is present), then a check is made to made sure the release string matches the first part of the Git release number (using the string command with the REGEX MATCH operation).  If it doesn't match, then a fatal error is produced.  This will catch a mismatch after a new tag is added if the release numbers are not also  updated before changes are pushed to GitHub.

One other minor change was made to the CMake file.  If the build type contains an empty string, it is now set to "Release" so that it is not empty.  Though technically this is the same as an empty string, at least now when CMake is run, the "Build type:" message does not show nothing.

[commit c0c027c07b]

Qt Application – Main Function

Another convention of Qt applications is that the main() function goes in the main.cpp source file, so the ibcp.cpp source file was renamed (and the CMake build file was updated).

While reviewing the CMake documentation, I discovered that there is a specific FindGit module, so it is not necessary to use the more generic find_program command.  The CMake build file was updated to use the find_package with the FindGit module (by using the Git argument).  There is no specific module for finding the Awk program, so this will remain using the generic find_program command.

Note: For now on instead of saying the changes have been pushed to GitHub, the specific commit ID (short form) will be put at the bottom of the post with a direct link to the commit on GitHub that is associated with the post (instead of a generic link to the Git repository, which is on the right under Downloads).  The post will also be given the GitHub label.  Recent posts are being updated to this convention.

I also noticed that the patch version number in the CMake file hasn't been updated for recent development tags.  This is not an issue when building from a source directory with the Git repository, but is when building from downloaded archives from GitHub.  So to set things straight going forward, tag v0.2-5 was added with the correct patch number in the CMake file.

[commit 381d30da1a]

Qt Application – Memory Errors

A simply program was created that containing a single main() function with a QApplication instance, a single shot timer to force the program to quit and a call to the Qt event processing executive.  This simply program also had the same memory issues, so this is some sort of issue with Qt, and probably explains why the External Errors are disabled by default in the Analyzer.

The valgrind utility has an option to disable (suppress) errors from being reported.  The ‑‑gen‑suppressions=all option can be used to generate a list of errors to suppress.  These error suppressions (extracted from the output - between the sets of braces) were put into the ibcp.supp file in the test sub-directory.  The memory test script was modified with the ‑‑suppression=$dir/ibcp.supp option where $dir is set to the test sub-directory in the source directory within the script.

This error suppression file can also be added to the Analyzer in QtCreator by going to Options... in the Tools menu and going to the Analyzer options page.  In the Memory Analysis Options section, the Add... button is used to select the ibcp.supp file.  (Note: this is only for running on Linux.)

The commented static linking commands in the CMake file were removed.  Since the executable is now going to require two Qt library files, there is no longer any reason to link the MinGW libraries required by the executable statically.  All the required dynamic link libraries will be included in future releases of the binary zip file for Windows.  If I read the licensing correctly, this is permitted if the libraries were not built from custom source.

[commit  af6faac469]

Qt Application – Building

Now that the program is using the QApplication class, the QtGui component needed to be added to the find package for Qt4 command in addition to the QtCore component in the CMake file. 

Upon testing these changes, there were no differences, however, the memory test reported a number of lost memory blocks.  It was not obvious where the problem was and was just reported on the line with the QApplication app instance.  After much experimentation, the problem was caused by statically linking libgcc and libstdc++.  Without the static linking, there were no more errors - at least when using Analyzer in QtCreator.

When running the memory test script, many more memory errors were reported, again against the QApplication app instance.  These were actually being reported in Analyzer, but were not listed because they were disabled (External Errors on the funnel looking icon on the Analyzer tool bar).

A commit was made with the changes for the QApplication instance and how the command line arguments are handled.  The static linking part of the CMake build file was temporarily commented (this will be dealt with next).

[commit  01a79dda20]

Qt Application – Event Loop

The Qt event loop is started with a call to the exec() member function of QApplication.  Normally when the users requests the program to exit from the GUI, the exec() function returns.  This occurs when the quit() function is called from one of the GUI elements (for example, the close on the application windows of Exit from the File menu).

Since no GUI elements have been implemented yet including any window, somehow the quit() function needs to be called.  This is accomplished with a single shot timer initiated with the call:

QTimer::singleShot(0, &app, SLOT(quit()));

This timer times out immediately and calls the quit() slot of the QApplication instance, but fortunately this does not occur until the exec() function is called.  Once exec() is called, it immediately returns, exiting the application.

Qt Application – Command Line Arguments

Up to now, the source code has been modified to use Qt support classes, but the program is not yet a Qt application.  A Qt application has a GUI (usually though it is possible to have a command line Qt application).  A GUI application has an event processing loop, which basically means that instead of processing sequentially from start to end, it processes events when they occur such as a keyboard press or a mouse click and calling the appropriate function to carry out the selected action.

A Qt application starts by creating an instance of a QApplication (or a QCoreApplication for a command line application).  Since this project will have a GUI, a QApplication instance will be created, but no GUI will be started if either the version or a test mode options are selected on the command line.

The command line arguments, via the argc and argv arguments of main(), are passed to the QApplication constructor because there are some Qt options that can be specified on the command line.  The arguments() member function of QApplication is used to obtain a QStringList of the remaining options (with the Qt options removed).  The first string in this list is still the name of the program.  The argc and argv arguments to the various functions were changed to QStringList &args, which is set to the arguments.

Internationalization (Qt)

Applications developed with Qt can support internationalization meaning support for multiple languages.  While I have no intention in implementing any language other than English, the door can be left open to add additional language translations later.

This is accomplished by adding the QObject::tr() function around all constant strings that would need to be translated.  The Qt linguist utilities use this as one way to identify strings that need a translation.  The Qt widget classes inherit this function from QObject and therefore just tr() is used without the scope.  This function can also be added to classes that don't inherit from QObject by adding a macro to the beginning of the class definition:

class MyClass
{
    Q_DECLARE_TR_FUNCTIONS(MyClass)
    ...
};

So, this was added to the Token class (so tr() could be used on the token status messages) and the Parser class (so tr() could be used on the parser error messages).

The scoped form QObject::tr() was added for the version, usage, error and test strings.  However, the GPL and test output strings were not changed so that the output of the regression tests would not change (otherwise the results will not match the expected result files).  The table initialization error messages were also not changed since these are development errors and will never occur in an official release.  There are alternatives to using scoped QObject::tr() form, but these need to wait until the program is turned into a full fledge Qt application (next up).

One other minor change was made to all class, struct and enum definitions where the opening brace was moved to a separate line instead of the end of the line, which is the same format used for if, for, while, etc. statements.  The opening brace on array initializers remain at the end of the line (for now).  And two unnamed enumerations were given names.

[commit  58b55f0b51]

Constant Access Getter Functions

Class member functions that do not modify the class instance should be defined to indicate this, which is coded as a trailing const on the function inside the class definition:

class MyClass {
    int value;
    ...
public:
    int value(void) const
    {
        return m_value;
    }
    int valueSquared(void) const;
}

If the function body is defined in class source file instead in the class definition in the header file, the trailing const is also necessary:

int MyClass::valueSquared(void) const
{
    return m_value * m_value;
}

The functions in the Table, Token and Translator classes that don't modify the instances were made constant functions.  The Parser class doesn't have any non-modifying access functions.  The getToken() function was also renamed to the more consistent token().  This is a good place to create another development tag: v0.2-4.

[commit  af69957e69]

Parser Errors – Resolved

It turned that many more places needed to be tested for parser errors.  The number of tests ballooned from 112 to 301 (43 places times seven parser errors).  A commit was made with all these tests.  While working on correcting the errors, there were only really two types of parser errors, an unrecognizable character and a numerical error.  So the tests were reduced to 86 with the six possible numerical errors spread across the 43 places.

The numerical parser errors were rephrased to the "expected such-and-such" format except for the "floating point constant is out of range" error ("expected valid floating point constant" didn't seem quite appropriate).  To determine which type of parser error occurs, the data type of the token is set to Double for numeric errors (previously the data type was set to None for all errors).

Numerical parser errors should only be used only when a numeric expression is expected.  In other words, when a number constant is expected, but there is something wrong with the number, then the appropriate numeric parser error should be reported.  In this case, the error should point where in the error is detected on the constant.  However, when a non-numeric expression is expected, the error should point to the beginning of the constant.  Consider the following two numerical parser errors (missing sign or digits in the exponent):

A = B + 1.2e
A$ = B$ + 1.2.e

In the first statement, the error should point to the character following the "e" indicating that a sign or digit(s) were expected in the exponent.  However, in the second statement, the same error does not make sense (pointing to the character after the "e").  The correct error should point to the "1" saying that a string expression was expected.

The remaining parser error (unrecognizable character) was treated as a normal bad token with the proper translator error being reported so the "unrecognizable character" error should never occur.  All the parser error reporting has been corrected.  More work is expected once more commands are implemented.

[commit 885389f640] [commit 6de8df90b] [commit  82ebc0beab]

Parser Errors – New Test

There are seven parser errors that can occur, but only two major types - an unrecognizable character error and some type of error in a numerical constant.  To test each of these seven errors, a new translator test (#14) was created for all the possible places during translation each of these errors can occur.  If no instances were missed, this is 16 places for each of the seven errors for a total of 112 test inputs.

The translator was temporarily modified to output the string "PARSER:" in front of the parser errors so they can easily be seen.  This does affect any of the current expected test results since none of the current translator tests have parser errors (an obvious oversight).

For now, the expected results for this new test contains the current output, but will be updated as these are corrected to the desired "expected such-and-such" error messages.  The numeric errors are appropriate if the translator was expecting a numerical constant (in other words, an operand), so the existing parser errors will be changed to the "expected such-and-such" format with the possible exception of the "constant is out of range" error.

Note that because of the recent changes to the regression test scripts (which now automatically detect tests), no modifications were needed to add this new test.

[commit  aa9271b8c9]

Translator Class Usage

Just like the Parser class, a single instance of the Translator class is created and a reference to it is passed around the program.  Unlike the Parser class, the Translator class has many more member variables, many of which are complex types (mainly stacks).  So that all these members do not need to be initialized for each new instance, the single instance design will remain for now.

There is a loop that performs the translation of an input line - the test translate input routine - which also prints the results of the translation.  This translation loop should really be handled within the Translator class, which should either return the translated output or an error.  The test routine should call this and then handle printing the results.

Therefore, a new setInput() function was implemented in the Translator class (the function name chosen to mirror the Parser class).  The functionality of the start() and getOperateState() functions was moved to this new function, which also instances its own parser.  A boolean success/fail flag is returned.

For the caller, upon success, it can proceed to obtain the output using the Translator output() function, renamed from getResults().  If an error is found, the Translator will clean up its internal variables and save the token at which the error was found with an error message.  The caller can access these through the new errorToken() and errorMessage() access functions.

The Translator will handle releasing the memory used by the error token.  There is a new internal function to set the error token.  If the error token pointer is already set, the old error token is freed.  The error token is also freed in the Translator's destructor if the pointer is set.  There is one problem with this scheme - Parser errors are returned directly, and these are not in the form "expected such-and-such" so errors could be confusing to the user.  This will be tackled next.

One other minor change was made, there was a setDefaultDataType() function in the Translator for setting the default data type of a token.  This was the remaining in-line access function still in the Translator header file and worked by examining the various members of the token.  No Translator variables were used, so it is more appropriate for this to be a Token class function, so was moved to the Token class as a new setDataType() function (taking no arguments, overloading the current function that takes a data type argument).

[commit 0dc86d1a4] [commit a88ed2f2bb]

Parser Class Usage

Currently, an instance of the Parser class is created for the program and a reference to it is passed around between the various routines.  The Parser class only has a few member variables and except for the reference to the table instance, these members are initialized for each new line that is parsed.  Therefore, it is not necessary to have a single parser instance for the program - a parser instance can be created as needed and destroyed when no longer needed.

The Parser class can be though of as a function call, though a complex one.  The parser is given an input line and returns one token at a time until either the end of the line is reached or an error is found.  A new function could be implemented to return a list of tokens for the line being parsed.  There are two locations where the parser is currently being used: the test parse input and test translate input routines.

Both of these callers work slightly different.  The test parse input routine just gets tokens until the line end or an error is found.  However, the test translate routine sets the Parser operands state from its own operand state (whether looking for an operand or not) before getting each token.  This is done so that the Parser can determine when it should be looking for a negative sign on a constant (operand state) or a minus operator (not operand state).

This implies a intimate use of the parser while a line is being translated.  So a function was not implemented to return a list of tokens.  The code was modified to instance a Parser when needed - in the two test routines - instead of passing a single instance reference of the Parser around.  These leads to the usage of the Translator class...