Wednesday, October 10, 2012

Testing Implementation Change

In considering how to modify the CMake build system to handle regression testing, I realized that there was a better design in how testing should be handled.  Currently all the test inputs are contained in a source file with the testing code.  The disadvantage is that every time a new test input or test is added, this source file needs to be modified.  Adding a new test requires additional work including several changes to the test source file and modifying both regression test scripts.  And finally, strange looking C string quoting is required for double quotes and back-slashes in the test strings.

A better design would be for the tests and their inputs to be in separate test files that would be read and processed by the test code.  To change or add test inputs, these test files would be modified instead of the source file.  To add a new test, a new test file would be added.  The regression test scripts would simply run the program for each test file present.  Another advantage is that the test code does not need to be recompiled when tests are modified or added.

The test files will also support simple comments.  Lines that start with the pound '#' character and blanks lines will be ignored by the test code.  This method of commenting was chosen since no BASIC lines will start with this character and editors that support syntax highlighting (for example, vim, Kate and probably notepad++) show these comments in a different color and/or font when the file is treated as a configuration file (which vim does automatically, and Kate remembers after the option is set once).