Saturday, October 13, 2012

Updated Regression Test Scripts

The regression test scripts were modified for the changes in the test code (and new command line).  The test numbering was removed and the scipts now determine which tests to run by the presence of the parser*.dat, expression*.dat and translator*.dat files in the test directory.  The regtest script (for Windows/MSYS and Linux) no longer references ibcp.exe but simply ibcp, which will work on both Windows and Linux, with the ".exe" part added automatically on Windows.

To be able to handle any build directory (in source or out of source), the regtest script uses the source directory directly.  This was accomplished by making CMake configure this, so regtest was renamed to regtest.in and references the CMake ibcp_SOURCE_DIR variable to create regtest in the binary (build) directory that uses the correct source directory.  Now a build and test consists of just three steps, running cmake (with the proper options and path to the build directory), running make and running regtest (in the build directory).

I considered dropping the Windows Command window regtest.bat batch file, but decided this should still be included so that the regression tests could be run from the downloaded binary along with the source directory without having to build the program.  It is still a crude alternative to the regtest bash script, but it gets the job done.  When run, the "Compare more files (Y/N)?" question still has to be answered with an N now three times with the addition of the expression test compares.

The tests with the regression tests script work on all three platforms, except one problem remains: the problem with parser test #3 giving different results on the three platforms (the Windows XP results are in the repository), which still needs to be resolved.  All the testing changes has be pushed to the repository on github, including a new .gitignore file, which tells git to ignore any files in the build and nbproject sub-directories.

No comments:

Post a Comment

All comments and feedback welcomed, whether positive or negative.
(Anonymous comments are allowed, but comments with URL links or unrelated comments will be removed.)