The regression test scripts were modified for the changes in the test code (and new command line). The test numbering was removed and the scipts now determine which tests to run by the presence of the parser*.dat, expression*.dat and translator*.dat files in the test directory. The regtest script (for Windows/MSYS and Linux) no longer references ibcp.exe but simply ibcp, which will work on both Windows and Linux, with the ".exe" part added automatically on Windows.
To be able to handle any build directory (in source or out of source), the regtest script uses the source directory directly. This was accomplished by making CMake configure this, so regtest was renamed to regtest.in and references the CMake ibcp_SOURCE_DIR variable to create regtest in the binary (build) directory that uses the correct source directory. Now a build and test consists of just three steps, running cmake (with the proper options and path to the build directory), running make and running regtest (in the build directory).
I considered dropping the Windows Command window regtest.bat batch file, but decided this should still be included so that the regression tests could be run from the downloaded binary along with the source directory without having to build the program. It is still a crude alternative to the regtest bash script, but it gets the job done. When run, the "Compare more files (Y/N)?" question still has to be answered with an N now three times with the addition of the expression test compares.
The tests with the regression tests script work on all three platforms, except one problem remains: the problem with parser test #3 giving different results on the three platforms (the Windows XP results are in the repository), which still needs to be resolved. All the testing changes has be pushed to the repository on github, including a new .gitignore file, which tells git to ignore any files in the build and nbproject sub-directories.
Saturday, October 13, 2012
New Test Code
The new test code design outlined on October 10 has been implemented and tested. The old test_parser() and test_translator() functions with their test inputs, were replaced with a single test_ibcp() function that reads an input file and calls the appropriate function based on the type. The ibcp program command lines options have changed as shown:
Early in the translator development, the translator only processed expressions. The first four tests only have expressions. This implementation was left in the code and was changed to be activated only if an expression only flag is passed to the translator. The old test code set this flag for the first four tests. To keep things simple, since the file name is being used to detect the type, this was extended by the new expression type.
The expected result files were renamed accordingly. Also, the translator tests were given two digit test numbers, with a leading 0 for the first nine tests, so that these would be processed in order by the regression test scripts. The new test input files were given a ".dat" extension. The code was also modified to not output the ".exe" with the program name in the header of the output so that Windows and Linux output matches. Now the regression test scripts need to be updated.
ibcp -t <testfile>The first command form specifies the test file, which contains the tests inputs. The test code determines the type of file by the beginning of its name (parser, expression, or translator). The file name may contain anything after the type, including any extension. Blank lines and comment lines (first character '#') are ignored. The second command form activates the interactive mode when the program asks for each input for one of the three types. The new expression type needs further explanation.
ibcp -tp|-te|-tt
Early in the translator development, the translator only processed expressions. The first four tests only have expressions. This implementation was left in the code and was changed to be activated only if an expression only flag is passed to the translator. The old test code set this flag for the first four tests. To keep things simple, since the file name is being used to detect the type, this was extended by the new expression type.
The expected result files were renamed accordingly. Also, the translator tests were given two digit test numbers, with a leading 0 for the first nine tests, so that these would be processed in order by the regression test scripts. The new test input files were given a ".dat" extension. The code was also modified to not output the ".exe" with the program name in the header of the output so that Windows and Linux output matches. Now the regression test scripts need to be updated.
Friday, October 12, 2012
CMake with NetBeans
There were problems getting the new test code to work properly. Using gdb from the command line was not the first choice (more time is spent looking up help on commands than actually debugging). So this left NetBeans, which I was hoping to delay if not permanently postpone the use of (for the reason the change in development alluded to earlier, more on this soon). NetBeans is supposed to work with CMake and it was time to find out.
In the spirit of staying current, the latest version (7.2) was downloaded from NetBeans.org. The C/C++ bundle was selected for both Windows and Linux (x86/x64). The defaults for the installer were used on all three platforms (XP, 7 and Mint 13) and available updates were installed.
On Windows XP and 7, getting NetBeans to work with CMake was problematic, in fact, I never did figure out how to get it to work correctly. NetBeans appeared to give CMake incorrect options and CMake said that the compiler was not able to be used. All attempts to get the options that worked were unsuccessful.
The solution found searching the Internet was to run CMake outside of NetBeans, then point NetBeans to the Makefile produced and this worked. CMake would have to be run outside of NetBeans from then on when needed - not ideal. After a bit of work, debugging was enabled and the program could be stepped through.
On Mint 13, NetBeans worked with CMake the way it was supposed to. NetBeans was able to run CMake and build the program. The program was built with debugging enabled by default. Debugging the problems in the new test code went without a hitch. Hit Continue... for details of the procedures on both platforms.
In the spirit of staying current, the latest version (7.2) was downloaded from NetBeans.org. The C/C++ bundle was selected for both Windows and Linux (x86/x64). The defaults for the installer were used on all three platforms (XP, 7 and Mint 13) and available updates were installed.
On Windows XP and 7, getting NetBeans to work with CMake was problematic, in fact, I never did figure out how to get it to work correctly. NetBeans appeared to give CMake incorrect options and CMake said that the compiler was not able to be used. All attempts to get the options that worked were unsuccessful.
The solution found searching the Internet was to run CMake outside of NetBeans, then point NetBeans to the Makefile produced and this worked. CMake would have to be run outside of NetBeans from then on when needed - not ideal. After a bit of work, debugging was enabled and the program could be stepped through.
On Mint 13, NetBeans worked with CMake the way it was supposed to. NetBeans was able to run CMake and build the program. The program was built with debugging enabled by default. Debugging the problems in the new test code went without a hitch. Hit Continue... for details of the procedures on both platforms.
Wednesday, October 10, 2012
Testing Implementation Change
In considering how to modify the CMake build system to handle regression testing, I realized that there was a better design in how testing should be handled. Currently all the test inputs are contained in a source file with the testing code. The disadvantage is that every time a new test input or test is added, this source file needs to be modified. Adding a new test requires additional work including several changes to the test source file and modifying both regression test scripts. And finally, strange looking C string quoting is required for double quotes and back-slashes in the test strings.
A better design would be for the tests and their inputs to be in separate test files that would be read and processed by the test code. To change or add test inputs, these test files would be modified instead of the source file. To add a new test, a new test file would be added. The regression test scripts would simply run the program for each test file present. Another advantage is that the test code does not need to be recompiled when tests are modified or added.
The test files will also support simple comments. Lines that start with the pound '#' character and blanks lines will be ignored by the test code. This method of commenting was chosen since no BASIC lines will start with this character and editors that support syntax highlighting (for example, vim, Kate and probably notepad++) show these comments in a different color and/or font when the file is treated as a configuration file (which vim does automatically, and Kate remembers after the option is set once).
A better design would be for the tests and their inputs to be in separate test files that would be read and processed by the test code. To change or add test inputs, these test files would be modified instead of the source file. To add a new test, a new test file would be added. The regression test scripts would simply run the program for each test file present. Another advantage is that the test code does not need to be recompiled when tests are modified or added.
The test files will also support simple comments. Lines that start with the pound '#' character and blanks lines will be ignored by the test code. This method of commenting was chosen since no BASIC lines will start with this character and editors that support syntax highlighting (for example, vim, Kate and probably notepad++) show these comments in a different color and/or font when the file is treated as a configuration file (which vim does automatically, and Kate remembers after the option is set once).
Monday, October 8, 2012
Additional CMake Corrections
Back at the end of September when Mint 12 (Ubuntu 11.10) with GCC 4.4 (needed for CUDA development) was still being used, a change to the CMake file to allow (with a warning) GCC versions less than 4.5, which do not support linking static libraries. This option was used to create an executable that did not require dynamic libraries (so the binaries posted on SourceForge would run without having to install any other software).
This change was considered temporary and the warnings produced with older GCC versions could be ignored and any executables produced couldn't be posted. To eliminate this compiler warning, an if statement was added around the statement that added the static library link options in the CMake file, so that this option is used for GCC versions 4.5 and later.
The problem with in source directory builds was due to the local string.h header file name conflicting with the standard header file of the same name, which was included in the local string.h header file - the local header file was being included instead of the standard header file.
There is a statement in the CMake file that adds the project's binary directory to the include directories searched so that the auto-generated include files can be found. So, with an in source directory build, the local string.h file was being found even with angle brackets on the include statement since the project directory was now being searched and is apparently being search before the standard directories.
With an in source directory build, if the project directory is not added to the include directories, the correct header file is found. Since it is not necessary to add the project directory if building in the source directory (the auto-generated header files will be in the same directory as the source files including them), so to correct this problem, a check was added to the CMake file that if the project binary (build) directory is the same as the project source directory, then it is not added to the include directories.
The old make file was also removed since it now being generated by CMake and there are problems when doing in source directory builds with it present since CMake modifies the make file that is under source control (git then indicates that is was modified). These two commits are now up on github.
This change was considered temporary and the warnings produced with older GCC versions could be ignored and any executables produced couldn't be posted. To eliminate this compiler warning, an if statement was added around the statement that added the static library link options in the CMake file, so that this option is used for GCC versions 4.5 and later.
The problem with in source directory builds was due to the local string.h header file name conflicting with the standard header file of the same name, which was included in the local string.h header file - the local header file was being included instead of the standard header file.
There is a statement in the CMake file that adds the project's binary directory to the include directories searched so that the auto-generated include files can be found. So, with an in source directory build, the local string.h file was being found even with angle brackets on the include statement since the project directory was now being searched and is apparently being search before the standard directories.
With an in source directory build, if the project directory is not added to the include directories, the correct header file is found. Since it is not necessary to add the project directory if building in the source directory (the auto-generated header files will be in the same directory as the source files including them), so to correct this problem, a check was added to the CMake file that if the project binary (build) directory is the same as the project source directory, then it is not added to the include directories.
The old make file was also removed since it now being generated by CMake and there are problems when doing in source directory builds with it present since CMake modifies the make file that is under source control (git then indicates that is was modified). These two commits are now up on github.
Sunday, October 7, 2012
Building The Latest With Command Line CMake
To use CMake from the command line on Windows, the directory containing cmake.exe must be added to the path in MSYS just like for the git programs. The directory /c/Program Files/Cmake 2.8/bin (Windows XP) or /c/Program Files (x86)/Cmake 2.8/bin (Windows 7) needs to be added. One thing I forgot to mention in the October 5, 2012 post was that in order for the "/c" part to work in MSYS, the /etc/fstab (C:\MinGW\msys\1.0\etc\fstab) file system table file needs to be modified by adding a new line containing "C:\ /c" to give access to the entire C: drive.
To start, first create the build directory. For this procedure, create ibcp-build in the same directory as the ibcp source directory and change to it. On Windows, just like in the CMake GUI, the generator needs to be selected from the command line. This is done with the ‑G option. So, use this command to generate the make file:
Now some additional build issues need to be resolved, including correcting the issues with parser test #3 on the different platforms, finding the problem with in-source directory builds, and correcting regression testing so that it can be run from the build directory without have to copy the executable to the source directory.
So far, Release 0.1.16 development has been mostly about the build system, but a couple of other minor changes were made, namely, parentheses are no longer stored with identifiers and support for negative constants was added to the parser (instead of treating the minus character as a unary operator). Therefore, before continuing with development (and a change is coming, stay tuned), a release will be made with the new CMake build system once the remaining issues are resolved.
To start, first create the build directory. For this procedure, create ibcp-build in the same directory as the ibcp source directory and change to it. On Windows, just like in the CMake GUI, the generator needs to be selected from the command line. This is done with the ‑G option. So, use this command to generate the make file:
cmake -G "MSYS Makefiles" ../ibcpOn Linux, the default generator will be used, so use the cmake ../ibcp command. To build, use the make command. By the way, to see the complete compile lines that make is using, use the make VERBOSE=1 command to build (which can be helpful in debugging compile problems like header files that can't be found).
Now some additional build issues need to be resolved, including correcting the issues with parser test #3 on the different platforms, finding the problem with in-source directory builds, and correcting regression testing so that it can be run from the build directory without have to copy the executable to the source directory.
So far, Release 0.1.16 development has been mostly about the build system, but a couple of other minor changes were made, namely, parentheses are no longer stored with identifiers and support for negative constants was added to the parser (instead of treating the minus character as a unary operator). Therefore, before continuing with development (and a change is coming, stay tuned), a release will be made with the new CMake build system once the remaining issues are resolved.
Building The Latest With CMake-GUI
Instructions for building with CMake were previously given on June 23, 2011 for Windows and June 21, 2011 for Linux, but will be briefly given here again using the CMake GUI for both.
On Windows, the CMake program needs to be installed. To keep up to date, the current version (2.8.9) was downloaded from Kitware and installed. There is only a 32-bit version, but this works fine on Windows 7 64-bit. The defaults of the installer were used. This version is close to the packages (2.8.7) mentioned on October 3, 2012 installed on Mint 13 (Ubuntu 12.04).
Start the CMake GUI, on Windows: Start, All Programs, CMake 2.8, CMake (cmake-gui), and on Linux: CMake from the launcher menu under Applications/Development or the cmake-gui command from a terminal in any directory). Once started, enter or browse to the ibcp directory where the source code is (on Windows, this will be C:/MinGW/msys/1.0/home/username/ibcp, and on Linux where it was cloned to, probably /home/username/ibcp). Then select the directory to build the binaries in (recommend using C:/MinGW/msys/1.0/home/username/ibcp-build or /home/username/ibcp-build). Click the Configure button, which brings up a dialog for which generator to use.
On Windows, the generator needs to be changed to MSYS Makefiles, and on Linux, the default Unix Makefiles is used. Click the Finish button to configure. The CmakeLists.txt file will be processed and the output will be written to the lower pane. Click the Generate button to create the make file. That's it, the program can now be built. From the command line, change to the ibcp-build directory and use the make command.
To test, copy the executable (ibcp.exe on Windows or ibcp on Linux) to the ibcp directory (on Linux, copy it as ibcp.exe). From the ibcp directory, the regtest script is used to run the tests. Currently, all tests succeed on Windows XP. On Windows 7, one test in parser test #3 fails due to how small a floating point number can be (see here). On Linux, several (5) tests in parser test #3 don't compare because exponents are output with only 2 digits instead of 3 on Windows. Next up, using CMake from the command line...
On Windows, the CMake program needs to be installed. To keep up to date, the current version (2.8.9) was downloaded from Kitware and installed. There is only a 32-bit version, but this works fine on Windows 7 64-bit. The defaults of the installer were used. This version is close to the packages (2.8.7) mentioned on October 3, 2012 installed on Mint 13 (Ubuntu 12.04).
Start the CMake GUI, on Windows: Start, All Programs, CMake 2.8, CMake (cmake-gui), and on Linux: CMake from the launcher menu under Applications/Development or the cmake-gui command from a terminal in any directory). Once started, enter or browse to the ibcp directory where the source code is (on Windows, this will be C:/MinGW/msys/1.0/home/username/ibcp, and on Linux where it was cloned to, probably /home/username/ibcp). Then select the directory to build the binaries in (recommend using C:/MinGW/msys/1.0/home/username/ibcp-build or /home/username/ibcp-build). Click the Configure button, which brings up a dialog for which generator to use.
On Windows, the generator needs to be changed to MSYS Makefiles, and on Linux, the default Unix Makefiles is used. Click the Finish button to configure. The CmakeLists.txt file will be processed and the output will be written to the lower pane. Click the Generate button to create the make file. That's it, the program can now be built. From the command line, change to the ibcp-build directory and use the make command.
To test, copy the executable (ibcp.exe on Windows or ibcp on Linux) to the ibcp directory (on Linux, copy it as ibcp.exe). From the ibcp directory, the regtest script is used to run the tests. Currently, all tests succeed on Windows XP. On Windows 7, one test in parser test #3 fails due to how small a floating point number can be (see here). On Linux, several (5) tests in parser test #3 don't compare because exponents are output with only 2 digits instead of 3 on Windows. Next up, using CMake from the command line...
Saturday, October 6, 2012
Another Windows AWK Script Problem
I spoke too soon, in doing some final testing with CMake building, the other awk script enums.awk (used to generate autoenums.h from table.cpp for the code enumeration and ibcp.cpp for the token status enumeration) also did not work correctly on Windows.
Again, the solution used in winfix0.1.15 with setting the record separator variables can't be used because it doesn't work on Linux. Therefore, a similar solution used for test_codes.awk was used in both parts of enums.awk, namely looking for a CR character and removing it if present and doing nothing if not. This change was committed and push to github.
While testing, another CMake problem was also discovered. For some reason, a compile error occurs when building in the ibcp directory (instead of in a different build directory). I'm not sure why this is happening. This is not recommended, but it should still work. It was also noticed that the old Makefile was still in the directory and needs to be removed.
Again, the solution used in winfix0.1.15 with setting the record separator variables can't be used because it doesn't work on Linux. Therefore, a similar solution used for test_codes.awk was used in both parts of enums.awk, namely looking for a CR character and removing it if present and doing nothing if not. This change was committed and push to github.
While testing, another CMake problem was also discovered. For some reason, a compile error occurs when building in the ibcp directory (instead of in a different build directory). I'm not sure why this is happening. This is not recommended, but it should still work. It was also noticed that the old Makefile was still in the directory and needs to be removed.
Obtaining New Commits
If the IBCP repository has not been cloned yet, then there is no reason to read this post. New commits, like the fix mentioned in the last post, can be obtained using the Git GUI. The fix commit was made on the cmake0.1.16 branch, so this branch should be checked out first. On the Remote menu, select Fetch from and select origin. This should fetch the new commit.
The fetch reads the new commit and updates the remotes/origin/cmake0.1.16 branch, but not the local cmake0.1.16. The remote branch needs to be merged with the local branch. This is a fast-forward merge (only the branch pointer is moved). On the Merge menu select Local Merge... and make sure it says Merge Into cmake0.1.16 (in other words, this branch is checked out). Select origin/cmake0.1.16 and click the Merge button.
For me, this worked on Linux and Windows 7, however, on Windows XP there was an error that the user name and email address was not set (it wasn't set on Windows 7 either, but no error occured). So if this error does occurs, on the Edit menu select Options... and enter a User Name and Email Address on at least the ibcp Repository (left) side . These are used when making commits. The merge then worked without incident.
From the command line, use the git merge origin/cmake0.1.16 command assuming cmake0.1.16 if currently checkout (which can be verified using the git status command). Next up, finally building this latest development branch with CMake...
The fetch reads the new commit and updates the remotes/origin/cmake0.1.16 branch, but not the local cmake0.1.16. The remote branch needs to be merged with the local branch. This is a fast-forward merge (only the branch pointer is moved). On the Merge menu select Local Merge... and make sure it says Merge Into cmake0.1.16 (in other words, this branch is checked out). Select origin/cmake0.1.16 and click the Merge button.
For me, this worked on Linux and Windows 7, however, on Windows XP there was an error that the user name and email address was not set (it wasn't set on Windows 7 either, but no error occured). So if this error does occurs, on the Edit menu select Options... and enter a User Name and Email Address on at least the ibcp Repository (left) side . These are used when making commits. The merge then worked without incident.
From the command line, use the git merge origin/cmake0.1.16 command assuming cmake0.1.16 if currently checkout (which can be verified using the git status command). Next up, finally building this latest development branch with CMake...
Windows AWK Script Problem Revisited
Before moving on how to build with CMake, A problem was discovered on Windows while writing the build procedure. Specifically with the test_codes.awk script that generates the test_codes.h header file (containing the strings of the code names used in the parser test code). This is the same problem corrected with the winfix0.1.15 branch off of the master branch.
The same fix can't be used for Linux as the script no longer works on Linux. The issue is that when git checks out files are Windows, all the source files are automatically converted to DOS (CRLF) format while on Linux, they are left in Unix (newline) format. Because the table.cpp source file (the source of the code names) is in DOS format on Windows, there is an extra CR at the end of each line, which is treated as a normal character. The way this was fixed in the winfix0.1.15 branch was to set the record separator (RS) variable to "\r\n" in the awk script so that CRLF characters are properly handled as line separators.
However, setting RS to "\r\n" on Linux breaks the awk script because now it sees a single huge line since there are no "\r\n" to be interpreted as line separators in the table.cpp source file that is in Unix format. Currently the script was subtracting 5 from the length of the field containing the "Xxx_Code" characters, removing the "_Code" part to get the "Xxx" part. With the extra CR character, the "_" was left on the string leaving "Xxx_" instead.
The length function was changed to the index function to search for the position of the "_Code" part. One was subtracted from this value to calculate the length of the "Xxx" part. This change works with both DOS and Unix format files. This change was committed and pushed to the github repository. Next up, how to obtain this latest commit for an already cloned repository...
The same fix can't be used for Linux as the script no longer works on Linux. The issue is that when git checks out files are Windows, all the source files are automatically converted to DOS (CRLF) format while on Linux, they are left in Unix (newline) format. Because the table.cpp source file (the source of the code names) is in DOS format on Windows, there is an extra CR at the end of each line, which is treated as a normal character. The way this was fixed in the winfix0.1.15 branch was to set the record separator (RS) variable to "\r\n" in the awk script so that CRLF characters are properly handled as line separators.
However, setting RS to "\r\n" on Linux breaks the awk script because now it sees a single huge line since there are no "\r\n" to be interpreted as line separators in the table.cpp source file that is in Unix format. Currently the script was subtracting 5 from the length of the field containing the "Xxx_Code" characters, removing the "_Code" part to get the "Xxx" part. With the extra CR character, the "_" was left on the string leaving "Xxx_" instead.
The length function was changed to the index function to search for the position of the "_Code" part. One was subtracted from this value to calculate the length of the "Xxx" part. This change works with both DOS and Unix format files. This change was committed and pushed to the github repository. Next up, how to obtain this latest commit for an already cloned repository...
Getting The Latest Development Branch
Using gitk, it can be seen that the cloned IBCP git repository contains all of the commits (shown with blue circles), branches (shown in green boxes), and releases (tags, shown in yellow boxes) created during the project's existence. The only local branch is master (shown in bold). The rest of the branches are known as remote branches (shown in tan with the orange branch boxes). The current commit is shown with a yellow circle (the currently selected commit is highlighted in blue).
All remote branches are preceded with a remotes name, followed by origin, denoting which remote the branch is from - origin is the name given to the remote repository the repository was cloned from. These remote branches show where the branches were the last time the remote repository was accessed. To switch to a branch using gitk, right-click on the branch and select Check out this branch. However, only local branches can be checked out this way.
To switch to the latest development branch, which is currently remote branch remotes/origin/cmake0.1.16, a local branch needs to be created. All branches besides master are remote branches (called tracking branches in the Git GUI). To create a local branch for cmake0.1.16, on the Branch menu, select Create..., enter cmake0.1.16 for the Name, select Tracking Branch, select origin/cmake0.1.15, uncheck the Fetch Tracking Branch option (this options causes the remote repository to be read, but this branch is already in the local repository, so it's not necessary to read again).
The Current Branch will now be local branch cmake0.1.16. In gitk, you can see that this branch is now checked out, by selecting Reload on the File menu (or pressing Ctrl+F5) and cmake0.1.16 will now be bold (previously master was bold).
Hit Continue... for instructions on doing these operations from the command line (for now on when command line is mentioned, this will mean either MSYS on Windows, or a Terminal on Linux and any differences between the too will be described). Next up, building this latest development branch with CMAKE...
All remote branches are preceded with a remotes name, followed by origin, denoting which remote the branch is from - origin is the name given to the remote repository the repository was cloned from. These remote branches show where the branches were the last time the remote repository was accessed. To switch to a branch using gitk, right-click on the branch and select Check out this branch. However, only local branches can be checked out this way.
To switch to the latest development branch, which is currently remote branch remotes/origin/cmake0.1.16, a local branch needs to be created. All branches besides master are remote branches (called tracking branches in the Git GUI). To create a local branch for cmake0.1.16, on the Branch menu, select Create..., enter cmake0.1.16 for the Name, select Tracking Branch, select origin/cmake0.1.15, uncheck the Fetch Tracking Branch option (this options causes the remote repository to be read, but this branch is already in the local repository, so it's not necessary to read again).
The Current Branch will now be local branch cmake0.1.16. In gitk, you can see that this branch is now checked out, by selecting Reload on the File menu (or pressing Ctrl+F5) and cmake0.1.16 will now be bold (previously master was bold).
Hit Continue... for instructions on doing these operations from the command line (for now on when command line is mentioned, this will mean either MSYS on Windows, or a Terminal on Linux and any differences between the too will be described). Next up, building this latest development branch with CMAKE...
Building On Windows
Once the IBCP git repository has been cloned, the program can be built using the MSYS command line. If the Git GUI was used to clone the repository as instructed, once the MSYS command line is started, there will be the ibcp directory. Change to this directory and build the same as Linux with the make command.
However, there a problem on Windows with the awk scripts - the record separator (RS and ORS) assignments to "\r\n" lines, that are needed to work with DOS (CRLF) text file format, were removed during the clean up and repair of the git repository so that they would work on correctly on Linux. However, this broke the scripts on Windows. This showed up when running the regtest script and all the parser tests failed (the translator tests did all succeed).
These assignments were put back and a new commit was made to new branch winfix0.1.15 off of master. So, to see the parser tests succeed on Windows, checkout out this branch with the git checkout winfix0.1.15 command. If the make was already performed, do a git clean ‑df command first and run make again. Now all tests should succeed when running the ./regtest script - at least on Windows XP.
However, on Windows 7, parser test 3 fails. The specific test that fails is the smallest constant out of range check for the value 1.234e‑308, which does not return an error on Windows 7, whereas on Window XP and Linux it does. Briefly investigating this, Windows 7 appears to allow even smaller values before reporting an out of range error, which don't occur until the value gets to around 1e‑323. This will be investigated further later on the latest development branch (assuming the problem still exists).
Making the awk scripts work on all platforms will be resolved on the latest development branch. Next up, building the latest development branch...
However, there a problem on Windows with the awk scripts - the record separator (RS and ORS) assignments to "\r\n" lines, that are needed to work with DOS (CRLF) text file format, were removed during the clean up and repair of the git repository so that they would work on correctly on Linux. However, this broke the scripts on Windows. This showed up when running the regtest script and all the parser tests failed (the translator tests did all succeed).
These assignments were put back and a new commit was made to new branch winfix0.1.15 off of master. So, to see the parser tests succeed on Windows, checkout out this branch with the git checkout winfix0.1.15 command. If the make was already performed, do a git clean ‑df command first and run make again. Now all tests should succeed when running the ./regtest script - at least on Windows XP.
However, on Windows 7, parser test 3 fails. The specific test that fails is the smallest constant out of range check for the value 1.234e‑308, which does not return an error on Windows 7, whereas on Window XP and Linux it does. Briefly investigating this, Windows 7 appears to allow even smaller values before reporting an out of range error, which don't occur until the value gets to around 1e‑323. This will be investigated further later on the latest development branch (assuming the problem still exists).
Making the awk scripts work on all platforms will be resolved on the latest development branch. Next up, building the latest development branch...
Friday, October 5, 2012
Obtain IBCP Repository on Windows
There are two ways to obtain the IBCP repositories on Windows. The first is using the MSYS command line, simply use the git clone command the same as on Linux:
Once the repository is cloned, the Git Gui (ibcp) window is presented, which will be mostly empty and just below the menu bar, there is a line that has the Current Branch: master indicating the current branch. While it doesn't appear that anything was downloaded, you can see the repository by selecting Visualize All Branch History in the Repository menu. This will start the gitk GUI.
From the gitk GUI, all the commits are in the upper pane, the differences of the currently selected commit is shown in the lower left pane and the files modified for this commit is shown in the lower right pane. Though not obvious, the sizes of each pane can be adjusted by maneuvering the mouse cursor to the area between the panes until the mouse changes to the double arrow, then click, hold and drag to the desired size. For the upper and lower panes, this area is between the Find next/prev line (used for finding commits) and the Search line (for searching the differences). The sizes of the three columns in the upper pane can also be adjusted.
The gitk GUI also has several options for searching through the commits and how to display the differences. Next up, building on Windows...
git clone https://github.com/thunder422/ibcp.gitThe second is using the Git GUI, first start the GUI by selecting Start, All Programs, Git, Git GUI. Select Clone Existing Repository. Enter https://github.com/thunder422/ibcp.git in the Source Location. Enter C:/MinGW/msys/1.0/home/username (where username is your user name) in the Target Directory (or browse you MSYS home directory and add "/ibcp" to the end in the Target Directory entry, but don't use the Make New Folder in the location browser dialog to create the ibcp directory or an error will cloning). Now click the Clone button.
Once the repository is cloned, the Git Gui (ibcp) window is presented, which will be mostly empty and just below the menu bar, there is a line that has the Current Branch: master indicating the current branch. While it doesn't appear that anything was downloaded, you can see the repository by selecting Visualize All Branch History in the Repository menu. This will start the gitk GUI.
From the gitk GUI, all the commits are in the upper pane, the differences of the currently selected commit is shown in the lower left pane and the files modified for this commit is shown in the lower right pane. Though not obvious, the sizes of each pane can be adjusted by maneuvering the mouse cursor to the area between the panes until the mouse changes to the double arrow, then click, hold and drag to the desired size. For the upper and lower panes, this area is between the Find next/prev line (used for finding commits) and the Search line (for searching the differences). The sizes of the three columns in the upper pane can also be adjusted.
The gitk GUI also has several options for searching through the commits and how to display the differences. Next up, building on Windows...
Windows Prerequisites For IBCP+Git
A couple of programs need to be installed on Windows to be able to obtain and build the IBCP program. Instructions for both Windows XP (32-bit) and Windows 7 (64-bit) will be given - adjust accordingly for other versions (these are the two versions of Windows I have available).
First the MinGW (Minimalist GNU for Windows) package needs to be installed. The version I am using, which is the latest as of this date, can be obtained from Sourceforge/MinGW. Download and run the mingw‑get‑inst-20120426.exe installer. In the installer, use the defaults (especially "Use pre-packaged repository catalogs" on the third dialog). On the seventh dialog "Select Components", select C++ Compiler, MSYS Basic System, and MinGW Developer Toolkit in addition to the C Compiler. This will download and install the GCC and MSYS programs. The version of GCC installed is 4.6.2 (close to the version 4.6.3 on Mint 13).
Next, the Git for Windows package needs to be installed. The Git‑1.7.11‑preview20120710.exe installer can be downloaded from msysgit Downloads. When installing this program at the Select Components dialog, under Windows Explorer integration select Simple context menu to install Git Bash and Git GUI. Select the defaults for the rest of the dialogs.
Now the git package needs to be integrated with the MSYS package. All that is necessary is that the PATH environment variable needs to be modified to include where the git programs are installed. For Windows XP, this is C:Program Files/Git/bin and for Windows 7, this is C:Program Files (x86)/Git/bin. Within the MSYS command line, edit the file /etc/profile (or with Notepad - the file is located at C:\MinGW\msys\1.0\etc\profile) and add the following line after all other lines that modify PATH:
If necessary, restart the MSYS command window, and confirm git is there by using the git ‑‑version command within the MSYS command window. Next up, obtaining the IBCP repository on Windows.
First the MinGW (Minimalist GNU for Windows) package needs to be installed. The version I am using, which is the latest as of this date, can be obtained from Sourceforge/MinGW. Download and run the mingw‑get‑inst-20120426.exe installer. In the installer, use the defaults (especially "Use pre-packaged repository catalogs" on the third dialog). On the seventh dialog "Select Components", select C++ Compiler, MSYS Basic System, and MinGW Developer Toolkit in addition to the C Compiler. This will download and install the GCC and MSYS programs. The version of GCC installed is 4.6.2 (close to the version 4.6.3 on Mint 13).
Next, the Git for Windows package needs to be installed. The Git‑1.7.11‑preview20120710.exe installer can be downloaded from msysgit Downloads. When installing this program at the Select Components dialog, under Windows Explorer integration select Simple context menu to install Git Bash and Git GUI. Select the defaults for the rest of the dialogs.
Now the git package needs to be integrated with the MSYS package. All that is necessary is that the PATH environment variable needs to be modified to include where the git programs are installed. For Windows XP, this is C:Program Files/Git/bin and for Windows 7, this is C:Program Files (x86)/Git/bin. Within the MSYS command line, edit the file /etc/profile (or with Notepad - the file is located at C:\MinGW\msys\1.0\etc\profile) and add the following line after all other lines that modify PATH:
Windows XP: export PATH="$PATH:/c/Program Files/Git/bin"
Windows 7: export PATH="$PATH:/c/Program Files (x86)/Git/bin"
(Update October 6, 2012) In order for the "/c" part to work in MSYS, the /etc/fstab (C:\MinGW\msys\1.0\etc\fstab) file system table file needs to be modified by adding a new line containing "C:\ /c" to give access to the entire C drive.
If necessary, restart the MSYS command window, and confirm git is there by using the git ‑‑version command within the MSYS command window. Next up, obtaining the IBCP repository on Windows.
Thursday, October 4, 2012
Accessing and Building on Linux
The git clone command is used to obtain a git repository. To obtain the IBCP git repository from github.com use the command:
At this point, the program can be built using the make command in the ibcp directory. There are a couple of compiler warnings, which were corrected in the 0.1.6 development branch and can be ignored. The parser and translator tests can be run with the ./regtest command. Currently on Mint 13 (64-bit) on Release 0.1.5 (master), parser tests 1 and 3 fail.
Parser test 1 fails due to a bug with parsing integers, which did not occur on 32-bit Windows. This problem was corrected in the 0.1.16 development branch. Parser test 3 also fails because of an integer parsing error and due to a change in output of exponentiation numbers (64-bit Linux outputs only 2 digit exponents, but Windows outputs 3, which may also be due to the particular compiler or library versions). The exponent issue has yet to be corrected.
Next up, how to build the latest development branch, which is also contained in the git repository and uses cmake to create a make file...
git clone https://github.com/thunder422/ibcp.gitThis will create the ibcp directory and populate ot with the source files for the master branch (which is currently Release 0.1.15). However, the entire repository is present (there is a hidden .git directory). The commits can be viewed using the git log command (there are many options, which is why installing the git documentation is strongly recommended). The repository can also be viewed in GUI form using the gitk command if installed.
At this point, the program can be built using the make command in the ibcp directory. There are a couple of compiler warnings, which were corrected in the 0.1.6 development branch and can be ignored. The parser and translator tests can be run with the ./regtest command. Currently on Mint 13 (64-bit) on Release 0.1.5 (master), parser tests 1 and 3 fail.
Parser test 1 fails due to a bug with parsing integers, which did not occur on 32-bit Windows. This problem was corrected in the 0.1.16 development branch. Parser test 3 also fails because of an integer parsing error and due to a change in output of exponentiation numbers (64-bit Linux outputs only 2 digit exponents, but Windows outputs 3, which may also be due to the particular compiler or library versions). The exponent issue has yet to be corrected.
Next up, how to build the latest development branch, which is also contained in the git repository and uses cmake to create a make file...
Wednesday, October 3, 2012
Linux Prerequisites For IBCP & Git
Several software packages are required to obtain and build the IBCP program from the git repository, which include:
g++ Needed to compile C++ programs (brings libstdc++6 with it)These can either be installed from the GUI package manager (for example the Synaptic Package Manager or Muon Package Manager) or from the command line using the apt-get command, as in sudo apt-get install g++ cmake git. There are several other packages that are nice to have, but are not required, which include:
git Needed to retrieve the IBCP Git repository
cmake Needed for the CMake utility
gitk a GUI for displaying a git repositoryI should note that I actually obtained git by cloning the official git source repository and building the latest version (1.7.12.1) myself though git needs to be installed before this can be done. Quite a few more packages are required to build all of the git package and documentation. However, the version in the repositories (1.7.9.5 in Mint 13/Ubuntu 12.04) is sufficient for working with the IBCP git repository. Next up, how to obtain and build the IBCP program.
git-gui a GUI for working with the git repository
git-man the manual pages for the git commands
git-doc alternate documentation for git
cmake-curses-gui a terminal GUI for cmake (ccmake)
cmake-qt-gui a nice Qt GUI for cmake (cmake-gui)
Subscribe to:
Posts (Atom)