Another Windows AWK Script Problem

I spoke too soon, in doing some final testing with CMake building, the other awk script enums.awk (used to generate autoenums.h from table.cpp for the code enumeration and ibcp.cpp for the token status enumeration) also did not work correctly on Windows.

Again, the solution used in winfix0.1.15 with setting the record separator variables can't be used because it doesn't work on Linux.  Therefore, a similar solution used for test_codes.awk was used in both parts of enums.awk, namely looking for a CR character and removing it if present and doing nothing if not.  This change was committed and push to github.

While testing, another CMake problem was also discovered.  For some reason, a compile error occurs when building in the ibcp directory (instead of in a different build directory).  I'm not sure why this is happening.  This is not recommended, but it should still work.  It was also noticed that the old Makefile was still in the directory and needs to be removed.

Obtaining New Commits

If the IBCP repository has not been cloned yet, then there is no reason to read this post.  New commits, like the fix mentioned in the last post, can be obtained using the Git GUI.  The fix commit was made on the cmake0.1.16 branch, so this branch should be checked out first.  On the Remote menu, select Fetch from and select origin.  This should fetch the new commit.

The fetch reads the new commit and updates the remotes/origin/cmake0.1.16 branch, but not the local cmake0.1.16.  The remote branch needs to be merged with the local branch.  This is a fast-forward merge (only the branch pointer is moved).  On the Merge menu select Local Merge... and make sure it says Merge Into cmake0.1.16 (in other words, this branch is checked out).  Select origin/cmake0.1.16 and click the Merge button.

For me, this worked on Linux and Windows 7, however, on Windows XP there was an error that the user name and email address was not set (it wasn't set on Windows 7 either, but no error occured).  So if this error does occurs, on the Edit menu select Options... and enter a User Name and Email Address on at least the ibcp Repository (left) side .  These are used when making commits.  The merge then worked without incident.

From the command line, use the git merge origin/cmake0.1.16 command assuming cmake0.1.16 if currently checkout (which can be verified using the git status command).  Next up, finally building this latest development branch with CMake... 

Windows AWK Script Problem Revisited

Before moving on how to build with CMake, A problem was discovered on Windows while writing the build procedure.  Specifically with the test_codes.awk script that generates the test_codes.h header file (containing the strings of the code names used in the parser test code).  This is the same problem corrected with the winfix0.1.15 branch off of the master branch.

The same fix can't be used for Linux as the script no longer works on Linux.  The issue is that when git checks out files are Windows, all the source files are automatically converted to DOS (CRLF) format while on Linux, they are left in Unix (newline) format.  Because the table.cpp source file (the source of the code names) is in DOS format on Windows, there is an extra CR at the end of each line, which is treated as a normal character.  The way this was fixed in the winfix0.1.15 branch was to set the record separator (RS) variable to "\r\n"  in the awk script so that CRLF characters are properly handled as line separators.

However, setting RS to "\r\n" on Linux breaks the awk script because now it sees a single huge line since there are no "\r\n" to be interpreted as line separators in the table.cpp source file that is in Unix format.  Currently  the script was subtracting 5 from the length of the field containing the "Xxx_Code" characters, removing the "_Code" part to get the "Xxx" part.  With the extra CR character, the "_" was left on the string leaving "Xxx_" instead.

The length function was changed to the index function to search for the position of the "_Code" part.  One was subtracted from this value to calculate the length of the "Xxx" part.  This change works with both DOS and Unix format files.  This change was committed and pushed to the github repository.  Next up, how to obtain this latest commit for an already cloned repository...

Getting The Latest Development Branch

Using gitk, it can be seen that the cloned IBCP git repository contains all of the commits (shown with blue circles), branches (shown in green boxes), and releases (tags, shown in yellow boxes) created during the project's existence.  The only local branch is master (shown in bold).  The rest of the branches are known as remote branches (shown in tan with the orange branch boxes).  The current commit is shown with a yellow circle (the currently selected commit is highlighted in blue).

All remote branches are preceded with a remotes name, followed by origin, denoting which remote the branch is from - origin is the name given to the remote repository the repository was cloned from.  These remote branches show where the branches were the last time the remote repository was accessed.  To switch to a branch using gitk, right-click on the branch and select Check out this branch.  However, only local branches can be checked out this way.

To switch to the latest development branch, which is currently remote branch remotes/origin/cmake0.1.16, a local branch needs to be created.  All branches besides master are remote branches (called tracking branches in the Git GUI).  To create a local branch for cmake0.1.16, on the Branch menu, select Create..., enter cmake0.1.16 for the Name, select Tracking Branch, select origin/cmake0.1.15, uncheck the Fetch Tracking Branch option (this options causes the remote repository to be read, but this branch is already in the local repository, so it's not necessary to read again).

The Current Branch will now be local branch cmake0.1.16.  In gitk, you can see that this branch is now checked out, by selecting Reload on the File menu (or pressing Ctrl+F5) and cmake0.1.16 will now be bold (previously master was bold).

Hit Continue... for instructions on doing these operations from the command line (for now on when command line is mentioned, this will mean either MSYS on Windows, or a Terminal on Linux and any differences between the too will be described).  Next up, building this latest development branch with CMAKE...

Building On Windows

Once the IBCP git repository has been cloned, the program can be built using the MSYS command line.  If the Git GUI was used to clone the repository as instructed, once the MSYS command line is started, there will be the ibcp directory.  Change to this directory and build the same as Linux with the make command.

However, there a problem on Windows with the awk scripts - the record separator (RS and ORS) assignments to "\r\n" lines, that are needed to work with DOS (CRLF) text file format, were removed during the clean up and repair of the git repository so that they would work on correctly on Linux.  However, this broke the scripts on Windows.  This showed up when running the regtest script and all the parser tests failed (the translator tests did all succeed).

These assignments were put back and a new commit was made to new branch winfix0.1.15 off of master.  So, to see the parser tests succeed on Windows, checkout out this branch with the git checkout winfix0.1.15 command.  If the make was already performed, do a git clean ‑df command first and run make again.  Now all tests should succeed when running the ./regtest script - at least on Windows XP.

However, on Windows 7, parser test 3 fails.  The specific test that fails is the smallest constant out of range check for the value 1.234e‑308, which does not return an error on Windows 7, whereas on Window XP and Linux it does.  Briefly investigating this, Windows 7 appears to allow even smaller values before reporting an out of range error, which don't occur until the value gets to around 1e‑323.  This will be investigated further later on the latest development branch (assuming the problem still exists). 

Making the awk scripts work on all platforms will be resolved on the latest development branch.  Next up, building the latest development branch...