The decode and translate now return RPN lists by value, either from a local variable or member variable as described in recent posts. For the functions that had an RPN list pointer as an argument, the argument was changed to a constant reference (constant because the RPN list is not modified). The callers that receive RPN lists no longer are responsible for deleting the instance, which will occur automatically when the variable goes out of scope, so all the delete calls were removed.
One issue that was discovered during compiling of functions taking a constant reference to an RPN list was an error when using the code size access function. This function only returned the code size member and did not modify the object, so it should haven declared as a constant function.
The translate input routine in the Tester class previously returned a null pointer to indicate an error. Since a pointer is no longer returned, the RPN list is cleared and an empty list is returned to indicate an error. An empty list could also mean an empty BASIC line (which is allowed), but empty lines are already being ignored and are not translated duing testing.
The null pointer initializer for the output list member was removed from the constructor of the Translator class since it is no longer a pointer - it is now initialized using the default constructor for the RPN List class. All uses accessing the pointer member were changed to regular member (dot) operators.
[branch cpp11 commit 5b9245cf0a]
Sunday, September 7, 2014
RPN List – Default Move Not Sufficient
The RPN list class contains other members in addition to the list including the code size, error column, error length and error message members. Since the list member is a standard list with a move operation, it is cleared by the default generated move constructor when calling the standard move in the return statement of the translate routine. All but the error message are integers with no default move operation so these members are not affected. The error message is currently a QString, which also does not have a move operation (the Qt5 classes do) and was also not affected.
When a new RPN list instance was allocated at the beginning of the translate routine, these other members were initialized according to the constructor (though only the error members were initialized). Now, at the return statement, the list of the RPN list member is cleared from the default move, but with no move operations for the other members, these members retained their values.
This was detected while running the tests. Once a line with an error was reported, that same error was reported for every line afterward (at least until there was a different error). This occurred because the error column member (used to test if the RPN list has an error) was not initialized for the next translation.
One solution would be to reset these other members at the at the beginning where the allocation was. The better solution of course is to add move constructor and assignment functions. The move constructor initializes each of the members from the other RPN list instance. In the function body, each of the members of the other RPN list instance are set to an initialized state. The list and error message members are cleared and the integer variables are set to appropriate default values. The move assignment swaps all the members before returning a reference to the instance.
When a new RPN list instance was allocated at the beginning of the translate routine, these other members were initialized according to the constructor (though only the error members were initialized). Now, at the return statement, the list of the RPN list member is cleared from the default move, but with no move operations for the other members, these members retained their values.
This was detected while running the tests. Once a line with an error was reported, that same error was reported for every line afterward (at least until there was a different error). This occurred because the error column member (used to test if the RPN list has an error) was not initialized for the next translation.
One solution would be to reset these other members at the at the beginning where the allocation was. The better solution of course is to add move constructor and assignment functions. The move constructor initializes each of the members from the other RPN list instance. In the function body, each of the members of the other RPN list instance are set to an initialized state. The list and error message members are cleared and the integer variables are set to appropriate default values. The move assignment swaps all the members before returning a reference to the instance.
RPN List – Return Non-Local Variable
Both the decode and translate routines generate an RPN list and instead of a pointer to the list will return the list by value, preferably by a move operation. The decode routine (of the Program Model class) contains a local RPN list variable, and so the value of the variable is returned by a move operation since the variable is going out of scope. However, the translate routine builds the RPN list in a member value of the Translator class (so all translator functions have access to the list). If this variable is returned by value, the value is copied because the variable is not going out of scope.
With the pointer implementation when a pointer was returned, since the translate routine no longer needed the list, it made a temporary copy of the member list pointer, set the member pointer to null and returned the temporary copy of the pointer (this transferred the list to the caller who was then responsible for deleting the list):
Defining a destructor indicates that something other than the default operation needs to be done for a copy or move (for example, dealing with an allocated resource). However, these rules are not completely enforced. For backward compatibility, GCC still generates default copy operations if a destructor is defined without a warning.
The upstream problem mentioned a few posts ago http://interactivebasiccompilerproject.blogspot.com/2014/09/rpn-list-base-class-to-member.html was thought to have occurred due the destructor of the list base class was not being called because the above return move statement was not clearing the list in the member variable. The problem was finally identified to be caused by the destructor (which wasn't doing anything) declaration was removed. With the destructor, there were no move operations, the standard move performed a copy. Removing the destructor allowed the default move operations to be generated. A default move operation can be generated using the new C++11 syntax:
With the pointer implementation when a pointer was returned, since the translate routine no longer needed the list, it made a temporary copy of the member list pointer, set the member pointer to null and returned the temporary copy of the pointer (this transferred the list to the caller who was then responsible for deleting the list):
RpnList *output = m_output;The Translator class RPN list member will be the actual list, not a pointer to a list. When returning from the translate routine, the RPN list is no longer needed, so it can be transferred to the caller with a move operation. This move does not occur by just returning the variable because it is not going out of scope. The move operation can be forced by using the standard move (which resets the member variable to an empty list and transfers its value to the receiving variable of the caller):
m_output = NULL;
return output;
return std::move(m_output);By default, C++ provides a default constructor, copy constructor, copy assignment, move constructor, move assignment and destructor for a class if a program uses them. If a constructor is declared, a default constructor is not generated. If a copy operation, move operation or destructor is declared, then no default copy operation, move operation or destructor is generated.
Defining a destructor indicates that something other than the default operation needs to be done for a copy or move (for example, dealing with an allocated resource). However, these rules are not completely enforced. For backward compatibility, GCC still generates default copy operations if a destructor is defined without a warning.
The upstream problem mentioned a few posts ago http://interactivebasiccompilerproject.blogspot.com/2014/09/rpn-list-base-class-to-member.html was thought to have occurred due the destructor of the list base class was not being called because the above return move statement was not clearing the list in the member variable. The problem was finally identified to be caused by the destructor (which wasn't doing anything) declaration was removed. With the destructor, there were no move operations, the standard move performed a copy. Removing the destructor allowed the default move operations to be generated. A default move operation can be generated using the new C++11 syntax:
ClassName(ClassName &&other) = default;This solved the issue of the member list not being cleared. However, the default move operations were not sufficient as there was another problem...
C++11 Move Constructors and Assignments
The actual instances of RPN lists will be returned from functions instead of a pointer to an allocated instance that requires the receiver to delete the instance when it is done with it (or a memory leak occurs). The new C++11 move constructors and assignments feature makes this efficient.
When a function is called that has a return value, temporary memory (usually on the stack) needs to set aside hold the return value because the variable holding the return value in called function goes out of scope and will be destroyed (destructor called) if not a plain old data (POD) type. Prior to C++11, the return value was copied from the temporary into a receiving variable using a copy constructor if not a POD type. If the variable was a large container like a list or vector, this was not efficient. This is why RPN lists were returned by a pointer.
However, with a C++11 move constructor, the return value is moved directly into the variable being constructed or assigned, each having a slightly different mechanism. For a container class like a list or vector, the only thing that is moved is the pointer to the data, not the actual data itself. Consider this example of a function that returns a list and code that calls the function:
A move assignment is similar, but is used when the receiving variable (that already exists) is being assigned to a return value, where the caller looks like this:
The above descriptions may not be exactly what occurs internally, but it is my interpretation and explains the basic concept behind the move mechanisms. One final mention is that if the object does not have a move constructor or assignment, then the old copy through a temporary mechanism (copy constructor) is used. There are reasons why an object may not have move operations, which will be covered in the next post.
One last note. The implicit sharing feature of the container classes of Qt is another mechanism to make returning instances more efficient. This feature acts like a shared smart pointer. The return value is copied to the temporary, and the use count is incremented. When the local variable is destroyed, the use count is decremented and nothing needs to be destroyed. Shared smart pointers is another mechanism (as already used for RPN item pointers).
When a function is called that has a return value, temporary memory (usually on the stack) needs to set aside hold the return value because the variable holding the return value in called function goes out of scope and will be destroyed (destructor called) if not a plain old data (POD) type. Prior to C++11, the return value was copied from the temporary into a receiving variable using a copy constructor if not a POD type. If the variable was a large container like a list or vector, this was not efficient. This is why RPN lists were returned by a pointer.
However, with a C++11 move constructor, the return value is moved directly into the variable being constructed or assigned, each having a slightly different mechanism. For a container class like a list or vector, the only thing that is moved is the pointer to the data, not the actual data itself. Consider this example of a function that returns a list and code that calls the function:
std::list<SomeType> someFunction()The new C++11 initializer syntax is used to show that the move constructor is called (but the old assignment syntax does the same thing). Without a move constructor, the local list variable destructor would be called after the return value was copied to a temporary. With the move constructor, the return value is moved directly into the receiving variable and the local variable is reset, and when the destructor for the local is called, it has nothing to do. With a container class, minimal values are copied (essentially a pointer with a little other possible housekeeping like a size). A move constructor has this basic layout:
{
std::list<SomeType> localList;
... list created here ...
return localList;
}
...
std::list<SomeType> list {someFunction()}; ← caller
ClassName(ClassName &&other) :First the members of the receiving variable are initialized with another instance (the return variable). In the body of the move constructor, the other members are initialized to defaults. If one of the members was a pointer to allocated data, the pointer is transferred, and the pointer in the return variable that is about to go out of scope is set to null, so when its destructor is called, it has nothing to delete.
member1{other.member1},
member2{other.member2},
...
{
other.member1{};
other.member2{};
...
}
A move assignment is similar, but is used when the receiving variable (that already exists) is being assigned to a return value, where the caller looks like this:
std::list<SomeType> list;In this case, the value in the receiving variable must be destroyed first and then the return value moved in. Without a move assignment, this is handled by the copy constructor, which destroys the old value, and copies the new value. The destructor for the local variable is called. A move assignment has this basic layout:
...
list = someFunction();
ClassName &operator=(ClassName &&other)For each member, the values of the members are swapped. The standard swap function does this efficiently with move operations where a member is moved into a temporary variable, the other member is moved into the current member, and the temporary moved into the other member. No destructors are called for any of the moves. Here, the other instance is the local variable of the function that is going out of scope. What this does is moves the value of the receiving variable into the local variable, which then gets destroyed. As with any assignment, the reference of the object is returned.
{
std::swap(member1, other.member1);
std::swap(member2, other.member2);
...
return *this;
}
The above descriptions may not be exactly what occurs internally, but it is my interpretation and explains the basic concept behind the move mechanisms. One final mention is that if the object does not have a move constructor or assignment, then the old copy through a temporary mechanism (copy constructor) is used. There are reasons why an object may not have move operations, which will be covered in the next post.
One last note. The implicit sharing feature of the container classes of Qt is another mechanism to make returning instances more efficient. This feature acts like a shared smart pointer. The return value is copied to the temporary, and the use count is incremented. When the local variable is destroyed, the use count is decremented and nothing needs to be destroyed. Shared smart pointers is another mechanism (as already used for RPN item pointers).
Saturday, September 6, 2014
RPN List – To Standard List
The RPN List class list member was changed from QList to std::list. The largest part of this change is due to standard lists not being indexed by a simple integer like a QList (which is implemented as an array internally), but with iterators as the standard list is a double linked list internally.
The significant parts of this change include iterating over the items in the list (changing regular indexed for loops to C++11 range-for loops) where the get item at index function was removed and access to the list's constant begin and end iterators was added; updating the RPN List class access functions for adding to or inserting into the list; and updating the INPUT command translate function to handle an iterator for inserting input parse codes into the list instead of an index. Click Continue... for details of these changes.
[branch cpp11 commit 9bca14b4b5]
The significant parts of this change include iterating over the items in the list (changing regular indexed for loops to C++11 range-for loops) where the get item at index function was removed and access to the list's constant begin and end iterators was added; updating the RPN List class access functions for adding to or inserting into the list; and updating the INPUT command translate function to handle an iterator for inserting input parse codes into the list instead of an index. Click Continue... for details of these changes.
[branch cpp11 commit 9bca14b4b5]
RPN List – Base Class To Member
The next step is to replace the use of pointers to RPN lists. Currently, an RPN list is allocated in the translator and decoder, a pointer to which is returned. It is then the caller's responsibility to delete the RPN list. This is the so called naked new and delete that can be problematic (cause memory leaks if not careful). This could be another use for a shared pointer. However, for this case, there is a another better C++11 solution. This change involves several distinct parts.
During these changes, a problem was found upstream with the RPN List class that was misinterpreted as being caused by using a list class as a base class. The list classes (QList or std::list) were not designed to be used as base classes. The problem was perceived to be due to these classes not having virtual destructors, but this was not the case. A bigger issue is that having the list class as a public base class opens up all of the public list class members for access to users of the RPN List class, which is not good object-oriented design.
Therefore, the first part of the next set of changes was to change from using the list class as a base class to having a list member variable. Access functions to the list member were added, which includes getting the list size, checking if the list is empty, getting the token of the last item, getting an item at an index, and clearing the list. There are already access functions for appending items to the list, comparing lists (equality operators), inserting into the list, and converting the list to text (for test output).
The Translator class also contained access functions to the output list (for the various external translate functions) and these were insured to call the RPN List class access functions instead of the list directly. Only the output last token function needed to be modified since it was accessing the list directly.
Finally, the RPN List class destructor was calling the clear function of the list, but this is no longer necessary as this will happen automatically when each member (including the new list member) is destroyed. This was necessary previously because of the problem mentioned above that the base list class destructor was not defined as virtual and therefore was not being called (the RPN List class destructor overrode the list class destructor).
[branch cpp11 commit 32ab971b45]
During these changes, a problem was found upstream with the RPN List class that was misinterpreted as being caused by using a list class as a base class. The list classes (QList or std::list) were not designed to be used as base classes. The problem was perceived to be due to these classes not having virtual destructors, but this was not the case. A bigger issue is that having the list class as a public base class opens up all of the public list class members for access to users of the RPN List class, which is not good object-oriented design.
Therefore, the first part of the next set of changes was to change from using the list class as a base class to having a list member variable. Access functions to the list member were added, which includes getting the list size, checking if the list is empty, getting the token of the last item, getting an item at an index, and clearing the list. There are already access functions for appending items to the list, comparing lists (equality operators), inserting into the list, and converting the list to text (for test output).
The Translator class also contained access functions to the output list (for the various external translate functions) and these were insured to call the RPN List class access functions instead of the list directly. Only the output last token function needed to be modified since it was accessing the list directly.
Finally, the RPN List class destructor was calling the clear function of the list, but this is no longer necessary as this will happen automatically when each member (including the new list member) is destroyed. This was necessary previously because of the problem mentioned above that the base list class destructor was not defined as virtual and therefore was not being called (the RPN List class destructor overrode the list class destructor).
[branch cpp11 commit 32ab971b45]
Thursday, September 4, 2014
Token Text – Standard String
Continuing with the change to the STL, the text function of the Token class was also modified to build the string into a standard string stream. Like with the text function of the RPN List class, the with indexes option argument was not being utilized, so it was removed.
For the Token class, the return value of the text function was changed to a standard string. Callers were modified accordingly for this change. Only one caller remains that expects a QString value, the delete operator function. This function will not been needed once smart (shared) pointers are used for token pointers.
The way the private text operand helper function was used in the text function was changed making a separate function unnecessary. This helper function surrounded the token text with vertical bars and only two call located were left once the with indexes feature was removed. Instead of setting the second string, the string was changed to a flag and at the end of the function, if set, the vertical bars and the token string are added to the string stream.
[branch cpp11 commit 47766746a1]
For the Token class, the return value of the text function was changed to a standard string. Callers were modified accordingly for this change. Only one caller remains that expects a QString value, the delete operator function. This function will not been needed once smart (shared) pointers are used for token pointers.
The way the private text operand helper function was used in the text function was changed making a separate function unnecessary. This helper function surrounded the token text with vertical bars and only two call located were left once the with indexes feature was removed. Instead of setting the second string, the string was changed to a flag and at the end of the function, if set, the vertical bars and the token string are added to the string stream.
[branch cpp11 commit 47766746a1]
Wednesday, September 3, 2014
RPN Item – Index Member and Text Function
The next step will be to use smart pointers for RPN list pointers. In keeping with the change to STL classes, the RPN list will be changed from a QList to a std::list. Even though QList is like std::vector internally (an allocated array), a double linked list container is more appropriate because there is one instance so far (the INPUT command) where RPN items are inserted into the list. Inserting into an array requires moving all the elements from the insertion point.
While reviewing the code for these changes, it was noticed that the index member of the RPN item was only being used for test output. This index is set when an RPN item is appended to the list. If an item is inserted into the list, the index of every item after the inserted item needs to be incremented. This index is only output on attached items as a check to make sure the correct item is attached.
Current, attached RPN items always occur before the item that are attached to, which means the indexes could be assigned to an item as it is converted to text. Therefore the index of each item will be assigned and temporarily held in an unordered map (RPN item pointer to index) instead of assigning an index when the item is appended (and incremented later for inserts). The index was removed from the RPN Item class.
The text function of the RPN List class was modified to handle the conversion of each item to text using access functions of the RPN Item class so that the indexes can be added (from the temporary map) instead of calling the text function of the RPN item (which would not have the temporary indexes). Since the text function of the RPN Item class is no longer needed, it was removed. Also, the with indexes option argument was not being utilized and so this feature was also removed.
The text function was also modified to build the text of the RPN List into a standard string stream (std::stringstream, which works like an output stream but puts the result into a string). This is easier for building a string than using the standard string which is feature limited. For now, when the final standard string is obtained from the string steam, it is converted to a QString (since that it how the caller uses it, for now). Also, the text from the token needs to be converted from a QString to a standard string before adding to the string stream.
[branch cpp11 commit da4fcfbb23]
While reviewing the code for these changes, it was noticed that the index member of the RPN item was only being used for test output. This index is set when an RPN item is appended to the list. If an item is inserted into the list, the index of every item after the inserted item needs to be incremented. This index is only output on attached items as a check to make sure the correct item is attached.
Current, attached RPN items always occur before the item that are attached to, which means the indexes could be assigned to an item as it is converted to text. Therefore the index of each item will be assigned and temporarily held in an unordered map (RPN item pointer to index) instead of assigning an index when the item is appended (and incremented later for inserts). The index was removed from the RPN Item class.
The text function of the RPN List class was modified to handle the conversion of each item to text using access functions of the RPN Item class so that the indexes can be added (from the temporary map) instead of calling the text function of the RPN item (which would not have the temporary indexes). Since the text function of the RPN Item class is no longer needed, it was removed. Also, the with indexes option argument was not being utilized and so this feature was also removed.
The text function was also modified to build the text of the RPN List into a standard string stream (std::stringstream, which works like an output stream but puts the result into a string). This is easier for building a string than using the standard string which is feature limited. For now, when the final standard string is obtained from the string steam, it is converted to a QString (since that it how the caller uses it, for now). Also, the text from the token needs to be converted from a QString to a standard string before adding to the string stream.
[branch cpp11 commit da4fcfbb23]
Monday, September 1, 2014
RPN Item Pointers As Shared Pointers
An attempt was made to replace all the token pointers with a shared pointer class (QSharedPointer). A lot of changes were required that created many compile issues to correct. Worst, when running the first time, there were many crashes that were not being easy to resolve. I believe some of the problems were related to the RPN items (that hold token pointers) stored in the RPN output list. So, these changes were temporarily stashed.
Instead of starting with the token pointers, the simpler RPN item pointers were changed to use QSharedPointer. This was successful, however, I decided to try using the STL std::shared_ptr class (C++11 only) instead . After some testing, it appears that STL classes are faster than the equivalent Qt classes.
Originally, STL classes were avoided in favor of Qt classes, since this is a Qt based GUI application. With the recent desire to use C++11 and considering that the STL classes were enhanced and optimized to use C++11, STL classes will now be used as much as possible except when dealing specifically with GUI elements requiring Qt classes or when Qt classes have features not available in the STL classes (for example, the STL std::string class does not have a case insensitive comparison, but the QString class does).
To start the changes for the RPN item pointers, all instances of RpnItem* were changed to RpnItemPtr, which was defined as an alias (this using syntax is new to C++11 and is just an easier form of the typedef statement):
The RPN item constructor was changed to use the new initializer syntax instead of setting the member inside the constructor body. This is more efficient because otherwise, the default constructor is first called for each non-plain member and then a value is copied into the member. The RPN list class also no longer needed a clear function to delete the RPN items (the items get deleted automatically).
The attached array (with a count) in the RPN item class was naively implemented as a plain C array, which required naked new and delete operations and was changed to a QList. This change eliminates the need for a count variable (QList maintains its size) and also eliminates the need to delete the attached RPN items (automatic when the RPN item is deleted automatically).
When the RPN item pointer was changed to the STL std::shared_ptr class, the changes with QSharedPointer were put on a branch and the branch abandoned. The change in shared pointer class only required minor changes, which included changing the alias above and adding an alias for RPN item pointer vector:
[branch cpp11 commit a2069aae24]
[branch rpnitem-qt-sharedptr commit 7840954d5e]
Instead of starting with the token pointers, the simpler RPN item pointers were changed to use QSharedPointer. This was successful, however, I decided to try using the STL std::shared_ptr class (C++11 only) instead . After some testing, it appears that STL classes are faster than the equivalent Qt classes.
Originally, STL classes were avoided in favor of Qt classes, since this is a Qt based GUI application. With the recent desire to use C++11 and considering that the STL classes were enhanced and optimized to use C++11, STL classes will now be used as much as possible except when dealing specifically with GUI elements requiring Qt classes or when Qt classes have features not available in the STL classes (for example, the STL std::string class does not have a case insensitive comparison, but the QString class does).
To start the changes for the RPN item pointers, all instances of RpnItem* were changed to RpnItemPtr, which was defined as an alias (this using syntax is new to C++11 and is just an easier form of the typedef statement):
using RpnItemPtr = QSharePointer<RpnItem>;Many functions that had an RPN item pointer as an argument were changed to a reference to a RpnItemPtr so that the use count of the isn't unnecessarily incremented when a copy is made for the function call argument (and then decremented when the function returns). If the item is copied inside the function, the use count will get incremented. RPN items no longer need to be deleted as they now will be done automatically when they they go out of scope or their container is deleted or goes out of scope.
The RPN item constructor was changed to use the new initializer syntax instead of setting the member inside the constructor body. This is more efficient because otherwise, the default constructor is first called for each non-plain member and then a value is copied into the member. The RPN list class also no longer needed a clear function to delete the RPN items (the items get deleted automatically).
The attached array (with a count) in the RPN item class was naively implemented as a plain C array, which required naked new and delete operations and was changed to a QList. This change eliminates the need for a count variable (QList maintains its size) and also eliminates the need to delete the attached RPN items (automatic when the RPN item is deleted automatically).
When the RPN item pointer was changed to the STL std::shared_ptr class, the changes with QSharedPointer were put on a branch and the branch abandoned. The change in shared pointer class only required minor changes, which included changing the alias above and adding an alias for RPN item pointer vector:
using RpnItemPtr = std::shared_ptr<RpnItem>;The latter alias is used for the attached member (just changed to a QList) was changed to a std::vector (which is more similar to QList than to std::list as QList is an allocated array, std::list a double-linked list). Also, the size of STL classes is accessed using the size member function compared to the count member functions of the Qt classes.
using RpnItemPtrVector = std::vector<RpnItemPtr>;
[branch cpp11 commit a2069aae24]
[branch rpnitem-qt-sharedptr commit 7840954d5e]
Sunday, August 31, 2014
C++ Smart Pointers
C++ smart pointers are not a feature of the C++ language put a utilization of the language. Consider the function below, which is translation of the C way of handling allocated resources:
Fortunately, very elaborate smart pointer classes have already been implemented. In the C++11 version of the STL, there is the std::unique_ptr class, which is a more elaborate implementation of the class above. Qt has the similar QScopedPointer class.
The C++11 STL also has the std::shared_ptr class, which is a smart pointer that can be shared. These have a use count, which is incremented each time the pointer is copied (for example, put into a different list or stack). When one of the copies is deleted, the use count is decremented. When the use count reaches zero, the resource is automatically released. Qt has the similar QSharePointer class. These shared smart pointers will be used for the tokens, which get shared (in the RPN output list, done stack, holding stack, pending parentheses token, etc.).
bool function()Care must be taken to release the resources that were allocated to prevent memory leaks. This puts the burden on the programmer to make sure resources are released. Consider that if the class instances were local variables, their destructors would be called automatically when they go out of scope. This is the basic idea of a smart pointer, consider this simple smart pointer class:
{
Class *a {new Class};
// ... do some work ...
if (error)
{
delete a1;
return false;
}
Class *b {new Class};
// ... do some more work ...
if (error)
{
delete b;
delete a;
return false;
}
// ... do final work ...
delete b;
delete a;
return true;
}
class SmartPtrThe class holds the pointer to the allocated resource. When the smart pointer instance goes out of scope, its destructor will release the resource it is holding. The smart pointer provides access to the pointer it is holding. This class can be made more elaborate and convenient (for example, allowing the pointer to be replaced, allowing the pointer to be cleared, providing the indirection and arrow operators so that it behaves like a pointer, etc.), but this definition is sufficient to show the basic concept of smart pointers. The original function can be rewritten as:
{
Class *ptr; // hold pointer to allocated resources
public:
SmartPtr(Class *p) : ptr {p} {}
~SmartPtr() { delete ptr; }
const Class *pointer() const { return ptr; }
};
bool function()Note that it is no longer necessary to release the resources. The new calls in the first function are known as naked, each of which requires a corresponding naked delete. In the second function, the smart pointers hold the allocated resources (the new is enclosed). When the smart pointer goes out of scope, it destructor is called releasing the resource.
{
SmartPtr a {new Class};
// ... do some work ...
if (error)
{
return false;
}
SmartPtr b {new Class};
// ... do some more work ...
if (error)
{
return false;
}
// ... do final work ...
return true;
}
Fortunately, very elaborate smart pointer classes have already been implemented. In the C++11 version of the STL, there is the std::unique_ptr class, which is a more elaborate implementation of the class above. Qt has the similar QScopedPointer class.
The C++11 STL also has the std::shared_ptr class, which is a smart pointer that can be shared. These have a use count, which is incremented each time the pointer is copied (for example, put into a different list or stack). When one of the copies is deleted, the use count is decremented. When the use count reaches zero, the resource is automatically released. Qt has the similar QSharePointer class. These shared smart pointers will be used for the tokens, which get shared (in the RPN output list, done stack, holding stack, pending parentheses token, etc.).
Saturday, August 30, 2014
Memory Testing Issues
The next step is to start using C++ smart pointers (described in the next post). One use for smart pointers will be to replace the complicated code used to track the handling of tokens and the reporting of various errors (for example, memory leaks for tokens not freed). Smart pointers will make this code unnecessary. Before starting I thought it was a good idea to make sure the memory test scripts still worked.
There were no memory errors reported when using Qt 4.8.1 (installed with Mint13 or Ubuntu 12.04) or Qt 4.8.2 (installed with the kubuntu backports to get the latest KDE). However, when building the application with Qt 4.8.6 (the latest Qt 4.8 that was installed from source), memory errors were reported. The CMake build file was set up to generate the error suppression file from a template using the detected version and directory of Qt. This file was not tested with newer versions of Qt.
It appeared that with Qt 4.8.6, memory errors were reported differently (which was also the case with Qt 4.8.4, also built from source). After examining the output for each version of Qt, a common error suppressions file template was created that works with each of these four versions of Qt (other versions were not tested).
As with GDB, valgrind (the memory testing utility) only supports the new debug symbols output from GCC 4.8.1 starting with version 3.8.0 (3.7.0 is installed in Mint13). Version 3.9.0 is the latest available from source code. However, version 3.7.0 appears to work fine, including within Qt Creator. The new errors suppressions file also works with valgrind 3.9.0. Click Continue... for details on installing 3.9.0 from source, but this shouldn't be necessary on Mint 13 (Unbuntu 12.04). The valgrind 3.10.0.SVN installed on Mint 17 (Ubuntu 14.04) reports errors differently and does not work with this error suppression file (for now 3.9.0 would have to be installed from source).
[branch cpp11 commit a2069aae24]
There were no memory errors reported when using Qt 4.8.1 (installed with Mint13 or Ubuntu 12.04) or Qt 4.8.2 (installed with the kubuntu backports to get the latest KDE). However, when building the application with Qt 4.8.6 (the latest Qt 4.8 that was installed from source), memory errors were reported. The CMake build file was set up to generate the error suppression file from a template using the detected version and directory of Qt. This file was not tested with newer versions of Qt.
It appeared that with Qt 4.8.6, memory errors were reported differently (which was also the case with Qt 4.8.4, also built from source). After examining the output for each version of Qt, a common error suppressions file template was created that works with each of these four versions of Qt (other versions were not tested).
As with GDB, valgrind (the memory testing utility) only supports the new debug symbols output from GCC 4.8.1 starting with version 3.8.0 (3.7.0 is installed in Mint13). Version 3.9.0 is the latest available from source code. However, version 3.7.0 appears to work fine, including within Qt Creator. The new errors suppressions file also works with valgrind 3.9.0. Click Continue... for details on installing 3.9.0 from source, but this shouldn't be necessary on Mint 13 (Unbuntu 12.04). The valgrind 3.10.0.SVN installed on Mint 17 (Ubuntu 14.04) reports errors differently and does not work with this error suppression file (for now 3.9.0 would have to be installed from source).
[branch cpp11 commit a2069aae24]
Sunday, August 24, 2014
Qt Creator (GDB) No Watch Variables With GCC 4.8
The next set of changes are rather complicated and required debugging. Upon reaching the first breakpoint, no variables were displayed in the Locals and Expressions debugging window in Qt Creator. After a little research, the problem was determined to be with GCC 4.8.1, which is using a new format (named DWARF-4) to write debugging symbols to the executable.
The problem is that GDB (debugger) does not support this newer format, at least older versions prior to GDB 7.5. Mint 13 (Ubuntu 12.04) has only version 7.4. This problem does not affect Windows with the programs installed as recently described (see the Windows tagged posts). The latest MSYS with MinGW has GDB 7.6.1 and MinGW-w64 has GDB 7.7. Mint 17 (Ubuntu 14.04) is also fine with GDB 7.7.
There are two ways to solve this problem. The GCC compiler has an option for generating the older DWARF-3 format debugging symbols. Instead of permanently adding this option to the CMake build file, the following option can be added to the CMake command line or in Arguments field in the Run CMake wizard within Qt Creator:
The problem is that GDB (debugger) does not support this newer format, at least older versions prior to GDB 7.5. Mint 13 (Ubuntu 12.04) has only version 7.4. This problem does not affect Windows with the programs installed as recently described (see the Windows tagged posts). The latest MSYS with MinGW has GDB 7.6.1 and MinGW-w64 has GDB 7.7. Mint 17 (Ubuntu 14.04) is also fine with GDB 7.7.
There are two ways to solve this problem. The GCC compiler has an option for generating the older DWARF-3 format debugging symbols. Instead of permanently adding this option to the CMake build file, the following option can be added to the CMake command line or in Arguments field in the Run CMake wizard within Qt Creator:
-DCMAKE_CXX_FLAGS_DEBUG='-g -gdwarf-3'The other solution is to build and install GDB from source code. Click Continue... for details on the procedure for this.
Thursday, August 21, 2014
Token – Status As Enumeration Class
I misinterpreted how the compiler generates code for switch statements (caused by looking at disassembled output instead of assembler output of the compiler). The compiler does not generate an if-elseif chain in circumstances where it doesn't simplify a switch statement to an array of return values (when the return values are simple types as previously described).
The compiler still generates an array for a switch statement, but instead of an array of return values (of simple types), it generates an array of pointers. Each pointer is the address to the code for the case. During run-time, the processor indexes on the expression of the switch into this array and jumps to the address. While this is not space efficient, it is run-time efficient.
Unfortunately, for the token status enumeration, a single C style string constants cannot be used because each must be the result of the tr() translation function. This was not necessary for the other enumeration to string functions because these are only used for testing (translation is not necessary).
The switch statement for converting a token status enumerator was put into the message function of the Token class. This function previously returned an element from the static message array (which was removed). The status enumeration was put into the Token class as an enumeration class therefore requiring the Token::Status:: scoping prefix on the status enumerators. The _TokenStatus suffix was removed from each (the bug statuses did not have this prefix, but now require the scoping prefix).
The generation of the token status enumeration was removed from the auto-generation awk script and the token source file was removed as a dependency to the auto-generated enumerations header file. Several of the token status enumerators were not being used and were removed. The only auto-generated code remaining is for the code enumeration and the code enumerator to code name array. These will be handled later when the Table class is redesigned.
[branch cpp11 commit cfed68f09b]
The compiler still generates an array for a switch statement, but instead of an array of return values (of simple types), it generates an array of pointers. Each pointer is the address to the code for the case. During run-time, the processor indexes on the expression of the switch into this array and jumps to the address. While this is not space efficient, it is run-time efficient.
Unfortunately, for the token status enumeration, a single C style string constants cannot be used because each must be the result of the tr() translation function. This was not necessary for the other enumeration to string functions because these are only used for testing (translation is not necessary).
The switch statement for converting a token status enumerator was put into the message function of the Token class. This function previously returned an element from the static message array (which was removed). The status enumeration was put into the Token class as an enumeration class therefore requiring the Token::Status:: scoping prefix on the status enumerators. The _TokenStatus suffix was removed from each (the bug statuses did not have this prefix, but now require the scoping prefix).
The generation of the token status enumeration was removed from the auto-generation awk script and the token source file was removed as a dependency to the auto-generated enumerations header file. Several of the token status enumerators were not being used and were removed. The only auto-generated code remaining is for the code enumeration and the code enumerator to code name array. These will be handled later when the Table class is redesigned.
[branch cpp11 commit cfed68f09b]
Wednesday, August 20, 2014
Token – Type As Enumeration Class
A series of changes were made to change the token type enumeration to an enumeration class because its size of enumerator was used to dimension several arrays. The changes were put into several individual commits on a work branch, which were later combined (squashed using the interactive git rebase command) into a single commit and merged back to the cpp11 branch before pushing to GitHub. Details of these changes follow.
1) Unrelated to the token type enumeration, the Table class convert code function was moved to the Token class. This function contained arguments for a token with a data type and the desired data type to convert to. It returned a conversion code. However, it did not actually access the table. Since codes are not part of the table (the table is only indexed by codes) and this function took a token pointer, it made more sense for this to be a Token member function.
2) Also unrelated to the token type enumeration, the auto-generated data type to string map was changed to a brute force static function with a switch statement in the test source file as described in the previous post. (The compiler in-lined this function when set to release build.) The generate map function was removed from the test names awk script and the dependency on the main header file was removed from the auto-generated test names source file from the CMake build file.
3) Two of the uses of the token type enumerator was the static has parentheses flag and precedence arrays used to determine if the token type has a parentheses and the precedence of the token type. These arrays were initialized by the static initialize function. These arrays were changed to unordered maps with initializer lists for their values. Note that the maps do not contain values for all the token types. When the map is accessed for one of these other token types, a new element will be added with the correct default values (false for the flag, zero for the precedence). Since these maps are initialized, the initialize function was no longer needed and was removed.
4) The token type enumeration was changed to an enumeration class and was moved into the Token class where it belongs. Outside of the Token class, the token type enumerators need a double scope as in the Constant_TokenType enumerator becomes Token::Type::Constant.
5) The auto-generated token type to string array was changed to a brute force function also. (The compiler also in-lined this function.) The find enumeration function in the test names awk script was now no longer used and so was removed and the dependency on the token header file was removed from the auto-generated test names source file from the CMake build file.
[branch cpp11 commit e7dc6a964a]
1) Unrelated to the token type enumeration, the Table class convert code function was moved to the Token class. This function contained arguments for a token with a data type and the desired data type to convert to. It returned a conversion code. However, it did not actually access the table. Since codes are not part of the table (the table is only indexed by codes) and this function took a token pointer, it made more sense for this to be a Token member function.
2) Also unrelated to the token type enumeration, the auto-generated data type to string map was changed to a brute force static function with a switch statement in the test source file as described in the previous post. (The compiler in-lined this function when set to release build.) The generate map function was removed from the test names awk script and the dependency on the main header file was removed from the auto-generated test names source file from the CMake build file.
3) Two of the uses of the token type enumerator was the static has parentheses flag and precedence arrays used to determine if the token type has a parentheses and the precedence of the token type. These arrays were initialized by the static initialize function. These arrays were changed to unordered maps with initializer lists for their values. Note that the maps do not contain values for all the token types. When the map is accessed for one of these other token types, a new element will be added with the correct default values (false for the flag, zero for the precedence). Since these maps are initialized, the initialize function was no longer needed and was removed.
4) The token type enumeration was changed to an enumeration class and was moved into the Token class where it belongs. Outside of the Token class, the token type enumerators need a double scope as in the Constant_TokenType enumerator becomes Token::Type::Constant.
5) The auto-generated token type to string array was changed to a brute force function also. (The compiler also in-lined this function.) The find enumeration function in the test names awk script was now no longer used and so was removed and the dependency on the token header file was removed from the auto-generated test names source file from the CMake build file.
[branch cpp11 commit e7dc6a964a]
Tuesday, August 19, 2014
Better Enumerators To Strings Solution
Several enumerations need to be converted to strings for output during testing. This was accomplished with an awk script that scanned source files for the enumerations and automatically generated a source file that contained C style string arrays of the enumerator names, which were indexed by an enumerator. With enumeration classes, the enumerators can not be used as indexes to arrays. Another solution was to generate an unordered map that can be indexed by the enumerators provided that a generic hash is defined for enumeration classes.
The method of automatically generating source code from source code is kludgy and should be eliminated. Also, changing a source file read by an awk script forces the entire project to be rebuilt since all the source files are generally dependent on the auto-generated source file. A different solution was needed.
Another solution is using a function that takes an enumerator value as input and returns the string using the brute force method with switch statement and returning a string for each case, for example:
However, if the return type is QString and each return value is QString("Value1"), the compiler no longer generates an array, but instead an if-elseif chain. Though note that the actual return value can still be a C style string where the QString constructor is called to create the QString return value. (This information was acquired by looking at the assembly code generated.)
On the other hand, if the return value is a standard string (STL std::string), then the compiler does generate an array. This probably has something to do with the fact that the standard string class supports move constructors (a C++11 addition not discussed here). The QString class does not support these (at least Qt4 classes no not, but Qt5 classes do). The problem with using standard strings is that the return values of these functions are used with Qt stream classes, which do not support standard strings. Therefore, C style strings will be used for now.
Functions like the example above will be used. These functions could be auto-generated, but again this practice will no longer be used. The nice thing about auto-generation is that it eliminates the mistake of missing an enumerator, because with all warnings enabled (and as errors), if an enumerator is missing, a compiler warning is issued for the switch statement that not values are handled (provided no default case is included). This is not a perfect solution, but probably the best within the limits of the C++ language.
There is one final issue with the function above. The compiler sees that there is no return value at the end of the function and issues a warning (error). The compiler is apparently not smart enough to realize that execution does not get past the switch statement since all cases return. To silence this warning, a return ""; statement is needed at the end of the function.
The method of automatically generating source code from source code is kludgy and should be eliminated. Also, changing a source file read by an awk script forces the entire project to be rebuilt since all the source files are generally dependent on the auto-generated source file. A different solution was needed.
Another solution is using a function that takes an enumerator value as input and returns the string using the brute force method with switch statement and returning a string for each case, for example:
const char *enumName(Enum value)
{
switch (value)
{
case Enum::Value1:
return "Value1";
case Enum::Value2:
return "Value2":
...
case Enum::ValueN:
return "ValueN":
}
}
A good optimizing compiler will convert this switch statement to an array instead of a series of an if-elseif chain, which the GCC compiler does at the higher optimization levels. However, there are some conditions that are required. First, the compiler will only generate a look up array if the function is static (local to the source file where used). Also, the return values need to be relatively simply types.However, if the return type is QString and each return value is QString("Value1"), the compiler no longer generates an array, but instead an if-elseif chain. Though note that the actual return value can still be a C style string where the QString constructor is called to create the QString return value. (This information was acquired by looking at the assembly code generated.)
On the other hand, if the return value is a standard string (STL std::string), then the compiler does generate an array. This probably has something to do with the fact that the standard string class supports move constructors (a C++11 addition not discussed here). The QString class does not support these (at least Qt4 classes no not, but Qt5 classes do). The problem with using standard strings is that the return values of these functions are used with Qt stream classes, which do not support standard strings. Therefore, C style strings will be used for now.
Functions like the example above will be used. These functions could be auto-generated, but again this practice will no longer be used. The nice thing about auto-generation is that it eliminates the mistake of missing an enumerator, because with all warnings enabled (and as errors), if an enumerator is missing, a compiler warning is issued for the switch statement that not values are handled (provided no default case is included). This is not a perfect solution, but probably the best within the limits of the C++ language.
There is one final issue with the function above. The compiler sees that there is no return value at the end of the function and issues a warning (error). The compiler is apparently not smart enough to realize that execution does not get past the switch statement since all cases return. To silence this warning, a return ""; statement is needed at the end of the function.
Sunday, August 17, 2014
More Enumeration Classes
Several more enumerations were changed to enumeration classes. While more of these included a size of enumerator, they were not being used. This included the translator test mode, translator reference type, dictionary entry type, error list type, program model operation type, and the table multiple enumerations. Two instances where the enum keyword was in front of the enumeration name were removed (this is something required for C but not C++).
This leaves several enumerations that are not as easy as adding the class keyword and renaming the enumerators like those changed above. These include the code (auto-generated), token status (auto-generated), sub-code (bit masks), table flag (bit masks), table search type, token type, and test option enumerations.
The auto-generated enumerations need a good C++ solution and not the current kludgy C solution using an external awk script. The bit mask enumerations will need operator functions implemented (bit-wise OR and AND), which will require using type casting. For the others, the size of enumerators are used for either dimensioning arrays or for looping over the enumerators.
[branch cpp11 commit e71fe6f0fe]
This leaves several enumerations that are not as easy as adding the class keyword and renaming the enumerators like those changed above. These include the code (auto-generated), token status (auto-generated), sub-code (bit masks), table flag (bit masks), table search type, token type, and test option enumerations.
The auto-generated enumerations need a good C++ solution and not the current kludgy C solution using an external awk script. The bit mask enumerations will need operator functions implemented (bit-wise OR and AND), which will require using type casting. For the others, the size of enumerators are used for either dimensioning arrays or for looping over the enumerators.
[branch cpp11 commit e71fe6f0fe]
Subscribe to:
Posts (Atom)