Dynamic properties

Introduction

The reflection of properties is special as these are in general declared to allow access to the reflected variables or functions at an arbitrary time, not just at the time of reflection. Usually this happens through the construction of a persistent PropertyNode by the property reflector, which holds references to these variables/accessors. An entity like a graphical property editor can then use the PropertyNode to permanently present the available properties to the user, read their current values or write to those property variables on changes (or call the respective accessor functions).

What if those variables are not persistent themselves? This might sound like an exotic case at first, but there are common examples: Consider a container of reflectable elements. Each element is a unique property (named 'item') of the container, allowing to change it (or its subproperties) at runtime. What if the container size is not constant, but changes over time? Beside the need to add properties when new elements are inserted, some of the elements may get erased or re-allocated to different memory locations, making uncontrolled access through original references dangerous: a list will free memory for erased elements, a vector may even have to relocate its existing elements when a new element is inserted. In both cases stored references to these elements (or their properties) will now point to memory not belonging to the container anymore, and any access very likely means undefined behaviour (e.g. segfault, data corruption).

For this reason, it is vital to manage dynamic properties, making sure that

1. the PropertyNode is updated when the set of properties of an object changes
2. the PropertyNode avoids invalid memory access

This is facilitated by the PropertyNode class and its descendants, together with the PropertySerializer.

When the PropertySerializer serializes an object, it creates a tree of PropertyNodes for the properties the object declares (by calling Reflector::property() in its reflect() method): The tree root corresponds to the object itself, each node holds a reference to the respective variable/accessor and PropertyNodes children for the properties of that object (this recursion ends with atomic elements).

PropertyNode synchronization

When the properties structure changes, making the PropertyNode incomplete or outdated, it can be updated by PropertyNode::synchronize(). This will automatically employ the PropertySerializer again, which will in turn call the reflected object's reflect() method to update the PropertyNode.

(This works best when the object who's property structure changes is also the owner of the created PropertyNode, and that is how it is implemented e.g. in Units).

PropertyNodeListener

During synchronization, the existing tree structure remains unchanged for properties that do not change either. If properties were added, new child PropertyNodes are added to the respective parents. If properties were removed, the corresponding nodes are deleted. In particular in the latter case, it is important to make changes known to "consumers" of the PropertyNode (e.g. a graphical property editor), to ensure those do not try to access the deleted node anymore.

For this purpose, the PropertyNode class defines a PropertyNodeListener interface. Implementations of PropertyListener must implement methods like beginAddChildren()/endAddChildren() and beginRemoveChildren()/endRemoveChildren(), and they can register themselves as listeners at the PropertyNode, using RootPropertyNode::registerListener(). When the PropertySerializer adds or removes nodes from the PropertyNode tree during reflection of an object, it will call the respective methods of all registered listeners. The implementation makes sure that endRemoveChildren() is signalled to all listeners immediately before the node(s) are deleted. Listeners must not try to access those nodes afterwards.

Listeners are registered once for the entire tree. To facilitate this, a special RootPropertyNode class is defined, which handles the listeners. If required, such a RootPropertyNode must be created separately by the PropertyNode's owner and added as an extra parent to the root (tip of the tree). Again, this is what is already implemented for Units. Access to this RootPropertyNode is possible from every PropertyNode in a tree by getRootNode(). This will either return the special root node (by recursively going up the tree through parents to the top), or NULL if none exists (imposing minor performance overhead for accessing the root node, but avoiding to store an extra pointer to it from every PropertyNode in the tree).

Access and locking

Proper use of the PropertyListener mechanism ensures that consumers of the PropertyNode are signalled changes during synchronization. However, when the underlying property variables become invalid, it is necessary to make sure they are not accessed even before synchronization can be performed.

Imagine the case of a list of integer values as property. There may be an operation that potentially changes the elements of the list. When that operation is performed (and possibly only when addition or removal of an element actually happened), the owner of both the list and the PropertyNode will synchronize the PropertyNode. But since all this operation may run in its own thread (most likely within a Unit::process() method), another module (e.g. a GUI with a property editor widget) might in parallel try to read the value of an element of the list through the PropertyNode, between the change of list elements and the subsequent synchronization. Without taking measures, it might end up trying to access an element that has already been removed from the list, resulting in undefined behaviour.

To avoid invalid access, in addition to signalling changes in the PropertyNode, it is necessary to lock access to the underlying data between the thread making changes to the property variable and the thread accessing it through the PropertyNode. This locking mechanism is also provided by the RootPropertyNode, which contains a mutex element and provides lock() and unlock() methods. All PropertyNode methods that internally access the stored variable reference for reading or writing will lock the RootPropertyNode's mutex first. When the variable's owner starts a method that will potentially remove elements or otherwise make stored references invalid, it should also lock the mutex for the time of this operation and until after the PropertyNode is synchronized (again we are assuming it also has hold of the PropertyNode). Obviously, locking works on the scope of the entire property tree, which was chosen as the more lightweight solution over maintaining individual mutex objects for each PropertyNode in the tree.

There are 2 more refinements to the locking mechanism described above: First, internally the PropertyNode methods will not really just lock the mutex (and wait until they have locked it). Instead, in order to avoid the risk of deadlocks, they will just try to lock the mutex with a short timeout. If locking fails to succeed within the time given, they will throw an exception, indicating that access has failed, and the caller must deal with it (for read access they will even keep a backup of the last successful read and return that value when locking fails). Second, locking only actually is attempted for properties that are explicitly marked to require it by setting the REFLECT_CTRLFLAG_VOLATILE flag in the property declaration (in the reflect() method). The PropertySerializer will mark the PropertyNode for the respective property and all its descendants (this property and its child properties and so on) as volatile, which will activate the locking on access. The REFLECT_CTRLFLAG_VOLATILE is set for all container elements in the mira serialization adapters for std and Qt containers, so any code using container type properties does not have to take care of that.

Making use of dynamic properties in own code

Example implementation with simple property provider and consumer:

#include <boost/interprocess/sync/scoped_lock.hpp>
#include <serialization/PropertyNode.h>

class PropertyProvider
{
public:
    PropertyProvider()
    {
        PropertySerializer s;
        mProperties = s.reflectProperties(getGlobalID(), this);
        mPropertiesRoot.addChild(node);
    }

    template<typename Reflector>
    void reflect(Reflector& r)
    {
        MIRA_REFLECT_BASE(r, Unit);
        r.property("Vector", mVector, "a vector that changes size", std::vector<int>()
                   /*, PropertHint(),
                   REFLECT_CTRLFLAG_VOLATILE*/); // volatile flag not required for std::vector,
                                                 // but for other dynamic structures it may be
                                                 // critical (see paragraph above)
    }

    virtual void process(const Timer& timer)
    {
        boost::interprocess::scoped_lock<RootPropertyNode> lock(mPropertiesRoot);

        mVector.push_back(mVector.size()); // just grow infinitely ...
        mProperties->synchronize();
    }

    PropertyNode* getProperties() { return mProperties; }

protected:
    std::vector<int> mVector;

    RootPropertyNode mPropertiesRoot;
    PropertyNode* mProperties;
};



class PropertyConsumer : public PropertyNodeListener
{
    ~PropertyConsumer()
    {
        foreach (auto p, mProperties)
        {
            RootPropertyNode* root = p->getRootNode();
            if (root)
                root->unregisterListener(this);
        }
    }

public:
    void addProperty(PropertyNode* property)
    {
        RootPropertyNode* root = property->getRootNode();
        if (root)
            root->registerListener(this);

        mProperties.push_back(property);
    }

public:
    void showProperties()
    {
        boost::mutex::scoped_lock lock(mMutex);
        foreach (auto p, mProperties)
            showProperty(p);
    }

protected:
    void showProperty(PropertyNode* property)
    {
        std::cout << property->fullID() << ": "
        try
        {
            std::cout <<  property->getAsString() << std::endl;
        }
        catch(XIO& ex)
        {
            std::cout << "exception:" << ex.what() << std::endl; // e.g. "Property not available"
        }

        foreach (auto p, property->children())
            showProperty(p);
    }

protected:
    virtual void beginAddChildren(const PropertyNode* node, int index, int count) { mMutex.lock(); }
    virtual void endAddChildren() { mMutex.unlock(); }

    virtual void beginRemoveChildren(const PropertyNode* node, int index, int count) { mMutex.lock(); }
    virtual void endRemoveChildren() { mMutex.unlock(); }

    virtual void beginMoveChildren(const PropertyNode* node, int index, int count, int destination) { mMutex.lock(); }
    virtual void endMoveChildren() { mMutex.unlock(); }
    
protected:
    std::vector<PropertyNode*> mProperties;

    boost::mutex mMutex; // prevent traversing the properties tree while it is being modified
};

For implementing dynamic properties in a MicroUnit or subclass, RootPropertyNode mPropertiesRoot as well as boost::shared_ptr<PropertyNode> mProperties are already defined, the implementation of dynamic properties boils down to

#include <boost/interprocess/sync/scoped_lock.hpp>

void MyUnit::myMethod()
{
    boost::interprocess::scoped_lock<RootPropertyNode> lock(mPropertiesRoot);

    // operation potentially removing or adding variables exposed as properties,
    // such as elements of a container property
    ...

    getProperties()->synchronize();
}