Previous: Units	Next: Publishing / Subscribing

Overview

The framework provides the functionality for different software modules to communicate with each other in an easy and safe way. The data is stored and exchanged via named channels. Each channel has a (globally) unique name and a type, i.e. when storing data in the channel the type information will be maintained.

The channel concept is designed to fulfill the following main design goals:

maintain type information of data that is stored in a channel to avoid dynamic_casts when retrieving the data
hide data locking from the user so that the user is not responsible for locking and unlocking of data
avoid unnecessary copying of data (images, etc.)
avoid blocking whenever accessing data (reading or writing)
allow read access to past data (history), in order to allow data synchronization
interpolate when accessing data in the past and allow the user to specify a user-defined interpolator
data can be exchanged between different processes via TCP

For a detailed list of requirements see Framework (Requirements).

Channels and Slots

A channel is a typed container that stores data and is used to exchange data between different software modules where some modules act as writers/producers of the data while others read the data.

Internally a channel is a queue of data. Each queue entry is called a slot. It is large enough to store one element of the data. The maximum number of slots can be specified via a configuration file (see <channel> tag).

The channel concept was designed to allow both a communication between modules within a single process (intra-process) and between different processes (inter-process) that even may be located on different machines. For the software modules it is fully transparent whether they communicate intra- or inter-process. This transparency was a major design goal for the channel concept since it allows to decide at configuration time or even runtime where to place a software module without the need to modify the module. This is an important advantage compared to other middleware (e.g. ROS).

Locking and Synchronization

Whenever data is exchanged through channels within the same process, the modules share the data by directly accessing the same memory where the data is located in to achieve maximal performance. The framework automatically takes care of locking the data and to protect it from concurrent access.

Whenever a module wants to write data, the framework first looks for a non-blocked slot, i.e. a slot where no one is reading from or writing to. This slot is locked automatically and the framework returns a write-accessor to the data, where the user can immediately start to write his/her data. If no free slot is available, a new slot is created and the channel's buffer will grow. As stated above, the maximum number of slots can be specified by the <channel> tag within the configuration file. It is set to 100 by default. After finishing writing, the slot becomes the newest element for the channel. When data is read from the channel, the framework read-locks the newest slot and returns a read-accessor.

This concept avoids blocking the writer/publisher of the data even while other readers are still consuming. Vice versa a reader will always be able to obtain data for reading without blocking. Additionally, the user can operate directly on the memory of the slot when reading or writing. Therefore, copying of data is avoided.

What Kind of Data can be Stored in Channels?

Channels can store and transmit virtually any kind of data and complex objects. The only requirements are that the data type must be default-constructible and it must be serializable, i.e. it must provide an intrusive or non-intrusive reflect method (see Serialization). The Framework uses the Serialization Framework to serialize the data into binary data when transmitting data between different processes or when writing data to a tape file.

Suppose you have a class containing an integer member and you want to use it in a channel. In this case you just have to add a reflect method reflecting the integer member.

class MyOwnChannelData
{
public:
    template<typename Reflector>
    void reflect(Reflector& r)
    {
        r.member("Member", someMember, "Comment on member");
    }
    
    int someMember;
};
// now you can use this class in channels
class MyUnit : public MicroUnit
{
...
    void initialize()
    {
        publish<MyOwnChannelData>("MyOwnChannel");
    }
...
};

Beside objects, channels can also store polymorphic pointers. See Polymorphic Channels for more details.

To summarize, some typical examples of data that can be stored is given in the following list:

built-in datatypes: int, float, etc.
STL-datatypes: string, vector, list
complex objects whose classes provide a reflect method
pointers to objects of polymorphic classes that are derived from mira::Object (see Polymorphic Channels)

Stamped Data

The data that is stored in the channel is Stamped both temporally and spatially, i.e. it is associated with a timestamp and a frame id that specifies the position within the global transformation tree.

Timestamp

The timestamp should always contain the time when the data was created or received. For example a unit that acts as a camera driver and publishes a channel that contains the camera image should always set the timestamp to the creation time of the current frame. For hardware driver units publishing sensor/hardware data this should be done as accurate as possible (e.g if the hardware sends an accurate and synchronized timestamp use this one instead of the time the data was received by the driver). The more accurate the timestamps of hardware data are, the better data from different sensors can be matched (e.g. when looking for the odometry data at the time a camera image was received).

When modifying data from sensors (resize an image from a camera channel, add information to sensor data) keep the original timestamp if possible. Only change the timestamp when creating completely new information out of existing data.

When data with a timestamp older than the newest data in the channel is published, it is sorted into the internal slot list according to the temporal order (see History of Data), but subscribed callbacks will not be called. If data with a timestamp that already exists in the channel is published, it gets dropped. In contrast to adding data in the past it is not allowed to alter data in the past.

For convenience: When a slot is requested by calling write() the timestamp of this slot automatically is set to Time::now().

Frame ID

The frame id is the name of the node in the transform tree the channel is linked with. This allows to maintain a spatial relationship between different channels that can be used to transform the position/coordinate frame of data of one channel to the coordinate frame of another channel at a given timestamp. You can think of one frame as if the data of your channel is centered/relative at/to its own coordinate system. When inserting this frame into the transform tree it gets linked to other coordinate frames and allows other users to ask for the position of a laser scan relative to the base frame of the robot or to the location of the robot in a map. For more information on the transformation framework see Transformation Framework.

Sequence ID

The sequence id is an unsigned int (32bit) value that can be freely set by the user. It can be used for tracking data from different publishers (e.g. by assigning every publisher a sequence id to filter messages from a specific publisher) and so on.

Stamped standard datatypes

A class or complex data type can be made "stamped" by using mira::Stamped. However for standard data types we can not use this functionality as Stamped uses inheritance and we can not derive a class from a standard data type (e.g. int) in C++. In this case the class mira::StampedPrimitive is used. For this reason stamped data of standard data types in channels behaves a bit different as stamped data of complex types. In fact the stamped data of a complex class A (Stamped) is derived from A and therefore allows access to all public members and member functions of A directly. Stamped data of standard data types holds the data internally and allows to be transformed to the underlying data type via the implicit cast operator. Both types of Stamped provide a value() method to access the underlying data directly. For Polymorphic Channels the Stamped data type is only supported for polymorphic classes that inherit directly or indirectly from mira::Object.

Access to the data

One can access the timestamp, frame id and sequence id of a stamped data object via the public members. The value of the underlying data type can be accessed by using the value() function or by casting (explicit or implicit) the stamped object to this type.

History of Data

The slots in the channel's queue are kept in the right temporal order, to allow read access to data with a timestamp in the past or even reading whole intervals of data. The framework tries to keep the number of used slots at the minimum required to facilitate the data exchange between producers and consumers. Therefore, as long as readers are only fetching new data at any time, usually only 1-3 slots exist in the channel (for concurrent reading and writing). Slots which were delivered to all subscribed readers are re-used by the writer (therefore, overwritten).

When a module requests access to data from the past (lets say 2 seconds ago) for the first time, no slot will exist anymore containing that data. However, the channel will notice the request and subsequently extend its internal history to provide as many slots as needed to represent the requested timespan (e.g. from 2 seconds ago until now). It will create new slots for every data that gets written to the channel until the requested history length is met. Every time a module requests access to an even older slot the history is extended again, until a maximum slot size is reached.

Polymorphic Channels

One of the biggest advantage of our channel concept is that channels are typed. This means that we don't need a meta description of the data in the channels. All that is needed is that types that are used in channels have an intrusive or nonintrusive reflect method (see What Kind of Data can be Stored in Channels?).

Typed channels also allow us to use pointers to polymorphic objects as data types. All that is needed is that the classes are inherited from mira::Object.

Imagine the following class hierarchy:

class A : public mira::Object...

class B : public A....

Then one could publish a channel containing B* and another one only interested in A* could subscribe to the same channel. This allows us to extend the class hierarchy without changing the subscriber that can only work with the base class (e.g. A).

The following rules apply to polymorphic channels:

Any pointer type to be used as channel type must be a pointer to a (direct or indirect) subclass of mira::Object. This is enforced at compile time. Any pointer type channel is polymorphic.

Publishers and subscribers of polymorphic channels can use different types within a class hierarchy to publish/subscribe. The channel type is 'fluid' as long as there is no publisher and no data written to the channel, but all types used to publish/subscribe must be bases/subclasses of each other. In particular they cannot be subscribed or published with a non-pointer type. The channel type cannot be queried as long as it is not fixed. The first publish() (or the first ChannelWrite::writeSerializedValue()) fixes the channel type, to the most derived of all publisher/subscriber types at that time. Once it is fixed, new subscribers/publishers can be registered with the fixed type or any of its base classes, but not with a more derived type. As a special case, the same authority can publish a channel multiple times, but only with the same type.

Polymorphic channels act like dynamic_cast<> when exchanging data (in fact internally dynamic_cast is employed): When publishing a Derived object, a Base subscriber will receive a Base pointer to the Derived object. When publishing a Base object, a Derived subscriber will receive a null pointer. Note that publishers can publish objects of different type than their publish type, but they will always do so through pointers to their publish type. The dynamic_cast semantics always apply to the actual object type.

Storing raw pointers on channels has the drawback that there is no clear lifetime policy for the objects these pointers refer to. In order to avoid accessing memory that has been freed already, either objects must be allocated on the heap (using new) and never deleted (which means leaking memory with each published object!), or there cannot be more than one subscriber, who will take ownership of the object. That is a significant limitation of the channel concept. Thus, the use of polymorphic channels is limited to very specific scenarios and cannot be recommended in general.

Example base publisher + derived subscriber:

Channel<Base*> channelBase       = subscribe<Base*>("Channel");    // promote to Base*
Channel<Derived*> channelDerived = subscribe<Derived*>("Channel"); // further promote to Derived*
Channel<Base*> channelBase2      = publish<Base*>("Channel");      // fix -> Derived*
Typename t = channelBase.getTypename();                            // --> "Derived*"
// different authority:
subscribe<SubDerived*>("Channel");                                 // --> XBadCast
publish<SubDerived*>("Channel");                                   // --> XBadCast
channelBase.post(new Base());
{
    ChannelRead<Base*> readBase = channelBase.read();
    const Base* b = readBase->value();                             // --> Base object
    ChannelRead<Derived*> readDerived = channelDerived.read);
    const Derived* b = readDerived->value();                       // --> nullptr
}
channelBase.post((Base*)(new Derived()));
{
    ChannelRead<Base*> readBase = channelBase.read();
    const Base* b = readBase->value();                             // --> Derived object
    ChannelRead<Derived*> readDerived = channelDerived.read);
    const Derived* b = readDerived->value();                       // --> Derived object
}

Example derived publisher + base subscriber:

Channel<Base*> channelBase       = subscribe<Base*>("Channel");     // promote to Base*
Channel<Derived*> channelDerived = subscribe<Derived*>("Channel");  // further promote to Derived*
Channel<SubDerived*> channelSub  = publish<SubDerived*>("Channel"); // fix -> SubDerived*
Typename t = channelBase.getTypename();                             // --> "SubDerived*"
// same authority:
Channel<Derived*> channelDerived2 = publish<Derived*>("Channel");   // --> XLogical
// other authority
Channel<Derived*> channelDerived3 = publish<Derived*>("Channel");
Typename t = channelDerived3.getTypename();                         // --> "SubDerived*"
channelSub.post(new SubDerived());
{
    ChannelRead<Base*> readBase = channelBase.read();
    const Base* a = readBase->value();                              // SubDerived object
    ChannelRead<Derived*> readDerived = channelDerived.read();
    const Derived* derived = readDerived->value();                  // SubDerived object
}
channelSub.post(new Derived());
{
    ChannelRead<Base*> readBase = channelBase.read();
    const Base* a = readBase->value();                              // Derived object
    ChannelRead<Derived*> readDerived = channelDerived.read();
    const Derived* derived = readDerived->value();                  // Derived object
}
channelSub.post(new Base());
{
    ChannelRead<Base*> readBase = channelBase.read();
    const Base* a = readBase->value();                              // Base object
    ChannelRead<Derived*> readDerived = channelDerived.read();
    const Derived* derived = readDerived->value();                  // nullptr
}

Events

Many application development frameworks use events to exchange data between modules or signal states or state and data changes (e.g. Windows Message, QT,...). Our channel concept can be used to exchange the same data and signals using the publisher/subscriber pattern. Event senders publish channels and notify subscribed event listeners by writing data to these channels.

Next: Publishing / Subscribing

Contents