Overview

A tape is a binary file that contains recorded data of one or multiple channels. Tapes can be recorded whenever channel messages should be stored in a temporal order for playing them back at a later time (e.g. when hardware parts that would normally provide that data are not available, or just for repeated experiments using the same data).

Because of the fact that data in tape files is indexed they provide fast access to stored messages within the tape file at any specific recording time.

In order to save space on your hard drive, each channel message in the tape can be compressed. Tapes use lossless compression, leading to a little overhead when compressing very small messages like single integer or floating point data. Therefore, compression should only be enabled for channels that carry a large amount of data (like range scans, images, ...).

Messages in tapes are clustered into message blocks for speeding up accessing/loading times. Each message block has a maximum size and consists of messages where their sizes sum up to the maximum block size. When accessing a single message in the tape, the whole block that contains the message will be read from file into memory. This speeds up the access to the next message in the temporal order.

Recording and playing tapes

Tapes can be recorded using either the TapeRecorder view in miracenter, using the miratape tool or using an instance of the mira::TapeRecorder class directly.

For playing back tapes one can use either the TapePlayer view in miracenter, using the miratape tool or using an instance of the mira::TapePlayer class directly.

For very low level access to tapes one can use the mira::Tape and mira::TapeVisitor classes.

Format of the tape files

The format of tapes is very simple. It consists of a header and several chunk fields with their respective headers.

// a tape with one message has the following content

<file_header><field_header><message_block><field_header><channel_info><field_header><message><field_header><message>...<field_header><message><field_header><index>

In the following the format is listed for all data fields including their size (bytes). Some fields do not have a fixed size, typically when containing strings, e.g. for names and types. These, in order to allow iteration of the binary data, store the size of the field before the actual field content. E.g. in the description below <name>(size(4) + size bytes) means the 'name' field has a variable length of 4 + size, where size is stored in the first 4 bytes, and the actual name is stored in the following [size] bytes.

The file header

The file header tells the version, time of recording and number of message blocks in that file. Additionally, it has a link to the first channel info field.

<version>(4)
<time of recording in nanoseconds since 00:00:00 01.01.1970>(8)
<time difference of recording machine to UTC time in nanoseconds>(8)
<number of message blocks>(4)
<offset to first channel info field>(8)

The field header

Each of the chunk fields - except the file header field - has a header. This header specifies the type and the size of the field.

<type of field>(1)
<size>(4)

The message block field

A message block field holds information about the number of message fields between the message block and the next message block or the end of the tape. It acts as a meta field describing the size of the block, the time of the first message and the time of the last message in this block - both relative to the start time of the tape given in nanoseconds.

<number of messages>(4)
<size of the message block(size of all message and info chunks it contains)>(4)
<time of first message in nanoseconds, relative to time of recording>(8)
<time of last message in nanoseconds, relative to time of recording>(8)

The channel info field

Whenever a tape is recorded and a message from a channel is recorded for the first time in this tape, a channel info field is created. It contains type information as well as name and links to the index table for that channel.

<offset to next channel info field>(8)
<time of first message in nanoseconds, relative to time of recording>(8)
<time of last message in nanoseconds, relative to time of recording>(8)
<name>(size(4) + size bytes)
<type>(size(4) + size bytes)
<meta data>(size(8) + size bytes)
<message data size (accumulated size of all messages' data elements)>(8)
<offset to index table>(8)

The message field

Each message field contains a single message. It also has information about the time of recording - relative to the start of recording given in nanoseconds - as well as its size and compression flags.

<time of message in nanoseconds, relative to time of recording>(8)
<name>(size(4) + size bytes)
<frameID> (size(4) + size bytes)
<sequenceID>(4)
<size of (compressed) data>(4)
<compression flag tells us if message is compressed>(1)
<if compression flag is set size of uncompressed data>(4)
<data>(size of (compressed) data bytes)

The index field

Each channel has its own index table. Index tables consist of multiple index entries. The table stores information about the offset of messages in the file as well as the offset to the message's message block.

<number of messages>(4)
for each message:
  <offset to message block containing the message>(8)
  <relative offset in message block to the message>(8)
  <time of message in nanoseconds, relative to time of recording>(8)