MIRA
|
A tape is a binary file that contains recorded/serialized data of one or multiple channels. More...
#include <fw/Tape.h>
Classes | |
struct | ChannelInfo |
Information about a channel in a tape. More... | |
struct | FileInfo |
Information about a tape file. More... | |
struct | Header |
Header containing type and size of the following packet. More... | |
struct | Message |
Struct for message data in a tape. More... | |
struct | MessageBlock |
Struct for a message block in a tape. More... | |
struct | MessageIndex |
Index entry for a message in the tape. More... | |
Public Types | |
enum | OpenMode { READ, WRITE, INFO } |
The open mode for a tape. More... | |
enum | HeaderType { MESSAGEBLOCK = 0x0A, CHANNELINFO = 0x0B, MESSAGE = 0x0C, INDEX = 0x0D } |
The type of the header specifying the following packet. More... | |
typedef std::map< std::string, ChannelInfo > | ChannelMap |
maps channel information to a channel names More... | |
typedef std::map< uint64, MessageBlock > | MessageBlockMap |
maps a message block to an offset More... | |
typedef std::multimap< Duration, Message > | MessageMap |
maps a message to an time offset More... | |
Public Member Functions | |
Tape () | |
~Tape () | |
void | open (const Path &file, OpenMode mode) |
Open the tape for reading or writing. More... | |
void | close () |
Close the tape. More... | |
void | repair (const Path &file, const Path &outFile, bool displayProgress=false) |
Try to repair the tape given by file and store repaired version in outFile. More... | |
void | waitForAlteredStartTime () |
If this method is called in write mode (must be called before writing the first message to the tape) all recorded data is written to file after setting the start time of the tape by calling alterStartTime(). More... | |
void | alterStartTime (const Time &startTime) |
Alters the start time of a tape in write mode. More... | |
template<typename T > | |
MIRA_DEPRECATED ("Please use int compressionLevel(=0/-1) instead of bool compress(=false/true)", std::size_t write(ChannelRead< T > value, bool compress, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase())) | |
Write the content of a channel into the tape. More... | |
template<typename T > | |
std::size_t | write (ChannelRead< T > value, int compressionLevel=0, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase()) |
Write the content of a channel into the tape. More... | |
template<typename T > | |
MIRA_DEPRECATED ("Please use int compressionLevel(=0/-1) instead of bool compress(=false/true)", std::size_t write(ChannelRead< T > value, std::list< BinarySerializerCodecPtr > &codecs, bool compress, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase())) | |
Write the content of a channel into the tape. More... | |
template<typename T > | |
std::size_t | write (ChannelRead< T > value, std::list< BinarySerializerCodecPtr > &codecs, int compressionLevel=0, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase()) |
Write the content of a channel into the tape. More... | |
template<typename T > | |
MIRA_DEPRECATED ("Please use int compressionLevel(=0/-1) instead of bool compress(=false/true)", std::size_t write(const std::string &channelID, const Stamped< T > &value, bool compress, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase())) | |
Write the stamped data into the tape. More... | |
template<typename T > | |
std::size_t | write (const std::string &channelID, const Stamped< T > &value, int compressionLevel=0, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase()) |
Write the stamped data into the tape. More... | |
MIRA_DEPRECATED ("Please use int compressionLevel(=0/-1) instead of bool compress(=false/true)", std::size_t write(const std::string &channelID, const std::string &typeName, const StampedHeader &header, const Buffer< uint8 > &data, bool compress, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase())) | |
Write the serialized data into the tape. More... | |
std::size_t | write (const std::string &channelID, const std::string &typeName, const StampedHeader &header, const Buffer< uint8 > &data, int compressionLevel=0, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase()) |
Write the serialized data into the tape. More... | |
MIRA_DEPRECATED ("Please use int compressionLevel(=0/-1) instead of bool compress(=false/true)", std::size_t write(const std::string &channelID, const std::string &typeName, const Time &time, const std::string &frameID, uint32 sequenceID, const Buffer< uint8 > &data, bool compress, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase())) | |
Low level write method to write serialized binary data to the tape. More... | |
std::size_t | write (const std::string &channelID, const std::string &typeName, const Time &time, const std::string &frameID, uint32 sequenceID, const Buffer< uint8 > &data, int compressionLevel=0, TypeMetaPtr meta=TypeMetaPtr(), const MetaTypeDatabase &metaDB=MetaTypeDatabase()) |
Low level write method to write serialized binary data to the tape. More... | |
void | setMaxMessageBlockSize (uint32 size) |
Set the maximum size of the chunks. More... | |
void | setSortWindowSize (const Duration &interval) |
Messages get sorted before they are written to a tape to guarantee a correct temporal order. More... | |
void | setStartTime (Time time) |
Set an alternative start time for recording. More... | |
Time | getStartTime () const |
Get the time when the recording started. More... | |
Time | getLocalStartTime () const |
Get the time when the recording started in the local timezone of the machine the recording took place (Time at place of recording). More... | |
Time | getEndTime () const |
Get the time of the last entry. More... | |
Duration | getTZOffset () const |
Get the time zone offset. More... | |
const Tape::ChannelMap & | getChannels () const |
Get a list of channels in the tape. More... | |
const Tape::MessageBlockMap & | getMessageBlocks () const |
Get a list of the tapes message blocks. More... | |
uint32 | getVersion () const |
Get the version of the tape. More... | |
void | readMessage (const MessageIndex &index, std::string &frameID, uint32 &sequenceID, Buffer< uint8 > &oData, Duration &oTime) |
Read a message from tape at the given index. More... | |
void | readMessage (const MessageIndex &index, std::string &frameID, uint32 &sequenceID, Buffer< uint8 > &oData, Duration &oTime, bool &oCompressed) |
Same as above but allows to obtain additional information whether the data was compressed within the tape or not. More... | |
Static Public Member Functions | |
static uint32 | getCurrentVersion () |
Returns the tape version that is supported by the local implementation. More... | |
Static Public Attributes | |
static const uint32 | sHeaderSize |
size of the header packet More... | |
static const uint32 | sMessageBlockSize |
Protected Member Functions | |
uint64 | getFileHeaderSize () const |
void | openInfo (const Path &file) |
void | openRead (const Path &file) |
void | closeRead () |
void | openWrite (const Path &file) |
void | closeWrite () |
void | openRepair (const Path &file) |
void | write (Duration time, const Message &message) |
void | readFileHeader () |
void | readChannelInfo () |
void | readMessageBlocks () |
void | readMessageBlock (uint64 offset) |
void | writeFileHeader () |
Tape::Header | readHeader () |
void | writeHeader (HeaderType type, uint32 size) |
void | writeChannelInfo (ChannelInfo &info) |
void | startMessageBlock (Duration time) |
void | finishMessageBlock () |
Protected Attributes | |
bool | mIsOpen |
bool | mWaitForAlteredStartTime |
OpenMode | mMode |
ChannelMap | mChannels |
std::set< std::string > | mWrittenChannels |
boost::optional< MessageBlock > | mCurrentMessageBlock |
boost::mutex | mMutex |
boost::mutex | mMessageMutex |
MessageBlockMap | mMessageBlocks |
Pairs of offset - message block. More... | |
FileInfo | mFile |
File informations. More... | |
uint64 | mLastInfo |
Offset to the last written info field. More... | |
uint32 | mMaxMessageBlockSize |
Maximum size of a message block. More... | |
MessageMap | mMessages |
Messages we have not yet written to file (in the sort window) More... | |
Duration | mSortWindowSize |
Size of the window used for sorting messages in a tape. More... | |
A tape is a binary file that contains recorded/serialized data of one or multiple channels.
The tape file can be played back later to review and analyze the recorded data e.g. for testing algorithms. The format of a tape (in the current version, see version history below) is as follows: (numbers in () list the size in bytes)
There is one file header at the start of the tape:
<FileHeader> -version (4) -time of recording in nanoseconds since 00:00:00 01.01.1970 (8) -time difference of recording machine to UTC time in nanoseconds (8) -number of message blocks (4) -offset to first channel info field (8)
All other field types start with a 5 byte header:
<FieldHeader> -type of field (1) -size (4)
There are 4 types of data fields, storing data and offsets to other fields:
<MessageBlockField> -number of messages (4) -block size (4) -time of first message in nanoseconds, relative to time of recording (8) -time of last message in nanoseconds, relative to time of recording (8)
<ChannelInfoField> -offset to next channel info field (8) -time of first message in nanoseconds, relative to time of recording (8) -time of last message in nanoseconds, relative to time of recording (8) -name (size (4) + size bytes) -type (size (4) + size bytes) -meta data (size (8) + size bytes) -total message data size (8) -offset to index table (8)
<MessageField> -time of message in nanoseconds, relative to time of recording (8) -name (size (4) + size bytes) -frameID (size(4) + size bytes) -sequenceID (4) -size of (compressed) data (4) -compression flag tells us if message is compressed (1) -if compression flag is set size of uncompressed data (4) -data (size of (compressed) data bytes)
<IndexField> -number of messages (4) for each message: -offset to block field (8) -relative offset in message block to message (8) -time of message in nanoseconds, relative to time of recording (8)
History of Tape versions:
typedef std::map<std::string, ChannelInfo> ChannelMap |
maps channel information to a channel names
typedef std::map<uint64, MessageBlock> MessageBlockMap |
maps a message block to an offset
typedef std::multimap<Duration, Message> MessageMap |
maps a message to an time offset
enum OpenMode |
enum HeaderType |
|
inline |
~Tape | ( | ) |
Open the tape for reading or writing.
[in] | file | The filename of the tape |
[in] | mode | The opening mode (read or write) |
void close | ( | ) |
Close the tape.
Try to repair the tape given by file and store repaired version in outFile.
[in] | file | The filename of the tape to be repaired |
[in] | outFile | The filename of the repaired tape |
[in] | displayProgress | If true, displays progress info |
void waitForAlteredStartTime | ( | ) |
If this method is called in write mode (must be called before writing the first message to the tape) all recorded data is written to file after setting the start time of the tape by calling alterStartTime().
Time offsets of all message written until alterStartTime() is called will be corrected relative to the new start time. This can be used to write "old" data to the tape first and specify the start time of recording later.
XIO | if the tape is not opened in write mode |
XInvalidConfig | if data was already written to the tape. |
void alterStartTime | ( | const Time & | startTime | ) |
Alters the start time of a tape in write mode.
Can only be called if waitForAlteredStartTime() was called before or no data has been written to the tape yet.
XIO | if the tape is not opened in write mode |
XInvalidConfig | if waitForAlteredStartTime() was not called before or data was already written to the tape. |
|
inline |
Write the content of a channel into the tape.
XIO | if the tape is not opened in write mode |
value | The channel that data gets serialized and written to tape |
compress | If true, compress data using zip compression (default compression level) |
meta | meta information about the type of the data |
meta | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Write the content of a channel into the tape.
XIO | if the tape is not opened in write mode |
value | The channel that data gets serialized and written to tape |
compressionLevel | libzip compression level (0=none, 1..9=best speed to best compression, -1=default compression) |
meta | meta information about the type of the data |
meta | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Write the content of a channel into the tape.
XIO | if the tape is not opened in write mode |
value | The channel that data gets serialized and written to tape |
codecs | A list of codecs. If this list contains a codec that matches the type of the channel it is used to encode its value. |
compress | If true, compress data using zip compression (default compression level) |
meta | meta information about the type of the data |
meta | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Write the content of a channel into the tape.
XIO | if the tape is not opened in write mode |
value | The channel that data gets serialized and written to tape |
codecs | A list of codecs. If this list contains a codec that matches the type of the channel it is used to encode its value. |
compressionLevel | libzip compression level (0=none, 1..9=best speed to best compression, -1=default compression) |
meta | meta information about the type of the data |
meta | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Write the stamped data into the tape.
XIO | if the tape is not opened in write mode |
channelID | The name of the channel |
value | The value that contains the stamped header information and the data that gets serialized and written to tape |
compress | If true, compress data using zip compression (default compression level) |
meta | meta information about the type of the data |
meta | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Write the stamped data into the tape.
XIO | if the tape is not opened in write mode |
channelID | The name of the channel |
value | The value that contains the stamped header information and the data that gets serialized and written to tape |
compressionLevel | libzip compression level (0=none, 1..9=best speed to best compression, -1=default compression) |
meta | meta information about the type of the data |
meta | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Write the serialized data into the tape.
The stamped information is passed via the stamped header parameter.
XIO | if the tape is not opened in write mode |
channelID | The name of the channel |
typeName | The typename of the data |
header | The stamped information for the data |
data | The serialized data that gets written to tape |
compress | If true, compress data using zip compression (default compression level) |
meta | meta information about the type of the data |
metaDB | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Write the serialized data into the tape.
The stamped information is passed via the stamped header parameter.
XIO | if the tape is not opened in write mode |
channelID | The name of the channel |
typeName | The typename of the data |
header | The stamped information for the data |
data | The serialized data that gets written to tape |
compressionLevel | libzip compression level (0=none, 1..9=best speed to best compression, -1=default compression) |
meta | meta information about the type of the data |
metaDB | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Low level write method to write serialized binary data to the tape.
XIO | if the tape is not opened in write mode |
channelID | The name of the channel |
typeName | The typename of the data |
time | Timestamp of the data |
frameID | Frame ID of the data |
sequenceID | Sequence ID of the data |
data | The serialized data that gets written to tape |
compress | If true, compress data using zip compression (default compression level) |
meta | meta information about the type of the data |
metaDB | database that contains meta information about the types used by the data type itself (e.g. class members) |
std::size_t write | ( | const std::string & | channelID, |
const std::string & | typeName, | ||
const Time & | time, | ||
const std::string & | frameID, | ||
uint32 | sequenceID, | ||
const Buffer< uint8 > & | data, | ||
int | compressionLevel = 0 , |
||
TypeMetaPtr | meta = TypeMetaPtr() , |
||
const MetaTypeDatabase & | metaDB = MetaTypeDatabase() |
||
) |
Low level write method to write serialized binary data to the tape.
XIO | if the tape is not opened in write mode |
channelID | The name of the channel |
typeName | The typename of the data |
time | Timestamp of the data |
frameID | Frame ID of the data |
sequenceID | Sequence ID of the data |
data | The serialized data that gets written to tape |
compressionLevel | libzip compression level (0=none, 1..9=best speed to best compression, -1=default compression) |
meta | meta information about the type of the data |
metaDB | database that contains meta information about the types used by the data type itself (e.g. class members) |
|
inline |
Set the maximum size of the chunks.
[in] | size | Size of a chunk in bytes |
|
inline |
Messages get sorted before they are written to a tape to guarantee a correct temporal order.
Messages are collected in a sorted temporary list. While new messages arrive older messages get written to file when they move out of the sort window. One can specify the size of the sorting window by calling this function.
|
inline |
Set an alternative start time for recording.
This is used when creating new tapes out of existing ones to use their start time instead of now.
[in] | time | Start time of recording |
|
inline |
Get the time when the recording started.
XIO | if the tape is not opened |
|
inline |
Get the time when the recording started in the local timezone of the machine the recording took place (Time at place of recording).
XIO | if the tape is not opened |
|
inline |
Get the time of the last entry.
XIO | if the tape is not opened |
|
inline |
Get the time zone offset.
XIO | if the tape is not opened |
|
inline |
Get a list of channels in the tape.
XIO | if the tape is not opened |
|
inline |
Get a list of the tapes message blocks.
XIO | if the tape is not opened |
|
inline |
Get the version of the tape.
XIO | if the tape is not opened |
|
static |
Returns the tape version that is supported by the local implementation.
Note, that this is the version that is used to generate new tape files. In contrast the getVersion() method returns the version of the opened tape file that may differ from that of the local implementation.
void readMessage | ( | const MessageIndex & | index, |
std::string & | frameID, | ||
uint32 & | sequenceID, | ||
Buffer< uint8 > & | oData, | ||
Duration & | oTime | ||
) |
Read a message from tape at the given index.
An index can be obtained from the ChannelMaps messages member
XIO | if the tape is not opened in read mode |
index | Read message at that index | |
frameID | Frame id of the message | |
sequenceID | Sequence id of the message | |
[out] | oData | Buffer to the data. Is resized and filled with the serialized data from the tape at index position |
[out] | oTime | The time offset to the start of recording when that message was recorded. |
void readMessage | ( | const MessageIndex & | index, |
std::string & | frameID, | ||
uint32 & | sequenceID, | ||
Buffer< uint8 > & | oData, | ||
Duration & | oTime, | ||
bool & | oCompressed | ||
) |
Same as above but allows to obtain additional information whether the data was compressed within the tape or not.
XIO | if the tape is not opened in read mode |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
static |
size of the header packet
|
static |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
Pairs of offset - message block.
|
protected |
File informations.
|
protected |
Offset to the last written info field.
|
protected |
Maximum size of a message block.
|
protected |
Messages we have not yet written to file (in the sort window)