An Open Repository for Real-Time Benchmarks

Formats

Benchmarks and case studies for Real-time analysis can be extremely complex and defined according to a variety of functional, task and message models, with semantics differences and possibly subtleties.

In addition, the problem of defining a model and a format for a set of tasks and messages clearly overlaps with the area of interest of system-level design, in which the industry and the academia have been at work for a long time producing a huge number of models (including formal models of computation), formalisms and languages and the associated formats for exchanging system models (and possibly the results of the analysis on them).

Examples of such languages and (XML-based) standards are

• The AUTOSAR standard with the ARXML format (http://www.autosar.org)
• The FIBEX automotive standard (http://www.asam.net)
• The OMG UML and SysML and the XMI interchange format (http://www.omg.org)

Here, we don't want to propose yet another language or modeling formalism with the associated file format and to be caught in a neverending fight about what format is best or most suitable. However, a few basic principles stand:

• Each benchmark must be described in clear, readable and (as far as possible) unambiguous way
• Text-based and XML-based standard formats belonging to the above list are acceptable
• In case another text-based format is used, it is the responsibility of whoever is providing the case study or benchmark to provide additional documentation explaining the format
• We do not expect the format to fully provide information about the semantics assumed for the model. However, given that the benchmarks are mostly aimed at the real-time systems community, it is highly recommended to also provide a reference to the paper (or technical report) that describes the semantics of the model.

Examples of information provided with a benchmark example