Customizing TTS Behaviour

Table of Contents

• Tests Driver
• Data display
• Equality and Ordering
• Precision Measurement
  • ULP Distance
  • IEEE Comparison
  • Relative Comparison
  • Absolute Comparison
• Data Generator

Tests Driver

By default, TTS provides an entry point function for the listed tests. However, it may be required to handle such an entry point. In this case, one can define the TTS_CUSTOM_DRIVER_FUNCTION preprocessor symbol to a name of their own entry-point function as shown below.

After defining the TTS_CUSTOM_DRIVER_FUNCTION symbol, tests can be added as usual. Then, a regular main function is to be defined. This function will then perform any special operations required, then call the specified entry point function. Finally, the main function will call tts::report which will aggregate test results and validate the whole tests with respect to expected numbers of failures and invalid tests.

#define TTS_MAIN
#define TTS_CUSTOM_DRIVER_FUNCTION custom_entry_point
#include <tts/tts.hpp>
#include <iostream>
 
TTS_CASE("Tautological test") { TTS_EXPECT_NOT(false == true); };
 
int main(int argc, char const** argv)
{
  std::cout << "Welcome to a special test driver !\n";
  custom_entry_point(argc, argv);
  return tts::report(0, 0);
}

Data display

By default, whenever TTS needs to display a value in a report, it uses std::to_string or, in the case of sequence-like types, a sequence of calls to std::to_string. In case no overload for std::to_string exists for a given type, a string will be built from the type name and its byte sequence.

#define TTS_MAIN                   // No need for main()
#include <tts/tts.hpp>
 
struct payload
{
  double         d;
  unsigned int   i, j;
  constexpr bool operator==(payload const&) const = default;
};
 
TTS_CASE("Display an unknown type")
{
  payload p {1.5, 0xAABBCCDD, 0x11223344};
  payload q {1.5, 0xFFEEDDCC, 0x99887766};
 
  TTS_EQUAL(p, q);
};

In the case a given type needs to be displayed in a specific manner, TTS allows to overload to_text in the type's namespace or as a friend function and will use it when necessary.

#define TTS_MAIN                   // No need for main()
#include <tts/tts.hpp>
 
struct payload
{
  double           d;
  unsigned int     i, j;
  constexpr bool   operator==(payload const&) const = default;
 
  friend tts::text to_text(payload const& p)
  {
    return "payload(" + tts::as_text(p.d) + ")[" + tts::as_text(p.i) + "][" + tts::as_text(p.j) +
           "]";
  }
};
 
TTS_CASE("Display a type with custom to_text")
{
  payload p {1.5, 0xAABBCCDD, 0x11223344};
  payload q {1.5, 0xFFEEDDCC, 0x99887766};
 
  TTS_EQUAL(p, q);
};

If needed, one can delegate a part of this string construction to the TTS internal string conversion function tts::as_text that will use all runtime options for display. tts::text can also be constructed from a formatting specification and other similar setup.

#define TTS_MAIN // No need for main()
#include <tts/tts.hpp>
 
namespace sample
{
  struct payload
  {
    double         d;
    unsigned int   i, j;
    constexpr bool operator==(payload const&) const = default;
  };
 
  tts::text to_text(payload const& p) { return tts::text("payload(%f)[%d][%d]", p.d, p.i, p.j); }
}
 
TTS_CASE("Display another type with custom to_text")
{
  sample::payload p {1.5, 0xAABBCCDD, 0x11223344};
  sample::payload q {1.5, 0xFFEEDDCC, 0x99887766};
 
  TTS_EQUAL(p, q);
};

Beware that, in this situation, command-line arguments controlling value display like -x or -s will not be applied to the formatted string.

Equality and Ordering

All equality-based checks in TTS use the compared value operator==. If needed, one can specialize the compare_equal function in the type's namespace or as a friend function to let TTS use a special comparison scheme.

#define TTS_MAIN                   // No need for main()
#include <tts/tts.hpp>
 
namespace sample
{
  template<typename T> struct box
  {
    T value;
  };
 
  template<typename T> bool compare_equal(box<T> const& l, box<T> const& r)
  {
    return l.value == r.value;
  }
};
 
TTS_CASE("Compare values with custom equality")
{
  sample::box<int> a {42};
  sample::box<int> b {13};
 
  TTS_EQUAL(a, a);
  TTS_NOT_EQUAL(a, b);
};

Similarly, TTS uses operator< to build all its ordering-based checks. If needed, one can specialize the compare_less function in the type's namespace or as a friend function to let TTS use a special ordering scheme.

#define TTS_MAIN // No need for main()
#include <tts/tts.hpp>
#include <cmath>
 
namespace sample
{
  template<typename T> struct absolute
  {
    T value;
  };
 
  template<typename T> bool compare_equal(absolute<T> const& l, absolute<T> const& r)
  {
    return std::abs(l.value) == std::abs(r.value);
  }
 
  template<typename T> bool compare_less(absolute<T> const& l, absolute<T> const& r)
  {
    return std::abs(l.value) < std::abs(r.value);
  }
};
 
TTS_CASE("Compare values with custom comparisons")
{
  sample::absolute<int> a {42};
  sample::absolute<int> b {-42};
  sample::absolute<int> c {-13};
 
  TTS_LESS(c, b);
  TTS_LESS_EQUAL(a, a);
  TTS_GREATER(b, c);
  TTS_GREATER_EQUAL(b, a);
};

Precision Measurement

ULP Distance

When dealing with floating point values, TTS uses its ulp_distance function to perform all ULP checks. If needed, one can specialize this function in the type's namespace or as a friend function to let TTS use a special ULP comparison scheme. One can also reuse the pre-existing tts::ulp_check to implement their own.

#define TTS_MAIN // No need for main()
#include <tts/tts.hpp>
 
namespace sample
{
  struct ratio
  {
    int n, d;
  };
 
  double ulp_distance(ratio a, ratio b)
  {
    auto ra = static_cast<float>(a.n) / a.d;
    auto rb = static_cast<float>(b.n) / b.d;
 
    return tts::ulp_check(ra, rb);
  }
};
 
TTS_CASE("Compare values with custom ULP distance computation")
{
  sample::ratio a {8388608, 8388608};
  sample::ratio b {8388618, 8388608};
 
  TTS_ULP_EQUAL(a, b, 10);
};

IEEE Comparison

IEEE comparison consists in checking for exact equality while considering all NaN/Invalid values of floating point values. One can specialize the ieee_equal function in the type's namespace or as a friend function to let TTS use a special IEEE comparison scheme. One can also reuse the pre-existing tts::ieee_check to implement their own.

#define TTS_MAIN // No need for main()
#include <tts/tts.hpp>
 
namespace sample
{
  struct ratio
  {
    int n, d;
  };
 
  double ieee_equal(ratio a, ratio b)
  {
    auto ra = static_cast<float>(a.n) / a.d;
    auto rb = static_cast<float>(b.n) / b.d;
    printf("%f vs %f\n", ra, rb);
    return tts::ieee_check(ra, rb);
  }
};
 
TTS_CASE("Compare values with custom IEEE distance computation")
{
  sample::ratio a {189, 265};
  sample::ratio b {0, 0};
 
  TTS_IEEE_EQUAL(a, a);
  TTS_IEEE_EQUAL(b, b);
};

Relative Comparison

Relative precision checks within TTS are done through the relative_distance function. If needed, one can specialize this function in the type's namespace or as a friend function to let TTS use a special relative precision scheme. One can also reuse the pre-existing tts::relative_check to implement their own.

#define TTS_MAIN // No need for main()
#include <tts/tts.hpp>
 
namespace sample
{
  struct ratio
  {
    int n, d;
  };
 
  double relative_distance(ratio a, ratio b)
  {
    auto ra = static_cast<float>(a.n) / a.d;
    auto rb = static_cast<float>(b.n) / b.d;
 
    return tts::relative_check(ra, rb);
  }
};
 
TTS_CASE("Compare values with custom relative distance computation")
{
  sample::ratio a {1, 77};
  sample::ratio b {3, 85};
 
  TTS_RELATIVE_EQUAL(a, b, 2.25);
};

Absolute Comparison

TTS** uses its absolute_distance function to perform all absolute precision checks. If needed, one can specialize this function in the type's namespace or as a friend function to let TTS use a special absolute precision scheme. One can also reuse the pre-existing tts::absolute_check to implement their own.

#define TTS_MAIN // No need for main()
#include <tts/tts.hpp>
 
namespace sample
{
  struct ratio
  {
    int n, d;
  };
 
  double absolute_distance(ratio a, ratio b)
  {
    auto ra = static_cast<float>(a.n) / a.d;
    auto rb = static_cast<float>(b.n) / b.d;
 
    return tts::absolute_check(ra, rb);
  }
};
 
TTS_CASE("Compare values with custom absolute distance computation")
{
  sample::ratio a {115, 77};
  sample::ratio b {397, 85};
 
  TTS_ABSOLUTE_EQUAL(a, b, 4);
};

Data Generator

Test cases based on data sets and range checks require one or more data generators to perform. If the pre-existing data generators are not suitable, define your own by providing a constexpr callable object with the following signature:

template<typename T>
T operator()(tts::type<T> target, auto index, auto count);

where:

• target is an instance of tts::type<T> representing the type of value to be generated.
• index is an integral value representing the index of the generated value.
• count is an integral value representing the total number of values to be generated.

For example, the following code defines a generator that will generate values alternating between -1 and 1 every n iterations.

#define TTS_MAIN // No need for main()
#include <tts/tts.hpp>
#include <array>
 
struct flip_values
{
  constexpr flip_values(int p = 1)
      : period_(p)
      , value_(-1)
  {
  }
 
  template<typename T> auto operator()(tts::type<T>, auto i, auto)
  {
    if((i % period_) == 0) value_ = -value_;
    return static_cast<T>(value_);
  }
 
  int period_, value_;
};
 
TTS_CASE_WITH("Test custom generator", (std::array<float, 10>, std::array<int, 4>), flip_values {3})
(auto const& args)
{
  for(std::size_t i = 0; i < args.size(); ++i)
  {
    TTS_EQUAL(args[ i ], ((i % 3 == 0) ? 1.0f : -1.0f));
  }
};