fuzzylite  6.0 A Fuzzy Logic Control Library in C++
fl::Benchmark Class Reference

The Benchmark class is designed to evaluate the performance of an Engine. More...

#include <Benchmark.h>

Collaboration diagram for fl::Benchmark:

## Public Types

enum  TimeUnit {
NanoSeconds, MicroSeconds, MilliSeconds, Seconds,
Minutes, Hours
}
Unit of time to utilize in the results. More...

enum  TableShape { Horizontal, Vertical }
Shape of the table of results. More...

Contents of the table of results. More...

enum  ErrorType { NonFinite, Accuracy, All }
Type of error between expected and obtained values. More...

typedef std::pair< std::string, std::string > Result
Result is a type definition for a pair of strings. More...

## Public Member Functions

Benchmark (const std::string &name="", Engine *engine=fl::null, scalar tolerance=fuzzylite::macheps())

virtual ~Benchmark ()

void setName (const std::string &name)
Sets the name of the benchmark. More...

std::string getName () const
Gets the name of the benchmark. More...

void setEngine (Engine *engine)
Sets the engine to benchmark. More...

EnginegetEngine () const
Gets the engine to benchmark. More...

void setExpected (const std::vector< std::vector< scalar > > &expected)
Sets the set of expected values from the engine, where the inner vector contains the input values and output values. More...

const std::vector< std::vector< scalar > > & getExpected () const
Gets the set of expected values from the engine, where the inner vector contains the input values and output values. More...

void setObtained (const std::vector< std::vector< scalar > > &obtained)
Sets the set of obtained values from the engine, where the inner vector contains the input values and output values. More...

const std::vector< std::vector< scalar > > & getObtained () const
Gets the set of obtained values from the engine, where the inner vector contains the input values and output values. More...

void setTimes (const std::vector< scalar > times)
Sets the vector of nanoseconds taken to produce the set of obtained values from the set of expected input values. More...

const std::vector< scalar > & getTimes () const
Gets the vector of nanoseconds taken to produce the set of obtained values from the set of expected input values. More...

void setTolerance (scalar tolerance)
Sets the tolerance above which the difference between an expected and obtained value from the engine is considered an error. More...

scalar getTolerance () const
Gets the tolerance above which the difference between an expected and obtained value from the engine is considered an error. More...

virtual void prepare (int values, FldExporter::ScopeOfValues scope)
Produces and loads into memory the set of expected values from the engine. More...

virtual void prepare (std::istream &reader, long numberOfLines=-1)
Reads and loads into memory the set of expected values from the engine. More...

virtual scalar runOnce ()
Runs the benchmark on the engine only once. More...

virtual std::vector< scalarrun (int times)
Runs the benchmark on the engine multiple times. More...

virtual void reset ()
Resets the benchmark to be ready to run again. More...

virtual bool canComputeErrors () const
Indicates whether errors can be computed based on the expected and obtained values from the benchmark. More...

virtual scalar meanSquaredError () const
Computes the mean squared error over all output variables considering only those cases where there is an accuracy error as defined in Benchmark::accuracyErrors(). More...

virtual scalar meanSquaredError (const OutputVariable *outputVariable) const
Computes the mean squared error of the given output variable considering only those cases where there is an accuracy error as defined in Benchmark::accuracyErrors(). More...

virtual int allErrors () const
Computes the number of errors over all the output variables caused by non-finite differences or accuracy differences. More...

virtual int allErrors (const OutputVariable *outputVariable) const
Computes the number of errors of the given output variable caused by non-finite differences or accuracy differences. More...

virtual int nonFiniteErrors () const
Computes the number of errors over all the output variables caused by non-finite differences (ie, infinity and NaN). More...

virtual int nonFiniteErrors (const OutputVariable *outputVariable) const
Computes the number of errors of the given output variable caused by non-finite differences (ie, infinity and NaN). More...

virtual int accuracyErrors () const
Computes the number of errors over all the output variables caused by a significant difference in accuracy. More...

virtual int accuracyErrors (const OutputVariable *outputVariable) const
Computes the number of errors over the given output variable caused by a significant difference in accuracy. More...

virtual int numberOfErrors (ErrorType errorType) const
Computes the number of errors of the given type over all the output variables. More...

virtual int numberOfErrors (ErrorType errorType, const OutputVariable *outputVariable) const
Computes the number of errors of the given type over the given output variable. More...

virtual std::vector< std::string > header (int runs, bool includeErrors=true)
Returns the header of a horizontal table of results. More...

virtual std::vector< Resultresults (TimeUnit timeUnit=NanoSeconds, bool includeTimes=true) const
Computes and returns the results from the benchmark aggregating the statistics of all the output variables. More...

virtual std::vector< Resultresults (const OutputVariable *outputVariable, TimeUnit timeUnit=NanoSeconds, bool includeTimes=true) const
Computes and returns the results from the benchmark for the given output variable. More...

virtual std::string format (std::vector< Result > results, TableShape shape, TableContents contents, const std::string &delimiter="\) const
Formats the results. More...

## Static Public Member Functions

static std::string stringOf (TimeUnit unit)
Returns the name of the time unit. More...

static scalar factorOf (TimeUnit unit)
Returns the factor of the given unit from NanoSeconds. More...

static scalar convert (scalar time, TimeUnit from, TimeUnit to)
Converts the time to different scales. More...

## Detailed Description

The Benchmark class is designed to evaluate the performance of an Engine.

Engine
Since
6.0

Definition at line 39 of file Benchmark.h.

## ◆ Result

 typedef std::pair fl::Benchmark::Result

Result is a type definition for a pair of strings.

Definition at line 364 of file Benchmark.h.

## ◆ ErrorType

Type of error between expected and obtained values.

Enumerator
NonFinite
Accuracy
All

Definition at line 74 of file Benchmark.h.

## ◆ TableContents

Contents of the table of results.

Enumerator
Body

Definition at line 67 of file Benchmark.h.

## ◆ TableShape

Shape of the table of results.

Enumerator
Horizontal
Vertical

Definition at line 60 of file Benchmark.h.

## ◆ TimeUnit

Unit of time to utilize in the results.

Enumerator
NanoSeconds
MicroSeconds
MilliSeconds
Seconds
Minutes
Hours

Definition at line 53 of file Benchmark.h.

## ◆ Benchmark()

 fl::Benchmark::Benchmark ( const std::string & name = "", Engine * engine = fl::null, scalar tolerance = fuzzylite::macheps() )
explicit

## ◆ ~Benchmark()

 virtual fl::Benchmark::~Benchmark ( )
virtual

## ◆ accuracyErrors() [1/2]

 virtual int fl::Benchmark::accuracyErrors ( ) const
virtual

Computes the number of errors over all the output variables caused by a significant difference in accuracy.

An error is counted when the absolute difference between the expected and obtained values is not smaller than the tolerance.

$$\text{E} = \sum_y \sum_i \epsilon_i^y, \text{where } \epsilon_i^y = \begin{cases} 0 & \text{if} |e_i^y - o^y_i| < \theta\\ 1 & \text{otherwise} \end{cases}$$, $$y$$ is the set of output variables, $$e$$ is the set of expected output values, $$o$$ is the set of obtained output values, and $$\theta$$ is the tolerance

Returns
the number of errors over all the output variables caused by a significant difference in accuracy

## ◆ accuracyErrors() [2/2]

 virtual int fl::Benchmark::accuracyErrors ( const OutputVariable * outputVariable ) const
virtual

Computes the number of errors over the given output variable caused by a significant difference in accuracy.

An error is counted when the absolute difference between the expected and obtained values is not smaller than the tolerance.

$$\text{E} = \sum_i \epsilon_i, \text{where } \epsilon_i = \begin{cases} 0 & \text{if} |e_i - o_i| < \theta\\ 1 & \text{otherwise} \end{cases}$$, $$e$$ is the set of expected output values, $$o$$ is the set of obtained output values, and $$\theta$$ is the tolerance

Parameters
 outputVariable is the output variable to account the errors for
Returns
the number of errors of the given output variable caused by a significant difference in accuracy

## ◆ allErrors() [1/2]

 virtual int fl::Benchmark::allErrors ( ) const
virtual

Computes the number of errors over all the output variables caused by non-finite differences or accuracy differences.

An error is counted when the difference between the expected and obtained values is not finite, or the absolute difference between the expected and obtained values is not smaller than the tolerance.

Returns
the number of errors over all the output variables caused by non-finite differences or accuracy differences

## ◆ allErrors() [2/2]

 virtual int fl::Benchmark::allErrors ( const OutputVariable * outputVariable ) const
virtual

Computes the number of errors of the given output variable caused by non-finite differences or accuracy differences.

An error is counted when the difference between the expected and obtained values is not finite, or the absolute difference between the expected and obtained values is not smaller than the tolerance.

Parameters
 outputVariable is the output variable to account the errors for
Returns
the number of errors of the given output variable caused by non-finite differences or accuracy differences

## ◆ canComputeErrors()

 virtual bool fl::Benchmark::canComputeErrors ( ) const
virtual

Indicates whether errors can be computed based on the expected and obtained values from the benchmark.

If the benchmark was prepared from a file reader and the file included columns of expected output values and the benchmark has been run at least once, then the benchmark can automatically compute the errors and will automatically include them in the results.

Returns
whether errors can be computed based on the expected and obtained values from the benchmark

## ◆ convert()

 static scalar fl::Benchmark::convert ( scalar time, TimeUnit from, TimeUnit to )
static

Converts the time to different scales.

Parameters
 time is the time to convert from is the units of the time to convert from to is the units of the time to convert to
Returns
the time in the units specified

## ◆ factorOf()

 static scalar fl::Benchmark::factorOf ( TimeUnit unit )
static

Returns the factor of the given unit from NanoSeconds.

Parameters
 unit is the time unit
Returns
the factor of the given unit from NanoSeconds

## ◆ format()

 virtual std::string fl::Benchmark::format ( std::vector< Result > results, TableShape shape, TableContents contents )
virtual

Formats the results.

Parameters
 results is the vector of results shape is the shape to present the table of results contents indicates the information to include in the table of results delimiter is the delimiter of the table of results
Returns
the formatted results from the benchmark

## ◆ getEngine()

 Engine* fl::Benchmark::getEngine ( ) const

Gets the engine to benchmark.

Returns
the engine to benchmark

## ◆ getExpected()

 const std::vector >& fl::Benchmark::getExpected ( ) const

Gets the set of expected values from the engine, where the inner vector contains the input values and output values.

Returns
the set of expected values from the engine

## ◆ getName()

 std::string fl::Benchmark::getName ( ) const

Gets the name of the benchmark.

Returns
name is the name of the benchmark

## ◆ getObtained()

 const std::vector >& fl::Benchmark::getObtained ( ) const

Gets the set of obtained values from the engine, where the inner vector contains the input values and output values.

Returns
the set of obtained values from the engine

## ◆ getTimes()

 const std::vector& fl::Benchmark::getTimes ( ) const

Gets the vector of nanoseconds taken to produce the set of obtained values from the set of expected input values.

Returns
the vector of nanoseconds taken to produce the set of obtained values from the set of expected input values

## ◆ getTolerance()

 scalar fl::Benchmark::getTolerance ( ) const

Gets the tolerance above which the difference between an expected and obtained value from the engine is considered an error.

Returns
the tolerance above which the difference between an expected and obtained value from the engine is considered an error

 virtual std::vector fl::Benchmark::header ( int runs, bool includeErrors = true )
virtual

Returns the header of a horizontal table of results.

Parameters
 runs is the number of times the benchmark will be run, hence producing the relevant number of columns for each run includeErrors indicates whether to include columns for computing the errors
Returns
the header of a horizontal table of results

## ◆ meanSquaredError() [1/2]

 virtual scalar fl::Benchmark::meanSquaredError ( ) const
virtual

Computes the mean squared error over all output variables considering only those cases where there is an accuracy error as defined in Benchmark::accuracyErrors().

Returns
the mean squared error over all the output variables.

## ◆ meanSquaredError() [2/2]

 virtual scalar fl::Benchmark::meanSquaredError ( const OutputVariable * outputVariable ) const
virtual

Computes the mean squared error of the given output variable considering only those cases where there is an accuracy error as defined in Benchmark::accuracyErrors().

Parameters
 outputVariable is the output variable to compute the errors for
Returns
the mean squared error over the given output variable.

## ◆ nonFiniteErrors() [1/2]

 virtual int fl::Benchmark::nonFiniteErrors ( ) const
virtual

Computes the number of errors over all the output variables caused by non-finite differences (ie, infinity and NaN).

An error is counted when the difference between the expected and obtained values is not finite.

Returns
the number of errors over all the output variables caused by non-finite differences

## ◆ nonFiniteErrors() [2/2]

 virtual int fl::Benchmark::nonFiniteErrors ( const OutputVariable * outputVariable ) const
virtual

Computes the number of errors of the given output variable caused by non-finite differences (ie, infinity and NaN).

An error is counted when the difference between the expected and obtained values is not finite.

Parameters
 outputVariable is the output variable to account the errors for
Returns
the number of errors of the given output variable caused by non-finite differences

## ◆ numberOfErrors() [1/2]

 virtual int fl::Benchmark::numberOfErrors ( ErrorType errorType ) const
virtual

Computes the number of errors of the given type over all the output variables.

Parameters
 errorType is the type of error to account for
Returns
the number of errors over all the output variables

## ◆ numberOfErrors() [2/2]

 virtual int fl::Benchmark::numberOfErrors ( ErrorType errorType, const OutputVariable * outputVariable ) const
virtual

Computes the number of errors of the given type over the given output variable.

Parameters
 errorType is the type of error to account for outputVariable is the output variable to account the errors for
Returns
the number of errors over the given output variable

## ◆ prepare() [1/2]

 virtual void fl::Benchmark::prepare ( int values, FldExporter::ScopeOfValues scope )
virtual

Produces and loads into memory the set of expected values from the engine.

Parameters
 values is the number of values to evaluate the engine upon scope is the scope of the values to generate
Exceptions
 Exception if the engine is not set

## ◆ prepare() [2/2]

 virtual void fl::Benchmark::prepare ( std::istream & reader, long numberOfLines = -1 )
virtual

Reads and loads into memory the set of expected values from the engine.

Parameters
 reader is the reader of a set of lines containing space-separated values numberOfLines is the maximum number of lines to read from the reader, and a value $f =(, -1]$f@ reads the entire file.

## ◆ reset()

 virtual void fl::Benchmark::reset ( )
virtual

Resets the benchmark to be ready to run again.

## ◆ results() [1/2]

 virtual std::vector fl::Benchmark::results ( TimeUnit timeUnit = NanoSeconds, bool includeTimes = true ) const
virtual

Computes and returns the results from the benchmark aggregating the statistics of all the output variables.

Parameters
 timeUnit is the unit of time of the results includeTimes indicates whether to include the times of each run
Returns
the results from the benchmark

## ◆ results() [2/2]

 virtual std::vector fl::Benchmark::results ( const OutputVariable * outputVariable, TimeUnit timeUnit = NanoSeconds, bool includeTimes = true ) const
virtual

Computes and returns the results from the benchmark for the given output variable.

Parameters
 outputVariable is the output variable to compute the statistics for timeUnit is the unit of time of the results includeTimes indicates whether to include the times of each run
Returns
the results from the benchmark

## ◆ run()

 virtual std::vector fl::Benchmark::run ( int times )
virtual

Runs the benchmark on the engine multiple times.

Parameters
 times is the number of times to run the benchmark on the engine
Returns
vector of the time in nanoseconds required by each run, which is also appended to the times stored in Benchmark::getTimes()

## ◆ runOnce()

 virtual scalar fl::Benchmark::runOnce ( )
virtual

Runs the benchmark on the engine only once.

Returns
the time in nanoseconds required by the run, which is also appended to the times stored in Benchmark::getTimes()

## ◆ setEngine()

 void fl::Benchmark::setEngine ( Engine * engine )

Sets the engine to benchmark.

Parameters
 engine is the engine to benchmark

## ◆ setExpected()

 void fl::Benchmark::setExpected ( const std::vector< std::vector< scalar > > & expected )

Sets the set of expected values from the engine, where the inner vector contains the input values and output values.

Parameters
 expected is the set of expected values from the engine

## ◆ setName()

 void fl::Benchmark::setName ( const std::string & name )

Sets the name of the benchmark.

Parameters
 name is the name of the benchmark

## ◆ setObtained()

 void fl::Benchmark::setObtained ( const std::vector< std::vector< scalar > > & obtained )

Sets the set of obtained values from the engine, where the inner vector contains the input values and output values.

Parameters
 obtained is the set of obtained values from the engine

## ◆ setTimes()

 void fl::Benchmark::setTimes ( const std::vector< scalar > times )

Sets the vector of nanoseconds taken to produce the set of obtained values from the set of expected input values.

Parameters
 times is the vector of nanoseconds taken to produce the set of obtained values from the set of expected input values

## ◆ setTolerance()

 void fl::Benchmark::setTolerance ( scalar tolerance )

Sets the tolerance above which the difference between an expected and obtained value from the engine is considered an error.

Parameters
 tolerance is the tolerance above which the difference between an expected and obtained value from the engine is considered an error

## ◆ stringOf()

 static std::string fl::Benchmark::stringOf ( TimeUnit unit )
static

Returns the name of the time unit.

Parameters
 unit is the time unit
Returns
the name of the time unit

The documentation for this class was generated from the following file: