Class ComputeFunctions

java.lang.Object

io.nosqlbench.nb.api.components.core.NBBaseComponentMetrics

io.nosqlbench.nb.api.components.core.NBBaseComponent

io.nosqlbench.engine.extensions.computefunctions.ComputeFunctions

All Implemented Interfaces:

NBComponent, NBComponentAdvisors, NBComponentEvents, NBComponentMetrics, NBComponentProps, NBComponentServices, NBComponentTimeline, NBProviderSearch, NBTokenWords, NBLabeledElement, AutoCloseable

public class ComputeFunctions
extends NBBaseComponent

A collection of compute functions related to vector search relevancy. These are based on arrays of indices of vectors, where the expected data is from known KNN test data, and the actual data is from a vector search query.

Variations of these functions have a limit parameter, which allows for derivation of relevancy measurements for a smaller query without having to run a separate test for each K value. If you are using test vectors from a computed KNN test data with for K=100, you can compute metrics "@K" for any size up to and including K=100. This simply uses a partial view of the result to do exactly what would have been done for a test where you actually query for that K limit. This assumes that the result rank is stable irrespective of the limit AND the results are passed to these functions as ranked in results.

Some of the methods apply K to the expected (relevant) indices, others to the actual (response) indices, depending on what is appropriate.

The array indices passed to these functions should not be sorted before-hand as a general rule.

Yet, no provision is made for duplicate entries. If you have duplicate indices in either array, these methods will yield incorrect results as they rely on the two-pointer method and do not elide duplicates internally.

Nested Class Summary

Nested classes/interfaces inherited from interface io.nosqlbench.nb.api.labels.NBLabeledElement

NBLabeledElement.BasicLabeledElement

Field Summary

Fields inherited from class io.nosqlbench.nb.api.components.core.NBBaseComponent

bufferOrphanedMetrics, closed_ns, error, errored_ns, labels, metricsBuffer, parent, started_epoch_ms, started_ns, state, teardown_ns

Fields inherited from interface io.nosqlbench.nb.api.components.core.NBComponent

EMPTY_COMPONENT

Fields inherited from interface io.nosqlbench.nb.api.components.core.NBComponentProps

HDRDIGITS, SUMMARY

Fields inherited from interface io.nosqlbench.nb.api.labels.NBLabeledElement

EMPTY

Constructor Summary

Constructors

Constructor	Description
ComputeFunctions(NBComponent parentComponent)

Method Summary

Modifier and Type	Method	Description
static double	average_precision(int[] relevant, int[] actual)
static double	average_precision(int[] relevant, int[] actual, int k)
static double	average_precision(long[] relevant, long[] actual)
static double	average_precision(long[] relevant, long[] actual, int k)
static double	F1(int[] relevant, int[] actual)
static double	F1(int[] relevant, int[] actual, int k)
static double	F1(long[] relevant, long[] actual)
static double	F1(long[] relevant, long[] actual, int k)
static double	precision(int[] relevant, int[] actual)
static double	precision(int[] relevant, int[] actual, int k)
static double	precision(long[] relevant, long[] actual)
static double	precision(long[] relevant, long[] actual, int k)
static double	recall(int[] relevant, int[] actual)	Compute the recall as the proportion of matching indices divided by the expected indices
static double	recall(int[] relevant, int[] actual, int k)
static double	recall(long[] relevant, long[] actual)	Compute the recall as the proportion of matching indices divided by the expected indices
static double	recall(long[] relevant, long[] actual, int k)
static double	reciprocal_rank(int[] relevant, int[] actual)
static double	reciprocal_rank(int[] relevant, int[] actual, int k)	RR as in M(RR)
static double	reciprocal_rank(long[] relevant, long[] actual)
static double	reciprocal_rank(long[] relevant, long[] actual, int k)	Reciprocal Rank - The multiplicative inverse of the first rank which is relevant.

Methods inherited from class io.nosqlbench.nb.api.components.core.NBBaseComponent

addAdvisor, addMetricsCloseable, attachChild, beforeDetach, close, create, detachChild, find, findParentService, getAdvisors, getChildren, getComponentOnlyLabels, getComponentProp, getComponentState, getLabels, getNanosSinceStart, getParent, getTokens, nanosof_close, nanosof_error, nanosof_start, nanosof_teardown, onError, onEvent, reportExecutionMetric, setComponentProp, started_epoch_ms, teardown, toString

Methods inherited from class io.nosqlbench.nb.api.components.core.NBBaseComponentMetrics

addComponentMetric, addListener, findComponentMetrics, findComponentMetrics, getComponentMetric, getComponentMetrics, removeListener

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface io.nosqlbench.nb.api.components.core.NBComponentAdvisors

getAdvisorResults

Methods inherited from interface io.nosqlbench.nb.api.components.core.NBComponentMetrics

addComponentMetric, findComponentMetrics, findComponentMetrics, findOneComponentMetric, findOneComponentMetric, getComponentMetric, getComponentMetrics

Methods inherited from interface io.nosqlbench.nb.api.labels.NBLabeledElement

description

Constructor Details

ComputeFunctions

public ComputeFunctions(NBComponent parentComponent)

Method Details

recall

public static double recall(long[] relevant,
 long[] actual)

Compute the recall as the proportion of matching indices divided by the expected indices

Parameters:

relevant - long array of indices

actual - long array of indices

Returns:

a fractional measure of matching vs expected indices

recall

public static double recall(long[] relevant,
 long[] actual,
 int k)

precision

public static double precision(long[] relevant,
 long[] actual)

precision

public static double precision(long[] relevant,
 long[] actual,
 int k)

recall

public static double recall(int[] relevant,
 int[] actual)

Compute the recall as the proportion of matching indices divided by the expected indices

Parameters:

relevant - int array of indices

actual - int array of indices

Returns:

a fractional measure of matching vs expected indices

recall

public static double recall(int[] relevant,
 int[] actual,
 int k)

precision

public static double precision(int[] relevant,
 int[] actual)

precision

public static double precision(int[] relevant,
 int[] actual,
 int k)

F1

public static double F1(int[] relevant,
 int[] actual)

F1

public static double F1(int[] relevant,
 int[] actual,
 int k)

F1

public static double F1(long[] relevant,
 long[] actual)

F1

public static double F1(long[] relevant,
 long[] actual,
 int k)

reciprocal_rank

public static double reciprocal_rank(long[] relevant,
 long[] actual,
 int k)

Reciprocal Rank - The multiplicative inverse of the first rank which is relevant.

reciprocal_rank

public static double reciprocal_rank(long[] relevant,
 long[] actual)

reciprocal_rank

public static double reciprocal_rank(int[] relevant,
 int[] actual,
 int k)

RR as in M(RR)

reciprocal_rank

public static double reciprocal_rank(int[] relevant,
 int[] actual)

average_precision

public static double average_precision(int[] relevant,
 int[] actual)

average_precision

public static double average_precision(int[] relevant,
 int[] actual,
 int k)

average_precision

public static double average_precision(long[] relevant,
 long[] actual)

average_precision

public static double average_precision(long[] relevant,
 long[] actual,
 int k)