public class RankedListEvaluator extends Object
A bundle of static methods for evaluating ranked lists. The class computes standard ranked retrieval metrics such as average precision (AP), precision at N documents (PN), etc.
Implemented metrics:
R-Prec
: R-PrecisionPN
: precision at N, where N can be any value (e.g.,
"P5", "P10", etc.)recall
: recallAPN
: average precision at N, where N can be any value
(e.g., "AP50" for AP at 50 hits)AP
: default average precision, on entire ranked list.num_ret
: number of hits returned.num_rel
: number of relevant hits retrieved.Constructor and Description |
---|
RankedListEvaluator() |
Modifier and Type | Method and Description |
---|---|
static double |
computeAP(Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs) |
static double |
computeAP(int n,
Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs)
Computes average precision (AP) of a ranked list.
|
static double |
computeDCG(int n,
double[] gain) |
static double |
computeNDCG(int n,
Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs)
Computes ndcg (NDCG) of a ranked list (1 query only !!)
|
static double |
computeNumRelevant(Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs) |
static double |
computeNumRetrieved(Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs) |
static double |
computePN(int n,
Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs)
Computes precision at N documents (PN), where N is a
user-specified value.
|
static double |
computeRecall(Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs)
Computes recall of a ranked list.
|
static double |
computeRPrecision(Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs)
Computes R-Precision, which is defined as precision at R, where
R is equal to the number of relevant documents.
|
static Map<String,Double> |
evaluate(Accumulator[] docs,
DocnoMapping mapping,
Set<String> reldocs,
String... metrics) |
static double |
roundTo4SigFigs(double d)
Returns a double value to four significant figures.
|
public static double computeAP(Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
public static double computeAP(int n, Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
docs
- the list of resultsreldocs
- set of the ids of the relevant documentspublic static double computeDCG(int n, double[] gain)
public static double computeNDCG(int n, Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
docs
- the list of resultsreldocs
- set of the ids of the relevant documentspublic static double computeNumRelevant(Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
public static double computeNumRetrieved(Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
public static double computePN(int n, Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
n
- the number of hits to considerdocs
- the list of resultsreldocs
- set of the ids of the relevant documentspublic static double computeRecall(Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
docs
- the list of resultsreldocs
- set of the ids of the relevant documentspublic static double computeRPrecision(Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs)
docs
- the list of resultsreldocs
- set of the ids of the relevant documentspublic static Map<String,Double> evaluate(Accumulator[] docs, DocnoMapping mapping, Set<String> reldocs, String... metrics)
public static double roundTo4SigFigs(double d)
trec_eval
, which reports values to four
significant figures.d
- the value to round.