RankerEval¶
A fast numpy/numba-based implementation of ranking metrics for information retrieval and recommendation. Coded with efficiency in mind and support for edge cases.
Find the full documentation here.
Features¶
- Wide array of evaluation metrics for information retrieval and top-N recommender systems:
- Binary labels: Recall, Precision, MAP, HitRate, MRR, MeanRanks, F1
- Numeric and binary labels: DCG, nDCG
- Minimal dependencies: Numpy and Numba (required), SciPy (optional)
- Flexible input formats: Supports arrays, lists and sparse matrices
- Built-in support for confidence intervals via bootstrapping
Usage¶
from rankereval import BinaryLabels, Rankings, Recall
y_true = BinaryLabels.from_positive_indices([[0,2], [0,1,2]])
y_pred = Rankings.from_ranked_indices([[2,1], [1]])
recall_at_3 = Recall(3).mean(y_true, y_pred)
print(recall_at_3)
To get confidence intervals (95% by default), specify conf_interval=True:
recall_at_3 = Recall(3).mean(y_true, y_pred, conf_interval=True)
print(recall_at_3["conf_interval"])
Input formats¶
RankerEval allows for a variety of input formats, e.g.,
# specify all labels as lists
y_true = BinaryLabels.from_matrix([[1,0,1], [1,1,1]])
# specify labels as numpy array
y_true = BinaryLabels.from_matrix(np.asarray([[1,0,1], [1,1,1]]))
# or use a sparse matrix
import scipy.sparse as sp
y_true = BinaryLabels.from_matrix(sp.coo_matrix([[1,0,1], [1,1,1]]))