org.aksw.limes.core.evaluation.evaluator

Class Evaluator

java.lang.Object

org.aksw.limes.core.evaluation.evaluator.Evaluator

public class Evaluator
extends Object

This evaluator is responsible for evaluating set of datasets that have source, target, gold standard and mappings against set of measures

Since:

1.0

Version:

1.0

Author:

Mofeed Hassan (mounir@informatik.uni-leipzig.de), Daniel Obraczka (obraczka@studserv.uni-leipzig.de)

Constructor Summary

Constructors

Constructor and Description
Evaluator()

Method Summary

Modifier and Type	Method and Description
ACache	cacheUnion(ACache a, ACache b)
List<EvaluationRun>	crossValidate(AMLAlgorithm algorithm, List<LearningParameter> parameter, Set<TaskData> datasets, int foldNumber, Set<EvaluatorType> qlMeasures, Set<IQuantitativeMeasure> qnMeasures)
List<EvaluationRun>	evaluate(List<TaskAlgorithm> TaskAlgorithms, Set<TaskData> datasets, Set<EvaluatorType> QlMeasures, Set<IQuantitativeMeasure> QnMeasures)
List<FoldData>	generateFolds(EvaluationData data, int foldNumber)
List<AMapping>	generateMappingFolds(AMapping refMap, ACache source, ACache target, int foldNumber)
static String	getRandomTargetInstance(ACache source, ACache target, List<String> values, Random random, HashMap<String,HashMap<String,Double>> refMap, String sourceInstance, int previousRandom)
AMapping	removeSubMap(AMapping mainMap, AMapping subMap)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Evaluator

public Evaluator()

Method Detail

evaluate

public List<EvaluationRun> evaluate(List<TaskAlgorithm> TaskAlgorithms,
                                    Set<TaskData> datasets,
                                    Set<EvaluatorType> QlMeasures,
                                    Set<IQuantitativeMeasure> QnMeasures)

Parameters:

TaskAlgorithms - the set of algorithms used to generate the predicted mappings

datasets - the set of the datasets to apply the algorithms on them. The should include source Cache, target Cache, goldstandard and predicted mapping

QlMeasures - the set of qualitative measures

QnMeasures - the set of quantitative measures

Returns:

List - contains list of multiple runs evaluation results corresponding to the algorithms, its implementation and used dataset

crossValidate

public List<EvaluationRun> crossValidate(AMLAlgorithm algorithm,
                                         List<LearningParameter> parameter,
                                         Set<TaskData> datasets,
                                         int foldNumber,
                                         Set<EvaluatorType> qlMeasures,
                                         Set<IQuantitativeMeasure> qnMeasures)

Parameters:

algorithm - the algorithm used to generate the predicted mappings

datasets - the set of the datasets to apply the algorithms on them. The should include source Cache, target Cache, goldstandard and predicted mapping

parameter - the parameters of the algorithm (will be set to default if this is null)

foldNumber - the number of subsamples to divide the data (k)

qlMeasures - the set of qualitative measures

qnMeasures - the set of quantitative measures

Returns:

List - contains list of multiple runs evaluation results corresponding to the algorithms, its implementation and used dataset

cacheUnion

public ACache cacheUnion(ACache a,
                         ACache b)

generateFolds

public List<FoldData> generateFolds(EvaluationData data,
                                    int foldNumber)

generateMappingFolds

public List<AMapping> generateMappingFolds(AMapping refMap,
                                           ACache source,
                                           ACache target,
                                           int foldNumber)

getRandomTargetInstance

public static String getRandomTargetInstance(ACache source,
                                             ACache target,
                                             List<String> values,
                                             Random random,
                                             HashMap<String,HashMap<String,Double>> refMap,
                                             String sourceInstance,
                                             int previousRandom)

removeSubMap

public AMapping removeSubMap(AMapping mainMap,
                             AMapping subMap)