org.aksw.qa.commons.qald

Class Qald7CreationTool

java.lang.Object

org.aksw.qa.commons.qald.Qald7CreationTool

public class Qald7CreationTool
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
(package private) int	badQuestionCounter
private static String	DBO_URI
private int	duplicate
static Set<Dataset>	HYBRID_SETS
static Set<Dataset>	MULTILINGUAL_TRAIN_TEST_SETS QALD1 and QALD2 not multilingual!
private static String	RES_URI
private ThreadedSPARQL	sparql

Constructor Summary

Constructors

Constructor and Description
Qald7CreationTool()
Qald7CreationTool(String sparqlEndpoint, int timeout)

Method Summary

Modifier and Type	Method and Description
private void	addSave(Map<String,List<Qald7Question>> map, String question, Qald7Question q)
private boolean	checkAnswertypeSet(IQuestion q)
private boolean	checkAtleastSixLanguages(IQuestion q)
private boolean	checkIsOnlydbo(String sparqlQuery)
private boolean	checkKeywordsPresent(IQuestion q)
private boolean	checkSparqlPresent(IQuestion q)
void	createFileReport(Set<Qald7Question> allQuestions, String pathAndFilenameWithExtension, Set<Fail> ignoreFlags)
void	createFileReportForTestQuestions(Set<Dataset> datasets, boolean autocorrectOnlydbo, String pathAndFilenameWithExtension, Set<Fail> ignoreFlags) Creates a file report to all bad questions in given datasets
private void	createQald7Dataset(Set<Qald7Question> allQuestions, String path, String filenameWithoutExtension)
void	createQald7HybridDataset(Set<Dataset> hybridDatasets, String path, String filenameWithoutExtension) Creates the hybrid datasets.
void	createQald7MultilingualTrainDataset(Set<Dataset> datasets, boolean fileReport, boolean autocorrectOnlydbo, String path, String filenameWithoutExtension) Creates the multilingual train datasets.
void	destroy() Call this if you dont need this object anymore.
private Set<Qald7Question>	extractBadQuestionsFromAnnotated(Set<Qald7Question> questions, Set<Fail> ignoreFlags)
private Set<Qald7Question>	extractGoodTrainQuestionsFromAnnotated(Set<Qald7Question> questions)
private Set<Qald7Question>	findAndSelectBestDuplicate(List<Qald7Question> questions)
Set<String>	getAnswersFromServer(IQuestion q) Returns answers from official dbpedia endpoint to the stored sparql in IQuestion
Set<Qald7Question>	getQald7HybridQuestions(Set<Dataset> datasets) Returns all Hybrid questions for Qald7 (Loads all previous qald hybrid questions and drops duplicates).
Set<Qald7Question>	getQald7MultilingualTrainQuestions(Set<Dataset> datasets, boolean autocorrectOnlydbo) Loads all questions from given datasets, checks question integrity (is the stored answerset still identical with the one returned for given sparql query, is a sparql present and parseable, are at least 6 languages available with keywords, is an answertype set,...
Set<Qald7Question>	loadAndAnnotateTrain(Set<Dataset> datasets, boolean autocorrectOnlyDBO)
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DBO_URI

private static final String DBO_URI

See Also:

Constant Field Values

RES_URI

private static final String RES_URI

See Also:

Constant Field Values

duplicate

private int duplicate

MULTILINGUAL_TRAIN_TEST_SETS

public static final Set<Dataset> MULTILINGUAL_TRAIN_TEST_SETS

QALD1 and QALD2 not multilingual!

HYBRID_SETS

public static final Set<Dataset> HYBRID_SETS

sparql

private final ThreadedSPARQL sparql

badQuestionCounter

int badQuestionCounter

Constructor Detail

Qald7CreationTool

public Qald7CreationTool()

Qald7CreationTool

public Qald7CreationTool(String sparqlEndpoint,
                         int timeout)

Method Detail

getQald7HybridQuestions

public Set<Qald7Question> getQald7HybridQuestions(Set<Dataset> datasets)

Returns all Hybrid questions for Qald7 (Loads all previous qald hybrid questions and drops duplicates). This will set "hybrid:true" in all questions!!!

Parameters:

datasets - All datasets from which questions should be extracted

Returns:

All available unique questions from given datasets

createQald7HybridDataset

public void createQald7HybridDataset(Set<Dataset> hybridDatasets,
                                     String path,
                                     String filenameWithoutExtension)

Creates the hybrid datasets. Three files will be stored in given location: QALD-Json, Extended-Json and xml

Parameters:

hybridDatasets - The sets questions are taken from.

path - The path to write the datasets to.

filenameWithoutExtension - The name of the new dataset

createQald7MultilingualTrainDataset

public void createQald7MultilingualTrainDataset(Set<Dataset> datasets,
                                                boolean fileReport,
                                                boolean autocorrectOnlydbo,
                                                String path,
                                                String filenameWithoutExtension)

Creates the multilingual train datasets. Three files will be stored in given location: QALD-Json, Extended-Json and xml

Parameters:

datasets - The sets questions are taken from.

autocorrectOnlydbo - Is a bad Onlydbo-flag a exclusion criterion (Question wont appear in file) for a question or should it be autofixed?

path - The path to write the datasets to.

filenameWithoutExtension - The name of the new dataset

getQald7MultilingualTrainQuestions

public Set<Qald7Question> getQald7MultilingualTrainQuestions(Set<Dataset> datasets,
                                                             boolean autocorrectOnlydbo)

Loads all questions from given datasets, checks question integrity (is the stored answerset still identical with the one returned for given sparql query, is a sparql present and parseable, are at least 6 languages available with keywords, is an answertype set,... ) Also, duplicates are filtered out, only the candidate with the least error flags @link Fail will be in returned set. So, returned Questions are all clean. To get a duplicate free, with Fail annotated dataset, use loadAndAnnotateTrain(Set, boolean)

Parameters:

datasets - The datasets from which the questions are gathered

autocorrectOnlydbo - Is a bad Onlydbo-flag a exclusion criterion (Question wont appear in file) for a question or should it be autofixed?

Returns:

All clean duplicate free questions from given datasets.

loadAndAnnotateTrain

public Set<Qald7Question> loadAndAnnotateTrain(Set<Dataset> datasets,
                                               boolean autocorrectOnlyDBO)

checkSparqlPresent

private boolean checkSparqlPresent(IQuestion q)

checkAnswertypeSet

private boolean checkAnswertypeSet(IQuestion q)

checkKeywordsPresent

private boolean checkKeywordsPresent(IQuestion q)

checkAtleastSixLanguages

private boolean checkAtleastSixLanguages(IQuestion q)

getAnswersFromServer

public Set<String> getAnswersFromServer(IQuestion q)
                                 throws ExecutionException

Returns answers from official dbpedia endpoint to the stored sparql in IQuestion

Parameters:

q - Question to be answered

Returns:

Answers as string set

Throws:

ExecutionException

addSave

private void addSave(Map<String,List<Qald7Question>> map,
                     String question,
                     Qald7Question q)

findAndSelectBestDuplicate

private Set<Qald7Question> findAndSelectBestDuplicate(List<Qald7Question> questions)

extractGoodTrainQuestionsFromAnnotated

private Set<Qald7Question> extractGoodTrainQuestionsFromAnnotated(Set<Qald7Question> questions)

extractBadQuestionsFromAnnotated

private Set<Qald7Question> extractBadQuestionsFromAnnotated(Set<Qald7Question> questions,
                                                            Set<Fail> ignoreFlags)

createQald7Dataset

private void createQald7Dataset(Set<Qald7Question> allQuestions,
                                String path,
                                String filenameWithoutExtension)

createFileReportForTestQuestions

public void createFileReportForTestQuestions(Set<Dataset> datasets,
                                             boolean autocorrectOnlydbo,
                                             String pathAndFilenameWithExtension,
                                             Set<Fail> ignoreFlags)

Creates a file report to all bad questions in given datasets

Parameters:

datasets - All datasets to be checked

autocorrectOnlydbo - Is a bad Onlydbo-flag a exclusion criterion (Question wont appear in file) for a question or should it be autofixed?

pathAndFilenameWithExtension - Path and name of new file report

skipQuestionsWithTooLittleLanguages - Normally, multilingual datasets have at least six languages. When this flag is set, all questions with less languages will be ignored, otherwise its an error Fail and the question goes into the report

createFileReport

public void createFileReport(Set<Qald7Question> allQuestions,
                             String pathAndFilenameWithExtension,
                             Set<Fail> ignoreFlags)

checkIsOnlydbo

private boolean checkIsOnlydbo(String sparqlQuery)
                        throws org.apache.jena.query.QueryParseException

Throws:

org.apache.jena.query.QueryParseException

destroy

public void destroy()

Call this if you dont need this object anymore. Closes the Threads around the server connection to the sparql server.

main

public static void main(String[] args)