org.dice_research.topicmodeling.lang.postagging

Class TreeTaggerWrapper

java.lang.Object

org.dice_research.topicmodeling.lang.postagging.AbstractPosTagger

org.dice_research.topicmodeling.lang.postagging.TreeTaggerWrapper

All Implemented Interfaces:

org.annolab.tt4j.TokenHandler<String>, org.dice_research.topicmodeling.automaton.AutomatonCallback, PosTagger

public class TreeTaggerWrapper
extends AbstractPosTagger
implements org.annolab.tt4j.TokenHandler<String>, org.dice_research.topicmodeling.automaton.AutomatonCallback

Field Summary

Fields

Modifier and Type	Field and Description
static String	ABBREVIATION_FILE_PROPERTY_KEY
private Set<String>	abbreviations
private org.dice_research.topicmodeling.automaton.BricsAutomatonManager	automaton
private String	currentText
private org.dice_research.topicmodeling.utils.doc.TermTokenizedText	currentTTText
private static org.slf4j.Logger	LOGGER
static String	MODEL_FILE_PROPERTY_KEY
private org.annolab.tt4j.TreeTaggerWrapper<String>	posTagger
private static String[]	TOKENIZER_PATTERN
private List<String>	tokens
static String	TREE_TAGGER_HOME_PROPERTY_KEY

Constructor Summary

Constructors

Modifier	Constructor and Description
private	TreeTaggerWrapper(org.annolab.tt4j.TreeTaggerWrapper<String> posTagger, Set<String> abbreviations)

Method Summary

Modifier and Type	Method and Description
private String	cleanLemma(String token, String lemma) Sometimes, the Treetagger returns more than one lemma.
static TreeTaggerWrapper	createTreeTaggerWrapper()
void	foundPattern(int patternId, int startPos, int length)
private void	setTermPropertiesFromPosTag(org.dice_research.topicmodeling.lang.Term term, String pos)
void	token(String token, String pos, String lemma)
protected org.dice_research.topicmodeling.utils.doc.TermTokenizedText	tokenizeText(String text)
protected org.dice_research.topicmodeling.utils.doc.TermTokenizedText	tokenizeText(String text, PosTaggingTermFilter filter)

Methods inherited from class org.dice_research.topicmodeling.lang.postagging.AbstractPosTagger

getFilter, setFilter, tokenize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

TREE_TAGGER_HOME_PROPERTY_KEY

public static final String TREE_TAGGER_HOME_PROPERTY_KEY

See Also:

Constant Field Values

MODEL_FILE_PROPERTY_KEY

public static final String MODEL_FILE_PROPERTY_KEY

See Also:

Constant Field Values

ABBREVIATION_FILE_PROPERTY_KEY

public static final String ABBREVIATION_FILE_PROPERTY_KEY

See Also:

Constant Field Values

TOKENIZER_PATTERN

private static final String[] TOKENIZER_PATTERN

posTagger

private org.annolab.tt4j.TreeTaggerWrapper<String> posTagger

currentTTText

private org.dice_research.topicmodeling.utils.doc.TermTokenizedText currentTTText

currentText

private String currentText

abbreviations

private Set<String> abbreviations

automaton

private org.dice_research.topicmodeling.automaton.BricsAutomatonManager automaton

tokens

private List<String> tokens

Constructor Detail

TreeTaggerWrapper

private TreeTaggerWrapper(org.annolab.tt4j.TreeTaggerWrapper<String> posTagger,
                          Set<String> abbreviations)

Method Detail

createTreeTaggerWrapper

public static TreeTaggerWrapper createTreeTaggerWrapper()

tokenizeText

protected org.dice_research.topicmodeling.utils.doc.TermTokenizedText tokenizeText(String text)

Specified by:

tokenizeText in class AbstractPosTagger

tokenizeText

protected org.dice_research.topicmodeling.utils.doc.TermTokenizedText tokenizeText(String text,
                                                                                   PosTaggingTermFilter filter)

Specified by:

tokenizeText in class AbstractPosTagger

token

public void token(String token,
                  String pos,
                  String lemma)

Specified by:

token in interface org.annolab.tt4j.TokenHandler<String>

cleanLemma

private String cleanLemma(String token,
                          String lemma)

Sometimes, the Treetagger returns more than one lemma. They are separated by a '|'. We will use the first lemma from this list.

setTermPropertiesFromPosTag

private void setTermPropertiesFromPosTag(org.dice_research.topicmodeling.lang.Term term,
                                         String pos)

foundPattern

public void foundPattern(int patternId,
                         int startPos,
                         int length)

Specified by:

foundPattern in interface org.dice_research.topicmodeling.automaton.AutomatonCallback