org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.ner

Class EntityBasedTokenizer

java.lang.Object

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.ner.AbstractNerPropagationPreprocessor

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.ner.EntityBasedTokenizer

All Implemented Interfaces:

EntityTokenSurfaceFormMappingSupplier, NerPropagationPreprocessor

Direct Known Subclasses:

EntityPropagator

public class EntityBasedTokenizer
extends AbstractNerPropagationPreprocessor
implements EntityTokenSurfaceFormMappingSupplier

Field Summary

Fields

Modifier and Type	Field and Description
private static String	CHARS_TO_INSERT
private static String	CHARS_TO_REPLACE
protected EntityTermMapping	surfaceFormsMapping

Fields inherited from interface org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.ner.EntityTokenSurfaceFormMappingSupplier

CORRECT_SURFACE_FORM_ID, ENTITY_TOKEN_GONE_THROUGH_POS_TAGGING_ID

Constructor Summary

Constructors

Constructor and Description
EntityBasedTokenizer()

Method Summary

Modifier and Type	Method and Description
EntityTermMapping	getLastEntityTokenSurfaceFormMapping() Returns the mapping created since the last call of this method.
protected org.dice_research.topicmodeling.lang.Term[]	getTokensAfterPosTagging(org.dice_research.topicmodeling.utils.doc.ner.NamedEntityInText entity, String surfaceForm)
protected String	processEntity(org.dice_research.topicmodeling.utils.doc.ner.NamedEntityInText entity, String surfaceForm, Set<String> tokens)

Methods inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.ner.AbstractNerPropagationPreprocessor

preprocessNamedEntities

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CHARS_TO_REPLACE

private static final String CHARS_TO_REPLACE

See Also:

Constant Field Values

CHARS_TO_INSERT

private static final String CHARS_TO_INSERT

See Also:

Constant Field Values

surfaceFormsMapping

protected EntityTermMapping surfaceFormsMapping

Constructor Detail

EntityBasedTokenizer

public EntityBasedTokenizer()

Method Detail

processEntity

protected String processEntity(org.dice_research.topicmodeling.utils.doc.ner.NamedEntityInText entity,
                               String surfaceForm,
                               Set<String> tokens)

Specified by:

processEntity in class AbstractNerPropagationPreprocessor

getTokensAfterPosTagging

protected org.dice_research.topicmodeling.lang.Term[] getTokensAfterPosTagging(org.dice_research.topicmodeling.utils.doc.ner.NamedEntityInText entity,
                                                                               String surfaceForm)

getLastEntityTokenSurfaceFormMapping

public EntityTermMapping getLastEntityTokenSurfaceFormMapping()

Description copied from interface: EntityTokenSurfaceFormMappingSupplier

Returns the mapping created since the last call of this method.

Specified by:

getLastEntityTokenSurfaceFormMapping in interface EntityTokenSurfaceFormMappingSupplier

Returns: