marmot.tokenize.openlp

Class OpenNlpTokenizer

java.lang.Object

marmot.tokenize.AbstractTokenizer

marmot.tokenize.openlp.OpenNlpTokenizer

All Implemented Interfaces:

java.io.Serializable, Tokenizer

public class OpenNlpTokenizer
extends AbstractTokenizer

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
OpenNlpTokenizer(opennlp.tools.tokenize.TokenizerModel model)

Method Summary

Modifier and Type	Method and Description
java.util.List<java.lang.String>	tokenize(java.lang.String untokenized)

Methods inherited from class marmot.tokenize.AbstractTokenizer

saveToFile

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpenNlpTokenizer

public OpenNlpTokenizer(opennlp.tools.tokenize.TokenizerModel model)

Method Detail

tokenize

public java.util.List<java.lang.String> tokenize(java.lang.String untokenized)