org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.splitter

Class SentenceBasedDocumentTextSplitter

java.lang.Object

org.dice_research.topicmodeling.preprocessing.docsupplier.AbstractDocumentSupplier

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.splitter.AbstractSplittingDocumentSupplierDecorator

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.splitter.SentenceBasedDocumentTextSplitter

All Implemented Interfaces:

DocumentSupplierDecorator, org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

public class SentenceBasedDocumentTextSplitter
extends AbstractSplittingDocumentSupplierDecorator

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOGGER
private opennlp.tools.sentdetect.SentenceDetectorME	sentenceDetector

Fields inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.splitter.AbstractSplittingDocumentSupplierDecorator

documentSource, queue

Fields inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.AbstractDocumentSupplier

nextDocumentId

Constructor Summary

Constructors

Constructor and Description
SentenceBasedDocumentTextSplitter(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource, opennlp.tools.sentdetect.SentenceDetectorME sentenceDetector)

Method Summary

Modifier and Type	Method and Description
private org.dice_research.topicmodeling.utils.doc.Document	createDocument(String documentText, int start, int end, org.dice_research.topicmodeling.utils.doc.TermTokenizedText tttext, org.dice_research.topicmodeling.utils.doc.Document document)
protected void	splitDocument(org.dice_research.topicmodeling.utils.doc.Document document) In this method the splitter should split up the given document and add all new documents to the AbstractSplittingDocumentSupplierDecorator.queue.
private void	splitDocument(org.dice_research.topicmodeling.utils.doc.Document document, String text)
private void	splitDocument(org.dice_research.topicmodeling.utils.doc.Document document, String text, List<org.dice_research.topicmodeling.lang.Term> terms)

Methods inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.splitter.AbstractSplittingDocumentSupplierDecorator

addToQueue, getDecoratedDocumentSupplier, getNextDocument, setDecoratedDocumentSupplier

Methods inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.AbstractDocumentSupplier

getNextDocumentId, setDocumentStartId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

convertToStream, setDocumentStartId

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

sentenceDetector

private opennlp.tools.sentdetect.SentenceDetectorME sentenceDetector

Constructor Detail

SentenceBasedDocumentTextSplitter

public SentenceBasedDocumentTextSplitter(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource,
                                         opennlp.tools.sentdetect.SentenceDetectorME sentenceDetector)

Method Detail

splitDocument

protected void splitDocument(org.dice_research.topicmodeling.utils.doc.Document document)

Description copied from class: AbstractSplittingDocumentSupplierDecorator

In this method the splitter should split up the given document and add all new documents to the AbstractSplittingDocumentSupplierDecorator.queue.

Specified by:

splitDocument in class AbstractSplittingDocumentSupplierDecorator

splitDocument

private void splitDocument(org.dice_research.topicmodeling.utils.doc.Document document,
                           String text,
                           List<org.dice_research.topicmodeling.lang.Term> terms)

splitDocument

private void splitDocument(org.dice_research.topicmodeling.utils.doc.Document document,
                           String text)

createDocument

private org.dice_research.topicmodeling.utils.doc.Document createDocument(String documentText,
                                                                          int start,
                                                                          int end,
                                                                          org.dice_research.topicmodeling.utils.doc.TermTokenizedText tttext,
                                                                          org.dice_research.topicmodeling.utils.doc.Document document)