org.dice_research.topicmodeling.preprocessing.docsupplier.decorator

Class NewsDeMarkupRemovingSupplierDecorator

java.lang.Object

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.NewsDeMarkupRemovingSupplierDecorator

All Implemented Interfaces:

Function<org.dice_research.topicmodeling.utils.doc.Document,org.dice_research.topicmodeling.utils.doc.Document>, DocumentSupplierDecorator, org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

public class NewsDeMarkupRemovingSupplierDecorator
extends AbstractDocumentSupplierDecorator

Field Summary

Fields

Modifier and Type	Field and Description
private dk.brics.automaton.RunAutomaton	charAutomaton
private dk.brics.automaton.RunAutomaton	tagAutomaton
private dk.brics.automaton.RunAutomaton	tooltipAutomaton

Fields inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

documentSource

Constructor Summary

Constructors

Constructor and Description
NewsDeMarkupRemovingSupplierDecorator(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource)

Method Summary

Modifier and Type	Method and Description
String	cleanText(String text)
private void	handleHtmlEncodedChar(StringBuilder cleanText, String text, int pos, int length)
private void	handleHtmlTag(StringBuilder cleanText, String text, int pos, int length)
private void	handleToolTip(StringBuilder cleanText, StringBuilder toolTips, String text, int pos, int length)
org.dice_research.topicmodeling.utils.doc.Document	prepareDocument(org.dice_research.topicmodeling.utils.doc.Document document)

Methods inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

apply, getDecoratedDocumentSupplier, getNextDocument, setDecoratedDocumentSupplier, setDocumentStartId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

convertToStream

Methods inherited from interface java.util.function.Function

andThen, compose, identity

Field Detail

tagAutomaton

private dk.brics.automaton.RunAutomaton tagAutomaton

charAutomaton

private dk.brics.automaton.RunAutomaton charAutomaton

tooltipAutomaton

private dk.brics.automaton.RunAutomaton tooltipAutomaton

Constructor Detail

NewsDeMarkupRemovingSupplierDecorator

public NewsDeMarkupRemovingSupplierDecorator(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource)

Method Detail

prepareDocument

public org.dice_research.topicmodeling.utils.doc.Document prepareDocument(org.dice_research.topicmodeling.utils.doc.Document document)

Specified by:

prepareDocument in class AbstractDocumentSupplierDecorator

cleanText

public String cleanText(String text)

handleHtmlTag

private void handleHtmlTag(StringBuilder cleanText,
                           String text,
                           int pos,
                           int length)

handleHtmlEncodedChar

private void handleHtmlEncodedChar(StringBuilder cleanText,
                                   String text,
                                   int pos,
                                   int length)

handleToolTip

private void handleToolTip(StringBuilder cleanText,
                           StringBuilder toolTips,
                           String text,
                           int pos,
                           int length)