org.dice_research.topicmodeling.preprocessing.docsupplier.decorator

Class UseNetTextExtractingSupplierDecorator

java.lang.Object

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.UseNetTextExtractingSupplierDecorator

All Implemented Interfaces:

Function<org.dice_research.topicmodeling.utils.doc.Document,org.dice_research.topicmodeling.utils.doc.Document>, DocumentSupplierDecorator, org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

public class UseNetTextExtractingSupplierDecorator
extends AbstractDocumentSupplierDecorator

Field Summary

Fields

Modifier and Type	Field and Description
private static String	PGP_MESSAGE_END
private static String	PGP_MESSAGE_START
private static Set<String>	USE_NET_HEADING_KEYS

Fields inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

documentSource

Constructor Summary

Constructors

Constructor and Description
UseNetTextExtractingSupplierDecorator(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource)

Method Summary

Modifier and Type	Method and Description
private int	getTextStartIndex(String originalText)
private String	identifyTextUsingPgpStatements(String text)
org.dice_research.topicmodeling.utils.doc.Document	prepareDocument(org.dice_research.topicmodeling.utils.doc.Document document)
private String	removeAddresses(String originalText)
private int	removeAuthorLine(String originalText, int lineStartIndex)

Methods inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

apply, getDecoratedDocumentSupplier, getNextDocument, setDecoratedDocumentSupplier, setDocumentStartId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

convertToStream

Methods inherited from interface java.util.function.Function

andThen, compose, identity

Field Detail

PGP_MESSAGE_START

private static final String PGP_MESSAGE_START

See Also:

Constant Field Values

PGP_MESSAGE_END

private static final String PGP_MESSAGE_END

See Also:

Constant Field Values

USE_NET_HEADING_KEYS

private static final Set<String> USE_NET_HEADING_KEYS

Constructor Detail

UseNetTextExtractingSupplierDecorator

public UseNetTextExtractingSupplierDecorator(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource)

Method Detail

prepareDocument

public org.dice_research.topicmodeling.utils.doc.Document prepareDocument(org.dice_research.topicmodeling.utils.doc.Document document)

Specified by:

prepareDocument in class AbstractDocumentSupplierDecorator

getTextStartIndex

private int getTextStartIndex(String originalText)

removeAuthorLine

private int removeAuthorLine(String originalText,
                             int lineStartIndex)

removeAddresses

private String removeAddresses(String originalText)

identifyTextUsingPgpStatements

private String identifyTextUsingPgpStatements(String text)