org.dice_research.topicmodeling.preprocessing.docsupplier.decorator

Class VocabularyReductionMappingApplyingSupplierDecorator

java.lang.Object

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.VocabularyReductionMappingApplyingSupplierDecorator

All Implemented Interfaces:

Function<org.dice_research.topicmodeling.utils.doc.Document,org.dice_research.topicmodeling.utils.doc.Document>, DocumentSupplierDecorator, org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

public class VocabularyReductionMappingApplyingSupplierDecorator
extends AbstractDocumentSupplierDecorator

Field Summary

Fields

Modifier and Type	Field and Description
private int[]	mapping
static int	REMOVED_WORD

Fields inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

documentSource

Constructor Summary

Constructors

Constructor and Description
VocabularyReductionMappingApplyingSupplierDecorator(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource, int[] mapping)

Method Summary

Modifier and Type	Method and Description
static int[]	createMapping(org.dice_research.topicmodeling.utils.vocabulary.Vocabulary vocabulary, com.carrotsearch.hppc.BitSet keptWords)
protected static int	lastSetBit(com.carrotsearch.hppc.BitSet bits, int from) Returns the last 1 bit in the given bit set before the given position (excluding) or -1 if there is no such bit.
protected org.dice_research.topicmodeling.utils.doc.Document	prepareDocument(org.dice_research.topicmodeling.utils.doc.Document document)
static org.dice_research.topicmodeling.utils.vocabulary.Vocabulary	updateVocabulary(org.dice_research.topicmodeling.utils.vocabulary.Vocabulary vocabulary, int[] mapping)
protected org.dice_research.topicmodeling.utils.doc.DocumentWordCounts	updateWordCounts(org.dice_research.topicmodeling.utils.doc.DocumentWordCounts counts)
protected org.dice_research.topicmodeling.utils.doc.DocumentTextWordIds	updateWordIds(org.dice_research.topicmodeling.utils.doc.DocumentTextWordIds wordIds)

Methods inherited from class org.dice_research.topicmodeling.preprocessing.docsupplier.decorator.AbstractDocumentSupplierDecorator

apply, getDecoratedDocumentSupplier, getNextDocument, setDecoratedDocumentSupplier, setDocumentStartId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier

convertToStream

Methods inherited from interface java.util.function.Function

andThen, compose, identity

Field Detail

REMOVED_WORD

public static final int REMOVED_WORD

See Also:

Constant Field Values

mapping

private int[] mapping

Constructor Detail

VocabularyReductionMappingApplyingSupplierDecorator

public VocabularyReductionMappingApplyingSupplierDecorator(org.dice_research.topicmodeling.preprocessing.docsupplier.DocumentSupplier documentSource,
                                                           int[] mapping)

Method Detail

createMapping

public static int[] createMapping(org.dice_research.topicmodeling.utils.vocabulary.Vocabulary vocabulary,
                                  com.carrotsearch.hppc.BitSet keptWords)

updateVocabulary

public static org.dice_research.topicmodeling.utils.vocabulary.Vocabulary updateVocabulary(org.dice_research.topicmodeling.utils.vocabulary.Vocabulary vocabulary,
                                                                                           int[] mapping)

lastSetBit

protected static int lastSetBit(com.carrotsearch.hppc.BitSet bits,
                                int from)

Returns the last 1 bit in the given bit set before the given position (excluding) or -1 if there is no such bit.

Parameters:

bits - the bit set in which should be searched

from - the position (excluding) from which the algorithm should start to search backwards

Returns:

the position of the bit or -1 if there is no such bit

prepareDocument

protected org.dice_research.topicmodeling.utils.doc.Document prepareDocument(org.dice_research.topicmodeling.utils.doc.Document document)

Specified by:

prepareDocument in class AbstractDocumentSupplierDecorator

updateWordIds

protected org.dice_research.topicmodeling.utils.doc.DocumentTextWordIds updateWordIds(org.dice_research.topicmodeling.utils.doc.DocumentTextWordIds wordIds)

updateWordCounts

protected org.dice_research.topicmodeling.utils.doc.DocumentWordCounts updateWordCounts(org.dice_research.topicmodeling.utils.doc.DocumentWordCounts counts)