Package org.aksw.palmetto.corpus.lucene

Class CachingWindowSupportingLuceneCorpusAdapter

java.lang.Object

org.aksw.palmetto.corpus.lucene.LuceneCorpusAdapter

org.aksw.palmetto.corpus.lucene.WindowSupportingLuceneCorpusAdapter

org.aksw.palmetto.corpus.lucene.CachingWindowSupportingLuceneCorpusAdapter

All Implemented Interfaces:

BooleanDocumentSupportingAdapter, CorpusAdapter, WindowSupportingAdapter

public class CachingWindowSupportingLuceneCorpusAdapter
extends WindowSupportingLuceneCorpusAdapter

An Extension of the WindowSupportingLuceneCorpusAdapter that caches word positions of the single documents. Preliminary tests show that it can improve the performance if the same words are requested very often. However, it reduces the performance if this is not the case.

Author:

Michael Röder (michael.roeder@uni-paderborn.de)

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	CachingWindowSupportingLuceneCorpusAdapter.CachedWordData

Field Summary

Fields

Modifier and Type	Field	Description
protected com.google.common.cache.LoadingCache<String,CachingWindowSupportingLuceneCorpusAdapter.CachedWordData>	cache
private static org.slf4j.Logger	LOGGER

Fields inherited from class org.aksw.palmetto.corpus.lucene.WindowSupportingLuceneCorpusAdapter

docLengthFieldName, histogram, HISTOGRAM_FILE_SUFFIX

Fields inherited from class org.aksw.palmetto.corpus.lucene.LuceneCorpusAdapter

contexts, dirReader, fieldName, reader

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	CachingWindowSupportingLuceneCorpusAdapter(org.apache.lucene.index.DirectoryReader dirReader, org.apache.lucene.index.AtomicReader[] reader, org.apache.lucene.index.AtomicReaderContext[] contexts, String textFieldName, String docLengthFieldName, int[][] histogram, int maxCacheSize)

Method Summary

Modifier and Type	Method	Description
static CachingWindowSupportingLuceneCorpusAdapter	create(String indexPath, String textFieldName, String docLengthFieldName, int maxCacheSize)
protected CachingWindowSupportingLuceneCorpusAdapter.CachedWordData	requestDocumentsWithWord(String word)
protected void	requestDocumentsWithWord(String word, com.carrotsearch.hppc.IntObjectOpenHashMap<com.carrotsearch.hppc.IntArrayList[]> positionsInDocs, com.carrotsearch.hppc.IntIntOpenHashMap docLengths, int wordId, int numberOfWords)

Methods inherited from class org.aksw.palmetto.corpus.lucene.WindowSupportingLuceneCorpusAdapter

addDocLength, create, gatherWordPositions, getDocumentSizeHistogram, requestWordPositionsInDocuments

Methods inherited from class org.aksw.palmetto.corpus.lucene.LuceneCorpusAdapter

close, create, getDocumentsWithWord, getDocumentsWithWordAsSet, getDocumentsWithWords, getDocumentsWithWordsAsSet, getNumberOfDocuments

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.aksw.palmetto.corpus.CorpusAdapter

close

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

cache

protected com.google.common.cache.LoadingCache<String,CachingWindowSupportingLuceneCorpusAdapter.CachedWordData> cache

Constructor Detail

CachingWindowSupportingLuceneCorpusAdapter

protected CachingWindowSupportingLuceneCorpusAdapter(org.apache.lucene.index.DirectoryReader dirReader,
                                                     org.apache.lucene.index.AtomicReader[] reader,
                                                     org.apache.lucene.index.AtomicReaderContext[] contexts,
                                                     String textFieldName,
                                                     String docLengthFieldName,
                                                     int[][] histogram,
                                                     int maxCacheSize)

Method Detail

requestDocumentsWithWord

protected void requestDocumentsWithWord(String word,
                                        com.carrotsearch.hppc.IntObjectOpenHashMap<com.carrotsearch.hppc.IntArrayList[]> positionsInDocs,
                                        com.carrotsearch.hppc.IntIntOpenHashMap docLengths,
                                        int wordId,
                                        int numberOfWords)

Overrides:

requestDocumentsWithWord in class WindowSupportingLuceneCorpusAdapter

requestDocumentsWithWord

protected CachingWindowSupportingLuceneCorpusAdapter.CachedWordData requestDocumentsWithWord(String word)

create

public static CachingWindowSupportingLuceneCorpusAdapter create(String indexPath,
                                                                String textFieldName,
                                                                String docLengthFieldName,
                                                                int maxCacheSize)
                                                         throws org.apache.lucene.index.CorruptIndexException,
                                                                IOException

Throws:

org.apache.lucene.index.CorruptIndexException

IOException