gate.persist
Class LuceneDataStoreImpl

java.lang.Object
  gate.util.AbstractFeatureBearer
      gate.persist.SerialDataStore
          gate.persist.LuceneDataStoreImpl

All Implemented Interfaces:

Searchable, SearchableDataStore, DataStore, CorpusListener, CreoleListener, FeatureBearer, NameBearer, Serializable, EventListener

public class LuceneDataStoreImpl
extends SerialDataStore
implements SearchableDataStore, CorpusListener, CreoleListener

See Also:

Serialized Form

Nested Class Summary

protected class	LuceneDataStoreImpl.IndexingTask
private class	LuceneDataStoreImpl.LabelledSoftReference Soft reference with an associated label.

Field Summary

protected ConcurrentMap<Object,LuceneDataStoreImpl.IndexingTask>	currentTasks Map keeping track of the most recent indexing task for each LR ID.
protected boolean	dataStoreClosing Indicates if the datastore is being closed.
protected ScheduledThreadPoolExecutor	executor Executor to run the indexing tasks
protected long	indexDelay Number of milliseconds we should wait after a sync before attempting to re-index a document.
protected Indexer	indexer Indexer to be used for indexing documents
protected Map	indexParameters Index Parameters
protected URL	indexURL URL of the index
protected Map<Object,LuceneDataStoreImpl.LabelledSoftReference>	lockObjects To store canonical lock objects for each LR ID.
protected ReferenceQueue<Object>	refQueue Reference queue with which the soft references in the lockObjects map will be registered.
protected Searcher	searcher Searcher to be used for searching the indexed documents
protected Map	searchParameters This is where we store the search parameters
private static long	serialVersionUID serial version UID

Fields inherited from class gate.persist.SerialDataStore

autoSaving, currentProtocolVersion, name, protocolVersionNumbers, randomiser, storageDir, versionFileName, versionNumber

Fields inherited from class gate.util.AbstractFeatureBearer

features

Fields inherited from interface gate.DataStore

DATASTORE_FEATURE_NAME, LR_ID_FEATURE_NAME

Constructor Summary

LuceneDataStoreImpl()

Method Summary

void	close() Close the data store.
void	datastoreClosed(CreoleEvent e) Called when a DataStore has been closed
void	datastoreCreated(CreoleEvent e) Called when a DataStore has been created
void	datastoreOpened(CreoleEvent e) Called when a DataStore has been opened
void	delete(String lrClassName, Object lrPersistenceId) Delete a resource from the data store.
void	documentAdded(CorpusEvent ce) This method is invoked whenever a document is added to a particular corpus
void	documentRemoved(CorpusEvent ce) This method is invoked whenever a document is removed from a corpus
long	getIndexDelay()
Indexer	getIndexer() Returns the Indexer
LanguageResource	getLr(String lrClassName, Object lrPersistenceId) Get a resource from the persistent store.
Searcher	getSearcher() Returns the Searcher
private Object	lockObjectForID(Object id) Obtain the lock object on which we must synchronize when loading or saving the LR with the given ID.
Hit[]	next(int numberOfPatterns) Returns the next numberOfPatterns
void	open() Open a connection to the data store.
private void	processRefQueue() Cleans up the lockObjects map by removing any entries whose SoftReference values have been cleared by the garbage collector.
protected void	queueForIndexing(Object lrID) Submits the given LR ID for indexing.
void	resourceLoaded(CreoleEvent e) Called when a new Resource has been loaded into the system
void	resourceRenamed(Resource resource, String oldName, String newName) Called when the creole register has renamed a resource.1
void	resourceUnloaded(CreoleEvent e) Called when a Resource has been removed from the system
boolean	search(String query, Map searchParameters) Search the datastore
void	setIndexDelay(long indexDelay) Sets the delay in milliseconds that we should wait after a sync before attempting to re-index a document.
void	setIndexer(Indexer indexer, Map indexParameters) Sets the Indexer to be used for indexing Datastore
void	setSearcher(Searcher searcher) This method is used to specify the searcher which is used for searchering the index
void	sync(LanguageResource lr) Save: synchonise the in-memory image of the LR with the persistent image.

Methods inherited from class gate.persist.SerialDataStore

addDatastoreListener, adopt, canReadLR, canWriteLR, constructPersistenceId, create, delete, equals, filterIgnoredFileNames, findLrIds, findLrIds, fireResourceAdopted, fireResourceDeleted, fireResourceWritten, getComment, getIconName, getLrIds, getLrName, getLrNames, getLrTypes, getName, getSecurityInfo, getSession, getStorageDir, getStorageUrl, getVersionFile, hashCode, isAutoSaving, isValidProtocolVersion, lockLr, random, removeDatastoreListener, setAutoSaving, setName, setSecurityInfo, setSession, setStorageDir, setStorageUrl, toString, unlockLr

Methods inherited from class gate.util.AbstractFeatureBearer

getFeatures, setFeatures

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface gate.DataStore

addDatastoreListener, adopt, canReadLR, canWriteLR, create, delete, findLrIds, findLrIds, getComment, getIconName, getLrIds, getLrName, getLrNames, getLrTypes, getSecurityInfo, getSession, getStorageUrl, isAutoSaving, lockLr, removeDatastoreListener, setAutoSaving, setSecurityInfo, setSession, setStorageUrl, unlockLr

Methods inherited from interface gate.util.FeatureBearer

getFeatures, setFeatures

Methods inherited from interface gate.util.NameBearer

getName, setName

Field Detail

serialVersionUID

private static final long serialVersionUID

serial version UID

See Also:

Constant Field Values

lockObjects

protected Map<Object,LuceneDataStoreImpl.LabelledSoftReference> lockObjects

To store canonical lock objects for each LR ID.

refQueue

protected ReferenceQueue<Object> refQueue

Reference queue with which the soft references in the lockObjects map will be registered.

dataStoreClosing

protected boolean dataStoreClosing

Indicates if the datastore is being closed.

executor

protected ScheduledThreadPoolExecutor executor

Executor to run the indexing tasks

currentTasks

protected ConcurrentMap<Object,LuceneDataStoreImpl.IndexingTask> currentTasks

Map keeping track of the most recent indexing task for each LR ID.

indexDelay

protected long indexDelay

Number of milliseconds we should wait after a sync before attempting to re-index a document. If sync is called again for the same document within this time then the timer for the re-indexing task is reset. Thus if several changes to the same document are made in quick succession it will only be re-indexed once. On the other hand, if the delay is set too long the document may never be indexed until the data store is closed. The default delay is 1000 (one second).

indexer

protected Indexer indexer

Indexer to be used for indexing documents

indexParameters

protected Map indexParameters

Index Parameters

indexURL

protected URL indexURL

URL of the index

searcher

protected Searcher searcher

Searcher to be used for searching the indexed documents

searchParameters

protected Map searchParameters

This is where we store the search parameters

Constructor Detail

LuceneDataStoreImpl

public LuceneDataStoreImpl()

Method Detail

close

public void close()
           throws PersistenceException

Close the data store.

Specified by:

close in interface DataStore

Overrides:

close in class SerialDataStore

Throws:

PersistenceException

open

public void open()
          throws PersistenceException

Open a connection to the data store.

Specified by:

open in interface DataStore

Overrides:

open in class SerialDataStore

Throws:

PersistenceException

lockObjectForID

private Object lockObjectForID(Object id)

Obtain the lock object on which we must synchronize when loading or saving the LR with the given ID.

Parameters:

id -

Returns:

processRefQueue

private void processRefQueue()

Cleans up the lockObjects map by removing any entries whose SoftReference values have been cleared by the garbage collector.

queueForIndexing

protected void queueForIndexing(Object lrID)

Submits the given LR ID for indexing. The task is delayed by 5 seconds, so multiple updates to the same LR in close succession do not un-necessarily trigger multiple re-indexing passes.

delete

public void delete(String lrClassName,
                   Object lrPersistenceId)
            throws PersistenceException

Delete a resource from the data store.

Specified by:

delete in interface DataStore

Overrides:

delete in class SerialDataStore

Parameters:

lrClassName - class name of the type of resource

lrPersistenceId - a data-store specific unique identifier for the resource

Throws:

PersistenceException

getLr

public LanguageResource getLr(String lrClassName,
                              Object lrPersistenceId)
                       throws PersistenceException,
                              SecurityException

Get a resource from the persistent store. Don't use this method - use Factory.createResource with DataStore and DataStoreInstanceId parameters set instead. (Sometimes I wish Java had "friend" declarations...)

Specified by:

getLr in interface DataStore

Overrides:

getLr in class SerialDataStore

Throws:

PersistenceException

SecurityException

sync

public void sync(LanguageResource lr)
          throws PersistenceException

Save: synchonise the in-memory image of the LR with the persistent image.

Specified by:

sync in interface DataStore

Overrides:

sync in class SerialDataStore

Throws:

PersistenceException

setIndexer

public void setIndexer(Indexer indexer,
                       Map indexParameters)
                throws IndexException

Sets the Indexer to be used for indexing Datastore

Specified by:

setIndexer in interface Searchable

indexParameters - - parameters required by the specific implementation of provided indexer

Throws:

IndexException

getIndexer

public Indexer getIndexer()

Description copied from interface: Searchable

Returns the Indexer

Specified by:

getIndexer in interface Searchable

Returns:

setSearcher

public void setSearcher(Searcher searcher)
                 throws SearchException

Description copied from interface: Searchable

This method is used to specify the searcher which is used for searchering the index

Specified by:

setSearcher in interface Searchable

Throws:

SearchException

getSearcher

public Searcher getSearcher()

Description copied from interface: Searchable

Returns the Searcher

Specified by:

getSearcher in interface Searchable

Returns:

setIndexDelay

public void setIndexDelay(long indexDelay)

Sets the delay in milliseconds that we should wait after a sync before attempting to re-index a document. If sync is called again for the same document within this time then the timer for the re-indexing task is reset. Thus if several changes to the same document are made in quick succession it will only be re-indexed once. On the other hand, if the delay is set too long the document may never be indexed until the data store is closed. The default delay is 1000ms (one second), which should be appropriate for usage in the GATE GUI.

getIndexDelay

public long getIndexDelay()

search

public boolean search(String query,
                      Map searchParameters)
               throws SearchException

Search the datastore

Specified by:

search in interface Searchable

searchParameters - - parameters required for searching an index (e.g. location of the index)

Returns:

true if the search was successful

Throws:

SearchException

next

public Hit[] next(int numberOfPatterns)
           throws SearchException

Returns the next numberOfPatterns

Specified by:

next in interface Searchable

Parameters:

numberOfPatterns -

Returns:

null if no patterns found

Throws:

SearchException

documentRemoved

public void documentRemoved(CorpusEvent ce)

This method is invoked whenever a document is removed from a corpus

Specified by:

documentRemoved in interface CorpusListener

documentAdded

public void documentAdded(CorpusEvent ce)

This method is invoked whenever a document is added to a particular corpus

Specified by:

documentAdded in interface CorpusListener

datastoreClosed

public void datastoreClosed(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a DataStore has been closed

Specified by:

datastoreClosed in interface CreoleListener

datastoreCreated

public void datastoreCreated(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a DataStore has been created

Specified by:

datastoreCreated in interface CreoleListener

datastoreOpened

public void datastoreOpened(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a DataStore has been opened

Specified by:

datastoreOpened in interface CreoleListener

resourceLoaded

public void resourceLoaded(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a new Resource has been loaded into the system

Specified by:

resourceLoaded in interface CreoleListener

resourceRenamed

public void resourceRenamed(Resource resource,
                            String oldName,
                            String newName)

Description copied from interface: CreoleListener

Called when the creole register has renamed a resource.1

Specified by:

resourceRenamed in interface CreoleListener

resourceUnloaded

public void resourceUnloaded(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a Resource has been removed from the system

Specified by:

resourceUnloaded in interface CreoleListener