Package org.aksw.commons.io.cache

Class ReadableChannelOverSliceWithCache<A>

java.lang.Object
org.aksw.commons.util.closeable.AutoCloseableBase
org.aksw.commons.util.closeable.AutoCloseableWithLeakDetectionBase
org.aksw.commons.io.cache.ReadableChannelOverSliceWithCache<A>
Type Parameters:
T -
All Implemented Interfaces:
Closeable, AutoCloseable, Channel, HasArrayOps<A>, ReadableChannel<A>, ReadableSource<A>
public class ReadableChannelOverSliceWithCache<A> extends AutoCloseableWithLeakDetectionBase implements ReadableChannel<A>
The class drives the iteration of items from the cache and triggers fetching of data as necessary. Thereby this class does not fetch the data directly, but it declares interest in data ranges. The SmartRangeCache will schedule loading of the region at least as long as interest is expressed.
Author:
raven

  • Field Details

    • slice

      protected Slice<A> slice

    • cache

      protected AdvancedRangeCacheImpl<A> cache

    • pageRange

      protected SliceAccessor<A> pageRange

    • requestRange

      protected com.google.common.collect.Range<Long> requestRange
      The original request range by this request. In general, the original request range has to be broken down into smaller ranges because of result size limits of the backend

    • nextCheckpointOffset

      protected long nextCheckpointOffset
      At a checkpoint the data fetching tasks for the next blocks are scheduled

    • currentOffset

      protected long currentOffset

    • maxReadAheadItemCount

      protected int maxReadAheadItemCount

    • workerToSlot

      protected Map<RangeRequestWorkerImpl<A>,Slot<Long>> workerToSlot

    • maxRedundantFetchSize

      protected long maxRedundantFetchSize

  • Constructor Details

    • ReadableChannelOverSliceWithCache

      public ReadableChannelOverSliceWithCache(AdvancedRangeCacheImpl<A> cache, com.google.common.collect.Range<Long> requestRange, int maxReadAheadItemCount)

    • ReadableChannelOverSliceWithCache

      public ReadableChannelOverSliceWithCache(AdvancedRangeCacheImpl<A> cache, long nextCheckpointOffset, LongSupplier offsetSupplier)

  • Method Details

    • getNextCheckpointOffset

      public long getNextCheckpointOffset()

    • getPageRange

      public SliceAccessor<A> getPageRange()

    • checkpoint

      protected void checkpoint(long n)
      Schedule ensured loading of the next 'n' items since the last checkpoint. Check whether there are any gaps ahead that require scheduling requests to the backend

    • clearPassedSlots

      public void clearPassedSlots()
      Clear all slots of workers with a value less than the currentOffset. These are workers for which we did not schedule any further tasks

    • scheduleWorkerToGaps

      protected void scheduleWorkerToGaps(com.google.common.collect.RangeSet<Long> gaps)

    • processGaps

      protected void processGaps(com.google.common.collect.RangeSet<Long> gaps, long start, long end)

    • isOpen

      public boolean isOpen()
      Specified by:
      isOpen in interface Channel

    • closeActual

      protected void closeActual()
      Overrides:
      closeActual in class AutoCloseableBase

    • read

      public int read(A tgt, int tgtOffset, int length) throws IOException
      Description copied from interface: ReadableSource
      Read method following the usual InputStream protocol.
      Specified by:
      read in interface ReadableSource<A>
      Parameters:
      tgt - The array into which to put the read data
      tgtOffset - Offset into array where to start writing
      length - Maximum number of items to read.
      Returns:
      The number of items read. Return -1 if end of data was reached, and 0 iff length was 0.
      Throws:
      IOException

    • getArrayOps

      public ArrayOps<A> getArrayOps()
      Specified by:
      getArrayOps in interface HasArrayOps<A>