Class BufferOverInputStream

java.lang.Object
org.aksw.commons.io.hadoop.binseach.bz2.BufferOverInputStream
All Implemented Interfaces:
AutoCloseable, ChannelFactory<Seekable>
@ThreadSafe public class BufferOverInputStream extends Object implements ChannelFactory<Seekable>
FIXME This class should be removed because it is superseded by BufferOverReadableChannel in aksw-commons-io! Implementation of a byte array that caches data in buckets from an InputStream. Instances of these class are thread safe, but the obtained channels are not; each channel should only be operated on by one thread. Differences to BufferedInputStream - this class caches all data read from the inputstream hence there is no mark / reset mechanism - buffer is split into buckets (no data copying required when allocating more space) - data is loaded on demand based on (possibly concurrent) requests to the seekable channels obtained with newChannel() Closest known-to-me Hadoop counterpart is BufferedFSInputStream (which is based on BufferedInputStream)
Author:
raven

  • Field Details

    • buckets

      protected byte[][] buckets
      The buffered data

    • activeEnd

      protected BufferOverInputStream.BucketPointer activeEnd
      End marker with two components (idx, pos) it is wrapped in an object to enable atomic replacement of the reference The pointer is monotonous in the sense that the end marker's logical linear location is only increased Reading an old version while a new one has been set will only cause a read to return on the old boundary, but a subsequent synchronized check for whether loading of additional data is needed is then made anyway

    • knownDataSize

      protected long knownDataSize
      The number of cached bytes. Corresponds to the linear representation of activeEnd.

    • dataSupplier

      protected InputStream dataSupplier
      Supplier for additional data

    • minReadSize

      protected int minReadSize

    • maxReadSize

      protected int maxReadSize
      Maximum number to read from the dataSupplier in one request

    • isDataSupplierConsumed

      protected boolean isDataSupplierConsumed
      Flag to indicate that the dataSupplier has been consumed This is the case when dataSupplier(buffer) returns -1

  • Constructor Details

    • BufferOverInputStream

      public BufferOverInputStream(int initialBucketSize, InputStream dataSupplier)

  • Method Details

    • getKnownDataSize

      public long getKnownDataSize()

    • isDataSupplierConsumed

      public boolean isDataSupplierConsumed()

    • create

      public static BufferOverInputStream create(InputStream in, int maxReadSize, int... preconfiguredBucketSizes)
      Parameters:
      in -
      maxReadSize - Maximum number of bytes to request form the input stream at once
      preconfiguredBucketSizes -
      maxReadSize -
      Returns:

    • getPosition

      public static long getPosition(byte[][] buckets, int idx, int pos)

    • getPointer

      public static BufferOverInputStream.BucketPointer getPointer(byte[][] buckets, BufferOverInputStream.BucketPointer end, long pos)
      Parameters:
      buckets -
      pos -
      Returns:
      Pointer to a valid location in the know data block or null

    • newChannel

      public Seekable newChannel()
      Specified by:
      newChannel in interface ChannelFactory<Seekable>

    • nextBucketSize

      protected int nextBucketSize()

    • doRead

      public int doRead(BufferOverInputStream.ByteArrayChannel reader, ByteBuffer dst)

    • loadDataUpTo

      protected void loadDataUpTo(long requestedPos)
      Preload data up to including the requested position. It is inclusive in order to allow for checking whether the requested position is in range.
      Parameters:
      requestedPos -

    • loadData

      protected void loadData(int needed)
      fetch a chunk from the input stream

    • ensureCapacityInActiveBucket

      protected void ensureCapacityInActiveBucket()

    • main

      public static void main(String[] args) throws Exception
      Throws:
      Exception

    • main2

      public static void main2(String[] args) throws Exception
      Throws:
      Exception

    • close

      public void close() throws Exception
      Specified by:
      close in interface AutoCloseable
      Throws:
      Exception