Class BufferOverReadableChannel<A>

java.lang.Object
org.aksw.commons.io.buffer.array.BufferOverReadableChannel<A>
All Implemented Interfaces:
ArrayReadable<A>, ArrayWritable<A>, BufferLike<A>, HasArrayOps<A>, Buffer<A>, ReadableChannelFactory<A>, ReadableChannelSource<A>, SeekableReadableChannelSource<A>
@ThreadSafe public class BufferOverReadableChannel<A> extends Object implements Buffer<A>
Implementation of a buffer that supports reading data from a channel. This implementation can serve as a plain buffer by simply providing a channel that provides no data. Instances of these class are thread safe, but the obtained channels are not; each channel should only be operated on by one thread. This class can be seen as a generalization of BufferedInputStream. Differences to BufferedInputStream: - this class caches all data read from the inputstream hence there is no mark / reset mechanism - buffer is split into buckets (no data copying required when allocating more space) - data is loaded on demand based on (possibly concurrent) requests to the seekable channels obtained with newChannel() Closest known-to-me Hadoop counterpart is BufferedFSInputStream (which is based on BufferedInputStream)
Author:
raven

  • Field Details

    • buckets

      protected A[] buckets
      The buffered data

    • arrayOps

      protected ArrayOps<A> arrayOps

    • activeEnd

      protected BufferOverReadableChannel.BucketPointer activeEnd
      End marker with two components (idx, pos) it is wrapped in an object to enable atomic replacement of the reference The pointer is monotonous in the sense that the end marker's logical linear location is only increased Reading an old version while a new one has been set will only cause a read to return on the old boundary, but a subsequent synchronized check for whether loading of additional data is needed is then made anyway

    • knownDataSize

      protected long knownDataSize
      The number of cached bytes. Corresponds to the linear representation of activeEnd.

    • dataSupplier

      protected ReadableChannel<A> dataSupplier

    • isDataSupplierConsumed

      protected boolean isDataSupplierConsumed

    • minReadSize

      protected int minReadSize

    • maxReadSize

      protected int maxReadSize
      Maximum number to read from the dataSupplier in one request

    • bufferView

      protected Buffer<A> bufferView

  • Constructor Details

    • BufferOverReadableChannel

      public BufferOverReadableChannel(ArrayOps<A> arrayOps, int initialBucketSize, ReadableChannel<A> dataSupplier, int minReadSize)

  • Method Details

    • getKnownDataSize

      public long getKnownDataSize()

    • getBuffer

      public Buffer<A> getBuffer()
      Returns a view of the buffered data only. Reading from that buffer must never consume data from the data supplier.

    • getDataSupplier

      public ReadableChannel<A> getDataSupplier()
      Return the data supplier. The data supplier usually needs to be closed when it is no longer needed.

    • setDataSupplier

      public void setDataSupplier(ReadableChannel<A> dataSupplier)
      Set a new data supplier whose data can be appended to this buffer on demand

    • getPosition

      public static <A> long getPosition(ArrayOps<A> arrayOps, A[] buckets, int idx, int pos)

    • getPointer

      public static <A> BufferOverReadableChannel.BucketPointer getPointer(ArrayOps<A> arrayOps, A[] buckets, BufferOverReadableChannel.BucketPointer end, long pos)
      Parameters:
      buckets -
      pos -
      Returns:
      Pointer to a valid location in the known data block or null

    • getPointerRel

      public static <A> BufferOverReadableChannel.BucketPointer getPointerRel(ArrayOps<A> arrayOps, A[] buckets, BufferOverReadableChannel.BucketPointer end, long startPos, BufferOverReadableChannel.BucketPointer startPtr, long pos)

    • isDataSupplierConsumed

      public boolean isDataSupplierConsumed()
      If this method returns true then further reads will not increase the known size unless a new data supplier is set.

    • nextBucketSize

      protected int nextBucketSize()

    • doRead

      public int doRead(ArraySink<A> dst, BufferOverReadableChannel.Channel reader)

    • loadDataUpTo

      protected void loadDataUpTo(long requestedPos)
      Preload data up to including the requested position. It is inclusive in order to allow for checking whether the requested position is in range.
      Parameters:
      requestedPos -

    • truncate

      public void truncate()
      Sets the end marker to position 0 effectively resizing the buffer to size 0. Allocated resources remain untouched.

    • loadFully

      public int loadFully(int amount, boolean exact)
      Transfer the given amount of data from the supplier to the end of the buffer. Returns the transferred amount or -1 if no data was transferred - should never return 0.
      Parameters:
      amount -
      exact - If true then the requested amount is not subject to adjustment by the minimum read length
      Returns:

    • loadData

      protected int loadData(int needed, boolean exact)
      fetch a chunk from the input stream

    • ensureCapacityInActiveBucket

      protected void ensureCapacityInActiveBucket()

    • getCapacity

      public long getCapacity()
      Description copied from interface: BufferLike
      Buffers with 'unlimited' capacity should return Long.MAX_VALUE
      Specified by:
      getCapacity in interface BufferLike<A>

    • write

      public void write(long offsetInBuffer, ReadableChannel<A> source, int amount) throws IOException
      Read a certain amount of data from a channel into this buffer starting at a certain position
      Throws:
      IOException

    • write

      public void write(long offsetInBuffer, A arrayWithItemsOfTypeT, int arrOffset, int arrLength) throws IOException
      Description copied from interface: ArrayWritable
      The method that needs to be implemented; all other methods default-delegate to this one.
      Specified by:
      write in interface ArrayWritable<A>
      Throws:
      IOException

    • getArrayOps

      public ArrayOps<A> getArrayOps()
      Specified by:
      getArrayOps in interface HasArrayOps<A>

    • readInto

      public int readInto(A tgt, int tgtOffset, long srcOffset, int length) throws IOException
      Specified by:
      readInto in interface ArrayReadable<A>
      Throws:
      IOException

    • newReadableChannel

      public SeekableReadableChannel<A> newReadableChannel() throws IOException
      Specified by:
      newReadableChannel in interface ArrayReadable<A>
      Specified by:
      newReadableChannel in interface ReadableChannelFactory<A>
      Specified by:
      newReadableChannel in interface SeekableReadableChannelSource<A>
      Throws:
      IOException

    • createForObjects

      public static <T> BufferOverReadableChannel<T[]> createForObjects(int initialCapacity)

    • createForBytes

      public static BufferOverReadableChannel<byte[]> createForBytes()

    • createForBytes

      public static BufferOverReadableChannel<byte[]> createForBytes(InputStream in)

    • createForBytes

      public static BufferOverReadableChannel<byte[]> createForBytes(InputStream in, int minReadSize)

    • createForBytes

      public static BufferOverReadableChannel<byte[]> createForBytes(ReadableChannel<byte[]> channel, int minReadSize)

    • main

      public static void main(String[] args) throws IOException
      Throws:
      IOException

    • main2

      public static void main2(String[] args) throws IOException
      Throws:
      IOException

    • newBufferedChannel

      public static <A> ReadableChannelSwitchable<A> newBufferedChannel(BufferOverReadableChannel<A> buffer)
      Create a switchable buffered channel where the buffering can be disabled later

    • debuffer

      public static <A> void debuffer(ReadableChannel<A> channel)
      Remove the buffer of channel created with newBufferedChannel