Package net.sansa_stack.hadoop.util
Class DeferredSeekablePushbackInputStream
- java.lang.Object
-
- java.io.InputStream
-
- java.io.FilterInputStream
-
- java.io.PushbackInputStream
-
- net.sansa_stack.hadoop.util.DeferredSeekablePushbackInputStream
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,SeekableDecorator
,org.apache.hadoop.fs.Seekable
public class DeferredSeekablePushbackInputStream extends PushbackInputStream implements SeekableDecorator
A wrapper for hadoop input streams created from codecs in ReadMode.BY_BLOCK: Defers reading by one byte such that position changes are advertised on the byte BEFORE the block boundary rather than on the byte AFTER it.
-
-
Field Summary
Fields Modifier and Type Field Description protected byte[]
fallbackBuffer
protected boolean
safeRead
Unsafe reads modify the byte after the reported number of read bytes in the read buffer.protected org.apache.hadoop.fs.Seekable
seekable
-
Fields inherited from class java.io.PushbackInputStream
buf, pos
-
Fields inherited from class java.io.FilterInputStream
in
-
-
Constructor Summary
Constructors Constructor Description DeferredSeekablePushbackInputStream(InputStream in)
DeferredSeekablePushbackInputStream(InputStream in, org.apache.hadoop.fs.Seekable seekable)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
afterSeek()
long
getPos()
org.apache.hadoop.fs.Seekable
getSeekable()
int
read()
int
read(byte[] b, int off, int len)
This method essentially delays reads by one byte.protected int
readInternal(byte[] b, int off, int len)
This method is assumed to be invoked with len >= 2void
seek(long l)
boolean
seekToNewSource(long l)
-
Methods inherited from class java.io.PushbackInputStream
available, close, mark, markSupported, reset, skip, unread, unread, unread
-
Methods inherited from class java.io.FilterInputStream
read
-
Methods inherited from class java.io.InputStream
nullInputStream, readAllBytes, readNBytes, readNBytes, transferTo
-
-
-
-
Field Detail
-
seekable
protected org.apache.hadoop.fs.Seekable seekable
-
safeRead
protected boolean safeRead
Unsafe reads modify the byte after the reported number of read bytes in the read buffer. Example: If buf := [a, b, c, d] and "read(buf)" returns 3, then buf[3] may no longer hold a 'd'. As this may cause data corruption, safe read mode uses an intermediate buffer with additional copying to prevent changing data outside of the reported range.
-
fallbackBuffer
protected byte[] fallbackBuffer
-
-
Constructor Detail
-
DeferredSeekablePushbackInputStream
public DeferredSeekablePushbackInputStream(InputStream in)
-
DeferredSeekablePushbackInputStream
public DeferredSeekablePushbackInputStream(InputStream in, org.apache.hadoop.fs.Seekable seekable)
-
-
Method Detail
-
getSeekable
public org.apache.hadoop.fs.Seekable getSeekable()
- Specified by:
getSeekable
in interfaceSeekableDecorator
-
read
public int read() throws IOException
- Overrides:
read
in classPushbackInputStream
- Throws:
IOException
-
read
public int read(byte[] b, int off, int len) throws IOException
This method essentially delays reads by one byte. Whereas hadoop's codec protocol reads one byte past block boundaries, this method buffers one byte on every read such that once the underlying stream reads accross the block boundary -- indicated by a position change -- only the byte before the block boundary is returned.- Overrides:
read
in classPushbackInputStream
- Returns:
- Throws:
IOException
-
readInternal
protected int readInternal(byte[] b, int off, int len) throws IOException
This method is assumed to be invoked with len >= 2- Throws:
IOException
-
afterSeek
protected void afterSeek()
-
seek
public void seek(long l) throws IOException
- Specified by:
seek
in interfaceorg.apache.hadoop.fs.Seekable
- Specified by:
seek
in interfaceSeekableDecorator
- Throws:
IOException
-
getPos
public long getPos() throws IOException
- Specified by:
getPos
in interfaceorg.apache.hadoop.fs.Seekable
- Specified by:
getPos
in interfaceSeekableDecorator
- Throws:
IOException
-
seekToNewSource
public boolean seekToNewSource(long l) throws IOException
- Specified by:
seekToNewSource
in interfaceorg.apache.hadoop.fs.Seekable
- Specified by:
seekToNewSource
in interfaceSeekableDecorator
- Throws:
IOException
-
-