Package net.sansa_stack.hadoop.util

Class DeferredSeekablePushbackInputStream

java.lang.Object

java.io.InputStream

java.io.FilterInputStream

java.io.PushbackInputStream

net.sansa_stack.hadoop.util.DeferredSeekablePushbackInputStream

All Implemented Interfaces:

Closeable, AutoCloseable, SeekableDecorator, org.apache.hadoop.fs.Seekable

public class DeferredSeekablePushbackInputStream
extends PushbackInputStream
implements SeekableDecorator

A wrapper for hadoop input streams created from codecs in ReadMode.BY_BLOCK: Defers reading by one byte such that position changes are advertised on the byte BEFORE the block boundary rather than on the byte AFTER it.

Field Summary

Fields

Modifier and Type	Field	Description
protected byte[]	fallbackBuffer
protected net.sansa_stack.hadoop.util.DeferredSeekablePushbackInputStream.ReadMode	readMode	Unsafe reads modify the byte after the reported number of read bytes in the read buffer.
protected org.apache.hadoop.fs.Seekable	seekable

Fields inherited from class java.io.PushbackInputStream

buf, pos

Fields inherited from class java.io.FilterInputStream

in

Constructor Summary

Constructors

Constructor	Description
DeferredSeekablePushbackInputStream(InputStream in)
DeferredSeekablePushbackInputStream(InputStream in, org.apache.hadoop.fs.Seekable seekable)

Method Summary

Modifier and Type	Method	Description
protected void	afterSeek()
void	close()
long	getPos()
org.apache.hadoop.fs.Seekable	getSeekable()
int	read()
int	read(byte[] b, int off, int len)	This method essentially delays reads by one byte.
protected int	readInternal(byte[] b, int off, int len)	This method is assumed to be invoked with len >= 2
void	seek(long l)
boolean	seekToNewSource(long l)

Methods inherited from class java.io.PushbackInputStream

available, mark, markSupported, reset, skip, unread, unread, unread

Methods inherited from class java.io.FilterInputStream

read

Methods inherited from class java.io.InputStream

nullInputStream, readAllBytes, readNBytes, readNBytes, skipNBytes, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

seekable

protected org.apache.hadoop.fs.Seekable seekable

readMode

protected net.sansa_stack.hadoop.util.DeferredSeekablePushbackInputStream.ReadMode readMode

Unsafe reads modify the byte after the reported number of read bytes in the read buffer. Example: If buf := [a, b, c, d] and "read(buf)" returns 3, then buf[3] may no longer hold a 'd'. As this may cause data corruption, safe read mode uses an intermediate buffer with additional copying to prevent changing data outside of the reported range.

fallbackBuffer

protected byte[] fallbackBuffer

Constructor Details

DeferredSeekablePushbackInputStream

public DeferredSeekablePushbackInputStream(InputStream in)

DeferredSeekablePushbackInputStream

public DeferredSeekablePushbackInputStream(InputStream in,
 org.apache.hadoop.fs.Seekable seekable)

Method Details

getSeekable

public org.apache.hadoop.fs.Seekable getSeekable()

Specified by:

getSeekable in interface SeekableDecorator

read

public int read()
         throws IOException

Overrides:

read in class PushbackInputStream

Throws:

IOException

read

public int read(byte[] b,
 int off,
 int len)
         throws IOException

This method essentially delays reads by one byte. Whereas hadoop's codec protocol reads one byte past block boundaries, this method buffers one byte on every read such that once the underlying stream reads accross the block boundary -- indicated by a position change -- only the byte before the block boundary is returned.

Overrides:

read in class PushbackInputStream

Returns:

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class PushbackInputStream

Throws:

IOException

readInternal

protected int readInternal(byte[] b,
 int off,
 int len)
                    throws IOException

This method is assumed to be invoked with len >= 2

Throws:

IOException

afterSeek

protected void afterSeek()

seek

public void seek(long l)
          throws IOException

Specified by:

seek in interface org.apache.hadoop.fs.Seekable

Specified by:

seek in interface SeekableDecorator

Throws:

IOException

getPos

public long getPos()
            throws IOException

Specified by:

getPos in interface org.apache.hadoop.fs.Seekable

Specified by:

getPos in interface SeekableDecorator

Throws:

IOException

seekToNewSource

public boolean seekToNewSource(long l)
                        throws IOException

Specified by:

seekToNewSource in interface org.apache.hadoop.fs.Seekable

Specified by:

seekToNewSource in interface SeekableDecorator

Throws:

IOException