Package net.sansa_stack.hadoop.format.jena.base

Class RecordReaderGenericRdfBase<U,G,A,T>

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,T>

net.sansa_stack.hadoop.core.RecordReaderGenericBase<U,G,A,T>

net.sansa_stack.hadoop.format.jena.base.RecordReaderGenericRdfBase<U,G,A,T>

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

RecordReaderGenericRdfAccumulatingBase, RecordReaderGenericRdfNonAccumulatingBase

public abstract class RecordReaderGenericRdfBase<U,G,A,T>
extends RecordReaderGenericBase<U,G,A,T>

Nested Class Summary

Nested classes/interfaces inherited from class net.sansa_stack.hadoop.core.RecordReaderGenericBase

RecordReaderGenericBase.ReadTooFarException

Field Summary

Fields

Modifier and Type	Field	Description
protected String	baseIri
protected final String	baseIriKey
protected final String	headerBytesKey
protected org.apache.jena.riot.Lang	lang
protected String	prefixesMaxLengthKey
protected org.apache.jena.riot.system.PrefixMap	prefixMap

Fields inherited from class net.sansa_stack.hadoop.core.RecordReaderGenericBase

accumulating, codec, currentKey, currentValue, datasetFlow, decompressor, EMPTY_BYTE_ARRAY, enableStats, isEncoded, isFirstSplit, maxExtraByteCount, maxRecordLength, maxRecordLengthKey, minRecordLength, minRecordLengthKey, postambleBytes, preambleBytes, probeElementCount, probeElementCountKey, probeRecordCount, probeRecordCountKey, rawStream, recordFlowCloseable, recordStartPattern, regionStartSearchReadOverRegionEnd, regionStartSearchReadOverSplitEnd, skipRecordCount, split, splitEnd, splitId, splitLength, splitName, splitStart, stream, tailByteBuffer, tailBytes, tailEltBuffer, tailElts, tailEltsTime, tailRecordOffset, totalEltCount, totalRecordCount

Constructor Summary

Constructors

Constructor	Description
RecordReaderGenericRdfBase(RecordReaderRdfConf conf, Accumulating<U,G,A,T> accumulating)

Method Summary

Modifier and Type	Method	Description
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext context)	Read out config paramaters (prefixes, length thresholds, ...) and examine the codec in order to set an internal flag whether the stream will be encoded or not.
protected org.apache.jena.riot.system.AsyncParserBuilder	setupParser(InputStream in, boolean isProbe)

Methods inherited from class net.sansa_stack.hadoop.core.RecordReaderGenericBase

abbreviate, abbreviate, abbreviateAsUTF8, aggregate, aggregate, close, convert, createMatcherFactory, createRecordFlow, detectTail, didHitSplitBound, effectiveInputStream, effectiveInputStreamSupp, findFirstPositionWithProbeSuccess, findNextRegion, getCurrentKey, getCurrentValue, getPos, getPosition, getProgress, getStats, initRecordFlow, lines, logClose, logUnexpectedClose, nextKeyValue, parse, parseFromSeekable, prober, setStreamToInterval, unbufferedStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

baseIriKey

protected final String baseIriKey

headerBytesKey

protected final String headerBytesKey

prefixesMaxLengthKey

protected String prefixesMaxLengthKey

baseIri

protected String baseIri

lang

protected org.apache.jena.riot.Lang lang

prefixMap

protected org.apache.jena.riot.system.PrefixMap prefixMap

Constructor Details

RecordReaderGenericRdfBase

public RecordReaderGenericRdfBase(RecordReaderRdfConf conf,
 Accumulating<U,G,A,T> accumulating)

Method Details

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit,
 org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws IOException

Description copied from class: RecordReaderGenericBase

Read out config paramaters (prefixes, length thresholds, ...) and examine the codec in order to set an internal flag whether the stream will be encoded or not.

Overrides:

initialize in class RecordReaderGenericBase<U,G,A,T>

Throws:

IOException

setupParser

protected org.apache.jena.riot.system.AsyncParserBuilder setupParser(InputStream in,
 boolean isProbe)