public class InMemorySink extends Object implements Sink
getCrawledRdfData() method. If the sink encounters a problem
during its usage, e.g., data is written using a URI for which the stream
already has been closed before, the sink becomes unhealthy. Note that this
status does not influence the functionality of the sink. The status of the
sink, i.e., whether it is healthy or not, can be accessed using the
isSinkHealthy() method.| Modifier and Type | Field and Description |
|---|---|
private Set<String> |
closedSinks
Set of URIs for which the sink has already been closed.
|
private boolean |
healthyness
The healthyness of the sink that is set to false if an error is encountered.
|
private static org.slf4j.Logger |
LOGGER |
private Map<String,org.apache.jena.rdf.model.Model> |
rdfData
In-memory map used to store the RDF data that is written to the sink.
|
private Map<String,List<byte[]>> |
unstrcuturedData
In-memory map used to store the unstructured data that is written to the
sink.
|
| Constructor and Description |
|---|
InMemorySink() |
| Modifier and Type | Method and Description |
|---|---|
void |
addData(CrawleableUri uri,
byte[] data)
Stores the given data for the given URI.
|
void |
addData(CrawleableUri uri,
InputStream stream)
Stores the data from the given stream for the given URI.
|
void |
addTriple(CrawleableUri uri,
org.apache.jena.graph.Triple triple)
Add a triple for the given uri.
|
void |
closeSinkForUri(CrawleableUri uri)
Closes the resources necessary for storing the data of the given URI.
|
Map<String,org.apache.jena.rdf.model.Model> |
getCrawledRdfData()
Returns the data written to the sink as a map with the crawled URI as key and
the RDF data as value.
|
Map<String,List<byte[]>> |
getCrawledUnstructuredData()
Returns the data written to the sink as a map with the crawled URI as key and
the unstructured data as value.
|
boolean |
isSinkHealthy()
Returns the status of the sink.
|
void |
openSinkForUri(CrawleableUri uri)
Opens the sink to process data for the given URI.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitaddMetaData, closeaddDataprivate static final org.slf4j.Logger LOGGER
private Map<String,org.apache.jena.rdf.model.Model> rdfData
private Map<String,List<byte[]>> unstrcuturedData
private boolean healthyness
public void addTriple(CrawleableUri uri, org.apache.jena.graph.Triple triple)
TripleBasedSinkaddTriple in interface TripleBasedSinkuri - The given uri.triple - The triple to add.public void openSinkForUri(CrawleableUri uri)
SinkBaseopenSinkForUri in interface SinkBaseuri - the URI for which data should be stored.public void closeSinkForUri(CrawleableUri uri)
SinkBasecloseSinkForUri in interface SinkBaseuri - the URI for which data has been stored and for which the resources
should be freed.public Map<String,org.apache.jena.rdf.model.Model> getCrawledRdfData()
public Map<String,List<byte[]>> getCrawledUnstructuredData()
public boolean isSinkHealthy()
public void addData(CrawleableUri uri, byte[] data)
UnstructuredDataSinkaddData in interface UnstructuredDataSinkuri - the URI for which the data should be storeddata - the data that should be storedpublic void addData(CrawleableUri uri, InputStream stream)
UnstructuredDataSinkaddData in interface UnstructuredDataSinkuri - the URI for which the data should be storedstream - the stream from which the data will be read that should be storedCopyright © 2017–2019. All rights reserved.