Package org.aksw.jenax.engine.qlever

Class RDFDatabaseBuilderQlever<X extends RDFDatabaseBuilderQlever<X>>

java.lang.Object

org.aksw.jenax.engine.qlever.RDFDatabaseBuilderQlever<X>

All Implemented Interfaces:

org.aksw.commons.util.obj.HasSelf<X>, org.aksw.jenax.dataaccess.sparql.creator.RDFDatabaseBuilder<X>

public class RDFDatabaseBuilderQlever<X extends RDFDatabaseBuilderQlever<X>>
extends Object
implements org.aksw.jenax.dataaccess.sparql.creator.RDFDatabaseBuilder<X>

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	RDFDatabaseBuilderQlever.ByteSourceSpec	Record to hold either a command for a ProcessBuilder that produces output or a ByteSource.
static final record	RDFDatabaseBuilderQlever.DockerDataArgumentBridge	hostFileWriterTask: The name of a regular file or a named pipe on the host.
static final record	RDFDatabaseBuilderQlever.FileAndCmd
static final record	RDFDatabaseBuilderQlever.FileArg	Record to capture arguments passed to this builder
static final record	RDFDatabaseBuilderQlever.FileSpec	A list of file binds and arguments to process them
static final record	RDFDatabaseBuilderQlever.InputSpec
static final record	RDFDatabaseBuilderQlever.QleverDbFileSet	Record to capture a set of files that make up a Qlever database.

Field Summary

Fields

Modifier and Type	Field	Description
protected List<RDFDatabaseBuilderQlever.FileArg>	args
protected String	containerFifoPath	Base path within the container where to mount any named pipes.
protected ContainerPathResolver	containerPathResolver	A resolver for host paths if the database builder is used from within docker (dind).
protected String	dockerImageName
protected String	dockerImageTag
protected List<Map.Entry<org.apache.jena.riot.Lang,Throwable>>	errorCollector
protected String	indexName
protected Path	outputFolder
protected org.aksw.jenax.arq.util.prefix.ShortNameMgr	shortNameMgr	Mapping from absolute file paths on the host names to file names.
protected String	stxxlMemory
static final List<org.apache.jena.riot.Lang>	supportedLangs
protected SysRuntime	sysRuntime

Constructor Summary

Constructors

Constructor	Description
RDFDatabaseBuilderQlever()

Method Summary

Modifier and Type	Method	Description
X	addPath(String source, org.apache.jena.graph.Node g)
protected void	addPath(Path source, org.apache.jena.graph.Node graph, List<String> encodings, org.apache.jena.riot.Lang lang)
RdfDatabaseQlever	build()	Determine the types of arguments: If all files are directly nq or ttl then use them as file arguments.
protected RDFDatabaseBuilderQlever.ByteSourceSpec	buildByteSourceCmd(List<StreamOp> args, org.apache.jena.riot.Lang lang)	On the host side: create a FileWriter for a file that can be mounted into the container bridge: create a bind of the host file to a container path.
protected RDFDatabaseBuilderQlever.FileAndCmd	buildCmdPart(String containerBasePath, StreamOp containerOp, String fileArg, org.apache.jena.graph.Node graph, org.apache.jena.riot.Lang lang)
protected RDFDatabaseBuilderQlever.DockerDataArgumentBridge	buildHostToContainerDataBridge(Supplier<Path> hostTempPathSupp, StreamOp op, org.apache.jena.graph.Node graph, org.apache.jena.riot.Lang lang)
protected RDFDatabaseBuilderQlever.InputSpec	buildInputSpec(Supplier<Path> hostTempPath)
StreamOp	convertArgToOp(RDFDatabaseBuilderQlever.FileArg arg)
protected FileWriterTask	createHostFileWriter(Supplier<Path> tempPathSupp, StreamOp sysCallOp)
String	getDockerImageName()
String	getDockerImageTag()
String	getFinalIndexName()
protected SysRuntime	getRuntime()
String	getStxxlMemory()
protected String	langToFormat(org.apache.jena.riot.Lang lang)
protected void	runContainerViaSysCall(CmdOp generatorCmd, org.apache.jena.riot.Lang lang)
protected void	runContainerViaSysCallWithInputStream(com.google.common.io.ByteSource byteSource, org.apache.jena.riot.Lang lang)
protected void	runContainerWithFileArgs(RDFDatabaseBuilderQlever.FileSpec fileSpec)
protected void	runContainerWithInputStream(com.google.common.io.ByteSource byteSource, org.apache.jena.riot.Lang lang)
X	setDockerImageName(String dockerImageName)
X	setDockerImageTag(String dockerImageTag)
X	setIndexName(String name)
X	setName(String name)
X	setOutputFolder(Path outputFolder)
X	setStxxlMemory(String stxxlMemory)
X	setSysRuntime(SysRuntime sysRuntime)
protected org.testcontainers.containers.GenericContainer<?>	setupContainer(String indexName, String cmdStr)
protected GenericContainer<?>	setupContainerSysCall(String fileName, org.apache.jena.riot.Lang lang)
protected StreamOpTransformToCmdOp	sysCallTransform()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.aksw.commons.util.obj.HasSelf

self

Methods inherited from interface org.aksw.jenax.dataaccess.sparql.creator.RDFDatabaseBuilder

addPath, addPath

Field Details

supportedLangs

public static final List<org.apache.jena.riot.Lang> supportedLangs

dockerImageName

protected String dockerImageName

dockerImageTag

protected String dockerImageTag

shortNameMgr

protected org.aksw.jenax.arq.util.prefix.ShortNameMgr shortNameMgr

Mapping from absolute file paths on the host names to file names.

sysRuntime

protected SysRuntime sysRuntime

outputFolder

protected Path outputFolder

args

protected List<RDFDatabaseBuilderQlever.FileArg> args

errorCollector

protected List<Map.Entry<org.apache.jena.riot.Lang,Throwable>> errorCollector

indexName

protected String indexName

stxxlMemory

protected String stxxlMemory

containerFifoPath

protected String containerFifoPath

Base path within the container where to mount any named pipes. Must end with '/'.

containerPathResolver

protected ContainerPathResolver containerPathResolver

A resolver for host paths if the database builder is used from within docker (dind).

Constructor Details

RDFDatabaseBuilderQlever

public RDFDatabaseBuilderQlever()

Method Details

setSysRuntime

public X setSysRuntime(SysRuntime sysRuntime)

setDockerImageName

public X setDockerImageName(String dockerImageName)

getDockerImageName

public String getDockerImageName()

setDockerImageTag

public X setDockerImageTag(String dockerImageTag)

getDockerImageTag

public String getDockerImageTag()

setName

public X setName(String name)

Specified by:

setName in interface org.aksw.jenax.dataaccess.sparql.creator.RDFDatabaseBuilder<X extends RDFDatabaseBuilderQlever<X>>

setIndexName

public X setIndexName(String name)

setOutputFolder

public X setOutputFolder(Path outputFolder)

Specified by:

setOutputFolder in interface org.aksw.jenax.dataaccess.sparql.creator.RDFDatabaseBuilder<X extends RDFDatabaseBuilderQlever<X>>

setStxxlMemory

public X setStxxlMemory(String stxxlMemory)

getStxxlMemory

public String getStxxlMemory()

addPath

public X addPath(String source,
 org.apache.jena.graph.Node g)
          throws IOException

Specified by:

addPath in interface org.aksw.jenax.dataaccess.sparql.creator.RDFDatabaseBuilder<X extends RDFDatabaseBuilderQlever<X>>

Throws:

IOException

addPath

protected void addPath(Path source,
 org.apache.jena.graph.Node graph,
 List<String> encodings,
 org.apache.jena.riot.Lang lang)

convertArgToOp

public StreamOp convertArgToOp(RDFDatabaseBuilderQlever.FileArg arg)

getRuntime

protected SysRuntime getRuntime()

sysCallTransform

protected StreamOpTransformToCmdOp sysCallTransform()

buildByteSourceCmd

protected RDFDatabaseBuilderQlever.ByteSourceSpec buildByteSourceCmd(List<StreamOp> args,
 org.apache.jena.riot.Lang lang)

On the host side: create a FileWriter for a file that can be mounted into the container bridge: create a bind of the host file to a container path. (the host file is not required to exist at this stage) On the container side: create a StreamOp that reads the container file.

Returns:

buildInputSpec

protected RDFDatabaseBuilderQlever.InputSpec buildInputSpec(Supplier<Path> hostTempPath)
                                                     throws NoSuchFileException

Throws:

NoSuchFileException

buildCmdPart

protected RDFDatabaseBuilderQlever.FileAndCmd buildCmdPart(String containerBasePath,
 StreamOp containerOp,
 String fileArg,
 org.apache.jena.graph.Node graph,
 org.apache.jena.riot.Lang lang)

createHostFileWriter

protected FileWriterTask createHostFileWriter(Supplier<Path> tempPathSupp,
 StreamOp sysCallOp)
                                       throws NoSuchFileException

Throws:

NoSuchFileException

buildHostToContainerDataBridge

protected RDFDatabaseBuilderQlever.DockerDataArgumentBridge buildHostToContainerDataBridge(Supplier<Path> hostTempPathSupp,
 StreamOp op,
 org.apache.jena.graph.Node graph,
 org.apache.jena.riot.Lang lang)
                                                                                    throws NoSuchFileException

Throws:

NoSuchFileException

setupContainer

protected org.testcontainers.containers.GenericContainer<?> setupContainer(String indexName,
 String cmdStr)
                                                                    throws NumberFormatException,
IOException,
InterruptedException

Throws:

NumberFormatException

IOException

InterruptedException

runContainerWithInputStream

protected void runContainerWithInputStream(com.google.common.io.ByteSource byteSource,
 org.apache.jena.riot.Lang lang)
                                    throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

runContainerWithFileArgs

protected void runContainerWithFileArgs(RDFDatabaseBuilderQlever.FileSpec fileSpec)
                                 throws NumberFormatException,
IOException,
InterruptedException

Throws:

NumberFormatException

IOException

InterruptedException

langToFormat

protected String langToFormat(org.apache.jena.riot.Lang lang)

setupContainerSysCall

protected GenericContainer<?> setupContainerSysCall(String fileName,
 org.apache.jena.riot.Lang lang)
                                             throws NumberFormatException,
IOException,
InterruptedException

Throws:

NumberFormatException

IOException

InterruptedException

runContainerViaSysCallWithInputStream

protected void runContainerViaSysCallWithInputStream(com.google.common.io.ByteSource byteSource,
 org.apache.jena.riot.Lang lang)
                                              throws InterruptedException,
IOException

Throws:

InterruptedException

IOException

runContainerViaSysCall

protected void runContainerViaSysCall(CmdOp generatorCmd,
 org.apache.jena.riot.Lang lang)
                               throws NumberFormatException,
IOException,
InterruptedException

Throws:

NumberFormatException

IOException

InterruptedException

getFinalIndexName

public String getFinalIndexName()

build

public RdfDatabaseQlever build()
                        throws IOException,
InterruptedException

Determine the types of arguments: If all files are directly nq or ttl then use them as file arguments. Otherwise, build a stream from the argument: - A mix of nq and ttl is not supported. - Use cat or a codec to decode files - Use a flag whether to use process substitution or command grouping.

Specified by:

build in interface org.aksw.jenax.dataaccess.sparql.creator.RDFDatabaseBuilder<X extends RDFDatabaseBuilderQlever<X>>

Returns:

Throws:

IOException

InterruptedException