edu.snu.coral.compiler.frontend.spark.source

Class SparkBoundedSourceVertex<T>

java.lang.Object

edu.snu.coral.common.dag.Vertex

edu.snu.coral.common.ir.vertex.IRVertex

edu.snu.coral.common.ir.vertex.SourceVertex<T>

edu.snu.coral.compiler.frontend.spark.source.SparkBoundedSourceVertex<T>

Type Parameters:

T - type of data to read.

All Implemented Interfaces:

Serializable

public final class SparkBoundedSourceVertex<T>
extends SourceVertex<T>

Bounded source vertex for Spark.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SparkBoundedSourceVertex(Dataset<T> dataset) Constructor.
SparkBoundedSourceVertex(ReadablesWrapper<T> readablesWrapper) Constructor.

Method Summary

Modifier and Type	Method and Description
SparkBoundedSourceVertex	getClone()
ReadablesWrapper<T>	getReadableWrapper(int desiredNumOfSplits) Gets parallel readable wrapper.

Methods inherited from class edu.snu.coral.common.ir.vertex.IRVertex

copyExecutionPropertiesTo, getExecutionProperties, getProperty, irVertexPropertiesToString, setProperty

Methods inherited from class edu.snu.coral.common.dag.Vertex

getId, getNumericId, propertiesToJSON

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkBoundedSourceVertex

public SparkBoundedSourceVertex(Dataset<T> dataset)

Constructor. Note that we have to first create our iterators here and supply them to our readables.

Parameters:

dataset - Dataset to read data from.

SparkBoundedSourceVertex

public SparkBoundedSourceVertex(ReadablesWrapper<T> readablesWrapper)

Constructor.

Parameters:

readablesWrapper - readables wrapper.

Method Detail

getClone

public SparkBoundedSourceVertex getClone()

Specified by:

getClone in class IRVertex

Returns:

a clone elemnt of the IRVertex.

getReadableWrapper

public ReadablesWrapper<T> getReadableWrapper(int desiredNumOfSplits)

Description copied from class: SourceVertex

Gets parallel readable wrapper.

Specified by:

getReadableWrapper in class SourceVertex<T>

Parameters:

desiredNumOfSplits - number of splits desired.

Returns:

the wrapper for a list of readers.