| Modifier and Type | Method and Description | 
|---|---|
IREdge | 
IREdge.setProperty(ExecutionProperty<?> executionProperty)
Set an executionProperty of the IREdge. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
void | 
IREdge.copyExecutionPropertiesTo(IREdge thatEdge)
Static function to copy executionProperties from an edge to the other. 
 | 
Boolean | 
IREdge.hasSameItineraryAs(IREdge edge)  | 
| Modifier and Type | Method and Description | 
|---|---|
static ExecutionPropertyMap | 
ExecutionPropertyMap.of(IREdge irEdge,
  DataCommunicationPatternProperty.Value commPattern)
Static initializer for irEdges. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
IREdge | 
LoopVertex.getEdgeWithLoop(IREdge edgeWithInternalVertex)  | 
| Modifier and Type | Method and Description | 
|---|---|
DAGBuilder<IRVertex,IREdge> | 
LoopVertex.getBuilder()  | 
DAG<IRVertex,IREdge> | 
LoopVertex.getDAG()  | 
Map<IRVertex,Set<IREdge>> | 
LoopVertex.getDagIncomingEdges()  | 
Map<IRVertex,Set<IREdge>> | 
LoopVertex.getDagOutgoingEdges()  | 
DAG<IRVertex,IREdge> | 
MetricCollectionBarrierVertex.getDAGSnapshot()
Access the DAG snapshot when triggering dynamic optimization. 
 | 
Map<IRVertex,Set<IREdge>> | 
LoopVertex.getIterativeIncomingEdges()  | 
Map<IRVertex,Set<IREdge>> | 
LoopVertex.getNonIterativeIncomingEdges()  | 
| Modifier and Type | Method and Description | 
|---|---|
void | 
LoopVertex.addDagIncomingEdge(IREdge edge)
Adds the incoming edge of the contained DAG. 
 | 
void | 
LoopVertex.addDagOutgoingEdge(IREdge edge)
Adds and outgoing edge of the contained DAG. 
 | 
void | 
LoopVertex.addIterativeIncomingEdge(IREdge edge)
Adds an iterative incoming edge, from the previous iteration, but connection internally. 
 | 
void | 
LoopVertex.addNonIterativeIncomingEdge(IREdge edge)
Adds a non-iterative incoming edge, from outside the previous iteration. 
 | 
IREdge | 
LoopVertex.getEdgeWithLoop(IREdge edgeWithInternalVertex)  | 
void | 
LoopVertex.mapEdgeWithLoop(IREdge edgeWithLoop,
               IREdge edgeWithInternalVertex)
Maps an edge from/to loop with the corresponding edge from/to internal vertex. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
void | 
MetricCollectionBarrierVertex.setDAGSnapshot(DAG<IRVertex,IREdge> dag)
This is to set the DAG snapshot at the end of the DataSkewCompositePass. 
 | 
LoopVertex | 
LoopVertex.unRollIteration(DAGBuilder<IRVertex,IREdge> dagBuilder)
Method for unrolling an iteration of the LoopVertex. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
Plan | 
Backend.compile(DAG<IRVertex,IREdge> dag)
Compiles a DAG to a physical execution plan. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
PhysicalPlan | 
CoralBackend.compile(DAG<IRVertex,IREdge> irDAG)
Compiles an IR DAG into a  
PhysicalPlan to be submitted to Runtime. | 
PhysicalPlan | 
CoralBackend.compile(DAG<IRVertex,IREdge> irDAG,
       PhysicalPlanGenerator physicalPlanGenerator)
Compiles an IR DAG into a  
PhysicalPlan to be submitted to Runtime. | 
| Constructor and Description | 
|---|
CoralPipelineVisitor(DAGBuilder<IRVertex,IREdge> builder,
                    org.apache.beam.sdk.options.PipelineOptions options)
Constructor of the BEAM Visitor. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
static <T> List<T> | 
SparkFrontendUtils.collect(DAG<IRVertex,IREdge> dag,
       Stack<LoopVertex> loopVertexStack,
       IRVertex lastVertex,
       org.apache.spark.serializer.Serializer serializer)
Collect data by running the DAG. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
static DAG<IRVertex,IREdge> | 
CompiletimeOptimizer.optimize(DAG<IRVertex,IREdge> dag,
        Policy optimizationPolicy,
        String dagDirectory)
Optimize function. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
static DAG<IRVertex,IREdge> | 
CompiletimeOptimizer.optimize(DAG<IRVertex,IREdge> dag,
        Policy optimizationPolicy,
        String dagDirectory)
Optimize function. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<IRVertex,IREdge> | 
DisaggregationEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewEdgeMetricCollectionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DefaultEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DefaultStagePartitioningPass.apply(DAG<IRVertex,IREdge> irDAG)  | 
DAG<IRVertex,IREdge> | 
SailfishEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
ScheduleGroupPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
CompressionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
PadoVertexExecutorPlacementPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewVertexPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
ShuffleEdgePushPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DefaultParallelismPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
ReviseInterStageEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
SailfishEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewEdgePartitionerPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
PadoEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
PadoEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
SailfishEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<IRVertex,IREdge> | 
DisaggregationEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewEdgeMetricCollectionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DefaultEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DefaultStagePartitioningPass.apply(DAG<IRVertex,IREdge> irDAG)  | 
DAG<IRVertex,IREdge> | 
SailfishEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
ScheduleGroupPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
CompressionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
PadoVertexExecutorPlacementPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewVertexPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
ShuffleEdgePushPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DefaultParallelismPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
ReviseInterStageEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
SailfishEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewEdgePartitionerPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
PadoEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
PadoEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
SailfishEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag)  | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<IRVertex,IREdge> | 
CompositePass.apply(DAG<IRVertex,IREdge> irVertexIREdgeDAG)  | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<IRVertex,IREdge> | 
CompositePass.apply(DAG<IRVertex,IREdge> irVertexIREdgeDAG)  | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<IRVertex,IREdge> | 
LoopUnrollingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewReshapingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
LoopOptimizations.LoopFusionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
LoopOptimizations.LoopInvariantCodeMotionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
SailfishRelayReshapingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
CommonSubexpressionEliminationPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
LoopExtractionPass.apply(DAG<IRVertex,IREdge> dag)  | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<IRVertex,IREdge> | 
LoopUnrollingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
DataSkewReshapingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
LoopOptimizations.LoopFusionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
LoopOptimizations.LoopInvariantCodeMotionPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
SailfishRelayReshapingPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
CommonSubexpressionEliminationPass.apply(DAG<IRVertex,IREdge> dag)  | 
DAG<IRVertex,IREdge> | 
LoopExtractionPass.apply(DAG<IRVertex,IREdge> dag)  | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<PhysicalStage,PhysicalStageEdge> | 
PhysicalPlanGenerator.apply(DAG<IRVertex,IREdge> irDAG)
Generates the  
PhysicalPlan to be executed. | 
DAG<Stage,StageEdge> | 
PhysicalPlanGenerator.stagePartitionIrDAG(DAG<IRVertex,IREdge> irDAG)
We take the stage-partitioned DAG and create actual stage and stage edge objects to create a DAG of stages. 
 | 
| Modifier and Type | Method and Description | 
|---|---|
DAG<IRVertex,IREdge> | 
Stage.getStageInternalDAG()  | 
| Modifier and Type | Method and Description | 
|---|---|
StageBuilder | 
StageBuilder.connectInternalVertices(IREdge edge)
Connects two  
IRVertex in this stage. | 
| Constructor and Description | 
|---|
Stage(String stageId,
     DAG<IRVertex,IREdge> stageInternalDAG,
     int scheduleGroupIndex)
Constructor. 
 | 
Copyright © 2018. All rights reserved.