Modifier and Type | Method and Description |
---|---|
IREdge |
IREdge.setProperty(ExecutionProperty<?> executionProperty)
Set an executionProperty of the IREdge.
|
Modifier and Type | Method and Description |
---|---|
void |
IREdge.copyExecutionPropertiesTo(IREdge thatEdge)
Static function to copy executionProperties from an edge to the other.
|
Boolean |
IREdge.hasSameItineraryAs(IREdge edge) |
Modifier and Type | Method and Description |
---|---|
static ExecutionPropertyMap |
ExecutionPropertyMap.of(IREdge irEdge,
DataCommunicationPatternProperty.Value commPattern)
Static initializer for irEdges.
|
Modifier and Type | Method and Description |
---|---|
IREdge |
LoopVertex.getEdgeWithLoop(IREdge edgeWithInternalVertex) |
Modifier and Type | Method and Description |
---|---|
DAGBuilder<IRVertex,IREdge> |
LoopVertex.getBuilder() |
DAG<IRVertex,IREdge> |
LoopVertex.getDAG() |
Map<IRVertex,Set<IREdge>> |
LoopVertex.getDagIncomingEdges() |
Map<IRVertex,Set<IREdge>> |
LoopVertex.getDagOutgoingEdges() |
DAG<IRVertex,IREdge> |
MetricCollectionBarrierVertex.getDAGSnapshot()
Access the DAG snapshot when triggering dynamic optimization.
|
Map<IRVertex,Set<IREdge>> |
LoopVertex.getIterativeIncomingEdges() |
Map<IRVertex,Set<IREdge>> |
LoopVertex.getNonIterativeIncomingEdges() |
Modifier and Type | Method and Description |
---|---|
void |
LoopVertex.addDagIncomingEdge(IREdge edge)
Adds the incoming edge of the contained DAG.
|
void |
LoopVertex.addDagOutgoingEdge(IREdge edge)
Adds and outgoing edge of the contained DAG.
|
void |
LoopVertex.addIterativeIncomingEdge(IREdge edge)
Adds an iterative incoming edge, from the previous iteration, but connection internally.
|
void |
LoopVertex.addNonIterativeIncomingEdge(IREdge edge)
Adds a non-iterative incoming edge, from outside the previous iteration.
|
IREdge |
LoopVertex.getEdgeWithLoop(IREdge edgeWithInternalVertex) |
void |
LoopVertex.mapEdgeWithLoop(IREdge edgeWithLoop,
IREdge edgeWithInternalVertex)
Maps an edge from/to loop with the corresponding edge from/to internal vertex.
|
Modifier and Type | Method and Description |
---|---|
void |
MetricCollectionBarrierVertex.setDAGSnapshot(DAG<IRVertex,IREdge> dag)
This is to set the DAG snapshot at the end of the DataSkewCompositePass.
|
LoopVertex |
LoopVertex.unRollIteration(DAGBuilder<IRVertex,IREdge> dagBuilder)
Method for unrolling an iteration of the LoopVertex.
|
Modifier and Type | Method and Description |
---|---|
Plan |
Backend.compile(DAG<IRVertex,IREdge> dag)
Compiles a DAG to a physical execution plan.
|
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
CoralBackend.compile(DAG<IRVertex,IREdge> irDAG)
Compiles an IR DAG into a
PhysicalPlan to be submitted to Runtime. |
PhysicalPlan |
CoralBackend.compile(DAG<IRVertex,IREdge> irDAG,
PhysicalPlanGenerator physicalPlanGenerator)
Compiles an IR DAG into a
PhysicalPlan to be submitted to Runtime. |
Constructor and Description |
---|
CoralPipelineVisitor(DAGBuilder<IRVertex,IREdge> builder,
org.apache.beam.sdk.options.PipelineOptions options)
Constructor of the BEAM Visitor.
|
Modifier and Type | Method and Description |
---|---|
static <T> List<T> |
SparkFrontendUtils.collect(DAG<IRVertex,IREdge> dag,
Stack<LoopVertex> loopVertexStack,
IRVertex lastVertex,
org.apache.spark.serializer.Serializer serializer)
Collect data by running the DAG.
|
Modifier and Type | Method and Description |
---|---|
static DAG<IRVertex,IREdge> |
CompiletimeOptimizer.optimize(DAG<IRVertex,IREdge> dag,
Policy optimizationPolicy,
String dagDirectory)
Optimize function.
|
Modifier and Type | Method and Description |
---|---|
static DAG<IRVertex,IREdge> |
CompiletimeOptimizer.optimize(DAG<IRVertex,IREdge> dag,
Policy optimizationPolicy,
String dagDirectory)
Optimize function.
|
Modifier and Type | Method and Description |
---|---|
DAG<IRVertex,IREdge> |
DisaggregationEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewEdgeMetricCollectionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DefaultEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DefaultStagePartitioningPass.apply(DAG<IRVertex,IREdge> irDAG) |
DAG<IRVertex,IREdge> |
SailfishEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
ScheduleGroupPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
CompressionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
PadoVertexExecutorPlacementPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewVertexPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
ShuffleEdgePushPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DefaultParallelismPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
ReviseInterStageEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
SailfishEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewEdgePartitionerPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
PadoEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
PadoEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
SailfishEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
Modifier and Type | Method and Description |
---|---|
DAG<IRVertex,IREdge> |
DisaggregationEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewEdgeMetricCollectionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DefaultEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DefaultStagePartitioningPass.apply(DAG<IRVertex,IREdge> irDAG) |
DAG<IRVertex,IREdge> |
SailfishEdgeUsedDataHandlingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
ScheduleGroupPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
CompressionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
PadoVertexExecutorPlacementPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewVertexPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
ShuffleEdgePushPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DefaultParallelismPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
ReviseInterStageEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
SailfishEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewEdgePartitionerPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
PadoEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
PadoEdgeDataFlowModelPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
SailfishEdgeDataStorePass.apply(DAG<IRVertex,IREdge> dag) |
Modifier and Type | Method and Description |
---|---|
DAG<IRVertex,IREdge> |
CompositePass.apply(DAG<IRVertex,IREdge> irVertexIREdgeDAG) |
Modifier and Type | Method and Description |
---|---|
DAG<IRVertex,IREdge> |
CompositePass.apply(DAG<IRVertex,IREdge> irVertexIREdgeDAG) |
Modifier and Type | Method and Description |
---|---|
DAG<IRVertex,IREdge> |
LoopUnrollingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewReshapingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
LoopOptimizations.LoopFusionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
LoopOptimizations.LoopInvariantCodeMotionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
SailfishRelayReshapingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
CommonSubexpressionEliminationPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
LoopExtractionPass.apply(DAG<IRVertex,IREdge> dag) |
Modifier and Type | Method and Description |
---|---|
DAG<IRVertex,IREdge> |
LoopUnrollingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
DataSkewReshapingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
LoopOptimizations.LoopFusionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
LoopOptimizations.LoopInvariantCodeMotionPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
SailfishRelayReshapingPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
CommonSubexpressionEliminationPass.apply(DAG<IRVertex,IREdge> dag) |
DAG<IRVertex,IREdge> |
LoopExtractionPass.apply(DAG<IRVertex,IREdge> dag) |
Modifier and Type | Method and Description |
---|---|
DAG<PhysicalStage,PhysicalStageEdge> |
PhysicalPlanGenerator.apply(DAG<IRVertex,IREdge> irDAG)
Generates the
PhysicalPlan to be executed. |
DAG<Stage,StageEdge> |
PhysicalPlanGenerator.stagePartitionIrDAG(DAG<IRVertex,IREdge> irDAG)
We take the stage-partitioned DAG and create actual stage and stage edge objects to create a DAG of stages.
|
Modifier and Type | Method and Description |
---|---|
DAG<IRVertex,IREdge> |
Stage.getStageInternalDAG() |
Modifier and Type | Method and Description |
---|---|
StageBuilder |
StageBuilder.connectInternalVertices(IREdge edge)
Connects two
IRVertex in this stage. |
Constructor and Description |
---|
Stage(String stageId,
DAG<IRVertex,IREdge> stageInternalDAG,
int scheduleGroupIndex)
Constructor.
|
Copyright © 2018. All rights reserved.