public final class DataSkewCompositePass extends CompositePass
MetricCollectionBarrierVertex before Shuffle edges,
to make a barrier before it, and to use the metrics to repartition the skewed data.
NOTE: we currently put the DataSkewCompositePass at the end of the list for each policies, as it needs to take a
snapshot at the end of the pass. This could be prevented by modifying other passes to take the snapshot of the DAG
at the end of each passes for metricCollectionVertices.| Constructor and Description |
|---|
DataSkewCompositePass()
Default constructor.
|
apply, getPassList, getPrerequisiteExecutionPropertiesCopyright © 2018. All rights reserved.