edu.snu.coral.runtime.executor.data

Class BlockManagerWorker

java.lang.Object

edu.snu.coral.runtime.executor.data.BlockManagerWorker

@ThreadSafe
public final class BlockManagerWorker
extends Object

Executor-side block manager.

Method Summary

Modifier and Type	Method and Description
void	commitBlock(String blockId, DataStoreProperty.Value blockStore, List<Long> partitionSizeInfo, String srcIRVertexId, int expectedReadTotal, UsedDataHandlingProperty.Value usedDataHandling) Notifies that all writes for a block is end.
void	createBlock(String blockId, DataStoreProperty.Value blockStore) Creates a new block.
void	onPullRequest(BlockOutputStream<?> outputStream) Respond to a pull request by another executor.
void	onPushNotification(BlockInputStream inputStream) Respond to a push notification by another executor.
Optional<List<Long>>	putPartitions(String blockId, Iterable<Partition> partitions, DataStoreProperty.Value blockStore) Store an iterable of data partitions to a block in the target BlockStore.
CompletableFuture<Iterator>	queryBlock(String blockId, String runtimeEdgeId, DataStoreProperty.Value blockStore, KeyRange keyRange) Inquiries the location of the specific block and routes the request to the local block manager worker or to the lower data plane.
void	removeBlock(String blockId, DataStoreProperty.Value blockStore) Remove the block from store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createBlock

public void createBlock(String blockId,
                        DataStoreProperty.Value blockStore)

Creates a new block. A stale data created by previous failed task should be handled during the creation of new block.

Parameters:

blockId - the ID of the block to create.

blockStore - the store to place the block.

queryBlock

public CompletableFuture<Iterator> queryBlock(String blockId,
                                              String runtimeEdgeId,
                                              DataStoreProperty.Value blockStore,
                                              KeyRange keyRange)

Inquiries the location of the specific block and routes the request to the local block manager worker or to the lower data plane. This can be invoked multiple times per blockId (maybe due to failures).

Parameters:

blockId - of the block.

runtimeEdgeId - id of the runtime edge that corresponds to the block.

blockStore - for the data storage.

keyRange - the key range descriptor

Returns:

the CompletableFuture of the block.

putPartitions

public Optional<List<Long>> putPartitions(String blockId,
                                          Iterable<Partition> partitions,
                                          DataStoreProperty.Value blockStore)

Store an iterable of data partitions to a block in the target BlockStore. Invariant: This should not be invoked after a block is committed. Invariant: This method may not support concurrent write for a single block. Only one thread have to write at once.

Parameters:

blockId - of the block.

partitions - to save to a block.

blockStore - to store the block.

Returns:

a Optional of the size of each written block.

commitBlock

public void commitBlock(String blockId,
                        DataStoreProperty.Value blockStore,
                        List<Long> partitionSizeInfo,
                        String srcIRVertexId,
                        int expectedReadTotal,
                        UsedDataHandlingProperty.Value usedDataHandling)

Notifies that all writes for a block is end.

Parameters:

blockId - the ID of the block.

blockStore - the store to save the block.

partitionSizeInfo - the size metric of partitions.

srcIRVertexId - the IR vertex ID of the source task.

expectedReadTotal - the expected number of read for this block.

usedDataHandling - how to handle the used block.

removeBlock

public void removeBlock(String blockId,
                        DataStoreProperty.Value blockStore)

Remove the block from store.

Parameters:

blockId - the ID of the block to remove.

blockStore - the store which contains the block.

onPullRequest

public void onPullRequest(BlockOutputStream<?> outputStream)

Respond to a pull request by another executor.

This method is executed by {edu.snu.coral.runtime.executor.data.blocktransfer.BlockTransport} thread. \ Never execute a blocking call in this method!

Parameters:

outputStream - BlockOutputStream

onPushNotification

public void onPushNotification(BlockInputStream inputStream)

Respond to a push notification by another executor.

A push notification is generated when a remote executor invokes .blocktransfer.BlockTransfer#initiatePush(String, boolean, String, String, HashRange) to transfer a block to another executor.

This method is executed by {edu.snu.coral.runtime.executor.data.blocktransfer.BlockTransport} thread. Never execute a blocking call in this method!

Parameters:

inputStream - BlockInputStream