Package org.apache.hadoop.hdfs.protocol.datatransfer

Class Sender

java.lang.Object
org.apache.hadoop.hdfs.protocol.datatransfer.Sender
All Implemented Interfaces:
DataTransferProtocol
@Private @Evolving public class Sender extends Object implements DataTransferProtocol
Sender

  • Constructor Details

    • Sender

      public Sender(DataOutputStream out)
      Create a sender for DataTransferProtocol with a output stream.

  • Method Details

    • readBlock

      public void readBlock(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, String clientName, long blockOffset, long length, boolean sendChecksum, CachingStrategy cachingStrategy) throws IOException
      Description copied from interface: DataTransferProtocol
      Read a block.
      Specified by:
      readBlock in interface DataTransferProtocol
      Parameters:
      blk - the block being read.
      blockToken - security token for accessing the block.
      clientName - client's name.
      blockOffset - offset of the block.
      length - maximum number of bytes for this read.
      sendChecksum - if false, the DN should skip reading and sending checksums
      cachingStrategy - The caching strategy to use.
      Throws:
      IOException

    • writeBlock

      public void writeBlock(ExtendedBlock blk, org.apache.hadoop.fs.StorageType storageType, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, String clientName, DatanodeInfo[] targets, org.apache.hadoop.fs.StorageType[] targetStorageTypes, DatanodeInfo source, BlockConstructionStage stage, int pipelineSize, long minBytesRcvd, long maxBytesRcvd, long latestGenerationStamp, org.apache.hadoop.util.DataChecksum requestedChecksum, CachingStrategy cachingStrategy, boolean allowLazyPersist, boolean pinning, boolean[] targetPinnings, String storageId, String[] targetStorageIds) throws IOException
      Description copied from interface: DataTransferProtocol
      Write a block to a datanode pipeline. The receiver datanode of this call is the next datanode in the pipeline. The other downstream datanodes are specified by the targets parameter. Note that the receiver DatanodeInfo is not required in the parameter list since the receiver datanode knows its info. However, the StorageType for storing the replica in the receiver datanode is a parameter since the receiver datanode may support multiple storage types.
      Specified by:
      writeBlock in interface DataTransferProtocol
      Parameters:
      blk - the block being written.
      storageType - for storing the replica in the receiver datanode.
      blockToken - security token for accessing the block.
      clientName - client's name.
      targets - other downstream datanodes in the pipeline.
      targetStorageTypes - target StorageTypes corresponding to the target datanodes.
      source - source datanode.
      stage - pipeline stage.
      pipelineSize - the size of the pipeline.
      minBytesRcvd - minimum number of bytes received.
      maxBytesRcvd - maximum number of bytes received.
      latestGenerationStamp - the latest generation stamp of the block.
      requestedChecksum - the requested checksum mechanism
      cachingStrategy - the caching strategy
      allowLazyPersist - hint to the DataNode that the block can be allocated on transient storage i.e. memory and written to disk lazily
      pinning - whether to pin the block, so Balancer won't move it.
      targetPinnings - whether to pin the block on target datanode
      storageId - optional StorageIDs designating where to write the block. An empty String or null indicates that this has not been provided.
      targetStorageIds - target StorageIDs corresponding to the target datanodes.
      Throws:
      IOException

    • transferBlock

      public void transferBlock(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, String clientName, DatanodeInfo[] targets, org.apache.hadoop.fs.StorageType[] targetStorageTypes, String[] targetStorageIds) throws IOException
      Description copied from interface: DataTransferProtocol
      Transfer a block to another datanode. The block stage must be either BlockConstructionStage.TRANSFER_RBW or BlockConstructionStage.TRANSFER_FINALIZED.
      Specified by:
      transferBlock in interface DataTransferProtocol
      Parameters:
      blk - the block being transferred.
      blockToken - security token for accessing the block.
      clientName - client's name.
      targets - target datanodes.
      targetStorageIds - StorageID designating where to write the block.
      Throws:
      IOException

    • requestShortCircuitFds

      public void requestShortCircuitFds(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, ShortCircuitShm.SlotId slotId, int maxVersion, boolean supportsReceiptVerification) throws IOException
      Description copied from interface: DataTransferProtocol
      Request short circuit access file descriptors from a DataNode.
      Specified by:
      requestShortCircuitFds in interface DataTransferProtocol
      Parameters:
      blk - The block to get file descriptors for.
      blockToken - Security token for accessing the block.
      slotId - The shared memory slot id to use, or null to use no slot id.
      maxVersion - Maximum version of the block data the client can understand.
      supportsReceiptVerification - True if the client supports receipt verification.
      Throws:
      IOException

    • releaseShortCircuitFds

      public void releaseShortCircuitFds(ShortCircuitShm.SlotId slotId) throws IOException
      Description copied from interface: DataTransferProtocol
      Release a pair of short-circuit FDs requested earlier.
      Specified by:
      releaseShortCircuitFds in interface DataTransferProtocol
      Parameters:
      slotId - SlotID used by the earlier file descriptors.
      Throws:
      IOException

    • requestShortCircuitShm

      public void requestShortCircuitShm(String clientName) throws IOException
      Description copied from interface: DataTransferProtocol
      Request a short circuit shared memory area from a DataNode.
      Specified by:
      requestShortCircuitShm in interface DataTransferProtocol
      Parameters:
      clientName - The name of the client.
      Throws:
      IOException

    • replaceBlock

      public void replaceBlock(ExtendedBlock blk, org.apache.hadoop.fs.StorageType storageType, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, String delHint, DatanodeInfo source, String storageId) throws IOException
      Description copied from interface: DataTransferProtocol
      Receive a block from a source datanode and then notifies the namenode to remove the copy from the original datanode. Note that the source datanode and the original datanode can be different. It is used for balancing purpose.
      Specified by:
      replaceBlock in interface DataTransferProtocol
      Parameters:
      blk - the block being replaced.
      storageType - the StorageType for storing the block.
      blockToken - security token for accessing the block.
      delHint - the hint for deleting the block in the original datanode.
      source - the source datanode for receiving the block.
      storageId - an optional storage ID to designate where the block is replaced to.
      Throws:
      IOException

    • copyBlock

      public void copyBlock(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken) throws IOException
      Description copied from interface: DataTransferProtocol
      Copy a block. It is used for balancing purpose.
      Specified by:
      copyBlock in interface DataTransferProtocol
      Parameters:
      blk - the block being copied.
      blockToken - security token for accessing the block.
      Throws:
      IOException

    • blockChecksum

      public void blockChecksum(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, BlockChecksumOptions blockChecksumOptions) throws IOException
      Description copied from interface: DataTransferProtocol
      Get block checksum (MD5 of CRC32).
      Specified by:
      blockChecksum in interface DataTransferProtocol
      Parameters:
      blk - a block.
      blockToken - security token for accessing the block.
      blockChecksumOptions - determines how the block-level checksum is computed from underlying block metadata.
      Throws:
      IOException

    • blockGroupChecksum

      public void blockGroupChecksum(StripedBlockInfo stripedBlockInfo, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, long requestedNumBytes, BlockChecksumOptions blockChecksumOptions) throws IOException
      Description copied from interface: DataTransferProtocol
      Get striped block group checksum (MD5 of CRC32).
      Specified by:
      blockGroupChecksum in interface DataTransferProtocol
      Parameters:
      stripedBlockInfo - a striped block info.
      blockToken - security token for accessing the block.
      requestedNumBytes - requested number of bytes in the block group to compute the checksum.
      blockChecksumOptions - determines how the block-level checksum is computed from underlying block metadata.
      Throws:
      IOException