Package org.apache.hadoop.hdfs.client.impl

Class BlockReaderFactory

java.lang.Object
org.apache.hadoop.hdfs.client.impl.BlockReaderFactory
All Implemented Interfaces:
ShortCircuitCache.ShortCircuitReplicaCreator
@Private public class BlockReaderFactory extends Object implements ShortCircuitCache.ShortCircuitReplicaCreator
Utility class to create BlockReader implementations.

  • Constructor Details

    • BlockReaderFactory

      public BlockReaderFactory(DfsClientConf conf)

  • Method Details

    • setFileName

      public BlockReaderFactory setFileName(String fileName)

    • setBlock

      public BlockReaderFactory setBlock(ExtendedBlock block)

    • setBlockToken

      public BlockReaderFactory setBlockToken(org.apache.hadoop.security.token.Token<BlockTokenIdentifier> token)

    • setStartOffset

      public BlockReaderFactory setStartOffset(long startOffset)

    • setVerifyChecksum

      public BlockReaderFactory setVerifyChecksum(boolean verifyChecksum)

    • setClientName

      public BlockReaderFactory setClientName(String clientName)

    • setDatanodeInfo

      public BlockReaderFactory setDatanodeInfo(DatanodeInfo datanode)

    • setStorageType

      public BlockReaderFactory setStorageType(org.apache.hadoop.fs.StorageType storageType)

    • setAllowShortCircuitLocalReads

      public BlockReaderFactory setAllowShortCircuitLocalReads(boolean allowShortCircuitLocalReads)

    • setClientCacheContext

      public BlockReaderFactory setClientCacheContext(ClientContext clientContext)

    • setLength

      public BlockReaderFactory setLength(long length)

    • setCachingStrategy

      public BlockReaderFactory setCachingStrategy(CachingStrategy cachingStrategy)

    • setInetSocketAddress

      public BlockReaderFactory setInetSocketAddress(InetSocketAddress inetSocketAddress)

    • setUserGroupInformation

      public BlockReaderFactory setUserGroupInformation(org.apache.hadoop.security.UserGroupInformation userGroupInformation)

    • setRemotePeerFactory

      public BlockReaderFactory setRemotePeerFactory(RemotePeerFactory remotePeerFactory)

    • setConfiguration

      public BlockReaderFactory setConfiguration(org.apache.hadoop.conf.Configuration configuration)

    • setFailureInjectorForTesting

      @VisibleForTesting public static void setFailureInjectorForTesting(BlockReaderFactory.FailureInjector injector)

    • build

      public BlockReader build() throws IOException
      Build a BlockReader with the given options. This function will do the best it can to create a block reader that meets all of our requirements. We prefer short-circuit block readers (BlockReaderLocal and BlockReaderLocalLegacy) over remote ones, since the former avoid the overhead of socket communication. If short-circuit is unavailable, our next fallback is data transfer over UNIX domain sockets, if dfs.client.domain.socket.data.traffic has been enabled. If that doesn't work, we will try to create a remote block reader that operates over TCP sockets. There are a few caches that are important here. The ShortCircuitCache stores file descriptor objects which have been passed from the DataNode. The DomainSocketFactory stores information about UNIX domain socket paths that we not been able to use in the past, so that we don't waste time retrying them over and over. (Like all the caches, it does have a timeout, though.) The PeerCache stores peers that we have used in the past. If we can reuse one of these peers, we avoid the overhead of re-opening a socket. However, if the socket has been timed out on the remote end, our attempt to reuse the socket may end with an IOException. For that reason, we limit our attempts at socket reuse to dfs.client.cached.conn.retry times. After that, we create new sockets. This avoids the problem where a thread tries to talk to a peer that it hasn't talked to in a while, and has to clean out every entry in a socket cache full of stale entries.
      Returns:
      The new BlockReader. We will not return null.
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken - If the block token was invalid. InvalidEncryptionKeyException If the encryption key was invalid. Other IOException If there was another problem.
      IOException

    • createShortCircuitReplicaInfo

      public ShortCircuitReplicaInfo createShortCircuitReplicaInfo()
      Fetch a pair of short-circuit block descriptors from a local DataNode.
      Specified by:
      createShortCircuitReplicaInfo in interface ShortCircuitCache.ShortCircuitReplicaCreator
      Returns:
      Null if we could not communicate with the datanode, a new ShortCircuitReplicaInfo object otherwise. ShortCircuitReplicaInfo objects may contain either an InvalidToken exception, or a ShortCircuitReplica object ready to use.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getFileName

      public static String getFileName(InetSocketAddress s, String poolId, long blockId)
      File name to print when accessing a block directly (from servlets)
      Parameters:
      s - Address of the block location
      poolId - Block pool ID of the block
      blockId - Block ID of the block
      Returns:
      string that has a file name for debug purposes