Package org.apache.hadoop.hdfs.server.federation.router

Class ConnectionPool

java.lang.Object
org.apache.hadoop.hdfs.server.federation.router.ConnectionPool
@Private @Evolving public class ConnectionPool extends Object
Maintains a pool of connections for each User (including tokens) + NN. The RPC client maintains a single socket, to achieve throughput similar to a NN, each request is multiplexed across multiple sockets/connections from a pool.

  • Constructor Details

    • ConnectionPool

      protected ConnectionPool(org.apache.hadoop.conf.Configuration config, String address, org.apache.hadoop.security.UserGroupInformation user, int minPoolSize, int maxPoolSize, float minActiveRatio, Class<?> proto, PoolAlignmentContext alignmentContext) throws IOException
      Throws:
      IOException

  • Method Details

    • getMaxSize

      protected int getMaxSize()
      Get the maximum number of connections allowed in this pool.
      Returns:
      Maximum number of connections.

    • getMinSize

      protected int getMinSize()
      Get the minimum number of connections in this pool.
      Returns:
      Minimum number of connections.

    • getMinActiveRatio

      protected float getMinActiveRatio()
      Get the minimum ratio of active connections in this pool.
      Returns:
      Minimum ratio of active connections.

    • getConnectionPoolId

      protected ConnectionPoolId getConnectionPoolId()
      Get the connection pool identifier.
      Returns:
      Connection pool identifier.

    • getClientIndex

      @VisibleForTesting public AtomicInteger getClientIndex()
      Get the clientIndex used to calculate index for lookup.
      Returns:
      Client index.

    • getPoolAlignmentContext

      public PoolAlignmentContext getPoolAlignmentContext()
      Get the alignment context for this pool.
      Returns:
      Alignment context

    • getConnection

      protected ConnectionContext getConnection()
      Return the next connection round-robin.
      Returns:
      Connection context.

    • addConnection

      public void addConnection(ConnectionContext conn)
      Add a connection to the current pool. It uses a Copy-On-Write approach.
      Parameters:
      conn - New connection to add to the pool.

    • removeConnections

      public List<ConnectionContext> removeConnections(int num)
      Remove connections from the current pool.
      Parameters:
      num - Number of connections to remove.
      Returns:
      Removed connections.

    • close

      protected void close()
      Close the connection pool.

    • getNumConnections

      protected int getNumConnections()
      Number of connections in the pool.
      Returns:
      Number of connections.

    • getNumActiveConnections

      protected int getNumActiveConnections()
      Number of active connections in the pool.
      Returns:
      Number of active connections.

    • getNumIdleConnections

      protected int getNumIdleConnections()
      Number of usable i.e. no active thread connections.
      Returns:
      Number of idle connections

    • getNumActiveConnectionsRecently

      protected int getNumActiveConnectionsRecently()
      Number of active connections recently in the pool.
      Returns:
      Number of active connections recently.

    • getLastActiveTime

      protected long getLastActiveTime()
      Get the last time the connection pool was used.
      Returns:
      Last time the connection pool was used.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getJSON

      public String getJSON()
      JSON representation of the connection pool.
      Returns:
      String representation of the JSON.

    • newConnection

      public ConnectionContext newConnection() throws IOException
      Create a new proxy wrapper for a client NN connection.
      Returns:
      Proxy for the target ClientProtocol that contains the user's security context.
      Throws:
      IOException - If it cannot get a new connection.

    • newConnection

      protected static <T> ConnectionContext newConnection(org.apache.hadoop.conf.Configuration conf, String nnAddress, org.apache.hadoop.security.UserGroupInformation ugi, Class<T> proto, boolean enableMultiSocket, int socketIndex, org.apache.hadoop.ipc.AlignmentContext alignmentContext) throws IOException
      Creates a proxy wrapper for a client NN connection. Each proxy contains context for a single user/security context. To maximize throughput it is recommended to use multiple connection per user+server, allowing multiple writes and reads to be dispatched in parallel.
      Type Parameters:
      T - Input type T.
      Parameters:
      conf - Configuration for the connection.
      nnAddress - Address of server supporting the ClientProtocol.
      ugi - User context.
      proto - Interface of the protocol.
      enableMultiSocket - Enable multiple socket or not.
      socketIndex - Index for FederationConnectionId.
      alignmentContext - Client alignment context.
      Returns:
      proto for the target ClientProtocol that contains the user's security context.
      Throws:
      IOException - If it cannot be created.