Package org.apache.hadoop.hdfs.server.federation.router

Class Router

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.service.CompositeService
org.apache.hadoop.hdfs.server.federation.router.Router
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.hdfs.server.common.TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>, org.apache.hadoop.service.Service
@Private @Evolving public class Router extends org.apache.hadoop.service.CompositeService implements org.apache.hadoop.hdfs.server.common.TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>
Router that provides a unified view of multiple federated HDFS clusters. It has two main roles: (1) federated interface and (2) NameNode heartbeat.

For the federated interface, the Router receives a client request, checks the State Store for the correct subcluster, and forwards the request to the active Namenode of that subcluster. The reply from the Namenode then flows in the opposite direction. The Routers are stateless and can be behind a load balancer. HDFS clients connect to the router using the same interfaces as are used to communicate with a namenode, namely the ClientProtocol RPC interface and the WebHdfs HTTP interface exposed by the router. RouterRpcServer RouterHttpServer

For NameNode heartbeat, the Router periodically checks the state of a NameNode (usually on the same server) and reports their high availability (HA) state and load/space status to the State Store. Note that this is an optional role as a Router can be independent of any subcluster. StateStoreService NamenodeHeartbeatService

  • Constructor Details

    • Router

      public Router()

  • Method Details

    • serviceInit

      protected void serviceInit(org.apache.hadoop.conf.Configuration configuration) throws Exception
      Overrides:
      serviceInit in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • serviceStart

      protected void serviceStart() throws Exception
      Overrides:
      serviceStart in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • serviceStop

      protected void serviceStop() throws Exception
      Overrides:
      serviceStop in class org.apache.hadoop.service.CompositeService
      Throws:
      Exception

    • shutDown

      public void shutDown()
      Shutdown the router.

    • createRpcServer

      public RouterRpcServer createRpcServer() throws IOException
      Create a new Router RPC server to proxy ClientProtocol requests.
      Returns:
      New Router RPC Server.
      Throws:
      IOException - If the router RPC server was not started.

    • getRpcServer

      public RouterRpcServer getRpcServer()
      Get the Router RPC server.
      Returns:
      Router RPC server.

    • setRpcServerAddress

      protected void setRpcServerAddress(InetSocketAddress address)
      Set the current RPC socket for the router.
      Parameters:
      address - RPC address.

    • getRpcServerAddress

      public InetSocketAddress getRpcServerAddress()
      Get the current RPC socket address for the router.
      Returns:
      InetSocketAddress

    • createAdminServer

      protected RouterAdminServer createAdminServer() throws IOException
      Create a new router admin server to handle the router admin interface.
      Returns:
      RouterAdminServer
      Throws:
      IOException - If the admin server was not successfully started.

    • setAdminServerAddress

      protected void setAdminServerAddress(InetSocketAddress address)
      Set the current Admin socket for the router.
      Parameters:
      address - Admin RPC address.

    • getAdminServerAddress

      public InetSocketAddress getAdminServerAddress()
      Get the current Admin socket address for the router.
      Returns:
      InetSocketAddress Admin address.

    • createHttpServer

      protected RouterHttpServer createHttpServer()
      Create an HTTP server for this Router.
      Returns:
      HTTP server for this Router.

    • getHttpServerAddress

      public InetSocketAddress getHttpServerAddress()
      Get the current HTTP socket address for the router.
      Returns:
      InetSocketAddress HTTP address.

    • verifyToken

      public void verifyToken(org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier tokenId, byte[] password) throws IOException
      Specified by:
      verifyToken in interface org.apache.hadoop.hdfs.server.common.TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>
      Throws:
      IOException

    • createNamenodeHeartbeatServices

      protected Collection<NamenodeHeartbeatService> createNamenodeHeartbeatServices()
      Create each of the services that will monitor a Namenode.
      Returns:
      List of heartbeat services.

    • createLocalNamenodeHeartbeatService

      @VisibleForTesting public NamenodeHeartbeatService createLocalNamenodeHeartbeatService()
      Create a new status updater for the local Namenode.
      Returns:
      Updater of the status for the local Namenode.

    • createNamenodeHeartbeatService

      protected NamenodeHeartbeatService createNamenodeHeartbeatService(String nsId, String nnId)
      Create a heartbeat monitor for a particular Namenode.
      Parameters:
      nsId - Identifier of the nameservice to monitor.
      nnId - Identifier of the namenode (HA) to monitor.
      Returns:
      Updater of the status for the specified Namenode.

    • createNamenodeHeartbeatService

      protected NamenodeHeartbeatService createNamenodeHeartbeatService(String nsId, String nnId, String resolvedHost)

    • updateRouterState

      public void updateRouterState(RouterServiceState newState)
      Update the router state and heartbeat to the state store.
      Parameters:
      newState - The new router state.

    • getRouterState

      public RouterServiceState getRouterState()
      Get the status of the router.
      Returns:
      Status of the router.

    • isRouterState

      public boolean isRouterState(RouterServiceState routerState)
      Compare router state.
      Parameters:
      routerState - the router service state.
      Returns:
      true if the given router state is same as the state maintained by the router object.

    • getStateStore

      public StateStoreService getStateStore()
      Get the State Store service.
      Returns:
      State Store service.

    • getRouterMetrics

      public RouterMetrics getRouterMetrics()
      Get the metrics system for the Router.
      Returns:
      Router metrics.

    • getRouterClientMetrics

      public RouterClientMetrics getRouterClientMetrics()
      Get the metrics system for the Router Client.
      Returns:
      Router Client metrics.

    • getMetrics

      public RBFMetrics getMetrics()
      Get the federation metrics.
      Returns:
      Federation metrics.

    • getNamenodeMetrics

      public NamenodeBeanMetrics getNamenodeMetrics() throws IOException
      Get the namenode metrics.
      Returns:
      the namenode metrics.
      Throws:
      IOException - if the namenode metrics are not initialized.

    • getSubclusterResolver

      public FileSubclusterResolver getSubclusterResolver()
      Get the subcluster resolver for files.
      Returns:
      Subcluster resolver for files.

    • getNamenodeResolver

      public ActiveNamenodeResolver getNamenodeResolver()
      Get the namenode resolver for a subcluster.
      Returns:
      The namenode resolver for a subcluster.

    • getRouterStateManager

      public RouterStore getRouterStateManager()
      Get the state store interface for the router heartbeats.
      Returns:
      RouterStore state store API handle.

    • getStartTime

      public long getStartTime()
      Get the start date of the Router.
      Specified by:
      getStartTime in interface org.apache.hadoop.service.Service
      Overrides:
      getStartTime in class org.apache.hadoop.service.AbstractService
      Returns:
      Start date of the router.

    • getRouterId

      public String getRouterId()
      Unique ID for the router, typically the hostname:port string for the router's RPC server. This ID may be null on router startup before the RPC server has bound to a port.
      Returns:
      Router identifier.

    • setRouterId

      public void setRouterId(String id)
      Sets a unique ID for this router.
      Parameters:
      id - Identifier of the Router.

    • isQuotaEnabled

      public boolean isQuotaEnabled()
      Check if the quota system is enabled in Router.
      Returns:
      True if the quota system is enabled in Router.

    • getQuotaManager

      public RouterQuotaManager getQuotaManager()
      Get route quota manager.
      Returns:
      RouterQuotaManager Quota manager.

    • getAdminServer

      public RouterAdminServer getAdminServer()
      Get router admin server.
      Returns:
      Null if admin is not enabled.

    • setConf

      @VisibleForTesting public void setConf(org.apache.hadoop.conf.Configuration conf)
      Set router configuration.
      Parameters:
      conf - the configuration.