Package org.apache.hadoop.yarn.server.federation.policies.router

Class AbstractRouterPolicy

java.lang.Object
org.apache.hadoop.yarn.server.federation.policies.AbstractConfigurableFederationPolicy
org.apache.hadoop.yarn.server.federation.policies.router.AbstractRouterPolicy
All Implemented Interfaces:
ConfigurableFederationPolicy, FederationRouterPolicy
Direct Known Subclasses:
HashBasedRouterPolicy, LoadBasedRouterPolicy, PriorityRouterPolicy, RejectRouterPolicy, UniformRandomRouterPolicy, WeightedRandomRouterPolicy
public abstract class AbstractRouterPolicy extends AbstractConfigurableFederationPolicy implements FederationRouterPolicy
Base abstract class for FederationRouterPolicy implementations, that provides common validation for reinitialization.

  • Constructor Details

    • AbstractRouterPolicy

      public AbstractRouterPolicy()

  • Method Details

    • validate

      public void validate(WeightedPolicyInfo newPolicyInfo) throws FederationPolicyInitializationException
      Description copied from class: AbstractConfigurableFederationPolicy
      Overridable validation step for the policy configuration.
      Overrides:
      validate in class AbstractConfigurableFederationPolicy
      Parameters:
      newPolicyInfo - the configuration to test.
      Throws:
      FederationPolicyInitializationException - if the configuration is not valid.

    • validate

      public void validate(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext appSubmissionContext) throws FederationPolicyException
      Throws:
      FederationPolicyException

    • chooseSubCluster

      protected abstract SubClusterId chooseSubCluster(String queue, Map<SubClusterId,SubClusterInfo> preSelectSubClusters) throws org.apache.hadoop.yarn.exceptions.YarnException
      This method is implemented by the specific policy, and it is used to route both reservations, and applications among a given set of sub-clusters.
      Parameters:
      queue - the queue for this application/reservation
      preSelectSubClusters - a pre-filter set of sub-clusters
      Returns:
      the chosen sub-cluster
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if the policy fails to choose a sub-cluster

    • prefilterSubClusters

      protected Map<SubClusterId,SubClusterInfo> prefilterSubClusters(org.apache.hadoop.yarn.api.records.ReservationId reservationId, Map<SubClusterId,SubClusterInfo> activeSubClusters) throws org.apache.hadoop.yarn.exceptions.YarnException
      Filter chosen SubCluster based on reservationId.
      Parameters:
      reservationId - the globally unique identifier for a reservation.
      activeSubClusters - the map of ids to info for all active subclusters.
      Returns:
      the chosen sub-cluster
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if the policy fails to choose a sub-cluster

    • getHomeSubcluster

      public SubClusterId getHomeSubcluster(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext appContext, List<SubClusterId> blackLists) throws org.apache.hadoop.yarn.exceptions.YarnException
      Simply picks from alphabetically-sorted active subclusters based on the hash of query name. Jobs of the same queue will all be routed to the same sub-cluster, as far as the number of active sub-cluster and their names remain the same.
      Specified by:
      getHomeSubcluster in interface FederationRouterPolicy
      Parameters:
      appContext - the ApplicationSubmissionContext that has to be routed to an appropriate subCluster for execution.
      blackLists - the list of subClusters as identified by SubClusterId to blackList from the selection of the home subCluster.
      Returns:
      a hash-based chosen SubClusterId that will be the "home" for this application.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if there are no active subclusters.

    • getReservationHomeSubcluster

      public SubClusterId getReservationHomeSubcluster(org.apache.hadoop.yarn.api.protocolrecords.ReservationSubmissionRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
      This method provides a wrapper of all policy functionalities for routing a reservation. Internally it manages configuration changes, and policy init/reinit.
      Specified by:
      getReservationHomeSubcluster in interface FederationRouterPolicy
      Parameters:
      request - the reservation to route.
      Returns:
      the id of the subcluster that will be the "home" for this reservation.
      Throws:
      org.apache.hadoop.yarn.exceptions.YarnException - if there are issues initializing policies, or no valid sub-cluster id could be found for this reservation.