Package org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies

Class DominantResourceFairnessPolicy

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.SchedulingPolicy
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.DominantResourceFairnessPolicy
@Private @Unstable public class DominantResourceFairnessPolicy extends SchedulingPolicy
Makes scheduling decisions by trying to equalize dominant resource usage. A schedulable's dominant resource usage is the largest ratio of resource usage to capacity among the resource types it is using.

  • Field Details

  • Constructor Details

    • DominantResourceFairnessPolicy

      public DominantResourceFairnessPolicy()

  • Method Details

    • getName

      public String getName()
      Specified by:
      getName in class SchedulingPolicy
      Returns:
      returns the name of SchedulingPolicy

    • getComparator

      public Comparator<Schedulable> getComparator()
      Description copied from class: SchedulingPolicy
      The comparator returned by this method is to be used for sorting the Schedulables in that queue.
      Specified by:
      getComparator in class SchedulingPolicy
      Returns:
      the comparator to sort by

    • getResourceCalculator

      public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
      Description copied from class: SchedulingPolicy
      The ResourceCalculator returned by this method should be used for any calculations involving resources.
      Specified by:
      getResourceCalculator in class SchedulingPolicy
      Returns:
      ResourceCalculator instance to use

    • computeShares

      public void computeShares(Collection<? extends Schedulable> schedulables, org.apache.hadoop.yarn.api.records.Resource totalResources)
      Description copied from class: SchedulingPolicy
      Computes and updates the shares of Schedulables as per the SchedulingPolicy, to be used later for scheduling decisions. The shares computed are instantaneous and only consider queues with running applications.
      Specified by:
      computeShares in class SchedulingPolicy
      Parameters:
      schedulables - Schedulables whose shares are to be updated
      totalResources - Total Resources in the cluster

    • computeSteadyShares

      public void computeSteadyShares(Collection<? extends FSQueue> queues, org.apache.hadoop.yarn.api.records.Resource totalResources)
      Description copied from class: SchedulingPolicy
      Computes and updates the steady shares of FSQueues as per the SchedulingPolicy. The steady share does not differentiate between queues with and without running applications under them. The steady share is not used for scheduling, it is displayed on the Web UI for better visibility.
      Specified by:
      computeSteadyShares in class SchedulingPolicy
      Parameters:
      queues - FSQueues whose shares are to be updated
      totalResources - Total Resources in the cluster

    • checkIfUsageOverFairShare

      public boolean checkIfUsageOverFairShare(org.apache.hadoop.yarn.api.records.Resource usage, org.apache.hadoop.yarn.api.records.Resource fairShare)
      Description copied from class: SchedulingPolicy
      Check if the resource usage is over the fair share under this policy.
      Specified by:
      checkIfUsageOverFairShare in class SchedulingPolicy
      Parameters:
      usage - Resource the resource usage
      fairShare - Resource the fair share
      Returns:
      true if check passes (is over) or false otherwise

    • getHeadroom

      public org.apache.hadoop.yarn.api.records.Resource getHeadroom(org.apache.hadoop.yarn.api.records.Resource queueFairShare, org.apache.hadoop.yarn.api.records.Resource queueUsage, org.apache.hadoop.yarn.api.records.Resource maxAvailable)
      Description copied from class: SchedulingPolicy
      Get headroom by calculating the min of clusterAvailable and (queueFairShare - queueUsage) resources that are applicable to this policy. For eg if only memory then leave other resources such as CPU to same as clusterAvailable.
      Specified by:
      getHeadroom in class SchedulingPolicy
      Parameters:
      queueFairShare - fairshare in the queue
      queueUsage - resources used in the queue
      maxAvailable - available resource in cluster for this queue
      Returns:
      calculated headroom

    • initialize

      public void initialize(FSContext fsContext)
      Description copied from class: SchedulingPolicy
      Initialize the scheduling policy with a FSContext object, which has a pointer to the cluster resources among other information.
      Overrides:
      initialize in class SchedulingPolicy
      Parameters:
      fsContext - a FSContext object which has a pointer to the cluster resources