Package org.apache.hadoop.yarn.server.resourcemanager.reservation

Class CapacityOverTimePolicy

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.reservation.NoOverCommitPolicy
org.apache.hadoop.yarn.server.resourcemanager.reservation.CapacityOverTimePolicy
All Implemented Interfaces:
SharingPolicy
@LimitedPrivate("yarn") @Unstable public class CapacityOverTimePolicy extends NoOverCommitPolicy
This policy enforces a time-extended notion of Capacity. In particular it guarantees that the allocation received in input when combined with all previous allocation for the user does not violate an instantaneous max limit on the resources received, and that for every window of time of length validWindow, the integral of the allocations for a user (sum of the currently submitted allocation and all prior allocations for the user) does not exceed validWindow * maxAvg. This allows flexibility, in the sense that an allocation can instantaneously use large portions of the available capacity, but prevents abuses by bounding the average use over time. By controlling maxInst, maxAvg, validWindow the administrator configuring this policy can obtain a behavior ranging from instantaneously enforced capacity (akin to existing queues), or fully flexible allocations (likely reserved to super-users, or trusted systems).

  • Constructor Details

    • CapacityOverTimePolicy

      public CapacityOverTimePolicy()

  • Method Details

    • init

      public void init(String reservationQueue, ReservationSchedulerConfiguration conf)
      Description copied from interface: SharingPolicy
      Initialize this policy.
      Specified by:
      init in interface SharingPolicy
      Overrides:
      init in class NoOverCommitPolicy
      Parameters:
      reservationQueue - the name of the queue for this plan
      conf - the system configuration

    • validate

      public void validate(Plan plan, ReservationAllocation reservation) throws PlanningException
      The validation algorithm walks over the RLE encoded allocation and checks that for all transition points (when the start or end of the checking window encounters a value in the RLE). At this point it checkes whether the integral computed exceeds the quota limit. Note that this might not find the exact time of a violation, but if a violation exists it will find it. The advantage is a much lower number of checks as compared to time-slot by time-slot checks.
      Specified by:
      validate in interface SharingPolicy
      Overrides:
      validate in class NoOverCommitPolicy
      Parameters:
      plan - the plan to validate against
      reservation - the reservation allocation to test.
      Throws:
      PlanningException - if the validation fails.

    • availableResources

      public RLESparseResourceAllocation availableResources(RLESparseResourceAllocation available, Plan plan, String user, org.apache.hadoop.yarn.api.records.ReservationId oldId, long start, long end) throws PlanningException
      Description copied from interface: SharingPolicy
      This method provide a (partial) instantaneous validation by applying business rules (such as max number of parallel containers allowed for a user). To provide the agent with more feedback the returned parameter is expressed in number of containers that can be fit in this time according to the business rules.
      Specified by:
      availableResources in interface SharingPolicy
      Overrides:
      availableResources in class NoOverCommitPolicy
      Parameters:
      available - the amount of resources that would be offered if not constrained by the policy
      plan - reference the the current Plan
      user - the username
      oldId - (optional) the id of a reservation being updated
      start - the start time for the range we are querying
      end - the end time for the range we are querying
      Returns:
      the available resources expressed as a RLESparseResourceAllocation
      Throws:
      PlanningException - throws if the request is not valid

    • getValidWindow

      public long getValidWindow()
      Description copied from interface: SharingPolicy
      Returns the time range before and after the current reservation considered by this policy. In particular, this informs the archival process for the Plan, i.e., reservations regarding times before (now - validWindow) can be deleted.
      Specified by:
      getValidWindow in interface SharingPolicy
      Overrides:
      getValidWindow in class NoOverCommitPolicy
      Returns:
      validWindow the window of validity considered by the policy.