Package org.apache.hadoop.yarn.server.resourcemanager.reservation

Interface PlanFollower

All Superinterfaces:
Runnable
All Known Implementing Classes:
AbstractSchedulerPlanFollower, CapacitySchedulerPlanFollower, FairSchedulerPlanFollower
public interface PlanFollower extends Runnable
A PlanFollower is a component that runs on a timer, and synchronizes the underlying ResourceScheduler with the Plan(s) and viceversa. While different implementations might operate differently, the key idea is to map the current allocation of resources for each active reservation in the plan(s), to a corresponding notion in the underlying scheduler (e.g., tuning capacity of queues, set pool weights, or tweak application priorities). The goal is to affect the dynamic allocation of resources done by the scheduler so that the jobs obtain access to resources in a way that is consistent with the reservations in the plan. A key conceptual step here is to convert the absolute-valued promises made in the reservations to appropriate relative priorities/queue sizes etc. Symmetrically the PlanFollower exposes changes in cluster conditions (as tracked by the scheduler) to the plan, e.g., the overall amount of physical resources available. The Plan in turn can react by replanning its allocations if appropriate. The implementation can assume that is run frequently enough to be able to observe and react to normal operational changes in cluster conditions on the fly (e.g., if cluster resources drop, we can update the relative weights of a queue so that the absolute promises made to the job at reservation time are respected). However, due to RM restarts and the related downtime, it is advisable for implementations to operate in a stateless way, and be able to synchronize the state of plans/scheduler regardless of how big is the time gap between executions.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    init(org.apache.hadoop.yarn.util.Clock clock, ResourceScheduler sched, Collection<Plan> plans)
    Init function that configures the PlanFollower, by providing:
    void
    setPlans(Collection<Plan> plans)
    Setter for the list of plans.
    void
    synchronizePlan(Plan plan, boolean shouldReplan)
    The function performing the actual synchronization operation for a given Plan.

    Methods inherited from interface java.lang.Runnable

    run

  • Method Details

    • init

      void init(org.apache.hadoop.yarn.util.Clock clock, ResourceScheduler sched, Collection<Plan> plans)
      Init function that configures the PlanFollower, by providing:
      Parameters:
      clock - a reference to the system clock.
      sched - a reference to the underlying scheduler
      plans - references to the plans we should keep synchronized at every time tick.

    • synchronizePlan

      void synchronizePlan(Plan plan, boolean shouldReplan)
      The function performing the actual synchronization operation for a given Plan. This is normally invoked by the run method, but it can be invoked synchronously to avoid race conditions when a user's reservation request start time is imminent.
      Parameters:
      plan - the Plan to synchronize
      shouldReplan - replan on reduction of plan capacity if true or proportionally scale down reservations if false

    • setPlans

      void setPlans(Collection<Plan> plans)
      Setter for the list of plans.
      Parameters:
      plans - the collection of Plans we operate on at every time tick.