Package org.apache.hadoop.mapreduce.v2.app.rm.preemption

Interface AMPreemptionPolicy

All Known Implementing Classes:
CheckpointAMPreemptionPolicy, KillAMPreemptionPolicy, NoopAMPreemptionPolicy
public interface AMPreemptionPolicy
Policy encoding the MRAppMaster response to preemption requests from the ResourceManager.
See Also:

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static class 
    AMPreemptionPolicy.Context
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID
    getCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId)
    Method to retrieve the latest checkpoint for a given TaskId
    void
    handleCompletedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
    Callback informing the policy of containers exiting cleanly.
    void
    handleFailedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
    Callback informing the policy of containers exiting with a failure.
    void
    init(AppContext context)
     
    boolean
    isPreempted(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
    This method is invoked by components interested to learn whether a certain task is being preempted.
    void
    preempt(AMPreemptionPolicy.Context context, org.apache.hadoop.yarn.api.records.PreemptionMessage preemptionRequests)
    Callback informing the policy of ResourceManager. requests for resources to return to the cluster.
    void
    reportSuccessfulPreemption(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
    This method is used to report to the policy that a certain task has been successfully preempted (for bookeeping, counters, etc..)
    void
    setCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId, org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID cid)
    Method to store the latest CheckpointID for a given TaskId.

  • Method Details

    • init

      void init(AppContext context)

    • preempt

      void preempt(AMPreemptionPolicy.Context context, org.apache.hadoop.yarn.api.records.PreemptionMessage preemptionRequests)
      Callback informing the policy of ResourceManager. requests for resources to return to the cluster. The policy may take arbitrary action to satisfy requests by checkpointing task state, returning containers, or ignoring requests. The RM may elect to enforce these requests by forcibly killing containers not returned after some duration.
      Parameters:
      context - Handle to the current state of running containers
      preemptionRequests - Request from RM for resources to return.

    • isPreempted

      boolean isPreempted(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
      This method is invoked by components interested to learn whether a certain task is being preempted.
      Parameters:
      attemptID - Task attempt to query
      Returns:
      true if this attempt is being preempted

    • reportSuccessfulPreemption

      void reportSuccessfulPreemption(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
      This method is used to report to the policy that a certain task has been successfully preempted (for bookeeping, counters, etc..)
      Parameters:
      attemptID - Task attempt that preempted

    • handleFailedContainer

      void handleFailedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
      Callback informing the policy of containers exiting with a failure. This allows the policy to implemnt cleanup/compensating actions.
      Parameters:
      attemptID - Task attempt that failed

    • handleCompletedContainer

      void handleCompletedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
      Callback informing the policy of containers exiting cleanly. This is reported to the policy for bookeeping purposes.
      Parameters:
      attemptID - Task attempt that completed

    • getCheckpointID

      org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID getCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId)
      Method to retrieve the latest checkpoint for a given TaskId
      Parameters:
      taskId - TaskID
      Returns:
      CheckpointID associated with this task or null

    • setCheckpointID

      void setCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId, org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID cid)
      Method to store the latest CheckpointID for a given TaskId. Assigning a null is akin to remove all previous checkpoints for this task.
      Parameters:
      taskId - TaskID
      cid - Checkpoint to assign or null to remove it.