Package org.apache.hadoop.mapreduce.v2.app.speculate

Interface TaskRuntimeEstimator

All Known Implementing Classes:
ExponentiallySmoothedTaskRuntimeEstimator, LegacyTaskRuntimeEstimator, NullTaskRuntimesEngine, SimpleExponentialTaskRuntimeEstimator
public interface TaskRuntimeEstimator

  • Method Details

    • enrollAttempt

      void enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus reportedStatus, long timestamp)

    • attemptEnrolledTime

      long attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)

    • updateAttempt

      void updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus reportedStatus, long timestamp)

    • contextualize

      void contextualize(org.apache.hadoop.conf.Configuration conf, AppContext context)

    • thresholdRuntime

      long thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)
      Find a maximum reasonable execution wallclock time. Includes the time already elapsed. Find a maximum reasonable execution time. Includes the time already elapsed. If the projected total execution time for this task ever exceeds its reasonable execution time, we may speculate it.
      Parameters:
      id - the TaskId of the task we are asking about
      Returns:
      the task's maximum reasonable runtime, or MAX_VALUE if we don't have enough information to rule out any runtime, however long.

    • estimatedRuntime

      long estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
      Estimate a task attempt's total runtime. Includes the time already elapsed.
      Parameters:
      id - the TaskAttemptId of the attempt we are asking about
      Returns:
      our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

    • estimatedNewAttemptRuntime

      long estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)
      Estimates how long a new attempt on this task will take if we start one now
      Parameters:
      id - the TaskId of the task we are asking about
      Returns:
      our best estimate of a new attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

    • runtimeEstimateVariance

      long runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id)
      Computes the width of the error band of our estimate of the task runtime as returned by estimatedRuntime(TaskAttemptId)
      Parameters:
      id - the TaskAttemptId of the attempt we are asking about
      Returns:
      our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

    • hasStagnatedProgress

      default boolean hasStagnatedProgress(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId id, long timeStamp)
      Returns true if the estimator has no updates records for a threshold time window. This helps to identify task attempts that are stalled at the beginning of execution.
      Parameters:
      id - the TaskAttemptId of the attempt we are asking about
      timeStamp - the time of the report we compare with
      Returns:
      true if the task attempt has no progress for a given time window