Package org.apache.hadoop.yarn.server.resourcemanager.scheduler

Class AppSchedulingInfo

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo
@Private @Unstable public class AppSchedulingInfo extends Object
This class keeps track of all the consumption of an application. This also keeps track of current running/completed containers for the application.

  • Field Details

  • Constructor Details

  • Method Details

    • getDefaultResourceRequestAppPlacementType

      public String getDefaultResourceRequestAppPlacementType()
      Set default App Placement Allocator.
      Returns:
      app placement class.

    • getApplicationId

      public org.apache.hadoop.yarn.api.records.ApplicationId getApplicationId()

    • getApplicationAttemptId

      public org.apache.hadoop.yarn.api.records.ApplicationAttemptId getApplicationAttemptId()

    • getUser

      public String getUser()

    • getNewContainerId

      public long getNewContainerId()

    • getQueueName

      public String getQueueName()

    • isPending

      public boolean isPending()

    • setUnmanagedAM

      public void setUnmanagedAM(boolean unmanagedAM)

    • isUnmanagedAM

      public boolean isUnmanagedAM()

    • getRequestedPartitions

      public Set<String> getRequestedPartitions()

    • getUpdateContext

      public ContainerUpdateContext getUpdateContext()

    • updateResourceRequests

      public boolean updateResourceRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> resourceRequests, boolean recoverPreemptedRequestForAContainer)
      The ApplicationMaster is updating resource requirements for the application, by asking for more resources and releasing resources acquired by the application.
      Parameters:
      resourceRequests - resource requests to be allocated
      recoverPreemptedRequestForAContainer - recover ResourceRequest/SchedulingRequest on preemption
      Returns:
      true if any resource was updated, false otherwise

    • updateResourceRequests

      public boolean updateResourceRequests(Map<org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey,Map<String,org.apache.hadoop.yarn.api.records.ResourceRequest>> dedupRequests, boolean recoverPreemptedRequestForAContainer)
      The ApplicationMaster is updating resource requirements for the application, by asking for more resources and releasing resources acquired by the application.
      Parameters:
      dedupRequests - (dedup) resource requests to be allocated
      recoverPreemptedRequestForAContainer - recover ResourceRequest/SchedulingRequest on preemption
      Returns:
      true if any resource was updated, false otherwise

    • updateSchedulingRequests

      public boolean updateSchedulingRequests(List<org.apache.hadoop.yarn.api.records.SchedulingRequest> schedulingRequests, boolean recoverPreemptedRequestForAContainer)
      The ApplicationMaster is updating resource requirements for the application, by asking for more resources and releasing resources acquired by the application.
      Parameters:
      schedulingRequests - resource requests to be allocated
      recoverPreemptedRequestForAContainer - recover ResourceRequest/SchedulingRequest on preemption
      Returns:
      true if any resource was updated, false otherwise

    • removeAppPlacement

      public void removeAppPlacement(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerRequestKey)

    • addRequestedPartition

      public void addRequestedPartition(String partition)

    • decPendingResource

      public void decPendingResource(String partition, org.apache.hadoop.yarn.api.records.Resource toDecrease)

    • updatePlacesBlacklistedByApp

      public void updatePlacesBlacklistedByApp(List<String> blacklistAdditions, List<String> blacklistRemovals)
      The ApplicationMaster is updating the placesBlacklistedByApp used for containers other than AMs.
      Parameters:
      blacklistAdditions - resources to be added to the userBlacklist
      blacklistRemovals - resources to be removed from the userBlacklist

    • updatePlacesBlacklistedBySystem

      public void updatePlacesBlacklistedBySystem(List<String> blacklistAdditions, List<String> blacklistRemovals)
      Update the list of places that are blacklisted by the system. Today the system only blacklists places when it sees that AMs failed there
      Parameters:
      blacklistAdditions - resources to be added to placesBlacklistedBySystem
      blacklistRemovals - resources to be removed from placesBlacklistedBySystem

    • getAndResetBlacklistChanged

      public boolean getAndResetBlacklistChanged()

    • getSchedulerKeys

      public Collection<org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey> getSchedulerKeys()

    • getAllResourceRequests

      public List<org.apache.hadoop.yarn.api.records.ResourceRequest> getAllResourceRequests()
      Used by REST API to fetch ResourceRequest
      Returns:
      All pending ResourceRequests.

    • getAllSchedulingRequests

      public List<org.apache.hadoop.yarn.api.records.SchedulingRequest> getAllSchedulingRequests()
      Fetch SchedulingRequests.
      Returns:
      All pending SchedulingRequests.

    • getRejectedRequest

      public List<org.apache.hadoop.yarn.api.records.RejectedSchedulingRequest> getRejectedRequest()

    • getNextPendingAsk

      public PendingAsk getNextPendingAsk()

    • getPendingAsk

      public PendingAsk getPendingAsk(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey)

    • getPendingAsk

      public PendingAsk getPendingAsk(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey, String resourceName)

    • isPlaceBlacklisted

      public boolean isPlaceBlacklisted(String resourceName, boolean blacklistedBySystem)
      Returns if the place (node/rack today) is either blacklisted by the application (user) or the system.
      Parameters:
      resourceName - the resourcename
      blacklistedBySystem - true if it should check amBlacklist
      Returns:
      true if its blacklisted

    • allocate

      public ContainerRequest allocate(NodeType type, SchedulerNode node, org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey, RMContainer containerAllocated)

    • checkForDeactivation

      public void checkForDeactivation()

    • move

      public void move(Queue newQueue)

    • stop

      public void stop()

    • setQueue

      public void setQueue(Queue queue)

    • getBlackListCopy

      public Set<String> getBlackListCopy()

    • transferStateFromPreviousAppSchedulingInfo

      public void transferStateFromPreviousAppSchedulingInfo(AppSchedulingInfo appInfo)

    • recoverContainer

      public void recoverContainer(RMContainer rmContainer, String partition)

    • checkAllocation

      public boolean checkAllocation(NodeType type, SchedulerNode node, org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey)

    • updateMetrics

      public static void updateMetrics(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, NodeType type, SchedulerNode node, RMContainer containerAllocated, String user, Queue queue)

    • getAppPlacementAllocator

      public <N extends SchedulerNode> AppPlacementAllocator<N> getAppPlacementAllocator(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerkey)

    • canDelayTo

      public boolean canDelayTo(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey, String resourceName)
      Can delay to next?.
      Parameters:
      schedulerKey - schedulerKey
      resourceName - resourceName
      Returns:
      If request exists, return {relaxLocality} Otherwise, return true.

    • precheckNode

      public boolean precheckNode(org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey, SchedulerNode schedulerNode, SchedulingMode schedulingMode, Optional<DiagnosticsCollector> dcOpt)
      Pre-check node to see if it satisfy the given schedulerKey and scheduler mode.
      Parameters:
      schedulerKey - schedulerKey
      schedulerNode - schedulerNode
      schedulingMode - schedulingMode
      dcOpt - optional diagnostics collector
      Returns:
      can use the node or not.

    • getApplicationSchedulingEnvs

      public Map<String,String> getApplicationSchedulingEnvs()
      Get scheduling envs configured for this application.
      Returns:
      a map of applicationSchedulingEnvs

    • getDefaultNodeLabelExpression

      public String getDefaultNodeLabelExpression()
      Get the defaultNodeLabelExpression for the application's current queue.
      Returns:
      defaultNodeLabelExpression

    • getRMContext

      public RMContext getRMContext()