Package org.apache.hadoop.yarn.server.resourcemanager.scheduler

Class SchedulerNode

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode
Direct Known Subclasses:
FiCaSchedulerNode, FSSchedulerNode
@Private @Unstable public abstract class SchedulerNode extends Object
Represents a YARN Cluster Node from the viewpoint of the scheduler.

  • Constructor Details

    • SchedulerNode

      public SchedulerNode(RMNode node, boolean usePortForNodeName, Set<String> labels)

    • SchedulerNode

      public SchedulerNode(RMNode node, boolean usePortForNodeName)

  • Method Details

    • getRMNode

      public RMNode getRMNode()

    • updateTotalResource

      public void updateTotalResource(org.apache.hadoop.yarn.api.records.Resource resource)
      Set total resources on the node.
      Parameters:
      resource - Total resources on the node.

    • setOvercommitTimeOut

      public void setOvercommitTimeOut(long timeOut)
      Set the timeout for the node to stop overcommitting the resources. After this time the scheduler will start killing containers until the resources are not overcommitted anymore. This may reset a previous timeout.
      Parameters:
      timeOut - Time out in milliseconds.

    • isOvercommitTimedOut

      public boolean isOvercommitTimedOut()
      Check if the time out has passed.
      Returns:
      If the node is overcommitted.

    • isOvercommitTimeOutSet

      public boolean isOvercommitTimeOutSet()
      Check if the node has a time out for overcommit resources.
      Returns:
      If the node has a time out for overcommit resources.

    • getNodeID

      public org.apache.hadoop.yarn.api.records.NodeId getNodeID()
      Get the ID of the node which contains both its hostname and port.
      Returns:
      The ID of the node.

    • getHttpAddress

      public String getHttpAddress()
      Get HTTP address for the node.
      Returns:
      HTTP address for the node.

    • getNodeName

      public String getNodeName()
      Get the name of the node for scheduling matching decisions.

      Typically this is the 'hostname' reported by the node, but it could be configured to be 'hostname:port' reported by the node via the YarnConfiguration.RM_SCHEDULER_INCLUDE_PORT_IN_NODE_NAME constant. The main usecase of this is YARN minicluster to be able to differentiate node manager instances by their port number.

      Returns:
      Name of the node for scheduling matching decisions.

    • getRackName

      public String getRackName()
      Get rackname.
      Returns:
      rackname

    • allocateContainer

      public void allocateContainer(RMContainer rmContainer)
      The Scheduler has allocated containers on this node to the given application.
      Parameters:
      rmContainer - Allocated container

    • allocateContainer

      protected void allocateContainer(RMContainer rmContainer, boolean launchedOnNode)
      The Scheduler has allocated containers on this node to the given application.
      Parameters:
      rmContainer - Allocated container
      launchedOnNode - True if the container has been launched

    • getUnallocatedResource

      public org.apache.hadoop.yarn.api.records.Resource getUnallocatedResource()
      Get unallocated resources on the node.
      Returns:
      Unallocated resources on the node

    • getAllocatedResource

      public org.apache.hadoop.yarn.api.records.Resource getAllocatedResource()
      Get allocated resources on the node.
      Returns:
      Allocated resources on the node

    • getTotalResource

      public org.apache.hadoop.yarn.api.records.Resource getTotalResource()
      Get total resources on the node.
      Returns:
      Total resources on the node.

    • isValidContainer

      public boolean isValidContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
      Check if a container is launched by this node.
      Parameters:
      containerId - containerId.
      Returns:
      If the container is launched by the node.

    • updateResourceForReleasedContainer

      protected void updateResourceForReleasedContainer(org.apache.hadoop.yarn.api.records.Container container)
      Update the resources of the node when releasing a container.
      Parameters:
      container - Container to release.

    • releaseContainer

      public void releaseContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, boolean releasedByNode)
      Release an allocated container on this node.
      Parameters:
      containerId - ID of container to be released.
      releasedByNode - whether the release originates from a node update.

    • containerStarted

      public void containerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId)
      Inform the node that a container has launched.
      Parameters:
      containerId - ID of the launched container

    • deductUnallocatedResource

      @VisibleForTesting public void deductUnallocatedResource(org.apache.hadoop.yarn.api.records.Resource resource)
      Deduct unallocated resources from the node. This is used when allocating a container.
      Parameters:
      resource - Resources to deduct.

    • reserveResource

      public abstract void reserveResource(SchedulerApplicationAttempt attempt, org.apache.hadoop.yarn.server.scheduler.SchedulerRequestKey schedulerKey, RMContainer container)
      Reserve container for the attempt on this node.
      Parameters:
      attempt - Application attempt asking for the reservation.
      schedulerKey - Priority of the reservation.
      container - Container reserving resources for.

    • unreserveResource

      public abstract void unreserveResource(SchedulerApplicationAttempt attempt)
      Unreserve resources on this node.
      Parameters:
      attempt - Application attempt that had done the reservation.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getNumContainers

      public int getNumContainers()
      Get number of active containers on the node.
      Returns:
      Number of active containers on the node.

    • getCopiedListOfRunningContainers

      public List<RMContainer> getCopiedListOfRunningContainers()
      Get the containers running on the node.
      Returns:
      A copy of containers running on the node.

    • getRunningContainersWithAMsAtTheEnd

      public List<RMContainer> getRunningContainersWithAMsAtTheEnd()
      Get the containers running on the node with AM containers at the end.
      Returns:
      A copy of running containers with AM containers at the end.

    • getContainersToKill

      public List<RMContainer> getContainersToKill()
      Get the containers running on the node ordered by which to kill first. It tries to kill AMs last, then GUARANTEED containers, and it kills OPPORTUNISTIC first. If the same time, it uses the creation time.
      Returns:
      A copy of the running containers ordered by which to kill first.

    • getLaunchedContainers

      protected List<RMContainer> getLaunchedContainers()
      Get the launched containers in the node.
      Returns:
      List of launched containers.

    • getContainer

      protected RMContainer getContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
      Get the container for the specified container ID.
      Parameters:
      containerId - The container ID
      Returns:
      The container for the specified container ID

    • getReservedContainer

      public RMContainer getReservedContainer()
      Get the reserved container in the node.
      Returns:
      Reserved container in the node.

    • setReservedContainer

      public void setReservedContainer(RMContainer reservedContainer)
      Set the reserved container in the node.
      Parameters:
      reservedContainer - Reserved container in the node.

    • recoverContainer

      public void recoverContainer(RMContainer rmContainer)
      Recover a container.
      Parameters:
      rmContainer - Container to recover.

    • getLabels

      public Set<String> getLabels()
      Get the labels for the node.
      Returns:
      Set of labels for the node.

    • updateLabels

      public void updateLabels(Set<String> labels)
      Update the labels for the node.
      Parameters:
      labels - Set of labels for the node.

    • getPartition

      public String getPartition()
      Get partition of which the node belongs to, if node-labels of this node is empty or null, it belongs to NO_LABEL partition. And since we only support one partition for each node (YARN-2694), first label will be its partition.
      Returns:
      Partition for the node.

    • setAggregatedContainersUtilization

      public void setAggregatedContainersUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization containersUtilization)
      Set the resource utilization of the containers in the node.
      Parameters:
      containersUtilization - Resource utilization of the containers.

    • getAggregatedContainersUtilization

      public org.apache.hadoop.yarn.api.records.ResourceUtilization getAggregatedContainersUtilization()
      Get the resource utilization of the containers in the node.
      Returns:
      Resource utilization of the containers.

    • setNodeUtilization

      public void setNodeUtilization(org.apache.hadoop.yarn.api.records.ResourceUtilization nodeUtilization)
      Set the resource utilization of the node. This includes the containers.
      Parameters:
      nodeUtilization - Resource utilization of the node.

    • getNodeUtilization

      public org.apache.hadoop.yarn.api.records.ResourceUtilization getNodeUtilization()
      Get the resource utilization of the node.
      Returns:
      Resource utilization of the node.

    • getLastHeartbeatMonotonicTime

      public long getLastHeartbeatMonotonicTime()

    • notifyNodeUpdate

      public void notifyNodeUpdate()
      This will be called for each node heartbeat.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • getNodeAttributes

      public Set<org.apache.hadoop.yarn.api.records.NodeAttribute> getNodeAttributes()

    • updateNodeAttributes

      public void updateNodeAttributes(Set<org.apache.hadoop.yarn.api.records.NodeAttribute> attributes)