Package org.apache.hadoop.yarn.server.resourcemanager.scheduler.constraint

Class AllocationTagsManager

java.lang.Object
org.apache.hadoop.yarn.server.resourcemanager.scheduler.constraint.AllocationTagsManager
@Private @Unstable public class AllocationTagsManager extends Object
In-memory mapping between applications/container-tags and nodes/racks. Required by constrained affinity/anti-affinity and cardinality placement.

  • Constructor Details

    • AllocationTagsManager

      public AllocationTagsManager(RMContext context)

  • Method Details

    • getPerAppNodeMappings

      @VisibleForTesting public Map<org.apache.hadoop.yarn.api.records.ApplicationId,AllocationTagsManager.TypeToCountedTags> getPerAppNodeMappings()

    • addContainer

      public void addContainer(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.ContainerId containerId, Set<String> allocationTags)
      Notify container allocated on a node.
      Parameters:
      nodeId - allocated node.
      containerId - container id.
      allocationTags - allocation tags, see SchedulingRequest.getAllocationTags() application_id will be added to allocationTags.

    • addTags

      public void addTags(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.ApplicationId applicationId, Set<String> allocationTags)

    • removeContainer

      public void removeContainer(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.ContainerId containerId, Set<String> allocationTags)
      Notify container removed.
      Parameters:
      nodeId - nodeId
      containerId - containerId.
      allocationTags - allocation tags for given container

    • removeTags

      public void removeTags(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.ApplicationId applicationId, Set<String> allocationTags)
      Helper method to just remove the tags associated with a container.
      Parameters:
      nodeId - nodeId.
      applicationId - application Id
      allocationTags - application Tags.

    • getNodeCardinality

      public long getNodeCardinality(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String tag) throws InvalidAllocationTagsQueryException
      Get Node cardinality for a specific tag. When applicationId is null, method returns aggregated cardinality
      Parameters:
      nodeId - nodeId, required.
      applicationId - applicationId. When null is specified, return aggregated cardinality among all nodes.
      tag - allocation tag, see SchedulingRequest.getAllocationTags(), If a specified tag doesn't exist, method returns 0.
      Returns:
      cardinality of specified query on the node.
      Throws:
      InvalidAllocationTagsQueryException - when illegal query parameter specified

    • getRackCardinality

      public long getRackCardinality(String rack, org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String tag) throws InvalidAllocationTagsQueryException
      Get Rack cardinality for a specific tag.
      Parameters:
      rack - rack, required.
      applicationId - applicationId. When null is specified, return aggregated cardinality among all nodes.
      tag - allocation tag, see SchedulingRequest.getAllocationTags(), If a specified tag doesn't exist, method returns 0.
      Returns:
      cardinality of specified query on the rack.
      Throws:
      InvalidAllocationTagsQueryException - when illegal query parameter specified

    • allocationTagExistsOnNode

      public boolean allocationTagExistsOnNode(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String tag) throws InvalidAllocationTagsQueryException
      Check if given tag exists on node.
      Parameters:
      nodeId - nodeId, required.
      applicationId - applicationId. When null is specified, return aggregation among all applications.
      tag - allocation tag, see SchedulingRequest.getAllocationTags(), When multiple tags specified. Returns cardinality depends on op. If a specified tag doesn't exist, 0 will be its cardinality. When null/empty tags specified, all tags (of the node/app) will be considered.
      Returns:
      cardinality of specified query on the node.
      Throws:
      InvalidAllocationTagsQueryException - when illegal query parameter specified

    • getNodeCardinalityByOp

      public long getNodeCardinalityByOp(org.apache.hadoop.yarn.api.records.NodeId nodeId, AllocationTags tags, LongBinaryOperator op) throws InvalidAllocationTagsQueryException
      Get cardinality for following conditions. External can pass-in a binary op to implement customized logic.
      Parameters:
      nodeId - nodeId, required.
      tags - AllocationTags, allocation tags under a specific namespace. See SchedulingRequest.getAllocationTags(), When multiple tags specified. Returns cardinality depends on op. If a specified tag doesn't exist, 0 will be its cardinality. When null/empty tags specified, all tags (of the node/app) will be considered.
      op - operator. Such as Long::max, Long::sum, etc. Required. This parameter only take effect when #values greater than 2.
      Returns:
      cardinality of specified query on the node.
      Throws:
      InvalidAllocationTagsQueryException - when illegal query parameter specified

    • getRackCardinalityByOp

      public long getRackCardinalityByOp(String rack, AllocationTags tags, LongBinaryOperator op) throws InvalidAllocationTagsQueryException
      Get cardinality for following conditions. External can pass-in a binary op to implement customized logic.
      Parameters:
      rack - rack, required.
      tags - AllocationTags, allocation tags under a specific namespace. See SchedulingRequest.getAllocationTags(), When multiple tags specified. Returns cardinality depends on op. If a specified tag doesn't exist, 0 will be its cardinality. When null/empty tags specified, all tags (of the rack/app) will be considered.
      op - operator. Such as Long::max, Long::sum, etc. Required. This parameter only take effect when #values greater than 2.
      Returns:
      cardinality of specified query on the rack.
      Throws:
      InvalidAllocationTagsQueryException - when illegal query parameter specified

    • getAllocationTagsWithCount

      public Map<String,Long> getAllocationTagsWithCount(org.apache.hadoop.yarn.api.records.NodeId nodeId)
      Returns a map whose key is the allocation tag and value is the count of allocations with this tag.
      Parameters:
      nodeId - nodeId.
      Returns:
      allocation tag to count mapping