Package org.apache.hadoop.yarn.server.resourcemanager.nodelabels

Class NodeAttributesManagerImpl

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.nodelabels.NodeAttributesManager
org.apache.hadoop.yarn.server.resourcemanager.nodelabels.NodeAttributesManagerImpl
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.service.Service
public class NodeAttributesManagerImpl extends org.apache.hadoop.yarn.nodelabels.NodeAttributesManager
Manager holding the attributes to Labels.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    NodeAttributesManagerImpl.Host
    A Host can have multiple Nodes.

    Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

    org.apache.hadoop.service.Service.STATE

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    EMPTY_ATTRIBUTE_VALUE
    If a user doesn't specify value for a label, then empty string is considered as default.
    protected static final org.slf4j.Logger
    LOG
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    NodeAttributesManagerImpl()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    activateNode(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.Resource resource)
     
    void
    addNodeAttributes(Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping)
     
    void
    deactivateNode(org.apache.hadoop.yarn.api.records.NodeId nodeId)
     
    Map<org.apache.hadoop.yarn.api.records.NodeAttribute,org.apache.hadoop.yarn.nodelabels.AttributeValue>
    getAttributesForNode(String hostName)
     
    Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,Map<String,org.apache.hadoop.yarn.nodelabels.AttributeValue>>
    getAttributesToNodes(Set<org.apache.hadoop.yarn.api.records.NodeAttributeKey> attributes)
     
    Set<org.apache.hadoop.yarn.api.records.NodeAttribute>
    getClusterNodeAttributes(Set<String> prefix)
     
    Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>>
    getNodesToAttributes(Set<String> hostNames)
     
    List<org.apache.hadoop.yarn.server.api.protocolrecords.NodeToAttributes>
    getNodeToAttributes(Set<String> prefix)
     
    org.apache.hadoop.yarn.api.records.Resource
    getResourceByAttribute(org.apache.hadoop.yarn.api.records.NodeAttribute attribute)
     
    protected void
    handleStoreEvent(NodeAttributesStoreEvent event)
     
    protected void
    initDispatcher(org.apache.hadoop.conf.Configuration conf)
     
    protected void
    initNodeAttributeStore(org.apache.hadoop.conf.Configuration conf)
     
    protected void
    internalUpdateAttributesOnNodes(Map<String,Map<org.apache.hadoop.yarn.api.records.NodeAttribute,org.apache.hadoop.yarn.nodelabels.AttributeValue>> nodeAttributeMapping, org.apache.hadoop.yarn.server.api.protocolrecords.AttributeMappingOperationType op, Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,org.apache.hadoop.yarn.nodelabels.RMNodeAttribute> newAttributesToBeAdded, String attributePrefix)
     
    protected String
    normalizeAttributeValue(String value)
     
    void
    refreshNodeAttributesToScheduler(org.apache.hadoop.yarn.api.records.NodeId nodeId)
    Refresh node attributes on a given node during RM recovery.
    void
    removeNodeAttributes(Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping)
     
    void
    replaceNodeAttributes(String prefix, Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping)
     
    protected void
    serviceStart()
     
    protected void
    serviceStop()
     
    void
    setRMContext(RMContext context)
     
    protected void
    startDispatcher()
     
    protected void
    stopDispatcher()
     
    void
    updateNodeResource(org.apache.hadoop.yarn.api.records.NodeId node, org.apache.hadoop.yarn.api.records.Resource newResource)
     
    protected Map<String,Map<org.apache.hadoop.yarn.api.records.NodeAttribute,org.apache.hadoop.yarn.nodelabels.AttributeValue>>
    validate(Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping, Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,org.apache.hadoop.yarn.nodelabels.RMNodeAttribute> newAttributesToBeAdded, boolean isRemoveOperation)
    Validate for attributes.

    Methods inherited from class org.apache.hadoop.service.AbstractService

    close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

    • LOG

      protected static final org.slf4j.Logger LOG

    • EMPTY_ATTRIBUTE_VALUE

      public static final String EMPTY_ATTRIBUTE_VALUE
      If a user doesn't specify value for a label, then empty string is considered as default.
      See Also:

  • Constructor Details

    • NodeAttributesManagerImpl

      public NodeAttributesManagerImpl()

  • Method Details

    • initDispatcher

      protected void initDispatcher(org.apache.hadoop.conf.Configuration conf)

    • startDispatcher

      protected void startDispatcher()

    • serviceStart

      protected void serviceStart() throws Exception
      Overrides:
      serviceStart in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • initNodeAttributeStore

      protected void initNodeAttributeStore(org.apache.hadoop.conf.Configuration conf) throws Exception
      Throws:
      Exception

    • internalUpdateAttributesOnNodes

      @VisibleForTesting protected void internalUpdateAttributesOnNodes(Map<String,Map<org.apache.hadoop.yarn.api.records.NodeAttribute,org.apache.hadoop.yarn.nodelabels.AttributeValue>> nodeAttributeMapping, org.apache.hadoop.yarn.server.api.protocolrecords.AttributeMappingOperationType op, Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,org.apache.hadoop.yarn.nodelabels.RMNodeAttribute> newAttributesToBeAdded, String attributePrefix)

    • validate

      protected Map<String,Map<org.apache.hadoop.yarn.api.records.NodeAttribute,org.apache.hadoop.yarn.nodelabels.AttributeValue>> validate(Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping, Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,org.apache.hadoop.yarn.nodelabels.RMNodeAttribute> newAttributesToBeAdded, boolean isRemoveOperation) throws IOException
      Validate for attributes.
      Parameters:
      nodeAttributeMapping - NodeAttribute Mapping
      newAttributesToBeAdded - new Attributes ToBeAdded
      isRemoveOperation - : to indicate whether it's a remove operation.
      Returns:
      Map of String to Map of NodeAttribute to AttributeValue
      Throws:
      IOException - : on invalid mapping in the current request or against already existing NodeAttributes.

    • normalizeAttributeValue

      protected String normalizeAttributeValue(String value)

    • getClusterNodeAttributes

      public Set<org.apache.hadoop.yarn.api.records.NodeAttribute> getClusterNodeAttributes(Set<String> prefix)
      Specified by:
      getClusterNodeAttributes in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager

    • getAttributesToNodes

      public Map<org.apache.hadoop.yarn.api.records.NodeAttributeKey,Map<String,org.apache.hadoop.yarn.nodelabels.AttributeValue>> getAttributesToNodes(Set<org.apache.hadoop.yarn.api.records.NodeAttributeKey> attributes)
      Specified by:
      getAttributesToNodes in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager

    • getResourceByAttribute

      public org.apache.hadoop.yarn.api.records.Resource getResourceByAttribute(org.apache.hadoop.yarn.api.records.NodeAttribute attribute)

    • getAttributesForNode

      public Map<org.apache.hadoop.yarn.api.records.NodeAttribute,org.apache.hadoop.yarn.nodelabels.AttributeValue> getAttributesForNode(String hostName)
      Specified by:
      getAttributesForNode in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager

    • getNodeToAttributes

      public List<org.apache.hadoop.yarn.server.api.protocolrecords.NodeToAttributes> getNodeToAttributes(Set<String> prefix)
      Specified by:
      getNodeToAttributes in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager

    • getNodesToAttributes

      public Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> getNodesToAttributes(Set<String> hostNames)
      Specified by:
      getNodesToAttributes in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager

    • activateNode

      public void activateNode(org.apache.hadoop.yarn.api.records.NodeId nodeId, org.apache.hadoop.yarn.api.records.Resource resource)

    • deactivateNode

      public void deactivateNode(org.apache.hadoop.yarn.api.records.NodeId nodeId)

    • updateNodeResource

      public void updateNodeResource(org.apache.hadoop.yarn.api.records.NodeId node, org.apache.hadoop.yarn.api.records.Resource newResource)

    • handleStoreEvent

      protected void handleStoreEvent(NodeAttributesStoreEvent event)

    • replaceNodeAttributes

      public void replaceNodeAttributes(String prefix, Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping) throws IOException
      Specified by:
      replaceNodeAttributes in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager
      Throws:
      IOException

    • addNodeAttributes

      public void addNodeAttributes(Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping) throws IOException
      Specified by:
      addNodeAttributes in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager
      Throws:
      IOException

    • removeNodeAttributes

      public void removeNodeAttributes(Map<String,Set<org.apache.hadoop.yarn.api.records.NodeAttribute>> nodeAttributeMapping) throws IOException
      Specified by:
      removeNodeAttributes in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager
      Throws:
      IOException

    • stopDispatcher

      protected void stopDispatcher()

    • serviceStop

      protected void serviceStop() throws Exception
      Overrides:
      serviceStop in class org.apache.hadoop.service.AbstractService
      Throws:
      Exception

    • setRMContext

      public void setRMContext(RMContext context)

    • refreshNodeAttributesToScheduler

      public void refreshNodeAttributesToScheduler(org.apache.hadoop.yarn.api.records.NodeId nodeId)
      Refresh node attributes on a given node during RM recovery.
      Specified by:
      refreshNodeAttributesToScheduler in class org.apache.hadoop.yarn.nodelabels.NodeAttributesManager
      Parameters:
      nodeId - Node Id