Package org.apache.hadoop.hdfs.server.blockmanagement

Class HostFileManager

java.lang.Object
org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager
org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
public class HostFileManager extends HostConfigManager
This class manages the include and exclude files for HDFS.

These files control which DataNodes the NameNode expects to see in the cluster. Loosely speaking, the include file, if it exists and is not empty, is a list of everything we expect to see. The exclude file is a list of everything we want to ignore if we do see it.

Entries may or may not specify a port. If they don't, we consider them to apply to every DataNode on that host. The code canonicalizes the entries into IP addresses.

The code ignores all entries that the DNS fails to resolve their IP addresses. This is okay because by default the NN rejects the registrations of DNs when it fails to do a forward and reverse lookup. Note that DNS resolutions are only done during the loading time to minimize the latency.

  • Constructor Summary

    Constructors
    Constructor
    Description
    HostFileManager()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.hadoop.conf.Configuration
    getConf()
     
    HostSet
    getExcludes()
    Return all datanodes that should be in decommissioned state.
    HostSet
    getIncludes()
    Return all the datanodes that are allowed to connect to the namenode.
    long
    getMaintenanceExpirationTimeInMS(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
    Get the maintenance expiration time in milli seconds.
    String
    getUpgradeDomain(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
    Get the upgrade domain of a datanode.
    boolean
    isExcluded(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
    Check if a datanode needs to be decommissioned.
    boolean
    isIncluded(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
    Check if a datanode is allowed to connect the namenode.
    void
    refresh()
    Reload the configuration.
    void
    setConf(org.apache.hadoop.conf.Configuration conf)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • HostFileManager

      public HostFileManager()

  • Method Details

    • setConf

      public void setConf(org.apache.hadoop.conf.Configuration conf)

    • getConf

      public org.apache.hadoop.conf.Configuration getConf()

    • refresh

      public void refresh() throws IOException
      Description copied from class: HostConfigManager
      Reload the configuration.
      Specified by:
      refresh in class HostConfigManager
      Throws:
      IOException

    • getIncludes

      public HostSet getIncludes()
      Description copied from class: HostConfigManager
      Return all the datanodes that are allowed to connect to the namenode.
      Specified by:
      getIncludes in class HostConfigManager
      Returns:
      Iterable of all datanodes

    • getExcludes

      public HostSet getExcludes()
      Description copied from class: HostConfigManager
      Return all datanodes that should be in decommissioned state.
      Specified by:
      getExcludes in class HostConfigManager
      Returns:
      Iterable of those datanodes

    • isIncluded

      public boolean isIncluded(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Check if a datanode is allowed to connect the namenode.
      Specified by:
      isIncluded in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      boolean if dn is allowed to connect the namenode.

    • isExcluded

      public boolean isExcluded(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Check if a datanode needs to be decommissioned.
      Specified by:
      isExcluded in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      boolean if dn needs to be decommissioned.

    • getUpgradeDomain

      public String getUpgradeDomain(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Get the upgrade domain of a datanode.
      Specified by:
      getUpgradeDomain in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      the upgrade domain of dn.

    • getMaintenanceExpirationTimeInMS

      public long getMaintenanceExpirationTimeInMS(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Get the maintenance expiration time in milli seconds.
      Specified by:
      getMaintenanceExpirationTimeInMS in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      the maintenance expiration time of dn.