Package org.apache.hadoop.hdfs.server.blockmanagement

Class CombinedHostFileManager

java.lang.Object
org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager
org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
public class CombinedHostFileManager extends HostConfigManager
This class manages datanode configuration using a json file. Please refer to CombinedHostsFileReader for the json format.

Entries may or may not specify a port. If they don't, we consider them to apply to every DataNode on that host. The code canonicalizes the entries into IP addresses.

The code ignores all entries that the DNS fails to resolve their IP addresses. This is okay because by default the NN rejects the registrations of DNs when it fails to do a forward and reverse lookup. Note that DNS resolutions are only done during the loading time to minimize the latency.

  • Constructor Details

    • CombinedHostFileManager

      public CombinedHostFileManager()

  • Method Details

    • getIncludes

      public Iterable<InetSocketAddress> getIncludes()
      Description copied from class: HostConfigManager
      Return all the datanodes that are allowed to connect to the namenode.
      Specified by:
      getIncludes in class HostConfigManager
      Returns:
      Iterable of all datanodes

    • getExcludes

      public Iterable<InetSocketAddress> getExcludes()
      Description copied from class: HostConfigManager
      Return all datanodes that should be in decommissioned state.
      Specified by:
      getExcludes in class HostConfigManager
      Returns:
      Iterable of those datanodes

    • setConf

      public void setConf(org.apache.hadoop.conf.Configuration conf)

    • getConf

      public org.apache.hadoop.conf.Configuration getConf()

    • refresh

      public void refresh() throws IOException
      Description copied from class: HostConfigManager
      Reload the configuration.
      Specified by:
      refresh in class HostConfigManager
      Throws:
      IOException

    • isIncluded

      public boolean isIncluded(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Check if a datanode is allowed to connect the namenode.
      Specified by:
      isIncluded in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      boolean if dn is allowed to connect the namenode.

    • isExcluded

      public boolean isExcluded(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Check if a datanode needs to be decommissioned.
      Specified by:
      isExcluded in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      boolean if dn needs to be decommissioned.

    • getUpgradeDomain

      public String getUpgradeDomain(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Get the upgrade domain of a datanode.
      Specified by:
      getUpgradeDomain in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      the upgrade domain of dn.

    • getMaintenanceExpirationTimeInMS

      public long getMaintenanceExpirationTimeInMS(org.apache.hadoop.hdfs.protocol.DatanodeID dn)
      Description copied from class: HostConfigManager
      Get the maintenance expiration time in milli seconds.
      Specified by:
      getMaintenanceExpirationTimeInMS in class HostConfigManager
      Parameters:
      dn - the DatanodeID of the datanode
      Returns:
      the maintenance expiration time of dn.