Package org.apache.hadoop.hdfs.tools

Class DFSck

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.hdfs.tools.DFSck
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
@Private public class DFSck extends org.apache.hadoop.conf.Configured implements org.apache.hadoop.util.Tool
This class provides rudimentary checking of DFS volumes for errors and sub-optimal conditions.

The tool scans all files and directories, starting from an indicated root path. The following abnormal conditions are detected and handled:

  • files with blocks that are completely missing from all datanodes.
    In this case the tool can perform one of the following actions:
    • move corrupted files to /lost+found directory on DFS (NamenodeFsck.doMove). Remaining data blocks are saved as a block chains, representing longest consecutive series of valid blocks.
    • delete corrupted files (NamenodeFsck.doDelete)
  • detect files with under-replicated or over-replicated blocks
Additionally, the tool collects a detailed overall DFS statistics, and optionally can print detailed statistics on block locations and replication factors of each file. The tool also provides and option to filter open files during the scan.

  • Constructor Summary

    Constructors
    Constructor
    Description
    DFSck(org.apache.hadoop.conf.Configuration conf)
    Filesystem checker.
    DFSck(org.apache.hadoop.conf.Configuration conf, PrintStream out)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    main(String[] args)
     
    int
    run(String[] args)
     

    Methods inherited from class org.apache.hadoop.conf.Configured

    getConf, setConf

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.conf.Configurable

    getConf, setConf

  • Constructor Details

    • DFSck

      public DFSck(org.apache.hadoop.conf.Configuration conf) throws IOException
      Filesystem checker.
      Parameters:
      conf - current Configuration
      Throws:
      IOException

    • DFSck

      public DFSck(org.apache.hadoop.conf.Configuration conf, PrintStream out) throws IOException
      Throws:
      IOException

  • Method Details