Package org.apache.hadoop.hdfs.server.common

Class Storage.StorageDirectory

java.lang.Object
org.apache.hadoop.hdfs.server.common.Storage.StorageDirectory
All Implemented Interfaces:
Storage.FormatConfirmable
Enclosing class:
Storage
@Private public static class Storage.StorageDirectory extends Object implements Storage.FormatConfirmable
One of the storage directories.

  • Constructor Details

    • StorageDirectory

      public StorageDirectory(File dir)

    • StorageDirectory

      public StorageDirectory(StorageLocation location)

    • StorageDirectory

      public StorageDirectory(File dir, Storage.StorageDirType dirType)

    • StorageDirectory

      public StorageDirectory(File dir, Storage.StorageDirType dirType, boolean isShared)
      Constructor
      Parameters:
      dir - directory corresponding to the storage
      dirType - storage directory type
      isShared - whether or not this dir is shared between two NNs. true disables locking on the storage directory, false enables locking

    • StorageDirectory

      public StorageDirectory(File dir, Storage.StorageDirType dirType, boolean isShared, org.apache.hadoop.fs.permission.FsPermission permission)

    • StorageDirectory

      public StorageDirectory(Storage.StorageDirType dirType, boolean isShared, StorageLocation location)
      Constructor
      Parameters:
      dirType - storage directory type
      isShared - whether or not this dir is shared between two NNs. true disables locking on the storage directory, false enables locking
      location - the StorageLocation for this directory

    • StorageDirectory

      public StorageDirectory(String bpid, Storage.StorageDirType dirType, boolean isShared, StorageLocation location)
      Constructor
      Parameters:
      bpid - the block pool id
      dirType - storage directory type
      isShared - whether or not this dir is shared between two NNs. true disables locking on the storage directory, false enables locking
      location - the StorageLocation for this directory

  • Method Details

    • setStorageUuid

      public void setStorageUuid(String storageUuid)

    • getStorageUuid

      public String getStorageUuid()

    • getRoot

      public File getRoot()
      Get root directory of this storage

    • getStorageDirType

      public Storage.StorageDirType getStorageDirType()
      Get storage directory type

    • getDirecorySize

      public long getDirecorySize()
      Get storage directory size.

    • read

      public void read(File from, Storage storage) throws IOException
      Throws:
      IOException

    • clearDirectory

      public void clearDirectory() throws IOException
      Clear and re-create storage directory.

      Removes contents of the current directory and creates an empty directory. This does not fully format storage directory. It cannot write the version file since it should be written last after all other storage type dependent files are written. Derived storage is responsible for setting specific storage values and writing the version file to disk.

      Throws:
      IOException

    • getCurrentDir

      public File getCurrentDir()
      Directory current contains latest files defining the file system meta-data.
      Returns:
      the directory path

    • getVersionFile

      public File getVersionFile()
      File VERSION contains the following fields:
      1. node type
      2. layout version
      3. namespaceID
      4. fs state creation time
      5. other fields specific for this node type
      The version file is always written last during storage directory updates. The existence of the version file indicates that all other files have been successfully written in the storage directory, the storage is valid and does not need to be recovered.
      Returns:
      the version file path

    • getPreviousVersionFile

      public File getPreviousVersionFile()
      File VERSION from the previous directory.
      Returns:
      the previous version file path

    • getPreviousDir

      public File getPreviousDir()
      Directory previous contains the previous file system state, which the system can be rolled back to.
      Returns:
      the directory path

    • getPreviousTmp

      public File getPreviousTmp()
      previous.tmp is a transient directory, which holds current file system state while the new state is saved into the new current during upgrade. If the saving succeeds previous.tmp will be moved to previous, otherwise it will be renamed back to current by the recovery procedure during startup.
      Returns:
      the directory path

    • getRemovedTmp

      public File getRemovedTmp()
      removed.tmp is a transient directory, which holds current file system state while the previous state is moved into current during rollback. If the moving succeeds removed.tmp will be removed, otherwise it will be renamed back to current by the recovery procedure during startup.
      Returns:
      the directory path

    • getFinalizedTmp

      public File getFinalizedTmp()
      finalized.tmp is a transient directory, which holds the previous file system state while it is being removed in response to the finalize request. Finalize operation will remove finalized.tmp when completed, otherwise the removal will resume upon the system startup.
      Returns:
      the directory path

    • getLastCheckpointTmp

      public File getLastCheckpointTmp()
      lastcheckpoint.tmp is a transient directory, which holds current file system state while the new state is saved into the new current during regular namespace updates. If the saving succeeds lastcheckpoint.tmp will be moved to previous.checkpoint, otherwise it will be renamed back to current by the recovery procedure during startup.
      Returns:
      the directory path

    • getPreviousCheckpoint

      public File getPreviousCheckpoint()
      previous.checkpoint is a directory, which holds the previous (before the last save) state of the storage directory. The directory is created as a reference only, it does not play role in state recovery procedures, and is recycled automatically, but it may be useful for manual recovery of a stale state of the system.
      Returns:
      the directory path

    • analyzeStorage

      public Storage.StorageState analyzeStorage(HdfsServerConstants.StartupOption startOpt, Storage storage) throws IOException
      Check consistency of the storage directory.
      Parameters:
      startOpt - a startup option.
      storage - The Storage object that manages this StorageDirectory.
      Returns:
      state Storage.StorageState of the storage directory
      Throws:
      InconsistentFSStateException - if directory state is not consistent and cannot be recovered.
      IOException

    • analyzeStorage

      public Storage.StorageState analyzeStorage(HdfsServerConstants.StartupOption startOpt, Storage storage, boolean checkCurrentIsEmpty) throws IOException
      Check consistency of the storage directory.
      Parameters:
      startOpt - a startup option.
      storage - The Storage object that manages this StorageDirectory.
      checkCurrentIsEmpty - if true, make sure current/ directory is empty before determining to format it.
      Returns:
      state Storage.StorageState of the storage directory
      Throws:
      InconsistentFSStateException - if directory state is not consistent and cannot be recovered.
      IOException

    • doRecover

      public void doRecover(Storage.StorageState curState) throws IOException
      Complete or recover storage state from previously failed transition.
      Parameters:
      curState - specifies what/how the state should be recovered
      Throws:
      IOException

    • hasSomeData

      public boolean hasSomeData() throws IOException
      Specified by:
      hasSomeData in interface Storage.FormatConfirmable
      Returns:
      true if the storage directory should prompt the user prior to formatting (i.e if the directory appears to contain some data)
      Throws:
      IOException - if the SD cannot be accessed due to an IO error

    • isShared

      public boolean isShared()

    • lock

      public void lock() throws IOException
      Lock storage to provide exclusive access.

      Locking is not supported by all file systems. E.g., NFS does not consistently support exclusive locks.

      If locking is supported we guarantee exclusive access to the storage directory. Otherwise, no guarantee is given.

      Throws:
      IOException - if locking fails

    • unlock

      public void unlock() throws IOException
      Unlock storage.
      Throws:
      IOException

    • toString

      public String toString()
      Specified by:
      toString in interface Storage.FormatConfirmable
      Overrides:
      toString in class Object
      Returns:
      a string representation of the formattable item, suitable for display to the user inside a prompt

    • isLockSupported

      public boolean isLockSupported() throws IOException
      Check whether underlying file system supports file locking.
      Returns:
      true if exclusive locks are supported or false otherwise.
      Throws:
      IOException
      See Also:

    • getStorageLocation

      public StorageLocation getStorageLocation()