Package org.apache.hadoop.hdfs.server.datanode

Class LocalReplica

java.lang.Object
org.apache.hadoop.hdfs.protocol.Block
org.apache.hadoop.hdfs.server.datanode.ReplicaInfo
org.apache.hadoop.hdfs.server.datanode.LocalReplica
All Implemented Interfaces:
Comparable<org.apache.hadoop.hdfs.protocol.Block>, Replica, org.apache.hadoop.io.Writable, org.apache.hadoop.util.LightWeightGSet.LinkedElement
Direct Known Subclasses:
FinalizedReplica, LocalReplicaInPipeline, ReplicaUnderRecovery, ReplicaWaitingToBeRecovered
public abstract class LocalReplica extends ReplicaInfo
This class is used for all replicas which are on local storage media and hence, are backed by files.

  • Method Details

    • getBlockFile

      @VisibleForTesting public File getBlockFile()
      Get the full path of this replica's data file.
      Returns:
      the full path of this replica's data file

    • getMetaFile

      @VisibleForTesting public File getMetaFile()
      Get the full path of this replica's meta file.
      Returns:
      the full path of this replica's meta file

    • getDir

      protected File getDir()
      Return the parent directory path where this replica is located.
      Returns:
      the parent directory path where this replica is located

    • parseBaseDir

      @VisibleForTesting public static LocalReplica.ReplicaDirInfo parseBaseDir(File dir, long blockId)

    • breakHardLinksIfNeeded

      public boolean breakHardLinksIfNeeded() throws IOException
      This function "breaks hardlinks" to the current replica file. When doing a DataNode upgrade, we create a bunch of hardlinks to each block file. This cleverly ensures that both the old and the new storage directories can contain the same block file, without using additional space for the data. However, when we want to append to the replica file, we need to "break" the hardlink to ensure that the old snapshot continues to contain the old data length. If we failed to do that, we could roll back to the previous/ directory during a downgrade, and find that the block contents were longer than they were at the time of upgrade.
      Specified by:
      breakHardLinksIfNeeded in class ReplicaInfo
      Returns:
      true only if data was copied.
      Throws:
      IOException

    • getBlockURI

      public URI getBlockURI()
      Description copied from class: ReplicaInfo
      Get the URI for where the data of this replica is stored.
      Specified by:
      getBlockURI in class ReplicaInfo
      Returns:
      URI for the location of replica data.

    • getDataInputStream

      public InputStream getDataInputStream(long seekOffset) throws IOException
      Description copied from class: ReplicaInfo
      Returns an InputStream to the replica's data.
      Specified by:
      getDataInputStream in class ReplicaInfo
      Parameters:
      seekOffset - the offset at which the read is started from.
      Returns:
      the InputStream to read the replica data.
      Throws:
      IOException - if an error occurs in opening a stream to the data.

    • getDataOutputStream

      public OutputStream getDataOutputStream(boolean append) throws IOException
      Description copied from class: ReplicaInfo
      Returns an OutputStream to the replica's data.
      Specified by:
      getDataOutputStream in class ReplicaInfo
      Parameters:
      append - indicates if the block should be opened for append.
      Returns:
      the OutputStream to write to the replica.
      Throws:
      IOException - if an error occurs in creating an OutputStream.

    • blockDataExists

      public boolean blockDataExists()
      Specified by:
      blockDataExists in class ReplicaInfo
      Returns:
      true if the replica's data exists.

    • deleteBlockData

      public boolean deleteBlockData()
      Description copied from class: ReplicaInfo
      Used to deletes the replica's block data.
      Specified by:
      deleteBlockData in class ReplicaInfo
      Returns:
      true if the replica's data is successfully deleted.

    • getBlockDataLength

      public long getBlockDataLength()
      Specified by:
      getBlockDataLength in class ReplicaInfo
      Returns:
      the length of the block on storage.

    • getMetadataURI

      public URI getMetadataURI()
      Description copied from class: ReplicaInfo
      Get the URI for where the metadata of this replica is stored.
      Specified by:
      getMetadataURI in class ReplicaInfo
      Returns:
      URI for the location of replica metadata.

    • getMetadataInputStream

      public LengthInputStream getMetadataInputStream(long offset) throws IOException
      Description copied from class: ReplicaInfo
      Returns an InputStream to the replica's metadata.
      Specified by:
      getMetadataInputStream in class ReplicaInfo
      Parameters:
      offset - the offset at which the read is started from.
      Returns:
      the LengthInputStream to read the replica metadata.
      Throws:
      IOException

    • getMetadataOutputStream

      public OutputStream getMetadataOutputStream(boolean append) throws IOException
      Description copied from class: ReplicaInfo
      Returns an OutputStream to the replica's metadata.
      Specified by:
      getMetadataOutputStream in class ReplicaInfo
      Parameters:
      append - indicates if the block metadata should be opened for append.
      Returns:
      the OutputStream to write to the replica's metadata.
      Throws:
      IOException - if an error occurs in creating an OutputStream.

    • metadataExists

      public boolean metadataExists()
      Specified by:
      metadataExists in class ReplicaInfo
      Returns:
      true if the replica's metadata exists.

    • deleteMetadata

      public boolean deleteMetadata()
      Description copied from class: ReplicaInfo
      Used to deletes the replica's metadata.
      Specified by:
      deleteMetadata in class ReplicaInfo
      Returns:
      true if the replica's metadata is successfully deleted.

    • getMetadataLength

      public long getMetadataLength()
      Specified by:
      getMetadataLength in class ReplicaInfo
      Returns:
      the length of the metadata on storage.

    • renameMeta

      public boolean renameMeta(URI destURI) throws IOException
      Description copied from class: ReplicaInfo
      Rename the metadata URI to that referenced by destURI.
      Specified by:
      renameMeta in class ReplicaInfo
      Parameters:
      destURI - the target URI.
      Returns:
      true if the rename is successful.
      Throws:
      IOException - if an exception occurs in the rename.

    • renameData

      public boolean renameData(URI destURI) throws IOException
      Description copied from class: ReplicaInfo
      Rename the data URI to that referenced by destURI.
      Specified by:
      renameData in class ReplicaInfo
      Parameters:
      destURI - the target URI.
      Returns:
      true if the rename is successful.
      Throws:
      IOException - if an exception occurs in the rename.

    • updateWithReplica

      public void updateWithReplica(StorageLocation replicaLocation)
      Description copied from class: ReplicaInfo
      Update this replica with the StorageLocation found.
      Specified by:
      updateWithReplica in class ReplicaInfo
      Parameters:
      replicaLocation - the StorageLocation found for this replica.

    • getPinning

      public boolean getPinning(org.apache.hadoop.fs.LocalFileSystem localFS) throws IOException
      Description copied from class: ReplicaInfo
      Check whether the block was pinned.
      Specified by:
      getPinning in class ReplicaInfo
      Parameters:
      localFS - the local filesystem to use.
      Returns:
      true if the block is pinned.
      Throws:
      IOException

    • setPinning

      public void setPinning(org.apache.hadoop.fs.LocalFileSystem localFS) throws IOException
      Description copied from class: ReplicaInfo
      Set a block to be pinned on this datanode so that it cannot be moved by Balancer/Mover.
      Specified by:
      setPinning in class ReplicaInfo
      Parameters:
      localFS - the local filesystem to use.
      Throws:
      IOException - if there is an exception in the pinning.

    • bumpReplicaGS

      public void bumpReplicaGS(long newGS) throws IOException
      Description copied from class: ReplicaInfo
      Bump a replica's generation stamp to a new one. Its on-disk meta file name is renamed to be the new one too.
      Specified by:
      bumpReplicaGS in class ReplicaInfo
      Parameters:
      newGS - new generation stamp
      Throws:
      IOException - if the change fails

    • truncateBlock

      public void truncateBlock(long newLength) throws IOException
      Specified by:
      truncateBlock in class ReplicaInfo
      Throws:
      IOException

    • compareWith

      public int compareWith(FsVolumeSpi.ScanInfo info)
      Specified by:
      compareWith in class ReplicaInfo

    • copyMetadata

      public void copyMetadata(URI destination) throws IOException
      Specified by:
      copyMetadata in class ReplicaInfo
      Throws:
      IOException

    • copyBlockdata

      public void copyBlockdata(URI destination) throws IOException
      Specified by:
      copyBlockdata in class ReplicaInfo
      Throws:
      IOException

    • getPinning

      public boolean getPinning(org.apache.hadoop.fs.LocalFileSystem localFS, org.apache.hadoop.fs.Path path) throws IOException
      Get pin status of a file by checking the sticky bit.
      Parameters:
      localFS - local file system
      path - path to be checked
      Returns:
      true if the file is pinned with sticky bit
      Throws:
      IOException

    • setPinning

      public void setPinning(org.apache.hadoop.fs.LocalFileSystem localFS, org.apache.hadoop.fs.Path path) throws IOException
      Set sticky bit on path to pin file.
      Parameters:
      localFS - local file system
      path - path to be pinned with sticky bit
      Throws:
      IOException

    • truncateBlock

      public static void truncateBlock(FsVolumeSpi volume, File blockFile, File metaFile, long oldlen, long newlen, FileIoProvider fileIoProvider) throws IOException
      Throws:
      IOException

    • fsyncDirectory

      public void fsyncDirectory() throws IOException
      Sync the parent directory changes to durable device.
      Throws:
      IOException