Package org.apache.hadoop.hdfs.server.namenode

Class EditLogInputStream

java.lang.Object
org.apache.hadoop.hdfs.server.namenode.EditLogInputStream
All Implemented Interfaces:
Closeable, AutoCloseable
Direct Known Subclasses:
EditLogFileInputStream
@Private @Evolving public abstract class EditLogInputStream extends Object implements Closeable
A generic abstract class to support reading edits log data from persistent storage. It should stream bytes from the storage exactly as they were written into the #EditLogOutputStream.

  • Constructor Summary

    Constructors
    Constructor
    Description
    EditLogInputStream()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    abstract void
    close()
    Close the stream.
    String
    getCurrentStreamName()
    Returns the name of the currently active underlying stream.
    abstract long
    getFirstTxId()
     
    abstract long
    getLastTxId()
     
    abstract String
    getName()
     
    abstract long
    getPosition()
    Get the "position" of in the stream.
    abstract int
    getVersion(boolean verifyVersion)
    Get the layout version of the data in the stream.
    abstract boolean
    isInProgress()
    Return true if this stream is in progress, false if it is finalized.
    abstract boolean
    isLocalLog()
    Returns true if we are currently reading the log from a local disk or an even faster data source (e.g. a byte buffer).
    abstract long
    length()
    Return the size of the current edits log or -1 if unknown.
    protected abstract FSEditLogOp
    nextOp()
    Get the next operation from the stream storage.
    protected FSEditLogOp
    nextValidOp()
    Get the next valid operation from the stream storage.
    FSEditLogOp
    readOp()
    Read an operation from the stream
    void
    resync()
    Position the stream so that a valid operation can be read from it with readOp().
    protected long
    scanNextOp()
    Go through the next operation from the stream storage.
    abstract void
    setMaxOpSize(int maxOpSize)
    Set the maximum opcode size in bytes.
    boolean
    skipUntil(long txid)
    Skip edit log operations up to a given transaction ID, or until the end of the edit log is reached.
    String
    toString()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • EditLogInputStream

      public EditLogInputStream()

  • Method Details

    • getCurrentStreamName

      public String getCurrentStreamName()
      Returns the name of the currently active underlying stream. The default implementation returns the same value as getName unless overridden by the subclass.
      Returns:
      String name of the currently active underlying stream

    • getName

      public abstract String getName()
      Returns:
      the name of the EditLogInputStream

    • getFirstTxId

      public abstract long getFirstTxId()
      Returns:
      the first transaction which will be found in this stream

    • getLastTxId

      public abstract long getLastTxId()
      Returns:
      the last transaction which will be found in this stream

    • close

      public abstract void close() throws IOException
      Close the stream.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException - if an error occurred while closing

    • readOp

      public FSEditLogOp readOp() throws IOException
      Read an operation from the stream
      Returns:
      an operation from the stream or null if at end of stream
      Throws:
      IOException - if there is an error reading from the stream

    • resync

      public void resync()
      Position the stream so that a valid operation can be read from it with readOp(). This method can be used to skip over corrupted sections of edit logs.

    • nextOp

      protected abstract FSEditLogOp nextOp() throws IOException
      Get the next operation from the stream storage.
      Returns:
      an operation from the stream or null if at end of stream
      Throws:
      IOException - if there is an error reading from the stream

    • scanNextOp

      protected long scanNextOp() throws IOException
      Go through the next operation from the stream storage.
      Returns:
      the txid of the next operation.
      Throws:
      IOException

    • nextValidOp

      protected FSEditLogOp nextValidOp()
      Get the next valid operation from the stream storage. This is exactly like nextOp, except that we attempt to skip over damaged parts of the edit log
      Returns:
      an operation from the stream or null if at end of stream

    • skipUntil

      public boolean skipUntil(long txid) throws IOException
      Skip edit log operations up to a given transaction ID, or until the end of the edit log is reached. After this function returns, the next call to readOp will return either end-of-file (null) or a transaction with a txid equal to or higher than the one we asked for.
      Parameters:
      txid - The transaction ID to read up until.
      Returns:
      Returns true if we found a transaction ID greater than or equal to 'txid' in the log.
      Throws:
      IOException

    • getVersion

      public abstract int getVersion(boolean verifyVersion) throws IOException
      Get the layout version of the data in the stream.
      Returns:
      the layout version of the ops in the stream.
      Throws:
      IOException - if there is an error reading the version

    • getPosition

      public abstract long getPosition()
      Get the "position" of in the stream. This is useful for debugging and operational purposes. Different stream types can have a different meaning for what the position is. For file streams it means the byte offset from the start of the file.
      Returns:
      the position in the stream

    • length

      public abstract long length() throws IOException
      Return the size of the current edits log or -1 if unknown.
      Returns:
      long size of the current edits log or -1 if unknown
      Throws:
      IOException

    • isInProgress

      public abstract boolean isInProgress()
      Return true if this stream is in progress, false if it is finalized.

    • setMaxOpSize

      public abstract void setMaxOpSize(int maxOpSize)
      Set the maximum opcode size in bytes.

    • isLocalLog

      public abstract boolean isLocalLog()
      Returns true if we are currently reading the log from a local disk or an even faster data source (e.g. a byte buffer).

    • toString

      public String toString()
      Overrides:
      toString in class Object