Package org.apache.hadoop.hdfs.server.namenode

Class EditLogFileInputStream

java.lang.Object
org.apache.hadoop.hdfs.server.namenode.EditLogInputStream
org.apache.hadoop.hdfs.server.namenode.EditLogFileInputStream
All Implemented Interfaces:
Closeable, AutoCloseable
@Private public class EditLogFileInputStream extends EditLogInputStream
An implementation of the abstract class EditLogInputStream, which reads edits from a file. That file may be either on the local disk or accessible via a URL.

  • Constructor Summary

    Constructors
    Constructor
    Description
    EditLogFileInputStream(File name, long firstTxId, long lastTxId, boolean isInProgress)
    Open an EditLogInputStream for the given file.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
    Close the stream.
    static EditLogInputStream
    fromByteString(org.apache.hadoop.thirdparty.protobuf.ByteString bytes, long startTxId, long endTxId, boolean inProgress)
    Create an EditLogInputStream from a ByteString, i.e. an in-memory collection of bytes.
    static EditLogInputStream
    fromUrl(org.apache.hadoop.hdfs.web.URLConnectionFactory connectionFactory, URL url, long startTxId, long endTxId, boolean inProgress)
    Open an EditLogInputStream for the given URL.
    long
    getFirstTxId()
     
    long
    getLastTxId()
     
    String
    getName()
     
    long
    getPosition()
    Get the "position" of in the stream.
    int
    getVersion(boolean verifyVersion)
    Get the layout version of the data in the stream.
    boolean
    isInProgress()
    Return true if this stream is in progress, false if it is finalized.
    boolean
    isLocalLog()
    Returns true if we are currently reading the log from a local disk or an even faster data source (e.g. a byte buffer).
    long
    length()
    Return the size of the current edits log or -1 if unknown.
    protected FSEditLogOp
    nextOp()
    Get the next operation from the stream storage.
    protected FSEditLogOp
    nextValidOp()
    Get the next valid operation from the stream storage.
    protected long
    scanNextOp()
    Go through the next operation from the stream storage.
    void
    setMaxOpSize(int maxOpSize)
    Set the maximum opcode size in bytes.
    String
    toString()
     

    Methods inherited from class org.apache.hadoop.hdfs.server.namenode.EditLogInputStream

    getCurrentStreamName, readOp, resync, skipUntil

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • EditLogFileInputStream

      public EditLogFileInputStream(File name, long firstTxId, long lastTxId, boolean isInProgress)
      Open an EditLogInputStream for the given file.
      Parameters:
      name - filename to open
      firstTxId - first transaction found in file
      lastTxId - last transaction id found in file

  • Method Details

    • fromUrl

      public static EditLogInputStream fromUrl(org.apache.hadoop.hdfs.web.URLConnectionFactory connectionFactory, URL url, long startTxId, long endTxId, boolean inProgress)
      Open an EditLogInputStream for the given URL.
      Parameters:
      connectionFactory - the URLConnectionFactory used to create the connection.
      url - the url hosting the log
      startTxId - the expected starting txid
      endTxId - the expected ending txid
      inProgress - whether the log is in-progress
      Returns:
      a stream from which edits may be read

    • fromByteString

      public static EditLogInputStream fromByteString(org.apache.hadoop.thirdparty.protobuf.ByteString bytes, long startTxId, long endTxId, boolean inProgress)
      Create an EditLogInputStream from a ByteString, i.e. an in-memory collection of bytes.
      Parameters:
      bytes - The byte string to read from
      startTxId - the expected starting transaction ID
      endTxId - the expected ending transaction ID
      inProgress - whether the log is in-progress
      Returns:
      An edit stream to read from

    • getFirstTxId

      public long getFirstTxId()
      Specified by:
      getFirstTxId in class EditLogInputStream
      Returns:
      the first transaction which will be found in this stream

    • getLastTxId

      public long getLastTxId()
      Specified by:
      getLastTxId in class EditLogInputStream
      Returns:
      the last transaction which will be found in this stream

    • getName

      public String getName()
      Specified by:
      getName in class EditLogInputStream
      Returns:
      the name of the EditLogInputStream

    • scanNextOp

      protected long scanNextOp() throws IOException
      Description copied from class: EditLogInputStream
      Go through the next operation from the stream storage.
      Overrides:
      scanNextOp in class EditLogInputStream
      Returns:
      the txid of the next operation.
      Throws:
      IOException

    • nextOp

      protected FSEditLogOp nextOp() throws IOException
      Description copied from class: EditLogInputStream
      Get the next operation from the stream storage.
      Specified by:
      nextOp in class EditLogInputStream
      Returns:
      an operation from the stream or null if at end of stream
      Throws:
      IOException - if there is an error reading from the stream

    • nextValidOp

      protected FSEditLogOp nextValidOp()
      Description copied from class: EditLogInputStream
      Get the next valid operation from the stream storage. This is exactly like nextOp, except that we attempt to skip over damaged parts of the edit log
      Overrides:
      nextValidOp in class EditLogInputStream
      Returns:
      an operation from the stream or null if at end of stream

    • getVersion

      public int getVersion(boolean verifyVersion) throws IOException
      Description copied from class: EditLogInputStream
      Get the layout version of the data in the stream.
      Specified by:
      getVersion in class EditLogInputStream
      Returns:
      the layout version of the ops in the stream.
      Throws:
      IOException - if there is an error reading the version

    • getPosition

      public long getPosition()
      Description copied from class: EditLogInputStream
      Get the "position" of in the stream. This is useful for debugging and operational purposes. Different stream types can have a different meaning for what the position is. For file streams it means the byte offset from the start of the file.
      Specified by:
      getPosition in class EditLogInputStream
      Returns:
      the position in the stream

    • close

      public void close() throws IOException
      Description copied from class: EditLogInputStream
      Close the stream.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Specified by:
      close in class EditLogInputStream
      Throws:
      IOException - if an error occurred while closing

    • length

      public long length() throws IOException
      Description copied from class: EditLogInputStream
      Return the size of the current edits log or -1 if unknown.
      Specified by:
      length in class EditLogInputStream
      Returns:
      long size of the current edits log or -1 if unknown
      Throws:
      IOException

    • isInProgress

      public boolean isInProgress()
      Description copied from class: EditLogInputStream
      Return true if this stream is in progress, false if it is finalized.
      Specified by:
      isInProgress in class EditLogInputStream

    • toString

      public String toString()
      Overrides:
      toString in class EditLogInputStream

    • setMaxOpSize

      public void setMaxOpSize(int maxOpSize)
      Description copied from class: EditLogInputStream
      Set the maximum opcode size in bytes.
      Specified by:
      setMaxOpSize in class EditLogInputStream

    • isLocalLog

      public boolean isLocalLog()
      Description copied from class: EditLogInputStream
      Returns true if we are currently reading the log from a local disk or an even faster data source (e.g. a byte buffer).
      Specified by:
      isLocalLog in class EditLogInputStream