Package org.apache.hadoop.hdfs.server.namenode

Class EditLogFileOutputStream

java.lang.Object
org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream
org.apache.hadoop.hdfs.server.namenode.EditLogFileOutputStream
All Implemented Interfaces:
Closeable, AutoCloseable
@Private public class EditLogFileOutputStream extends EditLogOutputStream
An implementation of the abstract class EditLogOutputStream, which stores edits in a local file.

  • Field Details

    • MIN_PREALLOCATION_LENGTH

      public static final int MIN_PREALLOCATION_LENGTH
      See Also:

  • Constructor Details

    • EditLogFileOutputStream

      public EditLogFileOutputStream(org.apache.hadoop.conf.Configuration conf, File name, int size) throws IOException
      Creates output buffers and file object.
      Parameters:
      conf - Configuration object
      name - File name to store edit log
      size - Size of flush buffer
      Throws:
      IOException

  • Method Details

    • write

      public void write(FSEditLogOp op) throws IOException
      Description copied from class: EditLogOutputStream
      Write edits log operation to the stream.
      Specified by:
      write in class EditLogOutputStream
      Parameters:
      op - operation
      Throws:
      IOException

    • writeRaw

      public void writeRaw(byte[] bytes, int offset, int length) throws IOException
      Write a transaction to the stream. The serialization format is:
      • the opcode (byte)
      • the transaction id (long)
      • the actual Writables for the transaction
      Specified by:
      writeRaw in class EditLogOutputStream
      Parameters:
      bytes - the bytes to write.
      offset - offset in the bytes to write from
      length - number of bytes to write
      Throws:
      IOException

    • create

      public void create(int layoutVersion) throws IOException
      Create empty edits logs file.
      Specified by:
      create in class EditLogOutputStream
      Parameters:
      layoutVersion - The LayoutVersion of the journal
      Throws:
      IOException

    • writeHeader

      @VisibleForTesting public static void writeHeader(int layoutVersion, DataOutputStream out) throws IOException
      Write header information for this EditLogFileOutputStream to the provided DataOutputSream.
      Parameters:
      layoutVersion - the LayoutVersion of the EditLog
      out - the output stream to write the header to.
      Throws:
      IOException - in the event of error writing to the stream.

    • close

      public void close() throws IOException
      Description copied from class: EditLogOutputStream
      Close the journal.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Specified by:
      close in class EditLogOutputStream
      Throws:
      IOException - if the journal can't be closed, or if there are unflushed edits

    • abort

      public void abort() throws IOException
      Description copied from class: EditLogOutputStream
      Close the stream without necessarily flushing any pending data. This may be called after a previous write or close threw an exception.
      Specified by:
      abort in class EditLogOutputStream
      Throws:
      IOException

    • setReadyToFlush

      public void setReadyToFlush() throws IOException
      All data that has been written to the stream so far will be flushed. New data can be still written to the stream while flushing is performed.
      Specified by:
      setReadyToFlush in class EditLogOutputStream
      Throws:
      IOException

    • flushAndSync

      public void flushAndSync(boolean durable) throws IOException
      Flush ready buffer to persistent store. currentBuffer is not flushed as it accumulates new log records while readyBuffer will be flushed and synced.
      Specified by:
      flushAndSync in class EditLogOutputStream
      Parameters:
      durable - if true, the edits should be made truly durable before returning
      Throws:
      IOException

    • shouldForceSync

      public boolean shouldForceSync()
      Description copied from class: EditLogOutputStream
      Implement the policy when to automatically sync the buffered edits log The buffered edits can be flushed when the buffer becomes full or a certain period of time is elapsed.
      Overrides:
      shouldForceSync in class EditLogOutputStream
      Returns:
      true if the number of buffered data exceeds the intial buffer size

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • isOpen

      public boolean isOpen()
      Returns:
      true if this stream is currently open.

    • setFileChannelForTesting

      @VisibleForTesting public void setFileChannelForTesting(FileChannel fc)

    • getFileChannelForTesting

      @VisibleForTesting public FileChannel getFileChannelForTesting()

    • setShouldSkipFsyncForTesting

      @VisibleForTesting public static void setShouldSkipFsyncForTesting(boolean skip)
      For the purposes of unit tests, we don't need to actually write durably to disk. So, we can skip the fsync() calls for a speed improvement.
      Parameters:
      skip - true if fsync should not be called