Package org.apache.hadoop.fs.impl.prefetch

Class CachingBlockManager

java.lang.Object
org.apache.hadoop.fs.impl.prefetch.BlockManager
org.apache.hadoop.fs.impl.prefetch.CachingBlockManager
All Implemented Interfaces:
Closeable, AutoCloseable
public abstract class CachingBlockManager extends BlockManager
Provides read access to the underlying file one block at a time. Improve read performance by prefetching and locall caching blocks.

  • Constructor Details

    • CachingBlockManager

      public CachingBlockManager(@Nonnull BlockManagerParameters blockManagerParameters)
      Constructs an instance of a CachingBlockManager.
      Parameters:
      blockManagerParameters - params for block manager.
      Throws:
      IllegalArgumentException - if bufferPoolSize is zero or negative.

  • Method Details

    • get

      public BufferData get(int blockNumber) throws IOException
      Gets the block having the given blockNumber.
      Overrides:
      get in class BlockManager
      Parameters:
      blockNumber - the number of the block to be read and returned.
      Returns:
      BufferData having data from the given block.
      Throws:
      IllegalArgumentException - if blockNumber is negative.
      IOException - if there an error reading the given block.

    • release

      public void release(BufferData data)
      Releases resources allocated to the given block.
      Overrides:
      release in class BlockManager
      Parameters:
      data - the BufferData to release.
      Throws:
      IllegalArgumentException - if data is null.

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class BlockManager

    • requestPrefetch

      public void requestPrefetch(int blockNumber)
      Requests optional prefetching of the given block. The block is prefetched only if we can acquire a free buffer.
      Overrides:
      requestPrefetch in class BlockManager
      Parameters:
      blockNumber - the id of the block to prefetch.
      Throws:
      IllegalArgumentException - if blockNumber is negative.

    • cancelPrefetches

      public void cancelPrefetches()
      Requests cancellation of any previously issued prefetch requests.
      Overrides:
      cancelPrefetches in class BlockManager

    • requestCaching

      public void requestCaching(BufferData data)
      Requests that the given block should be copied to the local cache. The block must not be accessed by the caller after calling this method because it will released asynchronously relative to the caller.
      Overrides:
      requestCaching in class BlockManager
      Parameters:
      data - the BufferData instance to optionally cache.
      Throws:
      IllegalArgumentException - if data is null.

    • createCache

      protected BlockCache createCache(int maxBlocksCount, DurationTrackerFactory trackerFactory)

    • cachePut

      protected void cachePut(int blockNumber, ByteBuffer buffer) throws IOException
      Throws:
      IOException

    • numAvailable

      public int numAvailable()
      Number of ByteBuffers available to be acquired.
      Returns:
      the number of available buffers.

    • numCached

      public int numCached()
      Number of caching operations completed.
      Returns:
      the number of cached buffers.

    • numCachingErrors

      public int numCachingErrors()
      Number of errors encountered when caching.
      Returns:
      the number of errors encountered when caching.

    • numReadErrors

      public int numReadErrors()
      Number of errors encountered when reading.
      Returns:
      the number of errors encountered when reading.

    • toString

      public String toString()
      Overrides:
      toString in class Object