Package org.apache.hadoop.fs.impl

Class TrackingByteBufferPool

java.lang.Object

org.apache.hadoop.fs.impl.TrackingByteBufferPool

All Implemented Interfaces:

AutoCloseable, ByteBufferPool

@VisibleForTesting
public final class TrackingByteBufferPool
extends Object
implements ByteBufferPool, AutoCloseable

A wrapper ByteBufferPool implementation that tracks whether all allocated buffers are released.

It throws the related exception at close() if any buffer remains un-released. It also clears the buffers at release so if they continued being used it'll generate errors.

To be used for testing..

The stacktraces of the allocation are not stored by default because it can significantly decrease the unit test performance. Configuring this class to log at DEBUG will trigger their collection.

See Also:

• Adapted from Parquet class org.apache.parquet.bytes.TrackingByteBufferAllocator.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	TrackingByteBufferPool.ByteBufferAllocationStacktraceException	Strack trace of allocation as saved in the tracking map.
static class	TrackingByteBufferPool.LeakDetectorHeapByteBufferPoolException
static final class	TrackingByteBufferPool.LeakedByteBufferException	Exception raised in close() if there was an unreleased buffer.
static final class	TrackingByteBufferPool.ReleasingUnallocatedByteBufferException	Exception raised in putBuffer(ByteBuffer) if the buffer to release was not in the hash map.

Method Summary

Modifier and Type	Method	Description
void	close()	Expect all buffers to be released -if not, log unreleased ones and then raise an exception with the stack trace of the first unreleased buffer.
boolean	containsBuffer(ByteBuffer buffer)	Check if the buffer is in the pool.
ByteBuffer	getBuffer(boolean direct, int size)	Get a buffer from the pool.
int	getBufferAllocations()
int	getBufferReleases()
void	putBuffer(ByteBuffer buffer)	Release a buffer back to the pool.
int	size()	Get the number of allocated buffers.
static TrackingByteBufferPool	wrap(ByteBufferPool allocator)	Wrap an existing allocator with this tracking allocator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.io.ByteBufferPool

release

Method Details

wrap

public static TrackingByteBufferPool wrap(ByteBufferPool allocator)

Wrap an existing allocator with this tracking allocator.

Parameters:

allocator - allocator to wrap.

Returns:

a new allocator.

getBufferAllocations

public int getBufferAllocations()

getBufferReleases

public int getBufferReleases()

getBuffer

public ByteBuffer getBuffer(boolean direct,
 int size)

Get a buffer from the pool.

This increments the bufferAllocations counter and stores the singleron or local allocation stack trace in the allocated map.

Specified by:

getBuffer in interface ByteBufferPool

Parameters:

direct - whether to allocate a direct buffer or not

size - size of the buffer to allocate

Returns:

a ByteBuffer instance

putBuffer

public void putBuffer(ByteBuffer buffer)
               throws TrackingByteBufferPool.ReleasingUnallocatedByteBufferException

Release a buffer back to the pool.

This increments the bufferReleases counter and removes the buffer from the allocated map.

If the buffer was not allocated by this pool, it throws TrackingByteBufferPool.ReleasingUnallocatedByteBufferException.

Specified by:

putBuffer in interface ByteBufferPool

Parameters:

buffer - buffer to release

Throws:

TrackingByteBufferPool.ReleasingUnallocatedByteBufferException - if the buffer was not allocated by this pool

containsBuffer

public boolean containsBuffer(ByteBuffer buffer)

Check if the buffer is in the pool.

Parameters:

buffer - buffer

Returns:

true if the buffer is in the pool

size

public int size()

Get the number of allocated buffers.

Returns:

number of allocated buffers

close

public void close()
           throws TrackingByteBufferPool.LeakedByteBufferException

Expect all buffers to be released -if not, log unreleased ones and then raise an exception with the stack trace of the first unreleased buffer.

Specified by:

close in interface AutoCloseable

Throws:

TrackingByteBufferPool.LeakedByteBufferException - if at least one buffer was not released