Package org.apache.hadoop.crypto

Class CryptoInputStream

java.lang.Object

java.io.InputStream

java.io.FilterInputStream

org.apache.hadoop.crypto.CryptoInputStream

All Implemented Interfaces:

Closeable, AutoCloseable, Channel, ReadableByteChannel, ByteBufferPositionedReadable, ByteBufferReadable, CanSetDropBehind, CanSetReadahead, CanUnbuffer, HasEnhancedByteBufferAccess, HasFileDescriptor, PositionedReadable, Seekable, IOStatisticsSource, StreamCapabilities

@Private
@Evolving
public class CryptoInputStream
extends FilterInputStream
implements Seekable, PositionedReadable, ByteBufferReadable, HasFileDescriptor, CanSetDropBehind, CanSetReadahead, HasEnhancedByteBufferAccess, ReadableByteChannel, CanUnbuffer, StreamCapabilities, ByteBufferPositionedReadable, IOStatisticsSource

CryptoInputStream decrypts data. It is not thread-safe. AES CTR mode is required in order to ensure that the plain text and cipher text have a 1:1 mapping. The decryption is buffer based. The key points of the decryption are (1) calculating the counter and (2) padding through stream position:

counter = base + pos/(algorithm blocksize); padding = pos%(algorithm blocksize);

The underlying stream offset is maintained as state.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.fs.StreamCapabilities

StreamCapabilities.StreamCapability

Field Summary

Fields inherited from class java.io.FilterInputStream

in

Fields inherited from interface org.apache.hadoop.fs.StreamCapabilities

ABORTABLE_STREAM, DROPBEHIND, HFLUSH, HSYNC, IOSTATISTICS, IOSTATISTICS_CONTEXT, PREADBYTEBUFFER, READAHEAD, READBYTEBUFFER, UNBUFFER, VECTOREDIO, VECTOREDIO_BUFFERS_SLICED

Constructor Summary

Constructors

Constructor	Description
CryptoInputStream(InputStream in, CryptoCodec codec, byte[] key, byte[] iv)
CryptoInputStream(InputStream in, CryptoCodec codec, int bufferSize, byte[] key, byte[] iv)
CryptoInputStream(InputStream in, CryptoCodec codec, int bufferSize, byte[] key, byte[] iv, long streamOffset)

Method Summary

Modifier and Type	Method	Description
int	available()
void	close()
FileDescriptor	getFileDescriptor()
IOStatistics	getIOStatistics()	Return a statistics instance.
long	getPos()	Get underlying stream position.
InputStream	getWrappedStream()
boolean	hasCapability(String capability)	Query the stream for a specific capability.
boolean	isOpen()
void	mark(int readLimit)
boolean	markSupported()
int	read()
int	read(byte[] b, int off, int len)	Decryption is buffer based.
int	read(long position, byte[] buffer, int offset, int length)	Positioned read.
int	read(long position, ByteBuffer buf)	Positioned read using ByteBuffers.
int	read(ByteBuffer buf)	ByteBuffer read.
ByteBuffer	read(ByteBufferPool bufferPool, int maxLength, EnumSet<ReadOption> opts)	Get a ByteBuffer containing file data.
void	readFully(long position, byte[] buffer)	Read number of bytes equal to the length of the buffer, from a given position within a file.
void	readFully(long position, byte[] buffer, int offset, int length)	Positioned read fully.
void	readFully(long position, ByteBuffer buf)	Positioned readFully using ByteBuffers.
void	releaseBuffer(ByteBuffer buffer)	Release a ByteBuffer which was created by the enhanced ByteBuffer read function.
void	reset()
void	seek(long pos)	Seek to a position.
boolean	seekToNewSource(long targetPos)	Seeks a different copy of the data.
void	setDropBehind(Boolean dropCache)	Configure whether the stream should drop the cache.
void	setReadahead(Long readahead)	Set the readahead on this stream.
long	skip(long n)	Skip n bytes
void	unbuffer()	Reduce the buffering.

Methods inherited from class java.io.FilterInputStream

read

Methods inherited from class java.io.InputStream

nullInputStream, readAllBytes, readNBytes, readNBytes, skipNBytes, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.fs.PositionedReadable

maxReadSizeForVectorReads, minSeekForVectorReads, readVectored, readVectored

Constructor Details

CryptoInputStream

public CryptoInputStream(InputStream in,
 CryptoCodec codec,
 int bufferSize,
 byte[] key,
 byte[] iv)
                  throws IOException

Throws:

IOException

CryptoInputStream

public CryptoInputStream(InputStream in,
 CryptoCodec codec,
 int bufferSize,
 byte[] key,
 byte[] iv,
 long streamOffset)
                  throws IOException

Throws:

IOException

CryptoInputStream

public CryptoInputStream(InputStream in,
 CryptoCodec codec,
 byte[] key,
 byte[] iv)
                  throws IOException

Throws:

IOException

Method Details

getWrappedStream

public InputStream getWrappedStream()

read

public int read(byte[] b,
 int off,
 int len)
         throws IOException

Decryption is buffer based. If there is data in outBuffer, then read it out of this buffer. If there is no data in outBuffer, then read more from the underlying stream and do the decryption.

Overrides:

read in class FilterInputStream

Parameters:

b - the buffer into which the decrypted data is read.

off - the buffer offset.

len - the maximum number of decrypted data bytes to read.

Returns:

int the total number of decrypted data bytes read into the buffer.

Throws:

IOException - raised on errors performing I/O.

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Channel

Specified by:

close in interface Closeable

Overrides:

close in class FilterInputStream

Throws:

IOException

read

public int read(long position,
 byte[] buffer,
 int offset,
 int length)
         throws IOException

Positioned read. It is thread-safe

Specified by:

read in interface PositionedReadable

Parameters:

position - position within file

buffer - destination buffer

offset - offset in the buffer

length - number of bytes to read

Returns:

actual number of bytes read; -1 means "none"

Throws:

IOException - IO problems.

read

public int read(long position,
 ByteBuffer buf)
         throws IOException

Positioned read using ByteBuffers. This method is thread-safe.

Specified by:

read in interface ByteBufferPositionedReadable

Parameters:

position - position within file

buf - the ByteBuffer to receive the results of the read operation.

Returns:

the number of bytes read, possibly zero, or -1 if reached end-of-stream

Throws:

IOException - if there is some error performing the read

readFully

public void readFully(long position,
 ByteBuffer buf)
               throws IOException

Positioned readFully using ByteBuffers. This method is thread-safe.

Specified by:

readFully in interface ByteBufferPositionedReadable

Parameters:

position - position within file

buf - the ByteBuffer to receive the results of the read operation.

Throws:

IOException - if there is some error performing the read

EOFException - the end of the data was reached before the read operation completed

See Also:

• ByteBufferPositionedReadable.read(long, ByteBuffer)

readFully

public void readFully(long position,
 byte[] buffer,
 int offset,
 int length)
               throws IOException

Positioned read fully. It is thread-safe

Specified by:

readFully in interface PositionedReadable

Parameters:

position - position within file

buffer - destination buffer

offset - offset in the buffer

length - number of bytes to read

Throws:

IOException - IO problems.

EOFException - the end of the data was reached before the read operation completed

readFully

public void readFully(long position,
 byte[] buffer)
               throws IOException

Description copied from interface: PositionedReadable

Read number of bytes equal to the length of the buffer, from a given position within a file. This does not change the current offset of a file, and is thread-safe. Warning: Not all filesystems satisfy the thread-safety requirement.

Specified by:

readFully in interface PositionedReadable

Parameters:

position - position within file

buffer - destination buffer

Throws:

IOException - IO problems.

EOFException - the end of the data was reached before the read operation completed

seek

public void seek(long pos)
          throws IOException

Seek to a position.

Specified by:

seek in interface Seekable

Parameters:

pos - offset from the start of the file.

Throws:

IOException - raised on errors performing I/O.

skip

public long skip(long n)
          throws IOException

Skip n bytes

Overrides:

skip in class FilterInputStream

Throws:

IOException

getPos

public long getPos()
            throws IOException

Get underlying stream position.

Specified by:

getPos in interface Seekable

Returns:

offset from the start of the file.

Throws:

IOException - raised on errors performing I/O.

read

public int read(ByteBuffer buf)
         throws IOException

ByteBuffer read.

Specified by:

read in interface ByteBufferReadable

Specified by:

read in interface ReadableByteChannel

Parameters:

buf - the ByteBuffer to receive the results of the read operation.

Returns:

the number of bytes read, possibly zero, or -1 if reach end-of-stream

Throws:

IOException - if there is some error performing the read

available

public int available()
              throws IOException

Overrides:

available in class FilterInputStream

Throws:

IOException

markSupported

public boolean markSupported()

Overrides:

markSupported in class FilterInputStream

mark

public void mark(int readLimit)

Overrides:

mark in class FilterInputStream

reset

public void reset()
           throws IOException

Overrides:

reset in class FilterInputStream

Throws:

IOException

seekToNewSource

public boolean seekToNewSource(long targetPos)
                        throws IOException

Description copied from interface: Seekable

Seeks a different copy of the data. Returns true if found a new source, false otherwise.

Specified by:

seekToNewSource in interface Seekable

Parameters:

targetPos - target position.

Returns:

true if found a new source, false otherwise.

Throws:

IOException - raised on errors performing I/O.

read

public ByteBuffer read(ByteBufferPool bufferPool,
 int maxLength,
 EnumSet<ReadOption> opts)
                throws IOException,
UnsupportedOperationException

Description copied from interface: HasEnhancedByteBufferAccess

Get a ByteBuffer containing file data. This ByteBuffer may come from the stream itself, via a call like mmap, or it may come from the ByteBufferFactory which is passed in as an argument.

Specified by:

read in interface HasEnhancedByteBufferAccess

Parameters:

bufferPool - If this is non-null, it will be used to create a fallback ByteBuffer when the stream itself cannot create one.

maxLength - The maximum length of buffer to return. We may return a buffer which is shorter than this.

opts - Options to use when reading.

Returns:

We will always return an empty buffer if maxLength was 0, whether or not we are at EOF. If maxLength > 0, we will return null if the stream has reached EOF. Otherwise, we will return a ByteBuffer containing at least one byte. You must free this ByteBuffer when you are done with it by calling releaseBuffer on it. The buffer will continue to be readable until it is released in this manner. However, the input stream's close method may warn about unclosed buffers.

Throws:

IOException - if there was an error reading.

UnsupportedOperationException - if factory was null, and we needed an external byte buffer.

releaseBuffer

public void releaseBuffer(ByteBuffer buffer)

Description copied from interface: HasEnhancedByteBufferAccess

Release a ByteBuffer which was created by the enhanced ByteBuffer read function. You must not continue using the ByteBuffer after calling this function.

Specified by:

releaseBuffer in interface HasEnhancedByteBufferAccess

Parameters:

buffer - The ByteBuffer to release.

setReadahead

public void setReadahead(Long readahead)
                  throws IOException,
UnsupportedOperationException

Description copied from interface: CanSetReadahead

Set the readahead on this stream.

Specified by:

setReadahead in interface CanSetReadahead

Parameters:

readahead - The readahead to use. null means to use the default.

Throws:

IOException - If there was an error changing the dropBehind setting. UnsupportedOperationException If this stream doesn't support setting readahead.

UnsupportedOperationException

setDropBehind

public void setDropBehind(Boolean dropCache)
                   throws IOException,
UnsupportedOperationException

Description copied from interface: CanSetDropBehind

Configure whether the stream should drop the cache.

Specified by:

setDropBehind in interface CanSetDropBehind

Parameters:

dropCache - Whether to drop the cache. null means to use the default value.

Throws:

IOException - If there was an error changing the dropBehind setting. UnsupportedOperationException If this stream doesn't support setting the drop-behind.

UnsupportedOperationException

getFileDescriptor

public FileDescriptor getFileDescriptor()
                                 throws IOException

Specified by:

getFileDescriptor in interface HasFileDescriptor

Returns:

the FileDescriptor

Throws:

IOException - raised on errors performing I/O.

read

public int read()
         throws IOException

Overrides:

read in class FilterInputStream

Throws:

IOException

isOpen

public boolean isOpen()

Specified by:

isOpen in interface Channel

unbuffer

public void unbuffer()

Description copied from interface: CanUnbuffer

Reduce the buffering. This will also free sockets and file descriptors held by the stream, if possible.

Specified by:

unbuffer in interface CanUnbuffer

hasCapability

public boolean hasCapability(String capability)

Description copied from interface: StreamCapabilities

Query the stream for a specific capability.

Specified by:

hasCapability in interface StreamCapabilities

Parameters:

capability - string to query the stream support for.

Returns:

True if the stream supports capability.

getIOStatistics

public IOStatistics getIOStatistics()

Description copied from interface: IOStatisticsSource

Return a statistics instance.

It is not a requirement that the same instance is returned every time. IOStatisticsSource.

If the object implementing this is Closeable, this method may return null if invoked on a closed object, even if it returns a valid instance when called earlier.

Specified by:

getIOStatistics in interface IOStatisticsSource

Returns:

an IOStatistics instance or null