Package org.apache.hadoop.hdfs.web

Class ByteRangeInputStream

java.lang.Object

java.io.InputStream

org.apache.hadoop.fs.FSInputStream

org.apache.hadoop.hdfs.web.ByteRangeInputStream

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.fs.PositionedReadable, org.apache.hadoop.fs.Seekable

public abstract class ByteRangeInputStream
extends org.apache.hadoop.fs.FSInputStream

To support HTTP byte streams, a new connection to an HTTP server needs to be created each time. This class hides the complexity of those multiple connections from the client. Whenever seek() is called, a new connection is made on the successive read(). The normal input stream functions are connected to the currently active input stream.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ByteRangeInputStream.URLOpener	This class wraps a URL and provides method to open connection.

Field Summary

Fields

Modifier and Type	Field	Description
protected long	currentPos
protected Long	fileLength
protected InputStream	in
protected final ByteRangeInputStream.URLOpener	originalURL
protected final ByteRangeInputStream.URLOpener	resolvedURL
protected long	startPos

Constructor Summary

Constructors

Constructor	Description
ByteRangeInputStream(ByteRangeInputStream.URLOpener o, ByteRangeInputStream.URLOpener r)	Create with the specified URLOpeners.

Method Summary

Modifier and Type	Method	Description
int	available()
void	close()
protected InputStream	getInputStream()
long	getPos()	Return the current offset from the start of the file
protected abstract URL	getResolvedUrl(HttpURLConnection connection)
protected org.apache.hadoop.hdfs.web.ByteRangeInputStream.InputStreamAndFileLength	openInputStream(long startOffset)
int	read()
int	read(byte[] b, int off, int len)
int	read(long position, byte[] buffer, int offset, int length)
void	readFully(long position, byte[] buffer, int offset, int length)
void	seek(long pos)	Seek to the given offset from the start of the file.
boolean	seekToNewSource(long targetPos)	Seeks a different copy of the data.

Methods inherited from class org.apache.hadoop.fs.FSInputStream

readFully, toString, validatePositionedReadArgs

Methods inherited from class java.io.InputStream

mark, markSupported, nullInputStream, read, readAllBytes, readNBytes, readNBytes, reset, skip, skipNBytes, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.fs.PositionedReadable

maxReadSizeForVectorReads, minSeekForVectorReads, readVectored, readVectored

Field Details

in

protected InputStream in

originalURL

protected final ByteRangeInputStream.URLOpener originalURL

resolvedURL

protected final ByteRangeInputStream.URLOpener resolvedURL

startPos

protected long startPos

currentPos

protected long currentPos

fileLength

protected Long fileLength

Constructor Details

ByteRangeInputStream

public ByteRangeInputStream(ByteRangeInputStream.URLOpener o,
 ByteRangeInputStream.URLOpener r)
                     throws IOException

Create with the specified URLOpeners. Original url is used to open the stream for the first time. Resolved url is used in subsequent requests.

Parameters:

o - Original url

r - Resolved url

Throws:

IOException

Method Details

getResolvedUrl

protected abstract URL getResolvedUrl(HttpURLConnection connection)
                               throws IOException

Throws:

IOException

getInputStream

@VisibleForTesting
protected InputStream getInputStream()
                              throws IOException

Throws:

IOException

openInputStream

@VisibleForTesting
protected org.apache.hadoop.hdfs.web.ByteRangeInputStream.InputStreamAndFileLength openInputStream(long startOffset)
                                                                                            throws IOException

Throws:

IOException

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

read

public int read(@Nonnull
 byte[] b,
 int off,
 int len)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

seek

public void seek(long pos)
          throws IOException

Seek to the given offset from the start of the file. The next read() will be from that location. Can't seek past the end of the file.

Specified by:

seek in interface org.apache.hadoop.fs.Seekable

Specified by:

seek in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException

read

public int read(long position,
 byte[] buffer,
 int offset,
 int length)
         throws IOException

Specified by:

read in interface org.apache.hadoop.fs.PositionedReadable

Overrides:

read in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException

readFully

public void readFully(long position,
 byte[] buffer,
 int offset,
 int length)
               throws IOException

Specified by:

readFully in interface org.apache.hadoop.fs.PositionedReadable

Overrides:

readFully in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException

getPos

public long getPos()
            throws IOException

Return the current offset from the start of the file

Specified by:

getPos in interface org.apache.hadoop.fs.Seekable

Specified by:

getPos in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException

seekToNewSource

public boolean seekToNewSource(long targetPos)
                        throws IOException

Seeks a different copy of the data. Returns true if found a new source, false otherwise.

Specified by:

seekToNewSource in interface org.apache.hadoop.fs.Seekable

Specified by:

seekToNewSource in class org.apache.hadoop.fs.FSInputStream

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class InputStream

Throws:

IOException

available

public int available()
              throws IOException

Overrides:

available in class InputStream

Throws:

IOException