Package org.apache.hadoop.hdfs

Class StripedDataStreamer

java.lang.Object

java.lang.Thread

org.apache.hadoop.util.Daemon

org.apache.hadoop.hdfs.StripedDataStreamer

All Implemented Interfaces:

Runnable

@Private
public class StripedDataStreamer
extends org.apache.hadoop.util.Daemon

This class extends DataStreamer to support writing striped blocks to datanodes. A DFSStripedOutputStream has multiple StripedDataStreamers. Whenever the streamers need to talk the namenode, only the fastest streamer sends an rpc call to the namenode and then populates the result for the other streamers.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.util.Daemon

org.apache.hadoop.util.Daemon.DaemonFactory

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields

Modifier and Type	Field	Description
protected org.apache.hadoop.security.token.Token<BlockTokenIdentifier>	accessToken
protected final org.apache.hadoop.hdfs.DataStreamer.BlockToWrite	block
protected long	bytesSent
protected final LinkedList<DFSPacket>	dataQueue
protected final DFSClient	dfsClient
protected final org.apache.hadoop.thirdparty.com.google.common.cache.LoadingCache<DatanodeInfo,DatanodeInfo>	excludedNodes
protected final String	src
protected final HdfsFileStatus	stat

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Method Summary

Modifier and Type	Method	Description
protected void	endBlock()
protected void	setPipeline(DatanodeInfo[] newNodes, org.apache.hadoop.fs.StorageType[] newStorageTypes, String[] newStorageIDs)
protected void	setPipeline(LocatedBlock lb)
protected void	setupPipelineForCreate()	Open a DataStreamer to a DataNode so that it can be written to.
protected boolean	setupPipelineInternal(DatanodeInfo[] nodes, org.apache.hadoop.fs.StorageType[] nodeStorageTypes, String[] nodeStorageIDs)
String	toString()
void	updatePipeline(long newGS)	update pipeline at the namenode
void	work()

Methods inherited from class org.apache.hadoop.util.Daemon

getRunnable, run, start

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, stop, suspend, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

block

protected final org.apache.hadoop.hdfs.DataStreamer.BlockToWrite block

accessToken

protected org.apache.hadoop.security.token.Token<BlockTokenIdentifier> accessToken

bytesSent

protected long bytesSent

dfsClient

protected final DFSClient dfsClient

src

protected final String src

stat

protected final HdfsFileStatus stat

dataQueue

protected final LinkedList<DFSPacket> dataQueue

excludedNodes

protected final org.apache.hadoop.thirdparty.com.google.common.cache.LoadingCache<DatanodeInfo,DatanodeInfo> excludedNodes

Method Details

endBlock

protected void endBlock()

setupPipelineForCreate

protected void setupPipelineForCreate()
                               throws IOException

Open a DataStreamer to a DataNode so that it can be written to. This happens when a file is created and each time a new block is allocated. Must get block ID and the IDs of the destinations from the namenode. Returns the list of target datanodes.

Throws:

IOException

setupPipelineInternal

protected boolean setupPipelineInternal(DatanodeInfo[] nodes,
 org.apache.hadoop.fs.StorageType[] nodeStorageTypes,
 String[] nodeStorageIDs)
                                 throws IOException

Throws:

IOException

toString

public String toString()

setPipeline

protected void setPipeline(LocatedBlock lb)

setPipeline

protected void setPipeline(DatanodeInfo[] newNodes,
 org.apache.hadoop.fs.StorageType[] newStorageTypes,
 String[] newStorageIDs)

work

public void work()

Overrides:

work in class org.apache.hadoop.util.Daemon

updatePipeline

@VisibleForTesting
public void updatePipeline(long newGS)
                    throws IOException

update pipeline at the namenode

Throws:

IOException