Package org.apache.hadoop.hdfs.server.aliasmap

Class InMemoryAliasMap

java.lang.Object

org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMap

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, InMemoryAliasMapProtocol

@Public
@Unstable
public class InMemoryAliasMap
extends Object
implements InMemoryAliasMapProtocol, org.apache.hadoop.conf.Configurable

InMemoryAliasMap is an implementation of the InMemoryAliasMapProtocol for use with LevelDB.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	InMemoryAliasMap.CheckedFunction2<T1,T2,R>	CheckedFunction is akin to Function but specifies an IOException.

Nested classes/interfaces inherited from interface org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMapProtocol

InMemoryAliasMapProtocol.IterationResult

Method Summary

Modifier and Type	Method	Description
void	close()
static void	completeBootstrapTransfer(File aliasMap)	Extract the aliasmap archive to complete the bootstrap process.
static org.apache.hadoop.hdfs.protocol.Block	fromBlockBytes(byte[] blockDbFormat)
static org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation	fromProvidedStorageLocationBytes(byte[] providedStorageLocationDbFormat)
String	getBlockPoolId()	Get the associated block pool id.
org.apache.hadoop.conf.Configuration	getConf()
static InMemoryAliasMap	init(org.apache.hadoop.conf.Configuration conf, String blockPoolID)
InMemoryAliasMapProtocol.IterationResult	list(Optional<org.apache.hadoop.hdfs.protocol.Block> marker)	List the next batch of FileRegions in the alias map starting from the given marker.
Optional<org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation>	read(org.apache.hadoop.hdfs.protocol.Block block)	Gets the ProvidedStorageLocation associated with the specified block.
void	setConf(org.apache.hadoop.conf.Configuration conf)
static byte[]	toProtoBufBytes(org.apache.hadoop.hdfs.protocol.Block block)
static byte[]	toProtoBufBytes(org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation)
static void	transferForBootstrap(javax.servlet.http.HttpServletResponse response, org.apache.hadoop.conf.Configuration conf, InMemoryAliasMap aliasMap)	Transfer this aliasmap for bootstrapping standby Namenodes.
void	write(org.apache.hadoop.hdfs.protocol.Block block, org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation)	Stores the block and it's associated ProvidedStorageLocation in the alias map.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

init

@Nonnull
public static InMemoryAliasMap init(org.apache.hadoop.conf.Configuration conf,
 String blockPoolID)
                             throws IOException

Throws:

IOException

list

public InMemoryAliasMapProtocol.IterationResult list(Optional<org.apache.hadoop.hdfs.protocol.Block> marker)
                                              throws IOException

Description copied from interface: InMemoryAliasMapProtocol

List the next batch of FileRegions in the alias map starting from the given marker. To retrieve all FileRegions stored in the alias map, multiple calls to this function might be required.

Specified by:

list in interface InMemoryAliasMapProtocol

Parameters:

marker - the next block to get fileregions from.

Returns:

the InMemoryAliasMapProtocol.IterationResult with a set of FileRegions and the next marker.

Throws:

IOException

read

@Nonnull
public Optional<org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation> read(@Nonnull
 org.apache.hadoop.hdfs.protocol.Block block)
                                                                       throws IOException

Description copied from interface: InMemoryAliasMapProtocol

Gets the ProvidedStorageLocation associated with the specified block.

Specified by:

read in interface InMemoryAliasMapProtocol

Parameters:

block - the block to lookup

Returns:

the associated ProvidedStorageLocation.

Throws:

IOException

write

public void write(@Nonnull
 org.apache.hadoop.hdfs.protocol.Block block,
 @Nonnull
 org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation)
           throws IOException

Description copied from interface: InMemoryAliasMapProtocol

Stores the block and it's associated ProvidedStorageLocation in the alias map.

Specified by:

write in interface InMemoryAliasMapProtocol

Throws:

IOException

getBlockPoolId

public String getBlockPoolId()

Description copied from interface: InMemoryAliasMapProtocol

Get the associated block pool id.

Specified by:

getBlockPoolId in interface InMemoryAliasMapProtocol

Returns:

the block pool id associated with the Namenode running the in-memory alias map.

close

public void close()
           throws IOException

Throws:

IOException

fromProvidedStorageLocationBytes

@Nonnull
public static org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation fromProvidedStorageLocationBytes(@Nonnull
 byte[] providedStorageLocationDbFormat)
                                                                                                throws org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException

Throws:

org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException

fromBlockBytes

@Nonnull
public static org.apache.hadoop.hdfs.protocol.Block fromBlockBytes(@Nonnull
 byte[] blockDbFormat)
                                                            throws org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException

Throws:

org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException

toProtoBufBytes

public static byte[] toProtoBufBytes(@Nonnull
 org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation providedStorageLocation)
                              throws IOException

Throws:

IOException

toProtoBufBytes

public static byte[] toProtoBufBytes(@Nonnull
 org.apache.hadoop.hdfs.protocol.Block block)
                              throws IOException

Throws:

IOException

transferForBootstrap

public static void transferForBootstrap(javax.servlet.http.HttpServletResponse response,
 org.apache.hadoop.conf.Configuration conf,
 InMemoryAliasMap aliasMap)
                                 throws IOException

Transfer this aliasmap for bootstrapping standby Namenodes. The map is transferred as a tar.gz archive. This archive needs to be extracted on the standby Namenode.

Parameters:

response - http response.

conf - configuration to use.

aliasMap - aliasmap to transfer.

Throws:

IOException

completeBootstrapTransfer

public static void completeBootstrapTransfer(File aliasMap)
                                      throws IOException

Extract the aliasmap archive to complete the bootstrap process. This method has to be called after the aliasmap archive is transfered from the primary Namenode.

Parameters:

aliasMap - location of the aliasmap.

Throws:

IOException