Package org.apache.hadoop.hdfs.server.namenode.ha

Class ObserverReadProxyProviderWithIPFailover<T extends ClientProtocol>

java.lang.Object

org.apache.hadoop.hdfs.server.namenode.ha.AbstractNNFailoverProxyProvider<T>

org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider<T>

org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProviderWithIPFailover<T>

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.io.retry.FailoverProxyProvider<T>

@Private
@Evolving
public class ObserverReadProxyProviderWithIPFailover<T extends ClientProtocol>
extends ObserverReadProxyProvider<T>

Extends ObserverReadProxyProvider to support NameNode IP failover. For Observer reads a client needs to know physical addresses of all NameNodes, so that it could switch between active and observer nodes for write and read requests. Traditional IPFailoverProxyProvider works with a virtual address of the NameNode. If active NameNode fails the virtual address is assigned to the standby NameNode, and IPFailoverProxyProvider, which keeps talking to the same virtual address is in fact now connects to the new physical server. To combine these behaviors ObserverReadProxyProviderWithIPFailover should both

1. Maintain all physical addresses of NameNodes in order to allow observer reads, and
2. Should rely on the virtual address of the NameNode in order to perform failover by assuming that the virtual address always points to the active NameNode.

An example of a configuration to leverage ObserverReadProxyProviderWithIPFailover should include the following values:

 fs.defaultFS = hdfs://mycluster
 dfs.nameservices = mycluster
 dfs.ha.namenodes.mycluster = ha1,ha2
 dfs.namenode.rpc-address.mycluster.ha1 = nn01-ha1.com:8020
 dfs.namenode.rpc-address.mycluster.ha2 = nn01-ha2.com:8020
 dfs.client.failover.ipfailover.virtual-address.mycluster =
     hdfs://nn01.com:8020
 dfs.client.failover.proxy.provider.mycluster =
     org.apache...ObserverReadProxyProviderWithIPFailover
 

Here nn01.com:8020 is the virtual address of the active NameNode, while nn01-ha1.com:8020 and nn01-ha2.com:8020 are the physically addresses the two NameNodes. With this configuration, client will use ObserverReadProxyProviderWithIPFailover, which creates proxies for both nn01-ha1 and nn01-ha2, used for read/write RPC calls, but for the failover, it relies on the virtual address nn01.com

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.namenode.ha.AbstractNNFailoverProxyProvider

AbstractNNFailoverProxyProvider.NNProxyInfo<T>

Nested classes/interfaces inherited from interface org.apache.hadoop.io.retry.FailoverProxyProvider

org.apache.hadoop.io.retry.FailoverProxyProvider.ProxyInfo<T extends Object>

Field Summary

Fields inherited from class org.apache.hadoop.hdfs.server.namenode.ha.AbstractNNFailoverProxyProvider

conf, factory, fallbackToSimpleAuth, ugi, xface

Constructor Summary

Constructors

Constructor	Description
ObserverReadProxyProviderWithIPFailover(org.apache.hadoop.conf.Configuration conf, URI uri, Class<T> xface, HAProxyFactory<T> factory)	By default ObserverReadProxyProviderWithIPFailover uses IPFailoverProxyProvider for failover.
ObserverReadProxyProviderWithIPFailover(org.apache.hadoop.conf.Configuration conf, URI uri, Class<T> xface, HAProxyFactory<T> factory, AbstractNNFailoverProxyProvider<T> failoverProxy)

Method Summary

Modifier and Type	Method	Description
boolean	useLogicalURI()	Inquire whether logical HA URI is used for the implementation.

Methods inherited from class org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider

close, getAlignmentContext, getProxy, performFailover

Methods inherited from class org.apache.hadoop.hdfs.server.namenode.ha.AbstractNNFailoverProxyProvider

createProxyIfNeeded, getFallbackToSimpleAuth, getInterface, getProxyAddresses, getRandomOrder, setFallbackToSimpleAuth

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ObserverReadProxyProviderWithIPFailover

public ObserverReadProxyProviderWithIPFailover(org.apache.hadoop.conf.Configuration conf,
 URI uri,
 Class<T> xface,
 HAProxyFactory<T> factory)

By default ObserverReadProxyProviderWithIPFailover uses IPFailoverProxyProvider for failover.

ObserverReadProxyProviderWithIPFailover

public ObserverReadProxyProviderWithIPFailover(org.apache.hadoop.conf.Configuration conf,
 URI uri,
 Class<T> xface,
 HAProxyFactory<T> factory,
 AbstractNNFailoverProxyProvider<T> failoverProxy)

Method Details

useLogicalURI

public boolean useLogicalURI()

Description copied from class: AbstractNNFailoverProxyProvider

Inquire whether logical HA URI is used for the implementation. If it is used, a special token handling may be needed to make sure a token acquired from a node in the HA pair can be used against the other node.

Overrides:

useLogicalURI in class ObserverReadProxyProvider<T extends ClientProtocol>

Returns:

true if logical HA URI is used. false, if not used.