All Classes and Interfaces
Class
Description
Base fairness policy that implements @RouterRpcFairnessPolicyController.
Locates the most active NN for a given nameservice ID or blockpool ID.
Add mount entry attributes to be used by Router admin.
API request for adding all mount table entries to the state store.
Protobuf implementation of the state store API object addMountTableEntriesRequest.
API response for adding multiple mount table entries to the state store.
Protobuf implementation of the state store API object addMountTableEntriesResponse.
API request for adding a mount table entry to the state store.
Protobuf implementation of the state store API object
AddMountTableEntryRequest.
API response for adding a mount table entry to the state store.
Protobuf implementation of the state store API object
AddMountTableEntryResponse.
Represents a function that accepts a value of type T and produces a result of type R.
An interface for asynchronous operations, providing utility methods
and constants related to asynchronous computations.
The AsyncApplyFunction interface represents a function that
asynchronously accepts a value of type T and produces a result
of type R.
The
AsyncBiFunction interface represents a bi-function that
asynchronously accepts two arguments and produces a result.The AsyncCatchFunction interface represents a function that handles exceptions
occurring within an asynchronous operation.
Provides asynchronous operations for erasure coding in HDFS Federation.
The AsyncForEachRun class is part of the asynchronous operation utilities
within the Hadoop Distributed File System (HDFS) Federation router.
Provides asynchronous operations for managing quotas in HDFS Federation.
This utility class encapsulates the logic required to initiate asynchronous RPCs,
handle responses, and propagate exceptions.
The AsyncRun interface represents an asynchronous operation that can be
executed in the context of the Hadoop Distributed File System (HDFS)
Federation router.
The AsyncUtil class provides a collection of utility methods to simplify
the implementation of asynchronous operations using Java's CompletableFuture.
Order the destinations based on available space.
Abstract base of a data record in the StateStore.
Record store that takes care of caching the records in memory.
The
CatchFunction interface represents a function that handles exceptions
occurring within an asynchronous operation.Context to track a connection in a
ConnectionPool.Implements a pool of connections for the
Router to be able to open
many connections to many Namenodes.Exception when can not get a non-null connection.
Maintains a pool of connections for each User (including tokens) + NN.
Identifier for a connection for a user to a namenode.
Consistent hash ring to distribute items across nodes (locations).
Order of the destinations when we have multiple of them.
Tool to start the
Router for Router-based federation.Data record indicating a specific name service ID has been disabled and
is no longer valid.
Protobuf implementation of the
DisabledNameservice record.State store record to track disabled name services.
Implementation of
DisabledNameserviceStore.API request for disabling a name service and updating its state in the
State Store.
Protobuf implementation of the state store API object
DisableNameserviceRequest.
API response for disabling a name service and updating its state in the
State Store.
Protobuf implementation of the state store API object
DisableNameserviceResponse.Distributed counter that relies on a SQL database to synchronize
between multiple clients.
API request for enabling a name service and updating its state in the
State Store.
Protobuf implementation of the state store API object
EnableNameserviceRequest.
API response for enabling a name service and updating its state in the
State Store.
Protobuf implementation of the state store API object
EnableNameserviceResponse.
API request for the Router entering safe mode state and updating
its state in State Store.
Protobuf implementation of the state store API object
EnterSafeModeRequest.
API response for the Router entering safe mode state and updating
its state in State Store.
Protobuf implementation of the state store API object
EnterSafeModeResponse.
Module that implements all the RPC calls in
ClientProtocol related to
Erasure Coding in the RouterRpcServer.JMX interface for the federation statistics.
Interface for a discovered NN and its current server endpoints.
Namenode state in the federation.
Represents information about a single nameservice/namespace in a federated
HDFS cluster.
FederationProtocolPBTranslator<P extends org.apache.hadoop.thirdparty.protobuf.GeneratedMessageV3,B extends org.apache.hadoop.thirdparty.protobuf.Message.Builder,T extends org.apache.hadoop.thirdparty.protobuf.MessageOrBuilder>
Helper class for setting/getting data elements in an object backed by a
protobuf implementation.
JMX interface for the RPC server.
Implementation of the RPC metrics collector.
Customizable RPC performance monitor.
Utilities for managing HDFS federation.
Interface to map a file path in the global name space to a specific
subcluster and path in an HDFS name space.
The
FinallyFunction interface represents a function that is used to perform
final actions after an asynchronous operation completes, regardless of whether the
operation was successful or resulted in an exception.API request for getting the destination subcluster of a file.
Protobuf implementation of the state store API object
GetDestinationRequest.
API response for getting the destination subcluster of a file.
Protobuf implementation of the state store API object
GetDestinationResponse.
API request for getting the disabled name services.
Protobuf implementation of the state store API object
GetDisabledNameservicesRequest.
API response for getting the disabled nameservices in the state store.
Protobuf implementation of the state store API object
GetDisabledNameservicesResponse.
API request for listing mount table entries present in the state store.
Protobuf implementation of the state store API object
GetMountTableEntriesRequest.
API response for listing mount table entries present in the state store.
Protobuf implementation of the state store API object
GetMountTableEntriesResponse.
API request for listing namenode registrations present in the state store.
Protobuf implementation of the state store API object
GetNamenodeRegistrationsRequest.
API response for listing namenode registrations present in the state store.
Protobuf implementation of the state store API object
GetNamenodeRegistrationsResponse.
API response for listing HDFS namespaces present in the state store.
Protobuf implementation of the state store API object
GetNamespaceInfoRequest.
API response for listing HDFS namespaces present in the state store.
Protobuf implementation of the state store API object
GetNamespaceInfoResponse.
API request for retrieving a single router registration present in the state
store.
Protobuf implementation of the state store API object
GetRouterRegistrationRequest.
API response for retrieving a single router registration present in the state
store.
Protobuf implementation of the state store API object
GetRouterRegistrationResponse.
API request for retrieving an all non-expired router registrations present in
the state store.
Protobuf implementation of the state store API object
GetRouterRegistrationsRequest.
API response for retrieving a all non-expired router registrations present in
the state store.
Protobuf implementation of the state store API object
GetRouterRegistrationsResponse.
API request for verifying if current Router state is safe mode.
Protobuf implementation of the state store API object
GetSafeModeRequest.
API response for verifying if current Router state is safe mode.
Protobuf implementation of the state store API object
GetSafeModeResponse.
Variation of HashResolver that only uses the first level of the path.
Order the destinations based on consistent hashing.
Detect if the Router is active and ready to serve requests.
LEADER_FOLLOWER can be used in cross-cluster disaster tolerance,
and the order of namespaces is always "leader,follower,follower...".
API request for the Router leaving safe mode state and updating
its state in State Store.
Protobuf implementation of the state store API object
LeaveSafeModeRequest.
API response for the Router leaving safe mode state and updating
its state in State Store.
Protobuf implementation of the state store API object
LeaveSafeModeResponse.
The local subcluster (where the writer is) should be tried first.
Implements a cached lookup of the most recently active namenode for a
particular nameservice.
Data schema for storing NN registration information in the
StateStoreService.Protobuf implementation of the MembershipState record.
Data schema for storing NN stats in the
StateStoreService.Protobuf implementation of the MembershipStats record.
Management API for NameNode registrations stored in
MembershipState records.Implementation of the
MembershipStore State Store API.Data schema for
MountTableStore data stored in the StateStoreService.Manage a mount table.
Protobuf implementation of the MountTable record.
Update mount table.
This service is invoked from
MountTableStore when there is change in
mount table entries and it updates mount table entry cache on local router as
well as on all remote routers.Base class for updating mount table cache on all the router.
Mount table to map between global paths and remote locations.
Management API for the HDFS mount table information stored in
MountTable records.Implementation of the
MountTableStore state store API.Mount table resolver that supports multiple locations for each mount entry.
Expose the Namenode metrics as the Router was one.
API request for registering a namenode with the state store.
Protobuf implementation of the state store API object
NamenodeHeartbeatRequest.
API response for registering a namenode with the state store.
Protobuf implementation of the state store API object
NamenodeHeartbeatResponse.
The
Router periodically checks the state of a Namenode (usually on
the same server) and reports their high availability (HA) state and
load/space status to the
StateStoreService
.Compares NNs in the same namespace and prioritizes by their status.
Status of the namenode.
Interface for enable/disable name service.
JMX interface for the RPC server of Nameservice.
Implementation of the Nameservice RPC metrics collector.
This exception is thrown when can not get any mount point for the input path.
Exception when no namenodes are available.
A pass through fairness policy that implements
RouterRpcFairnessPolicyController and allows any number
of handlers to connect to any specific downstream name service.Implementation of the State Store metrics which does not do anything.
Policy that decides which should be the first location accessed given
multiple destinations.
A map of the properties and target destinations (name space + path) for
a path in the global/federated name space.
A record implementation using Protobuf.
Service to periodically execute a runnable.
An alignment context shared by all connections in a
ConnectionPool.Proportion fairness policy extending
AbstractRouterRpcFairnessPolicyController
and fetching proportion of handlers from configuration for all available name services,
based on the proportion and the total number of handlers, calculate the handlers of all ns.Check if a record matches a query.
Encapsulates a state store query result that includes a set of records and a
time stamp for the result.
Module that implements the quota relevant RPC calls
ClientProtocol.setQuota(String, long, long, StorageType)
and
ClientProtocol.getQuotaUsage(String)
in the RouterRpcServer.Order the destinations randomly.
Config fields for router-based hdfs federation.
Implementation of the Router metrics collector.
Store records in the State Store.
API request for refreshing mount table cached entries from state store.
Protobuf implementation of the state store API object
RefreshMountTableEntriesRequest.
API response for refreshing mount table entries cache from state store.
Protobuf implementation of the state store API object
RefreshMountTableEntriesResponse.
API request for refreshing super user groups on router.
Protobuf implementation of the state store API object
RefreshSuperUserGroupsConfigurationRequest.
API response for refreshing super user groups on router.
Protobuf implementation of the state store API object
RefreshSuperUserGroupsConfigurationResponse.
A location in a remote namespace consisting of a nameservice ID and a HDFS
path (destination).
Base class for objects that are unique to a namespace.
Determines the remote client protocol method and the parameter list for a
specific location.
A dynamically assignable parameter that is location-specific.
Result from a remote location.
API request for removing a mount table path present in the state store.
Protobuf implementation of the state store API object
RemoveMountTableEntryRequest.
API response for removing a mount table path present in the state store.
Protobuf implementation of the state store API object
RemoveMountTableEntryResponse.
Router that provides a unified view of multiple federated HDFS clusters.
This class provides some Federation administrative access shell commands.
Protocol used by routeradmin to communicate with statestore.
Protocol that a clients use to communicate with the NameNode.
This class is used on the server side.
This class forwards RouterAdminProtocol calls as RPC calls to the RouterAdmin server
while translating from the parameter types used in RouterAdminProtocol to the
new PB types.
This class is responsible for handling all the Admin calls to the HDFS
router.
Module that implements all the asynchronous RPC calls in
ClientProtocol related to Cache Admin
in the RouterRpcServer.Module that implements all the async RPC calls in
ClientProtocol in the
RouterRpcServer.Module that implements all the asynchronous RPC calls in
NamenodeProtocol in the
RouterRpcServer.The
RouterAsyncRpcClient class extends the functionality of the base
RouterRpcClient class to provide asynchronous remote procedure call (RPC)
capabilities for communication with the Hadoop Distributed File System (HDFS)
NameNodes in a federated environment.When router async rpc enabled, it is recommended to use this fairness controller.
Module that implements all the asynchronous RPC calls related to snapshots in
ClientProtocol in the RouterRpcServer.Module that implements all the asynchronous RPC calls in
ClientProtocol related to
Storage Policy in the RouterRpcServer.Module that implements all the asynchronous RPC calls in
RefreshUserMappingsProtocol GetUserMappingsProtocol in the
RouterRpcServer.Module that implements all the RPC calls in
ClientProtocol related to Cache Admin
in the RouterRpcServer.Client to connect to the
Router via the admin protocol.This class is for maintaining the various Router Client activity statistics
and publishing them through the metrics interfaces.
Module that implements all the RPC calls in
ClientProtocol in the
RouterRpcServer.For
GetLisiting to sort results.This class forwards NN's ClientProtocol calls as RPC calls to the NN server
while translating from the parameter types used in ClientProtocol to the
new PB types.
Copy data through distcp in router-based federation cluster.
Balance data in router-based federation cluster.
Rename across router federation namespaces based on federation balance.
Wrapper for the Router to offer the Namenode FSCK.
This class is used in Namesystem's web server to do fsck on namenode.
Generic methods for managing Router.
API request for registering a router with the state store.
Protobuf implementation of the state store API object
RouterHeartbeatRequest.
API response for registering a router with the state store.
Protobuf implementation of the state store API object
RouterHeartbeatResponse.
Service to periodically update the Router current state in the State Store.
Web interface for the
Router.JMX interface for the router specific metrics.
This class is for maintaining the various Router activity statistics
and publishing them through the metrics interfaces.
Service to manage the metrics of the Router.
Module that implements all the RPC calls in
NamenodeProtocol in the
RouterRpcServer.A servlet to print out the network topology from router.
Class that helps in checking permissions in Router-based federation.
HDFSPolicyProvider for RBF protocols.Router quota manager in Router.
Service to periodically update the
RouterQuotaUsage
cached information in the Router.The subclass of
QuotaUsage used in Router-based federation.Build the instance based on the builder.
Thrown by FileSubclusterResolver when a path can't be resolved.
The order resolver that depends upon the Router service.
A client proxy for Router to NN communication using the NN ClientProtocol.
The
RouterRpcClient.ExecutionStatus class is a utility class used to track the status of
execution operations performed by the RouterRpcClient.Interface to define handlers assignment for specific name services.
Metrics and monitoring interface for the router RPC server.
This class is responsible for handling all of the RPC calls to the It is
created, started, and stopped by
Router.Service to periodically check if the
StateStoreService cached information in the Router is up to date.Manager to hold underlying delegation token secret manager implementations.
States of the Router.
Module that implements all the RPC calls related to snapshots in
ClientProtocol in the RouterRpcServer.Entry to log the state of a
Router in the
StateStoreService.This is the router implementation to hold the state Ids for all
namespaces.
Interface of managing the Router state.
Protobuf implementation of the RouterState record.
Module that implements all the RPC calls in
ClientProtocol related to
Storage Policy in the RouterRpcServer.Management API for
RouterState records in the state store.Implementation of the
RouterStore state store API.Module that implements all the RPC calls in
RefreshUserMappingsProtocol GetUserMappingsProtocol in the
RouterRpcServer.WebHDFS Router implementation.
Interface to provide SQL connections to the
SQLDelegationTokenSecretManagerImpl.An implementation of
SQLDelegationTokenSecretManager that
persists TokenIdentifiers and DelegationKeys in a SQL database.Interface to handle retries when
SQLDelegationTokenSecretManagerImpl
throws expected errors.Base implementation of a State Store driver.
Interface for a cached copy of the State Store.
Service to periodically update the
StateStoreService
cached information in the
Router.Service to periodically monitor the connection of the StateStore
StateStoreService data store and to re-open the connection
to the data store if required.Driver class for an implementation of a
StateStoreService
provider.StateStoreDriver implementation based on files.StateStoreDriver implementation based on a local file.
StateStoreDriver implementation based on a filesystem.JMX interface for the State Store metrics.
Implementations of the JMX interface for the State Store metrics.
StateStoreDriver implementation based on MySQL.
State store operation result with list of failed records.
Operations for a driver to manage records in the State Store.
State Store driver that stores a serialization of the records.
Serializer to store and retrieve data in the State Store.
Protobuf implementation of the State Store serializer.
A service to initialize a
StateStoreDriver and maintain the connection to the data store.Thrown when the state store is not reachable or available.
Set of utility functions used to work with the State Store.
Entry to track the version of the State Store data stored in the State Store
by a Router.
Protobuf implementation of the StateStoreVersion record.
StateStoreDriver driver implementation that uses ZooKeeper as a
backend.Static fairness policy extending @AbstractRouterRpcFairnessPolicyController
and fetching handlers from configuration for all available name services.
Exception when timing out waiting for the reply of a subcluster.
The ThreadLocalContext class is designed to capture and transfer the context of a
thread-local environment within the Hadoop Distributed File System (HDFS) federation
router operations.
API request for updating the destination of an existing mount point in the
state store.
Protobuf implementation of the state store API object
UpdateMountTableEntryRequest.
API response for updating the destination of an existing mount point in the
state store.
Protobuf implementation of the state store API object
UpdateMountTableEntryResponse.
API request for overriding an existing namenode registration in the state
store.
Protobuf implementation of the state store API object
OverrideNamenodeRegistrationRequest.
API response for overriding an existing namenode registration in the state
store.
Protobuf implementation of the state store API object
OverrideNamenodeRegistrationResponse.
Zookeeper based router delegation token store implementation.