Package org.apache.hadoop.yarn.server.router.rmadmin

Class FederationRMAdminInterceptor

java.lang.Object

org.apache.hadoop.yarn.server.router.rmadmin.AbstractRMAdminRequestInterceptor

org.apache.hadoop.yarn.server.router.rmadmin.FederationRMAdminInterceptor

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.tools.GetUserMappingsProtocol, org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocol, RMAdminRequestInterceptor

public class FederationRMAdminInterceptor
extends AbstractRMAdminRequestInterceptor

Field Summary

Fields inherited from class org.apache.hadoop.yarn.server.router.rmadmin.AbstractRMAdminRequestInterceptor

user

Fields inherited from interface org.apache.hadoop.tools.GetUserMappingsProtocol

versionID

Constructor Summary

Constructors

Constructor	Description
FederationRMAdminInterceptor()

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.yarn.server.api.protocolrecords.AddToClusterNodeLabelsResponse	addToClusterNodeLabels(org.apache.hadoop.yarn.server.api.protocolrecords.AddToClusterNodeLabelsRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.BatchSaveFederationQueuePoliciesResponse	batchSaveFederationQueuePolicies(org.apache.hadoop.yarn.server.api.protocolrecords.BatchSaveFederationQueuePoliciesRequest request)	Batch Save the Queue Policies for the Federation.
org.apache.hadoop.yarn.server.api.protocolrecords.CheckForDecommissioningNodesResponse	checkForDecommissioningNodes(org.apache.hadoop.yarn.server.api.protocolrecords.CheckForDecommissioningNodesRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationApplicationResponse	deleteFederationApplication(org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationApplicationRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationQueuePoliciesResponse	deleteFederationPoliciesByQueues(org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationQueuePoliciesRequest request)	Delete Policies based on the provided queue list.
org.apache.hadoop.yarn.server.api.protocolrecords.DeregisterSubClusterResponse	deregisterSubCluster(org.apache.hadoop.yarn.server.api.protocolrecords.DeregisterSubClusterRequest request)	In YARN Federation mode, We allow users to mark subClusters With no heartbeat for a long time as SC_LOST state.
protected org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocol	getAdminRMProxyForSubCluster(org.apache.hadoop.yarn.server.federation.store.records.SubClusterId subClusterId)
ThreadPoolExecutor	getExecutorService()
org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade	getFederationFacade()
org.apache.hadoop.yarn.server.api.protocolrecords.GetSubClustersResponse	getFederationSubClusters(org.apache.hadoop.yarn.server.api.protocolrecords.GetSubClustersRequest request)	Get federation subcluster list.
String[]	getGroupsForUser(String user)
protected Map<org.apache.hadoop.yarn.server.federation.store.records.SubClusterIdInfo,Float>	getSubClusterWeightMap(String policyWeight)	Get the Map of SubClusterWeight.
void	init(String userName)	Initializes the RMAdminRequestInterceptor.
org.apache.hadoop.yarn.server.api.protocolrecords.QueryFederationQueuePoliciesResponse	listFederationQueuePolicies(org.apache.hadoop.yarn.server.api.protocolrecords.QueryFederationQueuePoliciesRequest request)	List the Queue Policies for the Federation.
org.apache.hadoop.yarn.server.api.protocolrecords.NodesToAttributesMappingResponse	mapAttributesToNodes(org.apache.hadoop.yarn.server.api.protocolrecords.NodesToAttributesMappingRequest request)
protected String	parsePolicyWeights(Map<org.apache.hadoop.yarn.server.federation.store.records.SubClusterIdInfo,Float> policyWeights)	Parses the policy weights from the provided policyWeights map.
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshAdminAclsResponse	refreshAdminAcls(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshAdminAclsRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshClusterMaxPriorityResponse	refreshClusterMaxPriority(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshClusterMaxPriorityRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResponse	refreshNodes(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesRequest request)	Refresh node requests.
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResourcesResponse	refreshNodesResources(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResourcesRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshQueuesResponse	refreshQueues(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshQueuesRequest request)	Refresh queue requests.
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshServiceAclsResponse	refreshServiceAcls(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshServiceAclsRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshSuperUserGroupsConfigurationResponse	refreshSuperUserGroupsConfiguration(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshSuperUserGroupsConfigurationRequest request)	Refresh SuperUserGroupsConfiguration requests.
org.apache.hadoop.yarn.server.api.protocolrecords.RefreshUserToGroupsMappingsResponse	refreshUserToGroupsMappings(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshUserToGroupsMappingsRequest request)	Refresh UserToGroupsMappings requests.
org.apache.hadoop.yarn.server.api.protocolrecords.RemoveFromClusterNodeLabelsResponse	removeFromClusterNodeLabels(org.apache.hadoop.yarn.server.api.protocolrecords.RemoveFromClusterNodeLabelsRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.ReplaceLabelsOnNodeResponse	replaceLabelsOnNode(org.apache.hadoop.yarn.server.api.protocolrecords.ReplaceLabelsOnNodeRequest request)
org.apache.hadoop.yarn.server.api.protocolrecords.SaveFederationQueuePolicyResponse	saveFederationQueuePolicy(org.apache.hadoop.yarn.server.api.protocolrecords.SaveFederationQueuePolicyRequest request)	Save the Queue Policy for the Federation.
void	setNextInterceptor(RMAdminRequestInterceptor next)	Sets the RMAdminRequestInterceptor in the chain.
org.apache.hadoop.yarn.server.api.protocolrecords.UpdateNodeResourceResponse	updateNodeResource(org.apache.hadoop.yarn.server.api.protocolrecords.UpdateNodeResourceRequest request)

Methods inherited from class org.apache.hadoop.yarn.server.router.rmadmin.AbstractRMAdminRequestInterceptor

getConf, getNextInterceptor, setConf, shutdown

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FederationRMAdminInterceptor

public FederationRMAdminInterceptor()

Method Details

init

public void init(String userName)

Description copied from class: AbstractRMAdminRequestInterceptor

Initializes the RMAdminRequestInterceptor.

Specified by:

init in interface RMAdminRequestInterceptor

Overrides:

init in class AbstractRMAdminRequestInterceptor

Parameters:

userName - the name of the client

getAdminRMProxyForSubCluster

@VisibleForTesting
protected org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocol getAdminRMProxyForSubCluster(org.apache.hadoop.yarn.server.federation.store.records.SubClusterId subClusterId)
                                                                                                        throws Exception

Throws:

Exception

setNextInterceptor

public void setNextInterceptor(RMAdminRequestInterceptor next)

Description copied from class: AbstractRMAdminRequestInterceptor

Sets the RMAdminRequestInterceptor in the chain.

Specified by:

setNextInterceptor in interface RMAdminRequestInterceptor

Overrides:

setNextInterceptor in class AbstractRMAdminRequestInterceptor

Parameters:

next - the RMAdminRequestInterceptor to set in the pipeline

refreshQueues

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshQueuesResponse refreshQueues(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshQueuesRequest request)
                                                                                      throws org.apache.hadoop.ipc.StandbyException,
org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Refresh queue requests. The Router supports refreshing all SubCluster queues at once, and also supports refreshing queues by SubCluster.

Parameters:

request - RefreshQueuesRequest, If subClusterId is not empty, it means that we want to refresh the queue of the specified subClusterId. If subClusterId is empty, it means we want to refresh all queues.

Returns:

RefreshQueuesResponse, There is no specific information in the response, as long as it is not empty, it means that the request is successful.

Throws:

org.apache.hadoop.ipc.StandbyException - exception thrown by non-active server.

org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.

IOException - io error occurs.

refreshNodes

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResponse refreshNodes(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesRequest request)
                                                                                    throws org.apache.hadoop.ipc.StandbyException,
org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Refresh node requests. The Router supports refreshing all SubCluster nodes at once, and also supports refreshing node by SubCluster.

Parameters:

request - RefreshNodesRequest, If subClusterId is not empty, it means that we want to refresh the node of the specified subClusterId. If subClusterId is empty, it means we want to refresh all nodes.

Returns:

RefreshNodesResponse, There is no specific information in the response, as long as it is not empty, it means that the request is successful.

Throws:

org.apache.hadoop.ipc.StandbyException - exception thrown by non-active server.

org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.

IOException - io error occurs.

refreshSuperUserGroupsConfiguration

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshSuperUserGroupsConfigurationResponse refreshSuperUserGroupsConfiguration(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshSuperUserGroupsConfigurationRequest request)
                                                                                                                                  throws org.apache.hadoop.ipc.StandbyException,
org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Refresh SuperUserGroupsConfiguration requests. The Router supports refreshing all subCluster SuperUserGroupsConfiguration at once, and also supports refreshing SuperUserGroupsConfiguration by SubCluster.

Parameters:

request - RefreshSuperUserGroupsConfigurationRequest, If subClusterId is not empty, it means that we want to refresh the superuser groups configuration of the specified subClusterId. If subClusterId is empty, it means we want to refresh all subCluster superuser groups configuration.

Returns:

RefreshSuperUserGroupsConfigurationResponse, There is no specific information in the response, as long as it is not empty, it means that the request is successful.

Throws:

org.apache.hadoop.ipc.StandbyException - exception thrown by non-active server.

org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.

IOException - io error occurs.

refreshUserToGroupsMappings

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshUserToGroupsMappingsResponse refreshUserToGroupsMappings(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshUserToGroupsMappingsRequest request)
                                                                                                                  throws org.apache.hadoop.ipc.StandbyException,
org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Refresh UserToGroupsMappings requests. The Router supports refreshing all subCluster UserToGroupsMappings at once, and also supports refreshing UserToGroupsMappings by subCluster.

Parameters:

request - RefreshUserToGroupsMappingsRequest, If subClusterId is not empty, it means that we want to refresh the user groups mapping of the specified subClusterId. If subClusterId is empty, it means we want to refresh all subCluster user groups mapping.

Returns:

RefreshUserToGroupsMappingsResponse, There is no specific information in the response, as long as it is not empty, it means that the request is successful.

Throws:

org.apache.hadoop.ipc.StandbyException - exception thrown by non-active server.

org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.

IOException - io error occurs.

refreshAdminAcls

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshAdminAclsResponse refreshAdminAcls(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshAdminAclsRequest request)
                                                                                            throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

refreshServiceAcls

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshServiceAclsResponse refreshServiceAcls(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshServiceAclsRequest request)
                                                                                                throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

updateNodeResource

public org.apache.hadoop.yarn.server.api.protocolrecords.UpdateNodeResourceResponse updateNodeResource(org.apache.hadoop.yarn.server.api.protocolrecords.UpdateNodeResourceRequest request)
                                                                                                throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

refreshNodesResources

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResourcesResponse refreshNodesResources(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResourcesRequest request)
                                                                                                      throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

addToClusterNodeLabels

public org.apache.hadoop.yarn.server.api.protocolrecords.AddToClusterNodeLabelsResponse addToClusterNodeLabels(org.apache.hadoop.yarn.server.api.protocolrecords.AddToClusterNodeLabelsRequest request)
                                                                                                        throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

removeFromClusterNodeLabels

public org.apache.hadoop.yarn.server.api.protocolrecords.RemoveFromClusterNodeLabelsResponse removeFromClusterNodeLabels(org.apache.hadoop.yarn.server.api.protocolrecords.RemoveFromClusterNodeLabelsRequest request)
                                                                                                                  throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

replaceLabelsOnNode

public org.apache.hadoop.yarn.server.api.protocolrecords.ReplaceLabelsOnNodeResponse replaceLabelsOnNode(org.apache.hadoop.yarn.server.api.protocolrecords.ReplaceLabelsOnNodeRequest request)
                                                                                                  throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

checkForDecommissioningNodes

public org.apache.hadoop.yarn.server.api.protocolrecords.CheckForDecommissioningNodesResponse checkForDecommissioningNodes(org.apache.hadoop.yarn.server.api.protocolrecords.CheckForDecommissioningNodesRequest request)
                                                                                                                    throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

refreshClusterMaxPriority

public org.apache.hadoop.yarn.server.api.protocolrecords.RefreshClusterMaxPriorityResponse refreshClusterMaxPriority(org.apache.hadoop.yarn.server.api.protocolrecords.RefreshClusterMaxPriorityRequest request)
                                                                                                              throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

mapAttributesToNodes

public org.apache.hadoop.yarn.server.api.protocolrecords.NodesToAttributesMappingResponse mapAttributesToNodes(org.apache.hadoop.yarn.server.api.protocolrecords.NodesToAttributesMappingRequest request)
                                                                                                        throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

getGroupsForUser

public String[] getGroupsForUser(String user)
                          throws IOException

Throws:

IOException

getFederationFacade

@VisibleForTesting
public org.apache.hadoop.yarn.server.federation.utils.FederationStateStoreFacade getFederationFacade()

getExecutorService

@VisibleForTesting
public ThreadPoolExecutor getExecutorService()

deregisterSubCluster

public org.apache.hadoop.yarn.server.api.protocolrecords.DeregisterSubClusterResponse deregisterSubCluster(org.apache.hadoop.yarn.server.api.protocolrecords.DeregisterSubClusterRequest request)
                                                                                                    throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

In YARN Federation mode, We allow users to mark subClusters With no heartbeat for a long time as SC_LOST state. If we include a specific subClusterId in the request, check for the specified subCluster. If subClusterId is empty, all subClusters are checked.

Parameters:

request - deregisterSubCluster request. The request contains the id of to deregister sub-cluster.

Returns:

Response from deregisterSubCluster.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - exceptions from yarn servers.

IOException - if an IO error occurred.

saveFederationQueuePolicy

public org.apache.hadoop.yarn.server.api.protocolrecords.SaveFederationQueuePolicyResponse saveFederationQueuePolicy(org.apache.hadoop.yarn.server.api.protocolrecords.SaveFederationQueuePolicyRequest request)
                                                                                                              throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Save the Queue Policy for the Federation.

Parameters:

request - saveFederationQueuePolicy Request.

Returns:

Response from saveFederationQueuePolicy.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - exceptions from yarn servers.

IOException - if an IO error occurred.

batchSaveFederationQueuePolicies

public org.apache.hadoop.yarn.server.api.protocolrecords.BatchSaveFederationQueuePoliciesResponse batchSaveFederationQueuePolicies(org.apache.hadoop.yarn.server.api.protocolrecords.BatchSaveFederationQueuePoliciesRequest request)
                                                                                                                            throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Batch Save the Queue Policies for the Federation.

Parameters:

request - BatchSaveFederationQueuePolicies Request

Returns:

Response from batchSaveFederationQueuePolicies.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - exceptions from yarn servers.

IOException - if an IO error occurred.

listFederationQueuePolicies

public org.apache.hadoop.yarn.server.api.protocolrecords.QueryFederationQueuePoliciesResponse listFederationQueuePolicies(org.apache.hadoop.yarn.server.api.protocolrecords.QueryFederationQueuePoliciesRequest request)
                                                                                                                   throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

List the Queue Policies for the Federation.

Parameters:

request - QueryFederationQueuePolicies Request.

Returns:

QueryFederationQueuePolicies Response.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.

IOException - io error occurs.

deleteFederationApplication

public org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationApplicationResponse deleteFederationApplication(org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationApplicationRequest request)
                                                                                                                  throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

getFederationSubClusters

public org.apache.hadoop.yarn.server.api.protocolrecords.GetSubClustersResponse getFederationSubClusters(org.apache.hadoop.yarn.server.api.protocolrecords.GetSubClustersRequest request)
                                                                                                  throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Get federation subcluster list.

Parameters:

request - GetSubClustersRequest Request.

Returns:

SubClusters Response.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - exceptions from yarn servers.

IOException - io error occurs.

deleteFederationPoliciesByQueues

public org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationQueuePoliciesResponse deleteFederationPoliciesByQueues(org.apache.hadoop.yarn.server.api.protocolrecords.DeleteFederationQueuePoliciesRequest request)
                                                                                                                         throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException

Delete Policies based on the provided queue list.

Parameters:

request - DeleteFederationQueuePoliciesRequest Request.

Returns:

If the deletion is successful, the queue deletion success message will be returned.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.

IOException - io error occurs.

parsePolicyWeights

protected String parsePolicyWeights(Map<org.apache.hadoop.yarn.server.federation.store.records.SubClusterIdInfo,Float> policyWeights)

Parses the policy weights from the provided policyWeights map. returns a string similar to the following: SC-1:0.7,SC-2:0.3

Parameters:

policyWeights - A map containing SubClusterIdInfo as keys and corresponding weight values.

Returns:

A string representation of the parsed policy weights.

getSubClusterWeightMap

protected Map<org.apache.hadoop.yarn.server.federation.store.records.SubClusterIdInfo,Float> getSubClusterWeightMap(String policyWeight)
                                                                                                             throws org.apache.hadoop.yarn.exceptions.YarnException

Get the Map of SubClusterWeight. This method can parse the Weight information of Router and the Weight information of AMRMProxy. An example of a parsed string is as follows: SC-1:0.7,SC-2:0.3

Parameters:

policyWeight - policyWeight.

Returns:

Map of SubClusterWeight.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - exceptions from yarn servers.