Package org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair

Class AllocationConfiguration

java.lang.Object

org.apache.hadoop.conf.Configuration

org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationSchedulerConfiguration

org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationConfiguration

All Implemented Interfaces:

Iterable<Map.Entry<String,String>>, org.apache.hadoop.io.Writable

public class AllocationConfiguration
extends ReservationSchedulerConfiguration

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.conf.Configuration

org.apache.hadoop.conf.Configuration.DeprecationDelta, org.apache.hadoop.conf.Configuration.IntegerRanges

Field Summary

Fields inherited from class org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationSchedulerConfiguration

DEFAULT_CAPACITY_OVER_TIME_MULTIPLIER, DEFAULT_RESERVATION_ADMISSION_POLICY, DEFAULT_RESERVATION_AGENT_NAME, DEFAULT_RESERVATION_ENFORCEMENT_WINDOW, DEFAULT_RESERVATION_MOVE_ON_EXPIRY, DEFAULT_RESERVATION_PLANNER_NAME, DEFAULT_RESERVATION_WINDOW, DEFAULT_SHOW_RESERVATIONS_AS_QUEUES

Constructor Summary

Constructors

Constructor	Description
AllocationConfiguration(QueueProperties queueProperties, AllocationFileParser allocationFileParser, ReservationQueueConfiguration globalReservationQueueConfig)	Create a fully initialised configuration for the scheduler.
AllocationConfiguration(FairScheduler scheduler)	Create a base scheduler configuration with just the defaults set.

Method Summary

Modifier and Type	Method	Description
float	getAverageCapacity(QueuePath queue)	Gets the average allowed capacity which will aggregated over the ReservationSchedulerConfiguration.getReservationWindow(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.QueuePath) by the the SharingPolicy to check aggregate used capacity
Map<FSQueueType,Set<String>>	getConfiguredQueues()
SchedulingPolicy	getDefaultSchedulingPolicy()
long	getEnforcementWindow(QueuePath queue)	Gets the time in milliseconds for which the Planner will verify the Plans satisfy the constraints
float	getFairSharePreemptionThreshold(String queueName)	Get a queue's fair share preemption threshold in the allocation file.
long	getFairSharePreemptionTimeout(String queueName)	Get a queue's fair share preemption timeout configured in the allocation file, in milliseconds.
float	getInstantaneousMaxCapacity(QueuePath queue)	Gets the maximum capacity at any time that the SharingPolicy allows
long	getMinSharePreemptionTimeout(String queueName)	Get a queue's min share preemption timeout configured in the allocation file, in milliseconds.
boolean	getMoveOnExpiry(QueuePath queue)	Gets whether the applications should be killed or moved to the parent queue when the ReservationDefinition expires
Map<String,Map<org.apache.hadoop.yarn.security.AccessType,org.apache.hadoop.security.authorize.AccessControlList>>	getQueueAcls()	Get the map of ACLs of all queues.
float	getQueueMaxAMShareDefault()
int	getQueueMaxAppsDefault()
String	getReplanner(QueuePath queue)	Gets the name of the Planner class associated with the queue
Map<org.apache.hadoop.yarn.api.records.ReservationACL,org.apache.hadoop.security.authorize.AccessControlList>	getReservationAcls(QueuePath queue)	Gets a map containing the AccessControlList of users for each ReservationACL acl on thee specified queue.
String	getReservationAdmissionPolicy(QueuePath queue)	Gets the name of the SharingPolicy class associated with the queue
String	getReservationAgent(QueuePath queue)	Gets the name of the ReservationAgent class associated with the queue
long	getReservationWindow(QueuePath queue)	Gets the length of time in milliseconds for which the SharingPolicy checks for validity
boolean	getShowReservationAsQueues(QueuePath queue)	Checks whether the reservation queues be hidden or visible
Map<String,Integer>	getUserMaxApps()
int	getUserMaxApps(String user)
int	getUserMaxAppsDefault()
void	initFSQueue(FSQueue queue)	Initialize a FSQueue with queue-specific properties and its metrics.
boolean	isPreemptable(String queueName)
boolean	isReservable(QueuePath queue)	Checks if the queue participates in reservation based scheduling
void	setAverageCapacity(int avgCapacity)
void	setReservationWindow(long window)

Methods inherited from class org.apache.hadoop.conf.Configuration

addDefaultResource, addDeprecation, addDeprecation, addDeprecation, addDeprecation, addDeprecations, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addTags, clear, dumpConfiguration, dumpConfiguration, dumpDeprecatedKeys, get, get, getAllPropertiesByTag, getAllPropertiesByTags, getBoolean, getClass, getClass, getClassByName, getClassByNameOrNull, getClasses, getClassLoader, getConfResourceAsInputStream, getConfResourceAsReader, getDouble, getEnum, getEnumSet, getFile, getFinalParameters, getFloat, getInstances, getInt, getInts, getLocalPath, getLong, getLongBytes, getPassword, getPasswordFromConfig, getPasswordFromCredentialProviders, getPattern, getPropertySources, getProps, getPropsWithPrefix, getRange, getRaw, getResource, getSocketAddr, getSocketAddr, getStorageSize, getStorageSize, getStringCollection, getStrings, getStrings, getTimeDuration, getTimeDuration, getTimeDuration, getTimeDuration, getTimeDurationHelper, getTimeDurations, getTrimmed, getTrimmed, getTrimmedStringCollection, getTrimmedStrings, getTrimmedStrings, getValByRegex, hasWarnedDeprecation, isDeprecated, isPropertyTag, iterator, main, onlyKeyExists, readFields, reloadConfiguration, reloadExistingConfigurations, set, set, setAllowNullValueProperties, setBoolean, setBooleanIfUnset, setClass, setClassLoader, setDeprecatedProperties, setDouble, setEnum, setFloat, setIfUnset, setInt, setLong, setPattern, setQuietMode, setRestrictSystemProperties, setRestrictSystemPropertiesDefault, setRestrictSystemProps, setSocketAddr, setStorageSize, setStrings, setTimeDuration, size, substituteCommonVariables, toString, unset, updateConnectAddr, updateConnectAddr, write, writeXml, writeXml, writeXml, writeXml

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Details

AllocationConfiguration

public AllocationConfiguration(QueueProperties queueProperties,
 AllocationFileParser allocationFileParser,
 ReservationQueueConfiguration globalReservationQueueConfig)
                        throws AllocationConfigurationException

Create a fully initialised configuration for the scheduler.

Parameters:

queueProperties - The list of queues and their properties from the configuration.

allocationFileParser - The allocation file parser

globalReservationQueueConfig - The reservation queue config

Throws:

AllocationConfigurationException - for any errors.

AllocationConfiguration

public AllocationConfiguration(FairScheduler scheduler)

Create a base scheduler configuration with just the defaults set. Should only be called to init a basic setup on scheduler init.

Parameters:

scheduler - The FairScheduler to create and initialise the placement policy.

Method Details

getQueueAcls

public Map<String,Map<org.apache.hadoop.yarn.security.AccessType,org.apache.hadoop.security.authorize.AccessControlList>> getQueueAcls()

Get the map of ACLs of all queues.

Returns:

the map of ACLs of all queues

getReservationAcls

public Map<org.apache.hadoop.yarn.api.records.ReservationACL,org.apache.hadoop.security.authorize.AccessControlList> getReservationAcls(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets a map containing the AccessControlList of users for each ReservationACL acl on thee specified queue.

Specified by:

getReservationAcls in class ReservationSchedulerConfiguration

Parameters:

queue - the queue with which to check a user's permissions.

Returns:

The a Map of ReservationACL to AccessControlList which contains a list of users that have the specified permission level.

getMinSharePreemptionTimeout

public long getMinSharePreemptionTimeout(String queueName)

Get a queue's min share preemption timeout configured in the allocation file, in milliseconds. Return -1 if not set.

Parameters:

queueName - queue name.

Returns:

min share preemption timeout, return -1f if not set.

getFairSharePreemptionTimeout

public long getFairSharePreemptionTimeout(String queueName)

Get a queue's fair share preemption timeout configured in the allocation file, in milliseconds. Return -1 if not set.

Parameters:

queueName - queue Name.

Returns:

fair share preemption timeout, return -1f if not set.

getFairSharePreemptionThreshold

public float getFairSharePreemptionThreshold(String queueName)

Get a queue's fair share preemption threshold in the allocation file. Return -1f if not set.

Parameters:

queueName - queue Name.

Returns:

preemption threshold, return -1f if not set.

isPreemptable

public boolean isPreemptable(String queueName)

getUserMaxApps

public int getUserMaxApps(String user)

getUserMaxApps

public Map<String,Integer> getUserMaxApps()

getQueueMaxAppsDefault

public int getQueueMaxAppsDefault()

getUserMaxAppsDefault

public int getUserMaxAppsDefault()

getQueueMaxAMShareDefault

public float getQueueMaxAMShareDefault()

getDefaultSchedulingPolicy

public SchedulingPolicy getDefaultSchedulingPolicy()

getConfiguredQueues

public Map<FSQueueType,Set<String>> getConfiguredQueues()

isReservable

public boolean isReservable(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Checks if the queue participates in reservation based scheduling

Specified by:

isReservable in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

true if the queue participates in reservation based scheduling

getReservationWindow

public long getReservationWindow(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets the length of time in milliseconds for which the SharingPolicy checks for validity

Overrides:

getReservationWindow in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

length in time in milliseconds for which to check the SharingPolicy

getAverageCapacity

public float getAverageCapacity(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets the average allowed capacity which will aggregated over the ReservationSchedulerConfiguration.getReservationWindow(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.QueuePath) by the the SharingPolicy to check aggregate used capacity

Overrides:

getAverageCapacity in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

average capacity allowed by the SharingPolicy

getInstantaneousMaxCapacity

public float getInstantaneousMaxCapacity(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets the maximum capacity at any time that the SharingPolicy allows

Overrides:

getInstantaneousMaxCapacity in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

maximum allowed capacity at any time

getReservationAdmissionPolicy

public String getReservationAdmissionPolicy(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets the name of the SharingPolicy class associated with the queue

Overrides:

getReservationAdmissionPolicy in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

the class name of the SharingPolicy

getReservationAgent

public String getReservationAgent(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets the name of the ReservationAgent class associated with the queue

Overrides:

getReservationAgent in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

the class name of the ReservationAgent

getShowReservationAsQueues

public boolean getShowReservationAsQueues(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Checks whether the reservation queues be hidden or visible

Overrides:

getShowReservationAsQueues in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

true if reservation queues should be visible

getReplanner

public String getReplanner(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets the name of the Planner class associated with the queue

Overrides:

getReplanner in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

the class name of the Planner

getMoveOnExpiry

public boolean getMoveOnExpiry(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets whether the applications should be killed or moved to the parent queue when the ReservationDefinition expires

Overrides:

getMoveOnExpiry in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

true if application should be moved, false if they need to be killed

getEnforcementWindow

public long getEnforcementWindow(QueuePath queue)

Description copied from class: ReservationSchedulerConfiguration

Gets the time in milliseconds for which the Planner will verify the Plans satisfy the constraints

Overrides:

getEnforcementWindow in class ReservationSchedulerConfiguration

Parameters:

queue - name of the queue

Returns:

the time in milliseconds for which to check constraints

setReservationWindow

@VisibleForTesting
public void setReservationWindow(long window)

setAverageCapacity

@VisibleForTesting
public void setAverageCapacity(int avgCapacity)

initFSQueue

public void initFSQueue(FSQueue queue)

Initialize a FSQueue with queue-specific properties and its metrics.

Parameters:

queue - the FSQueue needed to be initialized