Package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf

Class FSSchedulerConfigurationStore

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.YarnConfigurationStore

org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore

All Implemented Interfaces:

AutoCloseable

public class FSSchedulerConfigurationStore
extends YarnConfigurationStore

A filesystem implementation of YarnConfigurationStore. Offer configuration storage in FileSystem

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.YarnConfigurationStore

YarnConfigurationStore.LogMutation

Field Summary

Fields

Modifier and Type	Field	Description
protected static final org.apache.hadoop.yarn.server.records.Version	CURRENT_VERSION_INFO
static final org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
FSSchedulerConfigurationStore()

Method Summary

Modifier and Type	Method	Description
void	close()	Closes the configuration store, releasing any required resources.
void	confirmMutation(YarnConfigurationStore.LogMutation pendingMutation, boolean isValid)	Should be called after logMutation.
void	format()	Format the persisted configuration.
long	getConfigVersion()	Get the last updated config version.
List<YarnConfigurationStore.LogMutation>	getConfirmedConfHistory(long fromId)	Get a list of confirmed configuration mutations starting from a given id.
protected org.apache.hadoop.yarn.server.records.Version	getConfStoreVersion()	Get schema version of persisted conf store, for detecting compatibility issues when changing conf store schema.
protected org.apache.hadoop.yarn.server.records.Version	getCurrentVersion()	Get the hard-coded schema version, for comparison against the schema version currently persisted.
protected LinkedList<YarnConfigurationStore.LogMutation>	getLogs()	Get a list of configuration mutations.
void	initialize(org.apache.hadoop.conf.Configuration fsConf, org.apache.hadoop.conf.Configuration vSchedConf, RMContext rmContext)	Initialize the configuration store, with schedConf as the initial scheduler configuration.
void	logMutation(YarnConfigurationStore.LogMutation logMutation)	Update and persist latest configuration in temp file.
org.apache.hadoop.conf.Configuration	retrieve()	Retrieve the persisted configuration.
protected void	storeVersion()	Persist the hard-coded schema version to the conf store.

Methods inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.YarnConfigurationStore

checkVersion

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

public static final org.slf4j.Logger LOG

CURRENT_VERSION_INFO

@VisibleForTesting
protected static final org.apache.hadoop.yarn.server.records.Version CURRENT_VERSION_INFO

Constructor Details

FSSchedulerConfigurationStore

public FSSchedulerConfigurationStore()

Method Details

initialize

public void initialize(org.apache.hadoop.conf.Configuration fsConf,
 org.apache.hadoop.conf.Configuration vSchedConf,
 RMContext rmContext)
                throws Exception

Description copied from class: YarnConfigurationStore

Initialize the configuration store, with schedConf as the initial scheduler configuration. If a persisted store already exists, use the scheduler configuration stored there, and ignore schedConf.

Specified by:

initialize in class YarnConfigurationStore

Parameters:

fsConf - configuration to initialize store with

vSchedConf - Initial key-value scheduler configuration to persist.

rmContext - RMContext for this configuration store

Throws:

IOException - if initialization fails

Exception

logMutation

public void logMutation(YarnConfigurationStore.LogMutation logMutation)
                 throws IOException

Update and persist latest configuration in temp file.

Specified by:

logMutation in class YarnConfigurationStore

Parameters:

logMutation - configuration change to be persisted in write ahead log

Throws:

IOException - throw IOE when write temp configuration file fail

confirmMutation

public void confirmMutation(YarnConfigurationStore.LogMutation pendingMutation,
 boolean isValid)
                     throws Exception

Description copied from class: YarnConfigurationStore

Should be called after logMutation. Gets the pending mutation last logged by logMutation and marks the mutation as persisted (no longer pending). If isValid is true, merge the mutation with the persisted configuration.

Specified by:

confirmMutation in class YarnConfigurationStore

Parameters:

pendingMutation - the log mutation to apply

isValid - if true, finalize temp configuration file if false, remove temp configuration file and rollback

Throws:

Exception - throw IOE when write temp configuration file fail

format

public void format()
            throws Exception

Description copied from class: YarnConfigurationStore

Format the persisted configuration.

Specified by:

format in class YarnConfigurationStore

Throws:

IOException - on failure to format

Exception

getConfigVersion

public long getConfigVersion()
                      throws Exception

Description copied from class: YarnConfigurationStore

Get the last updated config version.

Specified by:

getConfigVersion in class YarnConfigurationStore

Returns:

Last updated config version.

Throws:

Exception - On version fetch failure.

retrieve

public org.apache.hadoop.conf.Configuration retrieve()
                                              throws IOException

Description copied from class: YarnConfigurationStore

Retrieve the persisted configuration.

Specified by:

retrieve in class YarnConfigurationStore

Returns:

configuration as key-value

Throws:

IOException - an I/O exception has occurred.

getConfirmedConfHistory

public List<YarnConfigurationStore.LogMutation> getConfirmedConfHistory(long fromId)

Description copied from class: YarnConfigurationStore

Get a list of confirmed configuration mutations starting from a given id.

Specified by:

getConfirmedConfHistory in class YarnConfigurationStore

Parameters:

fromId - id from which to start getting mutations, inclusive

Returns:

list of configuration mutations

getLogs

protected LinkedList<YarnConfigurationStore.LogMutation> getLogs()

Description copied from class: YarnConfigurationStore

Get a list of configuration mutations.

Specified by:

getLogs in class YarnConfigurationStore

Returns:

list of configuration mutations.

getConfStoreVersion

protected org.apache.hadoop.yarn.server.records.Version getConfStoreVersion()
                                                                     throws Exception

Description copied from class: YarnConfigurationStore

Get schema version of persisted conf store, for detecting compatibility issues when changing conf store schema.

Specified by:

getConfStoreVersion in class YarnConfigurationStore

Returns:

Schema version currently used by the persisted configuration store.

Throws:

Exception - On version fetch failure

storeVersion

protected void storeVersion()
                     throws Exception

Description copied from class: YarnConfigurationStore

Persist the hard-coded schema version to the conf store.

Specified by:

storeVersion in class YarnConfigurationStore

Throws:

Exception - On storage failure

getCurrentVersion

protected org.apache.hadoop.yarn.server.records.Version getCurrentVersion()

Description copied from class: YarnConfigurationStore

Get the hard-coded schema version, for comparison against the schema version currently persisted.

Specified by:

getCurrentVersion in class YarnConfigurationStore

Returns:

Current hard-coded schema version

close

public void close()
           throws IOException

Description copied from class: YarnConfigurationStore

Closes the configuration store, releasing any required resources.

Specified by:

close in interface AutoCloseable

Specified by:

close in class YarnConfigurationStore

Throws:

IOException - on failure to close