Package org.apache.hadoop.ipc

Class DecayRpcScheduler

java.lang.Object

org.apache.hadoop.ipc.DecayRpcScheduler

All Implemented Interfaces:

DecayRpcSchedulerMXBean, RpcScheduler, MetricsSource

public class DecayRpcScheduler
extends Object
implements RpcScheduler, DecayRpcSchedulerMXBean, MetricsSource

The decay RPC scheduler tracks the cost of incoming requests in a map, then decays the costs at a fixed time interval. The scheduler is optimized for large periods (on the order of seconds), as it offloads work to the decay sweep.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DecayRpcScheduler.DecayTask	This TimerTask will call decayCurrentCosts until the scheduler has been garbage collected.
static final class	DecayRpcScheduler.MetricsProxy	MetricsProxy is a singleton because we may init multiple schedulers and we want to clean up resources when a new scheduler replaces the old one.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	DECAYSCHEDULER_METRICS_TOP_USER_COUNT
static final int	DECAYSCHEDULER_METRICS_TOP_USER_COUNT_DEFAULT
static final String	DECAYSCHEDULER_UNKNOWN_IDENTITY
static final Boolean	IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_ENABLE_DEFAULT
static final String	IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_ENABLE_KEY
static final String	IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_THRESHOLDS_KEY
static final String	IPC_DECAYSCHEDULER_SERVICE_USERS_KEY	Service users will always be scheduled into the highest-priority queue.
static final String	IPC_DECAYSCHEDULER_THRESHOLDS_KEY	Thresholds are specified as integer percentages, and specify which usage range each queue will be allocated to.
static final String	IPC_FCQ_DECAYSCHEDULER_FACTOR_KEY	Deprecated.
static final String	IPC_FCQ_DECAYSCHEDULER_PERIOD_KEY	Deprecated.
static final String	IPC_FCQ_DECAYSCHEDULER_THRESHOLDS_KEY	Deprecated.
static final double	IPC_SCHEDULER_DECAYSCHEDULER_FACTOR_DEFAULT
static final String	IPC_SCHEDULER_DECAYSCHEDULER_FACTOR_KEY	Decay factor controls how much each cost is suppressed by on each sweep.
static final long	IPC_SCHEDULER_DECAYSCHEDULER_PERIOD_DEFAULT
static final String	IPC_SCHEDULER_DECAYSCHEDULER_PERIOD_KEY	Period controls how many milliseconds between each decay sweep.
static final org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
DecayRpcScheduler(int numLevels, String ns, Configuration conf)	Create a decay scheduler.

Method Summary

Modifier and Type	Method	Description
void	addResponseTime(String callName, Schedulable schedulable, ProcessingDetails details)	Store a processing time value for an RPC call into this scheduler.
double[]	getAverageResponseTime()
String	getCallVolumeSummary()
DecayRpcSchedulerDetailedMetrics	getDecayRpcSchedulerDetailedMetrics()
void	getMetrics(MetricsCollector collector, boolean all)	Get metrics from the metrics source
int	getPriorityLevel(Schedulable obj)	Compute the appropriate priority for a schedulable based on past requests.
long[]	getResponseTimeCountInLastWindow()
String	getSchedulingDecisionSummary()
long	getTotalCallVolume()
long	getTotalRawCallVolume()
long	getTotalServiceUserCallVolume()
long	getTotalServiceUserRawCallVolume()
int	getUniqueIdentityCount()
boolean	shouldBackOff(Schedulable obj)
void	stop()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.ipc.RpcScheduler

addResponseTime

Field Details

IPC_SCHEDULER_DECAYSCHEDULER_PERIOD_KEY

public static final String IPC_SCHEDULER_DECAYSCHEDULER_PERIOD_KEY

Period controls how many milliseconds between each decay sweep.

See Also:

• Constant Field Values

IPC_SCHEDULER_DECAYSCHEDULER_PERIOD_DEFAULT

public static final long IPC_SCHEDULER_DECAYSCHEDULER_PERIOD_DEFAULT

See Also:

• Constant Field Values

IPC_FCQ_DECAYSCHEDULER_PERIOD_KEY

@Deprecated
public static final String IPC_FCQ_DECAYSCHEDULER_PERIOD_KEY

Deprecated.

See Also:

• Constant Field Values

IPC_SCHEDULER_DECAYSCHEDULER_FACTOR_KEY

public static final String IPC_SCHEDULER_DECAYSCHEDULER_FACTOR_KEY

Decay factor controls how much each cost is suppressed by on each sweep. Valid numbers are > 0 and < 1. Decay factor works in tandem with period to control how long the scheduler remembers an identity.

See Also:

• Constant Field Values

IPC_SCHEDULER_DECAYSCHEDULER_FACTOR_DEFAULT

public static final double IPC_SCHEDULER_DECAYSCHEDULER_FACTOR_DEFAULT

See Also:

• Constant Field Values

IPC_FCQ_DECAYSCHEDULER_FACTOR_KEY

@Deprecated
public static final String IPC_FCQ_DECAYSCHEDULER_FACTOR_KEY

Deprecated.

See Also:

• Constant Field Values

IPC_DECAYSCHEDULER_THRESHOLDS_KEY

public static final String IPC_DECAYSCHEDULER_THRESHOLDS_KEY

Thresholds are specified as integer percentages, and specify which usage range each queue will be allocated to. For instance, specifying the list 10, 40, 80 implies 4 queues, with - q3 from 80% up - q2 from 40 up to 80 - q1 from 10 up to 40 - q0 otherwise.

See Also:

• Constant Field Values

IPC_FCQ_DECAYSCHEDULER_THRESHOLDS_KEY

@Deprecated
public static final String IPC_FCQ_DECAYSCHEDULER_THRESHOLDS_KEY

Deprecated.

See Also:

• Constant Field Values

IPC_DECAYSCHEDULER_SERVICE_USERS_KEY

public static final String IPC_DECAYSCHEDULER_SERVICE_USERS_KEY

Service users will always be scheduled into the highest-priority queue. They are specified as a comma-separated list.

See Also:

• Constant Field Values

DECAYSCHEDULER_UNKNOWN_IDENTITY

public static final String DECAYSCHEDULER_UNKNOWN_IDENTITY

See Also:

• Constant Field Values

IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_ENABLE_KEY

public static final String IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_ENABLE_KEY

See Also:

• Constant Field Values

IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_ENABLE_DEFAULT

public static final Boolean IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_ENABLE_DEFAULT

IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_THRESHOLDS_KEY

public static final String IPC_DECAYSCHEDULER_BACKOFF_RESPONSETIME_THRESHOLDS_KEY

See Also:

• Constant Field Values

DECAYSCHEDULER_METRICS_TOP_USER_COUNT

public static final String DECAYSCHEDULER_METRICS_TOP_USER_COUNT

See Also:

• Constant Field Values

DECAYSCHEDULER_METRICS_TOP_USER_COUNT_DEFAULT

public static final int DECAYSCHEDULER_METRICS_TOP_USER_COUNT_DEFAULT

See Also:

• Constant Field Values

LOG

public static final org.slf4j.Logger LOG

Constructor Details

DecayRpcScheduler

public DecayRpcScheduler(int numLevels,
 String ns,
 Configuration conf)

Create a decay scheduler.

Parameters:

numLevels - number of priority levels

ns - config prefix, so that we can configure multiple schedulers in a single instance.

conf - configuration to use.

Method Details

getPriorityLevel

public int getPriorityLevel(Schedulable obj)

Compute the appropriate priority for a schedulable based on past requests.

Specified by:

getPriorityLevel in interface RpcScheduler

Parameters:

obj - the schedulable obj to query and remember

Returns:

the level index which we recommend scheduling in

shouldBackOff

public boolean shouldBackOff(Schedulable obj)

Specified by:

shouldBackOff in interface RpcScheduler

addResponseTime

public void addResponseTime(String callName,
 Schedulable schedulable,
 ProcessingDetails details)

Description copied from interface: RpcScheduler

Store a processing time value for an RPC call into this scheduler.

Specified by:

addResponseTime in interface RpcScheduler

Parameters:

callName - The name of the call.

schedulable - The schedulable representing the incoming call.

details - The details of processing time.

getUniqueIdentityCount

public int getUniqueIdentityCount()

Specified by:

getUniqueIdentityCount in interface DecayRpcSchedulerMXBean

getTotalCallVolume

public long getTotalCallVolume()

Specified by:

getTotalCallVolume in interface DecayRpcSchedulerMXBean

getTotalRawCallVolume

public long getTotalRawCallVolume()

getTotalServiceUserCallVolume

public long getTotalServiceUserCallVolume()

getTotalServiceUserRawCallVolume

public long getTotalServiceUserRawCallVolume()

getResponseTimeCountInLastWindow

public long[] getResponseTimeCountInLastWindow()

Specified by:

getResponseTimeCountInLastWindow in interface DecayRpcSchedulerMXBean

getAverageResponseTime

public double[] getAverageResponseTime()

Specified by:

getAverageResponseTime in interface DecayRpcSchedulerMXBean

getMetrics

public void getMetrics(MetricsCollector collector,
 boolean all)

Description copied from interface: MetricsSource

Get metrics from the metrics source

Specified by:

getMetrics in interface MetricsSource

Parameters:

collector - to contain the resulting metrics snapshot

all - if true, return all metrics even if unchanged.

getSchedulingDecisionSummary

public String getSchedulingDecisionSummary()

Specified by:

getSchedulingDecisionSummary in interface DecayRpcSchedulerMXBean

getCallVolumeSummary

public String getCallVolumeSummary()

Specified by:

getCallVolumeSummary in interface DecayRpcSchedulerMXBean

getDecayRpcSchedulerDetailedMetrics

@VisibleForTesting
public DecayRpcSchedulerDetailedMetrics getDecayRpcSchedulerDetailedMetrics()

stop

public void stop()

Specified by:

stop in interface RpcScheduler