Package org.apache.hadoop.fs.statistics.impl

Class IOStatisticsBinding

java.lang.Object
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding
public final class IOStatisticsBinding extends Object
Support for implementing IOStatistics interfaces.

  • Field Details

  • Method Details

    • fromStorageStatistics

      public static IOStatistics fromStorageStatistics(StorageStatistics storageStatistics)
      Create IOStatistics from a storage statistics instance. This will be updated as the storage statistics change.
      Parameters:
      storageStatistics - source data.
      Returns:
      an IO statistics source.

    • dynamicIOStatistics

      public static DynamicIOStatisticsBuilder dynamicIOStatistics()
      Create a builder for dynamic IO Statistics.
      Returns:
      a builder to be completed.

    • emptyStatistics

      public static IOStatistics emptyStatistics()
      Get the shared instance of the immutable empty statistics object.
      Returns:
      an empty statistics object.

    • emptyStatisticsStore

      public static IOStatisticsStore emptyStatisticsStore()
      Get the shared instance of the immutable empty statistics store.
      Returns:
      an empty statistics object.

    • wrap

      public static IOStatisticsSource wrap(IOStatistics statistics)
      Take an IOStatistics instance and wrap it in a source.
      Parameters:
      statistics - statistics.
      Returns:
      a source which will return the values

    • iostatisticsStore

      public static IOStatisticsStoreBuilder iostatisticsStore()
      Create a builder for an IOStatisticsStore.
      Returns:
      a builder instance.

    • entryToString

      public static <E> String entryToString(Map.Entry<String,E> entry)
      Convert an entry to the string format used in logging.
      Type Parameters:
      E - entry type
      Parameters:
      entry - entry to evaluate
      Returns:
      formatted string

    • entryToString

      public static <E> String entryToString(String name, E value)
      Convert entry values to the string format used in logging.
      Type Parameters:
      E - type of values.
      Parameters:
      name - statistic name
      value - stat value
      Returns:
      formatted string

    • passthroughFn

      public static <E extends Serializable> E passthroughFn(E src)
      A passthrough copy operation suitable for immutable types, including numbers.
      Type Parameters:
      E - type of values.
      Parameters:
      src - source object
      Returns:
      the source object

    • snapshotMap

      public static <E extends Serializable> Map<String,E> snapshotMap(Map<String,E> source)
      Take a snapshot of a supplied map, where the copy option simply uses the existing value. For this to be safe, the map must refer to immutable objects.
      Type Parameters:
      E - type of values.
      Parameters:
      source - source map
      Returns:
      a new map referencing the same values.

    • snapshotMap

      public static <E extends Serializable> ConcurrentHashMap<String,E> snapshotMap(Map<String,E> source, Function<E,E> copyFn)
      Take a snapshot of a supplied map, using the copy function to replicate the source values.
      Type Parameters:
      E - type of values.
      Parameters:
      source - source map
      copyFn - function to copy the value
      Returns:
      a concurrent hash map referencing the same values.

    • aggregateMaps

      public static <E> void aggregateMaps(Map<String,E> dest, Map<String,E> other, BiFunction<E,E,E> aggregateFn, Function<E,E> copyFn)
      Aggregate two maps so that the destination.
      Type Parameters:
      E - type of values
      Parameters:
      dest - destination map.
      other - other map
      aggregateFn - function to aggregate the values.
      copyFn - function to copy the value

    • aggregateCounters

      public static Long aggregateCounters(Long l, Long r)
      Aggregate two counters.
      Parameters:
      l - left value
      r - right value
      Returns:
      the aggregate value

    • aggregateGauges

      public static Long aggregateGauges(Long l, Long r)
      Add two gauges.
      Parameters:
      l - left value
      r - right value
      Returns:
      aggregate value

    • aggregateMinimums

      public static Long aggregateMinimums(Long l, Long r)
      Aggregate two minimum values.
      Parameters:
      l - left
      r - right
      Returns:
      the new minimum.

    • aggregateMaximums

      public static Long aggregateMaximums(Long l, Long r)
      Aggregate two maximum values.
      Parameters:
      l - left
      r - right
      Returns:
      the new minimum.

    • aggregateMeanStatistics

      public static MeanStatistic aggregateMeanStatistics(MeanStatistic l, MeanStatistic r)
      Aggregate the mean statistics. This returns a new instance.
      Parameters:
      l - left value
      r - right value
      Returns:
      aggregate value

    • maybeUpdateMaximum

      public static void maybeUpdateMaximum(AtomicLong dest, long sample)
      Update a maximum value tracked in an atomic long. This is thread safe -it uses compareAndSet to ensure that Thread T1 whose sample is greater than the current value never overwrites an update from thread T2 whose sample was also higher -and which completed first.
      Parameters:
      dest - destination for all changes.
      sample - sample to update.

    • maybeUpdateMinimum

      public static void maybeUpdateMinimum(AtomicLong dest, long sample)
      Update a maximum value tracked in an atomic long. This is thread safe -it uses compareAndSet to ensure that Thread T1 whose sample is greater than the current value never overwrites an update from thread T2 whose sample was also higher -and which completed first.
      Parameters:
      dest - destination for all changes.
      sample - sample to update.

    • trackFunctionDuration

      public static <A, B> FunctionRaisingIOE<A,B> trackFunctionDuration(@Nullable DurationTrackerFactory factory, String statistic, FunctionRaisingIOE<A,B> inputFn)
      Given an IOException raising function/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
      Type Parameters:
      A - type of argument to the input function.
      B - return type.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      inputFn - input function
      Returns:
      a new function which tracks duration and failure.

    • trackJavaFunctionDuration

      public static <A, B> Function<A,B> trackJavaFunctionDuration(@Nullable DurationTrackerFactory factory, String statistic, Function<A,B> inputFn)
      Given a java function/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
      Type Parameters:
      A - type of argument to the input function.
      B - return type.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      inputFn - input function
      Returns:
      a new function which tracks duration and failure.

    • trackDuration

      public static <B> B trackDuration(DurationTrackerFactory factory, String statistic, CallableRaisingIOE<B> input) throws IOException
      Given an IOException raising callable/lambda expression, execute it and update the relevant statistic.
      Type Parameters:
      B - return type.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      input - input callable.
      Returns:
      the result of the operation.
      Throws:
      IOException - raised on errors performing I/O.

    • trackDurationOfInvocation

      public static void trackDurationOfInvocation(DurationTrackerFactory factory, String statistic, InvocationRaisingIOE input) throws IOException
      Given an IOException raising callable/lambda expression, execute it and update the relevant statistic.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      input - input callable.
      Throws:
      IOException - IO failure.

    • measureDurationOfInvocation

      public static Duration measureDurationOfInvocation(DurationTrackerFactory factory, String statistic, InvocationRaisingIOE input) throws IOException
      Given an IOException raising callable/lambda expression, execute it and update the relevant statistic, returning the measured duration. trackDurationOfInvocation(DurationTrackerFactory, String, InvocationRaisingIOE) with the duration returned for logging etc.; added as a new method to avoid linking problems with any code calling the existing method.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      input - input callable.
      Returns:
      the duration of the operation, as measured by the duration tracker.
      Throws:
      IOException - IO failure.

    • trackDurationOfOperation

      public static <B> CallableRaisingIOE<B> trackDurationOfOperation(@Nullable DurationTrackerFactory factory, String statistic, CallableRaisingIOE<B> input)
      Given an IOException raising callable/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
      Type Parameters:
      B - return type.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      input - input callable.
      Returns:
      a new callable which tracks duration and failure.

    • invokeTrackingDuration

      public static <B> B invokeTrackingDuration(DurationTracker tracker, CallableRaisingIOE<B> input) throws IOException
      Given an IOException raising callable/lambda expression, execute it, updating the tracker on success/failure.
      Type Parameters:
      B - return type.
      Parameters:
      tracker - duration tracker.
      input - input callable.
      Returns:
      the result of the invocation
      Throws:
      IOException - on failure.

    • trackDurationConsumer

      public static <B> ConsumerRaisingIOE<B> trackDurationConsumer(@Nullable DurationTrackerFactory factory, String statistic, ConsumerRaisingIOE<B> input)
      Given an IOException raising Consumer, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
      Type Parameters:
      B - return type.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      input - input callable.
      Returns:
      a new consumer which tracks duration and failure.

    • trackDurationOfCallable

      public static <B> Callable<B> trackDurationOfCallable(@Nullable DurationTrackerFactory factory, String statistic, Callable<B> input)
      Given a callable/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
      Type Parameters:
      B - return type.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      input - input callable.
      Returns:
      a new callable which tracks duration and failure.

    • trackDurationOfSupplier

      public static <B> B trackDurationOfSupplier(@Nullable DurationTrackerFactory factory, String statistic, Supplier<B> input)
      Given a Java supplier, evaluate it while tracking the duration of the operation and success/failure.
      Type Parameters:
      B - return type.
      Parameters:
      factory - factory of duration trackers
      statistic - statistic key
      input - input callable.
      Returns:
      the output of the supplier.

    • createTracker

      public static DurationTracker createTracker(@Nullable DurationTrackerFactory factory, String statistic)
      Create the tracker. If the factory is null, a stub tracker is returned.
      Parameters:
      factory - tracker factory
      statistic - statistic to track
      Returns:
      a duration tracker.

    • pairedTrackerFactory

      public static DurationTrackerFactory pairedTrackerFactory(DurationTrackerFactory first, DurationTrackerFactory second)
      Create a DurationTrackerFactory which aggregates the tracking of two other factories.
      Parameters:
      first - first tracker factory
      second - second tracker factory
      Returns:
      a factory

    • publishAsStorageStatistics

      public static StorageStatistics publishAsStorageStatistics(String name, String scheme, IOStatistics source)
      Publish the IOStatistics as a set of storage statistics. This is dynamic.
      Parameters:
      name - storage statistics name.
      scheme - FS scheme; may be null.
      source - IOStatistics source.
      Returns:
      a dynamic storage statistics object.