Package org.apache.hadoop.hdfs.server.datanode.checker

Class DatasetVolumeChecker

java.lang.Object

org.apache.hadoop.hdfs.server.datanode.checker.DatasetVolumeChecker

public class DatasetVolumeChecker
extends Object

A class that encapsulates running disk checks against each volume of an FsDatasetSpi and allows retrieving a list of failed volumes. This splits out behavior that was originally implemented across DataNode, FsDatasetImpl and FsVolumeList.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	DatasetVolumeChecker.Callback	A callback interface that is supplied the result of running an async disk check on multiple volumes.

Field Summary

Fields

Modifier and Type	Field	Description
static final org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
DatasetVolumeChecker(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.util.Timer timer)

Method Summary

Modifier and Type	Method	Description
Set<FsVolumeSpi>	checkAllVolumes(FsDatasetSpi<? extends FsVolumeSpi> dataset)	Run checks against all volumes of a dataset.
boolean	checkVolume(FsVolumeSpi volume, DatasetVolumeChecker.Callback callback)	Check a single volume asynchronously, returning a ListenableFuture that can be used to retrieve the final result.
long	getNumSkippedChecks()	Return the number of checks skipped because the minimum gap since the last check had not elapsed.
long	getNumSyncDatasetChecks()	Return the number of checkAllVolumes(org.apache.hadoop.hdfs.server.datanode.fsdataset.FsDatasetSpi<? extends org.apache.hadoop.hdfs.server.datanode.fsdataset.FsVolumeSpi>) invocations.
long	getNumVolumeChecks()	Return the number of checkVolume(org.apache.hadoop.hdfs.server.datanode.fsdataset.FsVolumeSpi, org.apache.hadoop.hdfs.server.datanode.checker.DatasetVolumeChecker.Callback) invocations.
void	shutdownAndWait(int gracePeriod, TimeUnit timeUnit)	Shutdown the checker and its associated ExecutorService.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

public static final org.slf4j.Logger LOG

Constructor Details

DatasetVolumeChecker

public DatasetVolumeChecker(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.util.Timer timer)
                     throws org.apache.hadoop.util.DiskChecker.DiskErrorException

Parameters:

conf - Configuration object.

timer - Timer object used for throttling checks.

Throws:

org.apache.hadoop.util.DiskChecker.DiskErrorException

Method Details

checkAllVolumes

public Set<FsVolumeSpi> checkAllVolumes(FsDatasetSpi<? extends FsVolumeSpi> dataset)
                                 throws InterruptedException

Run checks against all volumes of a dataset. This check may be performed at service startup and subsequently at regular intervals to detect and handle failed volumes.

Parameters:

dataset - - FsDatasetSpi to be checked.

Returns:

set of failed volumes.

Throws:

InterruptedException

checkVolume

public boolean checkVolume(FsVolumeSpi volume,
 DatasetVolumeChecker.Callback callback)

Check a single volume asynchronously, returning a ListenableFuture that can be used to retrieve the final result. If the volume cannot be referenced then it is already closed and cannot be checked. No error is propagated to the callback.

Parameters:

volume - the volume that is to be checked.

callback - callback to be invoked when the volume check completes.

Returns:

true if the check was scheduled and the callback will be invoked. false otherwise.

shutdownAndWait

public void shutdownAndWait(int gracePeriod,
 TimeUnit timeUnit)

Shutdown the checker and its associated ExecutorService. See ExecutorService.awaitTermination(long, java.util.concurrent.TimeUnit) for the interpretation of the parameters.

getNumVolumeChecks

public long getNumVolumeChecks()

Return the number of checkVolume(org.apache.hadoop.hdfs.server.datanode.fsdataset.FsVolumeSpi, org.apache.hadoop.hdfs.server.datanode.checker.DatasetVolumeChecker.Callback) invocations.

getNumSyncDatasetChecks

public long getNumSyncDatasetChecks()

Return the number of checkAllVolumes(org.apache.hadoop.hdfs.server.datanode.fsdataset.FsDatasetSpi<? extends org.apache.hadoop.hdfs.server.datanode.fsdataset.FsVolumeSpi>) invocations.

getNumSkippedChecks

public long getNumSkippedChecks()

Return the number of checks skipped because the minimum gap since the last check had not elapsed.