Package org.apache.hadoop.hdfs.server.datanode

Class DataStorage

java.lang.Object

org.apache.hadoop.hdfs.server.common.StorageInfo

org.apache.hadoop.hdfs.server.common.Storage

org.apache.hadoop.hdfs.server.datanode.DataStorage

@Private
public class DataStorage
extends Storage

Data storage information file.

See Also:

• Storage

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DataStorage.VolumeBuilder	VolumeBuilder holds the metadata (e.g., the storage directories) of the prepared volume returned from prepareVolume(DataNode, StorageLocation, List).

Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.common.Storage

Storage.FormatConfirmable, Storage.StorageDirectory, Storage.StorageDirType, Storage.StorageState

Field Summary

Fields

Modifier and Type	Field	Description
static final String	BLOCK_SUBDIR_PREFIX
static final String	STORAGE_DIR_FINALIZED
static final String	STORAGE_DIR_LAZY_PERSIST
static final String	STORAGE_DIR_RBW
static final String	STORAGE_DIR_TMP

Fields inherited from class org.apache.hadoop.hdfs.server.common.Storage

LAST_PRE_UPGRADE_LAYOUT_VERSION, LAST_UPGRADABLE_HADOOP_VERSION, LAST_UPGRADABLE_LAYOUT_VERSION, LAYOUT_VERSIONS_203, LOG, STORAGE_1_BBW, STORAGE_DIR_CURRENT, STORAGE_DIR_PREVIOUS, STORAGE_FILE_LOCK, STORAGE_PREVIOUS_CKPT, STORAGE_TMP_FINALIZED, STORAGE_TMP_LAST_CKPT, STORAGE_TMP_PREVIOUS, STORAGE_TMP_REMOVED

Fields inherited from class org.apache.hadoop.hdfs.server.common.StorageInfo

clusterID, cTime, layoutVersion, namespaceID, STORAGE_FILE_VERSION, storageType

Constructor Summary

Constructors

Constructor	Description
DataStorage(StorageInfo storageInfo)

Method Summary

Modifier and Type	Method	Description
void	clearRollingUpgradeMarker(String bpid)
void	clearTrash(String bpid)
static boolean	createStorageID(Storage.StorageDirectory sd, boolean regenerateStorageIds, org.apache.hadoop.conf.Configuration conf)	Create an ID for this storage.
void	enableTrash(String bpid)	Enable trash for the specified block pool storage.
static boolean	fullyDelete(File dir)	Deprecated.
BlockPoolSliceStorage	getBPStorage(String bpid)
String	getDatanodeUuid()
String	getTrashDirectoryForReplica(String bpid, ReplicaInfo info)	If rolling upgrades are in progress then do not delete block files immediately.
boolean	isPreUpgradableLayout(Storage.StorageDirectory sd)	Return true if the layout of the given storage directory is from a version of Hadoop prior to the introduction of the "current" and "previous" directories which allow upgrade and rollback.
DataStorage.VolumeBuilder	prepareVolume(DataNode datanode, StorageLocation location, List<NamespaceInfo> nsInfos)	Prepare a storage directory.
void	setDatanodeUuid(String newDatanodeUuid)
protected void	setFieldsFromProperties(Properties props, Storage.StorageDirectory sd)	Get common storage fields.
protected void	setPropertiesFromFields(Properties props, Storage.StorageDirectory sd)	Set common storage fields into the given properties object.
void	setRollingUpgradeMarker(String bpid)
boolean	trashEnabled(String bpid)

Methods inherited from class org.apache.hadoop.hdfs.server.common.Storage

addStorageDir, checkVersionUpgradable, confirmFormat, containsStorageDir, containsStorageDir, containsStorageDir, deleteDir, dirIterable, dirIterator, dirIterator, dirIterator, dirIterator, getBuildVersion, getFiles, getNamespaceInfo, getNumStorageDirs, getRegistrationID, getSingularStorageDir, getStorageDir, getStorageDirs, is203LayoutVersion, listStorageDirectories, nativeCopyFileUnbuffered, rename, unlockAll, writeAll, writeProperties, writeProperties, writeProperties

Methods inherited from class org.apache.hadoop.hdfs.server.common.StorageInfo

checkStorageType, getClusterID, getClusterIdFromColonSeparatedString, getCTime, getLayoutVersion, getNamespaceID, getNsIdFromColonSeparatedString, getProperty, getServiceLayoutFeatureMap, getServiceLayoutVersion, readPreviousVersionProperties, readProperties, readPropertiesFile, setClusterId, setcTime, setLayoutVersion, setNamespaceID, setServiceLayoutVersion, setStorageInfo, toColonSeparatedString, toMapString, toString, versionSupportsFederation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

BLOCK_SUBDIR_PREFIX

public static final String BLOCK_SUBDIR_PREFIX

See Also:

• Constant Field Values

STORAGE_DIR_RBW

public static final String STORAGE_DIR_RBW

See Also:

• Constant Field Values

STORAGE_DIR_FINALIZED

public static final String STORAGE_DIR_FINALIZED

See Also:

• Constant Field Values

STORAGE_DIR_LAZY_PERSIST

public static final String STORAGE_DIR_LAZY_PERSIST

See Also:

• Constant Field Values

STORAGE_DIR_TMP

public static final String STORAGE_DIR_TMP

See Also:

• Constant Field Values

Constructor Details

DataStorage

public DataStorage(StorageInfo storageInfo)

Method Details

getBPStorage

public BlockPoolSliceStorage getBPStorage(String bpid)

getDatanodeUuid

public String getDatanodeUuid()

setDatanodeUuid

public void setDatanodeUuid(String newDatanodeUuid)

createStorageID

public static boolean createStorageID(Storage.StorageDirectory sd,
 boolean regenerateStorageIds,
 org.apache.hadoop.conf.Configuration conf)

Create an ID for this storage.

Returns:

true if a new storage ID was generated.

enableTrash

public void enableTrash(String bpid)

Enable trash for the specified block pool storage. Even if trash is enabled by the caller, it is superseded by the 'previous' directory if a layout upgrade is in progress.

clearTrash

public void clearTrash(String bpid)

trashEnabled

public boolean trashEnabled(String bpid)

setRollingUpgradeMarker

public void setRollingUpgradeMarker(String bpid)
                             throws IOException

Throws:

IOException

clearRollingUpgradeMarker

public void clearRollingUpgradeMarker(String bpid)
                               throws IOException

Throws:

IOException

getTrashDirectoryForReplica

public String getTrashDirectoryForReplica(String bpid,
 ReplicaInfo info)

If rolling upgrades are in progress then do not delete block files immediately. Instead we move the block files to an intermediate 'trash' directory. If there is a subsequent rollback, then the block files will be restored from trash.

Returns:

trash directory if rolling upgrade is in progress, null otherwise.

prepareVolume

public DataStorage.VolumeBuilder prepareVolume(DataNode datanode,
 StorageLocation location,
 List<NamespaceInfo> nsInfos)
                                        throws IOException

Prepare a storage directory. It creates a builder which can be used to add to the volume. If the volume cannot be added, it is OK to discard the builder later.

Parameters:

datanode - DataNode object.

location - the StorageLocation for the storage directory.

nsInfos - an array of namespace infos.

Returns:

a VolumeBuilder that holds the metadata of this storage directory and can be added to DataStorage later.

Throws:

IOException - if encounters I/O errors. Note that if there is IOException, the state of DataStorage is not modified.

setPropertiesFromFields

protected void setPropertiesFromFields(Properties props,
 Storage.StorageDirectory sd)
                                throws IOException

Description copied from class: Storage

Set common storage fields into the given properties object. Should be overloaded if additional fields need to be set.

Overrides:

setPropertiesFromFields in class Storage

Parameters:

props - the Properties object to write into

Throws:

IOException

setFieldsFromProperties

protected void setFieldsFromProperties(Properties props,
 Storage.StorageDirectory sd)
                                throws IOException

Description copied from class: StorageInfo

Get common storage fields. Should be overloaded if additional fields need to be get.

Overrides:

setFieldsFromProperties in class StorageInfo

Parameters:

props - properties

Throws:

IOException - on error

isPreUpgradableLayout

public boolean isPreUpgradableLayout(Storage.StorageDirectory sd)
                              throws IOException

Description copied from class: Storage

Return true if the layout of the given storage directory is from a version of Hadoop prior to the introduction of the "current" and "previous" directories which allow upgrade and rollback.

Specified by:

isPreUpgradableLayout in class Storage

Throws:

IOException

fullyDelete

@Deprecated
public static boolean fullyDelete(File dir)

Deprecated.

Prefer FileIoProvider#fullydelete.

Parameters:

dir -

Returns: