Package org.apache.hadoop.hdfs

Class ViewDistributedFileSystem

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.hdfs.DistributedFileSystem

org.apache.hadoop.hdfs.ViewDistributedFileSystem

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.crypto.key.KeyProviderTokenIssuer, org.apache.hadoop.fs.BatchListingOperations, org.apache.hadoop.fs.BulkDeleteSource, org.apache.hadoop.fs.LeaseRecoverable, org.apache.hadoop.fs.PathCapabilities, org.apache.hadoop.fs.SafeMode, org.apache.hadoop.fs.WithErasureCoding, org.apache.hadoop.security.token.DelegationTokenIssuer

@Private
@Unstable
public class ViewDistributedFileSystem
extends DistributedFileSystem

The ViewDistributedFileSystem is an extended class to DistributedFileSystem with additional mounting functionality. The goal is to have better API compatibility for HDFS users when using mounting filesystem(ViewFileSystemOverloadScheme). The ViewFileSystemOverloadSchemeViewFileSystemOverloadScheme is a new filesystem with inherited mounting functionality from ViewFileSystem. For the user who is using ViewFileSystemOverloadScheme by setting fs.hdfs.impl=org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme, now they can set fs.hdfs.impl=org.apache.hadoop.hdfs.ViewDistributedFileSystem. So, that the hdfs users will get closely compatible API with mount functionality. For the rest of all other schemes can continue to use ViewFileSystemOverloadScheme class directly for mount functionality. Please note that ViewFileSystemOverloadScheme provides only ViewFileSystemViewFileSystem APIs. If user configured this class but no mount point configured? Then it will simply work as existing DistributedFileSystem class. If user configured both fs.hdfs.impl to this class and mount configurations, then users will be able to make calls the APIs available in this class, they are nothing but DFS APIs, but they will be delegated to viewfs functionality. Please note, APIs without any path in arguments( ex: isInSafeMode), will be delegated to default filesystem only, that is the configured fallback link. If you want to make these API calls on specific child filesystem, you may want to initialize them separately and call. In ViewDistributedFileSystem, we strongly recommend to configure linkFallBack when you add mount links and it's recommended to point be to your base cluster, usually your current fs.defaultFS if that's pointing to hdfs.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.DistributedFileSystem

DistributedFileSystem.HdfsDataOutputStreamBuilder

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.FileSystem.DirectoryEntries, org.apache.hadoop.fs.FileSystem.Statistics

Field Summary

Fields inherited from class org.apache.hadoop.fs.FileSystem

DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX

Fields inherited from interface org.apache.hadoop.security.token.DelegationTokenIssuer

TOKEN_LOG

Constructor Summary

Constructors

Constructor	Description
ViewDistributedFileSystem()

Method Summary

Modifier and Type	Method	Description
void	access(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode)
long	addCacheDirective(CacheDirectiveInfo info)
long	addCacheDirective(CacheDirectiveInfo info, EnumSet<CacheFlag> flags)	Add a new CacheDirective.
void	addCachePool(CachePoolInfo info)	Add a cache pool.
AddErasureCodingPolicyResponse[]	addErasureCodingPolicies(ErasureCodingPolicy[] policies)	Add Erasure coding policies to HDFS.
void	allowSnapshot(org.apache.hadoop.fs.Path path)
org.apache.hadoop.fs.FSDataOutputStream	append(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress)
org.apache.hadoop.fs.FSDataOutputStream	append(org.apache.hadoop.fs.Path f, EnumSet<org.apache.hadoop.fs.CreateFlag> flag, int bufferSize, org.apache.hadoop.util.Progressable progress)	Append to an existing file (optional operation).
org.apache.hadoop.fs.FSDataOutputStream	append(org.apache.hadoop.fs.Path f, EnumSet<org.apache.hadoop.fs.CreateFlag> flag, int bufferSize, org.apache.hadoop.util.Progressable progress, InetSocketAddress[] favoredNodes)	Append to an existing file (optional operation).
DistributedFileSystem.HdfsDataOutputStreamBuilder	appendFile(org.apache.hadoop.fs.Path path)	Create a DistributedFileSystem.HdfsDataOutputStreamBuilder to append a file on DFS.
org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.PartialListing<org.apache.hadoop.fs.LocatedFileStatus>>	batchedListLocatedStatusIterator(List<org.apache.hadoop.fs.Path> paths)
org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.PartialListing<org.apache.hadoop.fs.FileStatus>>	batchedListStatusIterator(List<org.apache.hadoop.fs.Path> paths)
protected URI	canonicalizeUri(URI uri)
void	close()
void	concat(org.apache.hadoop.fs.Path trg, org.apache.hadoop.fs.Path[] psrcs)	Move blocks from srcs to trg and delete srcs afterwards.
org.apache.hadoop.fs.FSDataOutputStream	create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
HdfsDataOutputStream	create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, InetSocketAddress[] favoredNodes)	Same as DistributedFileSystem.create(Path, FsPermission, boolean, int, short, long, Progressable) with the addition of favoredNodes that is a hint to where the namenode should place the file blocks.
org.apache.hadoop.fs.FSDataOutputStream	create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> cflags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt)
void	createEncryptionZone(org.apache.hadoop.fs.Path path, String keyName)
DistributedFileSystem.HdfsDataOutputStreamBuilder	createFile(org.apache.hadoop.fs.Path path)	Create a HdfsDataOutputStreamBuilder to create a file on DFS.
org.apache.hadoop.fs.FSDataOutputStream	createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)	Same as create(), except fails if parent directory doesn't already exist.
protected HdfsPathHandle	createPathHandle(org.apache.hadoop.fs.FileStatus st, org.apache.hadoop.fs.Options.HandleOpt... opts)	Create a handle to an HDFS file.
org.apache.hadoop.fs.Path	createSnapshot(org.apache.hadoop.fs.Path path, String snapshotName)
void	createSymlink(org.apache.hadoop.fs.Path target, org.apache.hadoop.fs.Path link, boolean createParent)
boolean	delete(org.apache.hadoop.fs.Path f)
boolean	delete(org.apache.hadoop.fs.Path f, boolean recursive)
void	deleteSnapshot(org.apache.hadoop.fs.Path path, String snapshotName)
void	disableErasureCodingPolicy(String ecPolicyName)	Disable erasure coding policy.
void	disallowSnapshot(org.apache.hadoop.fs.Path path)
void	enableErasureCodingPolicy(String ecPolicyName)	Enable erasure coding policy.
void	finalizeUpgrade()	Finalize previously upgraded files system state.
protected org.apache.hadoop.fs.Path	fixRelativePart(org.apache.hadoop.fs.Path p)
org.apache.hadoop.fs.permission.AclStatus	getAclStatus(org.apache.hadoop.fs.Path path)
org.apache.hadoop.security.token.DelegationTokenIssuer[]	getAdditionalTokenIssuers()
Map<String,String>	getAllErasureCodingCodecs()	Retrieve all the erasure coding codecs and coders supported by this file system.
Collection<ErasureCodingPolicyInfo>	getAllErasureCodingPolicies()	Gets all erasure coding policies from all available child file systems.
Collection<BlockStoragePolicy>	getAllStoragePolicies()
long	getBytesWithFutureGenerationStamps()	Returns number of bytes within blocks with future generation stamp.
String	getCanonicalServiceName()	Get a canonical service name for this file system.
org.apache.hadoop.fs.FileSystem[]	getChildFileSystems()
DFSClient	getClient()
org.apache.hadoop.fs.ContentSummary	getContentSummary(org.apache.hadoop.fs.Path f)
long	getCorruptBlocksCount()	Returns count of blocks with at least one replica marked corrupt.
DatanodeInfo[]	getDataNodeStats()
DatanodeInfo[]	getDataNodeStats(HdfsConstants.DatanodeReportType type)
long	getDefaultBlockSize(org.apache.hadoop.fs.Path f)
protected int	getDefaultPort()
short	getDefaultReplication(org.apache.hadoop.fs.Path f)
org.apache.hadoop.security.token.Token<DelegationTokenIdentifier>	getDelegationToken(String renewer)	If no mount points configured, it works same as DistributedFileSystem.getDelegationToken(String).
ECTopologyVerifierResult	getECTopologyResultForPolicies(String... policyNames)	Verifies if the given policies are supported in the given cluster setup.
ErasureCodingPolicy	getErasureCodingPolicy(org.apache.hadoop.fs.Path path)	Get erasure coding policy information for the specified path.
EncryptionZone	getEZForPath(org.apache.hadoop.fs.Path path)
org.apache.hadoop.fs.BlockLocation[]	getFileBlockLocations(org.apache.hadoop.fs.FileStatus fs, long start, long len)
org.apache.hadoop.fs.BlockLocation[]	getFileBlockLocations(org.apache.hadoop.fs.Path p, long start, long len)	The returned BlockLocation will have different formats for replicated and erasure coded file.
org.apache.hadoop.fs.FileChecksum	getFileChecksum(org.apache.hadoop.fs.Path f)
org.apache.hadoop.fs.FileChecksum	getFileChecksum(org.apache.hadoop.fs.Path f, long length)
org.apache.hadoop.fs.FileEncryptionInfo	getFileEncryptionInfo(org.apache.hadoop.fs.Path path)
org.apache.hadoop.fs.FileStatus	getFileLinkStatus(org.apache.hadoop.fs.Path f)
org.apache.hadoop.fs.FileStatus	getFileStatus(org.apache.hadoop.fs.Path f)	Returns the stat information about the file.
DFSHedgedReadMetrics	getHedgedReadMetrics()	Returns only default cluster getHedgedReadMetrics.
org.apache.hadoop.fs.Path	getHomeDirectory()
DFSInotifyEventInputStream	getInotifyEventStream()
DFSInotifyEventInputStream	getInotifyEventStream(long lastReadTxid)
org.apache.hadoop.crypto.key.KeyProvider	getKeyProvider()
URI	getKeyProviderUri()
org.apache.hadoop.fs.Path	getLinkTarget(org.apache.hadoop.fs.Path path)
long	getLowRedundancyBlocksCount()	Returns aggregated count of blocks with less redundancy.
long	getMissingBlocksCount()	Returns count of blocks with no good replicas left.
long	getMissingReplOneBlocksCount()	Returns count of blocks with replication factor 1 and have lost the only replica.
org.apache.hadoop.fs.viewfs.ViewFileSystem.MountPoint[]	getMountPoints()
long	getPendingDeletionBlocksCount()	Returns count of blocks pending on deletion.
org.apache.hadoop.fs.QuotaUsage	getQuotaUsage(org.apache.hadoop.fs.Path f)
String	getScheme()	Return the protocol scheme for the FileSystem.
org.apache.hadoop.fs.FsServerDefaults	getServerDefaults()
org.apache.hadoop.fs.FsServerDefaults	getServerDefaults(org.apache.hadoop.fs.Path f)
DatanodeInfo[]	getSlowDatanodeStats()	Retrieve stats for slow running datanodes.
SnapshotDiffReport	getSnapshotDiffReport(org.apache.hadoop.fs.Path snapshotDir, String fromSnapshot, String toSnapshot)	Get the difference between two snapshots, or between a snapshot and the current tree of a directory.
SnapshottableDirectoryStatus[]	getSnapshottableDirListing()	Get the list of snapshottable directories that are owned by the current user.
org.apache.hadoop.fs.FsStatus	getStatus()
org.apache.hadoop.fs.FsStatus	getStatus(org.apache.hadoop.fs.Path p)
BlockStoragePolicy[]	getStoragePolicies()	Deprecated.
org.apache.hadoop.fs.BlockStoragePolicySpi	getStoragePolicy(org.apache.hadoop.fs.Path src)
org.apache.hadoop.fs.Path	getTrashRoot(org.apache.hadoop.fs.Path path)	Get the root directory of Trash for a path in HDFS. 1.
Collection<org.apache.hadoop.fs.FileStatus>	getTrashRoots(boolean allUsers)	Get all the trash roots of HDFS for current user or for all the users. 1.
URI	getUri()
long	getUsed()
org.apache.hadoop.fs.Path	getWorkingDirectory()
byte[]	getXAttr(org.apache.hadoop.fs.Path path, String name)
Map<String,byte[]>	getXAttrs(org.apache.hadoop.fs.Path path)
Map<String,byte[]>	getXAttrs(org.apache.hadoop.fs.Path path, List<String> names)
boolean	hasPathCapability(org.apache.hadoop.fs.Path path, String capability)	HDFS client capabilities.
void	initialize(URI uri, org.apache.hadoop.conf.Configuration conf)
boolean	isFileClosed(org.apache.hadoop.fs.Path src)	Get the close status of a file
boolean	isInSafeMode()	Utility function that returns if the NameNode is in safemode or not.
org.apache.hadoop.fs.RemoteIterator<CacheDirectiveEntry>	listCacheDirectives(CacheDirectiveInfo filter)	List cache directives.
org.apache.hadoop.fs.RemoteIterator<CachePoolEntry>	listCachePools()	List all cache pools.
org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path>	listCorruptFileBlocks(org.apache.hadoop.fs.Path path)
org.apache.hadoop.fs.RemoteIterator<EncryptionZone>	listEncryptionZones()	Returns the results from default DFS (fallback).
org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus>	listLocatedStatus(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.PathFilter filter)	The BlockLocation of returned LocatedFileStatus will have different formats for replicated and erasure coded file.
org.apache.hadoop.fs.RemoteIterator<OpenFileEntry>	listOpenFiles()	Deprecated.
org.apache.hadoop.fs.RemoteIterator<OpenFileEntry>	listOpenFiles(EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes)	Deprecated.
org.apache.hadoop.fs.RemoteIterator<OpenFileEntry>	listOpenFiles(EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes, String path)
org.apache.hadoop.fs.RemoteIterator<ZoneReencryptionStatus>	listReencryptionStatus()	Returns the results from default DFS (fallback).
org.apache.hadoop.fs.FileStatus[]	listStatus(org.apache.hadoop.fs.Path p)	List all the entries of a directory Note that this operation is not atomic for a large directory.
org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus>	listStatusIterator(org.apache.hadoop.fs.Path p)	Returns a remote iterator so that followup calls are made on demand while consuming the entries.
List<String>	listXAttrs(org.apache.hadoop.fs.Path path)
void	metaSave(String pathname)
boolean	mkdir(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission)	Create a directory, only when the parent directories exist.
boolean	mkdirs(org.apache.hadoop.fs.Path dir)
boolean	mkdirs(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission)	Create a directory and its parent directories.
void	modifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
void	modifyCacheDirective(CacheDirectiveInfo info)
void	modifyCacheDirective(CacheDirectiveInfo info, EnumSet<CacheFlag> flags)	Modify a CacheDirective.
void	modifyCachePool(CachePoolInfo info)	Modify an existing cache pool.
org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.PathHandle fd, int bufferSize)	Opens an FSDataInputStream with the indicated file ID extracted from the PathHandle.
org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.Path f, int bufferSize)
protected HdfsDataOutputStream	primitiveCreate(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission absolutePermission, EnumSet<org.apache.hadoop.fs.CreateFlag> flag, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt)
protected boolean	primitiveMkdir(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission absolutePermission)
void	provisionEZTrash(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsPermission trashPermission)
org.apache.hadoop.fs.Path	provisionSnapshotTrash(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsPermission trashPermission)	HDFS only.
boolean	recoverLease(org.apache.hadoop.fs.Path f)	Start the lease recovery of a file
void	reencryptEncryptionZone(org.apache.hadoop.fs.Path zone, HdfsConstants.ReencryptAction action)
void	refreshNodes()	Refreshes the list of hosts and excluded hosts from the configured files.
void	removeAcl(org.apache.hadoop.fs.Path path)
void	removeAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
void	removeCacheDirective(long id)	Remove a CacheDirectiveInfo.
void	removeCachePool(String poolName)	Remove a cache pool.
void	removeDefaultAcl(org.apache.hadoop.fs.Path path)
void	removeErasureCodingPolicy(String ecPolicyName)	Remove erasure coding policy.
void	removeXAttr(org.apache.hadoop.fs.Path path, String name)
boolean	rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)
void	rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, org.apache.hadoop.fs.Options.Rename... options)	This rename operation is guaranteed to be atomic.
void	renameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName)
protected org.apache.hadoop.fs.Path	resolveLink(org.apache.hadoop.fs.Path f)
org.apache.hadoop.fs.Path	resolvePath(org.apache.hadoop.fs.Path f)
boolean	restoreFailedStorage(String arg)	enable/disable/check restoreFaileStorage.
long	rollEdits()	Rolls the edit log on the active NameNode.
RollingUpgradeInfo	rollingUpgrade(HdfsConstants.RollingUpgradeAction action)	Rolling upgrade: prepare/finalize/query.
void	satisfyStoragePolicy(org.apache.hadoop.fs.Path src)	Set the source path to satisfy storage policy.
void	saveNamespace()	Save namespace image.
boolean	saveNamespace(long timeWindow, long txGap)	Save namespace image.
void	setAcl(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
void	setBalancerBandwidth(long bandwidth)	Requests the namenode to tell all datanodes to use a new, non-persistent bandwidth value for dfs.datanode.balance.bandwidthPerSec.
void	setErasureCodingPolicy(org.apache.hadoop.fs.Path path, String ecPolicyName)	Set the source path to the specified erasure coding policy.
void	setOwner(org.apache.hadoop.fs.Path f, String username, String groupname)
void	setPermission(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission)
void	setQuota(org.apache.hadoop.fs.Path src, long namespaceQuota, long storagespaceQuota)	Set a directory's quotas
void	setQuotaByStorageType(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.StorageType type, long quota)	Set the per type storage quota of a directory.
boolean	setReplication(org.apache.hadoop.fs.Path f, short replication)
boolean	setSafeMode(HdfsConstants.SafeModeAction action)	Enter, leave or get safe mode.
boolean	setSafeMode(HdfsConstants.SafeModeAction action, boolean isChecked)	Enter, leave or get safe mode.
void	setStoragePolicy(org.apache.hadoop.fs.Path src, String policyName)	Set the source path to the specified storage policy.
void	setTimes(org.apache.hadoop.fs.Path f, long mtime, long atime)
void	setVerifyChecksum(boolean verifyChecksum)
void	setWorkingDirectory(org.apache.hadoop.fs.Path dir)
void	setWriteChecksum(boolean writeChecksum)
void	setXAttr(org.apache.hadoop.fs.Path path, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag)
org.apache.hadoop.fs.RemoteIterator<SnapshotDiffReportListing>	snapshotDiffReportListingRemoteIterator(org.apache.hadoop.fs.Path snapshotDir, String fromSnapshot, String toSnapshot)	Returns a remote iterator so that followup calls are made on demand while consuming the SnapshotDiffReportListing entries.
boolean	supportsSymlinks()
boolean	truncate(org.apache.hadoop.fs.Path f, long newLength)
void	unsetErasureCodingPolicy(org.apache.hadoop.fs.Path path)	Unset the erasure coding policy from the source path.
void	unsetStoragePolicy(org.apache.hadoop.fs.Path src)
boolean	upgradeStatus()	Get status of upgrade - finalized or not.

Methods inherited from class org.apache.hadoop.hdfs.DistributedFileSystem

append, createMultipartUploader, getDefaultBlockSize, getDefaultReplication, getEnclosingRoot, getErasureCodingPolicyName, getLocatedBlocks, getSnapshotDiffReportListing, getSnapshotListing, isSnapshotTrashRootEnabled, msync, setSafeMode, setSafeMode, toString

Methods inherited from class org.apache.hadoop.fs.FileSystem

append, append, append, areSymlinksEnabled, cancelDeleteOnExit, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, createBulkDelete, createDataInputStreamBuilder, createDataInputStreamBuilder, createDataOutputStreamBuilder, createNewFile, createNonRecursive, createNonRecursive, createSnapshot, deleteOnExit, enableSymlinks, exists, get, get, get, getAllStatistics, getBlockSize, getCanonicalUri, getDefaultUri, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getInitialWorkingDirectory, getLength, getLocal, getName, getNamed, getPathHandle, getReplication, getStatistics, getStatistics, getStorageStatistics, getUsed, globStatus, globStatus, isDirectory, isFile, listFiles, listLocatedStatus, listStatus, listStatus, listStatus, listStatusBatch, makeQualified, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, open, openFile, openFile, openFileWithOptions, openFileWithOptions, primitiveMkdir, printStatistics, processDeleteOnExit, setDefaultUri, setDefaultUri, setXAttr, startLocalOutput

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.security.token.DelegationTokenIssuer

addDelegationTokens

Constructor Details

ViewDistributedFileSystem

public ViewDistributedFileSystem()

Method Details

initialize

public void initialize(URI uri,
 org.apache.hadoop.conf.Configuration conf)
                throws IOException

Overrides:

initialize in class DistributedFileSystem

Throws:

IOException

getUri

public URI getUri()

Overrides:

getUri in class DistributedFileSystem

getScheme

public String getScheme()

Description copied from class: DistributedFileSystem

Return the protocol scheme for the FileSystem.

Overrides:

getScheme in class DistributedFileSystem

Returns:

hdfs

getWorkingDirectory

public org.apache.hadoop.fs.Path getWorkingDirectory()

Overrides:

getWorkingDirectory in class DistributedFileSystem

setWorkingDirectory

public void setWorkingDirectory(org.apache.hadoop.fs.Path dir)

Overrides:

setWorkingDirectory in class DistributedFileSystem

getHomeDirectory

public org.apache.hadoop.fs.Path getHomeDirectory()

Overrides:

getHomeDirectory in class DistributedFileSystem

getHedgedReadMetrics

public DFSHedgedReadMetrics getHedgedReadMetrics()

Returns only default cluster getHedgedReadMetrics.

Overrides:

getHedgedReadMetrics in class DistributedFileSystem

Returns:

object of DFSHedgedReadMetrics

getFileBlockLocations

public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.FileStatus fs,
 long start,
 long len)
                                                           throws IOException

Overrides:

getFileBlockLocations in class DistributedFileSystem

Throws:

IOException

getFileBlockLocations

public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.Path p,
 long start,
 long len)
                                                           throws IOException

Description copied from class: DistributedFileSystem

The returned BlockLocation will have different formats for replicated and erasure coded file. Please refer to FileSystem.getFileBlockLocations(FileStatus, long, long) for more details.

Overrides:

getFileBlockLocations in class DistributedFileSystem

Throws:

IOException

setVerifyChecksum

public void setVerifyChecksum(boolean verifyChecksum)

Overrides:

setVerifyChecksum in class DistributedFileSystem

recoverLease

public boolean recoverLease(org.apache.hadoop.fs.Path f)
                     throws IOException

Description copied from class: DistributedFileSystem

Start the lease recovery of a file

Specified by:

recoverLease in interface org.apache.hadoop.fs.LeaseRecoverable

Overrides:

recoverLease in class DistributedFileSystem

Parameters:

f - a file

Returns:

true if the file is already closed

Throws:

IOException - if an error occurs

open

public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path f,
 int bufferSize)
                                            throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

open in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

open

public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.PathHandle fd,
 int bufferSize)
                                            throws IOException

Description copied from class: DistributedFileSystem

Opens an FSDataInputStream with the indicated file ID extracted from the PathHandle.

Overrides:

open in class DistributedFileSystem

Parameters:

fd - Reference to entity in this FileSystem.

bufferSize - the size of the buffer to be used.

Throws:

org.apache.hadoop.fs.InvalidPathHandleException - If PathHandle constraints do not hold

IOException - On I/O errors

createPathHandle

protected HdfsPathHandle createPathHandle(org.apache.hadoop.fs.FileStatus st,
 org.apache.hadoop.fs.Options.HandleOpt... opts)

Description copied from class: DistributedFileSystem

Create a handle to an HDFS file.

Overrides:

createPathHandle in class DistributedFileSystem

Parameters:

st - HdfsFileStatus instance from NameNode

opts - Standard handle arguments

Returns:

A handle to the file.

append

public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f,
 int bufferSize,
 org.apache.hadoop.util.Progressable progress)
                                               throws IOException

Overrides:

append in class DistributedFileSystem

Throws:

IOException

append

public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f,
 EnumSet<org.apache.hadoop.fs.CreateFlag> flag,
 int bufferSize,
 org.apache.hadoop.util.Progressable progress)
                                               throws IOException

Description copied from class: DistributedFileSystem

Append to an existing file (optional operation).

Overrides:

append in class DistributedFileSystem

Parameters:

f - the existing file to be appended.

flag - Flags for the Append operation. CreateFlag.APPEND is mandatory to be present.

bufferSize - the size of the buffer to be used.

progress - for reporting progress if it is not null.

Returns:

Returns instance of FSDataOutputStream

Throws:

IOException

append

public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f,
 EnumSet<org.apache.hadoop.fs.CreateFlag> flag,
 int bufferSize,
 org.apache.hadoop.util.Progressable progress,
 InetSocketAddress[] favoredNodes)
                                               throws IOException

Description copied from class: DistributedFileSystem

Append to an existing file (optional operation).

Overrides:

append in class DistributedFileSystem

Parameters:

f - the existing file to be appended.

flag - Flags for the Append operation. CreateFlag.APPEND is mandatory to be present.

bufferSize - the size of the buffer to be used.

progress - for reporting progress if it is not null.

favoredNodes - Favored nodes for new blocks

Returns:

Returns instance of FSDataOutputStream

Throws:

IOException

create

public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission permission,
 boolean overwrite,
 int bufferSize,
 short replication,
 long blockSize,
 org.apache.hadoop.util.Progressable progress)
                                               throws IOException

Overrides:

create in class DistributedFileSystem

Throws:

IOException

create

public HdfsDataOutputStream create(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission permission,
 boolean overwrite,
 int bufferSize,
 short replication,
 long blockSize,
 org.apache.hadoop.util.Progressable progress,
 InetSocketAddress[] favoredNodes)
                            throws IOException

Description copied from class: DistributedFileSystem

Same as DistributedFileSystem.create(Path, FsPermission, boolean, int, short, long, Progressable) with the addition of favoredNodes that is a hint to where the namenode should place the file blocks. The favored nodes hint is not persisted in HDFS. Hence it may be honored at the creation time only. And with favored nodes, blocks will be pinned on the datanodes to prevent balancing move the block. HDFS could move the blocks during replication, to move the blocks from favored nodes. A value of null means no favored nodes for this create

Overrides:

create in class DistributedFileSystem

Throws:

IOException

create

public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission permission,
 EnumSet<org.apache.hadoop.fs.CreateFlag> cflags,
 int bufferSize,
 short replication,
 long blockSize,
 org.apache.hadoop.util.Progressable progress,
 org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt)
                                               throws IOException

Overrides:

create in class DistributedFileSystem

Throws:

IOException

primitiveCreate

protected HdfsDataOutputStream primitiveCreate(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission absolutePermission,
 EnumSet<org.apache.hadoop.fs.CreateFlag> flag,
 int bufferSize,
 short replication,
 long blockSize,
 org.apache.hadoop.util.Progressable progress,
 org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt)
                                        throws IOException

Overrides:

primitiveCreate in class DistributedFileSystem

Throws:

IOException

createNonRecursive

public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission permission,
 EnumSet<org.apache.hadoop.fs.CreateFlag> flags,
 int bufferSize,
 short replication,
 long blockSize,
 org.apache.hadoop.util.Progressable progress)
                                                           throws IOException

Description copied from class: DistributedFileSystem

Same as create(), except fails if parent directory doesn't already exist.

Overrides:

createNonRecursive in class DistributedFileSystem

Throws:

IOException

setReplication

public boolean setReplication(org.apache.hadoop.fs.Path f,
 short replication)
                       throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

setReplication in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

setStoragePolicy

public void setStoragePolicy(org.apache.hadoop.fs.Path src,
 String policyName)
                      throws IOException

Description copied from class: DistributedFileSystem

Set the source path to the specified storage policy.

Overrides:

setStoragePolicy in class DistributedFileSystem

Parameters:

src - The source path referring to either a directory or a file.

policyName - The name of the storage policy.

Throws:

IOException

unsetStoragePolicy

public void unsetStoragePolicy(org.apache.hadoop.fs.Path src)
                        throws IOException

Overrides:

unsetStoragePolicy in class DistributedFileSystem

Throws:

IOException

getStoragePolicy

public org.apache.hadoop.fs.BlockStoragePolicySpi getStoragePolicy(org.apache.hadoop.fs.Path src)
                                                            throws IOException

Overrides:

getStoragePolicy in class DistributedFileSystem

Throws:

IOException

getAllStoragePolicies

public Collection<BlockStoragePolicy> getAllStoragePolicies()
                                                     throws IOException

Overrides:

getAllStoragePolicies in class DistributedFileSystem

Throws:

IOException

getBytesWithFutureGenerationStamps

public long getBytesWithFutureGenerationStamps()
                                        throws IOException

Description copied from class: DistributedFileSystem

Returns number of bytes within blocks with future generation stamp. These are bytes that will be potentially deleted if we forceExit from safe mode.

Overrides:

getBytesWithFutureGenerationStamps in class DistributedFileSystem

Returns:

number of bytes.

Throws:

IOException

getStoragePolicies

@Deprecated
public BlockStoragePolicy[] getStoragePolicies()
                                        throws IOException

Deprecated.

Description copied from class: DistributedFileSystem

Deprecated. Prefer FileSystem.getAllStoragePolicies()

Overrides:

getStoragePolicies in class DistributedFileSystem

Throws:

IOException

concat

public void concat(org.apache.hadoop.fs.Path trg,
 org.apache.hadoop.fs.Path[] psrcs)
            throws IOException

Description copied from class: DistributedFileSystem

Move blocks from srcs to trg and delete srcs afterwards. The file block sizes must be the same.

Overrides:

concat in class DistributedFileSystem

Parameters:

trg - existing file to append to

psrcs - list of files (same block size, same replication)

Throws:

IOException

rename

public boolean rename(org.apache.hadoop.fs.Path src,
 org.apache.hadoop.fs.Path dst)
               throws IOException

Overrides:

rename in class DistributedFileSystem

Throws:

IOException

rename

public void rename(org.apache.hadoop.fs.Path src,
 org.apache.hadoop.fs.Path dst,
 org.apache.hadoop.fs.Options.Rename... options)
            throws IOException

Description copied from class: DistributedFileSystem

This rename operation is guaranteed to be atomic.

Overrides:

rename in class DistributedFileSystem

Throws:

IOException

truncate

public boolean truncate(org.apache.hadoop.fs.Path f,
 long newLength)
                 throws IOException

Overrides:

truncate in class DistributedFileSystem

Throws:

IOException

delete

public boolean delete(org.apache.hadoop.fs.Path f,
 boolean recursive)
               throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

delete in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

getContentSummary

public org.apache.hadoop.fs.ContentSummary getContentSummary(org.apache.hadoop.fs.Path f)
                                                      throws IOException

Overrides:

getContentSummary in class DistributedFileSystem

Throws:

IOException

getQuotaUsage

public org.apache.hadoop.fs.QuotaUsage getQuotaUsage(org.apache.hadoop.fs.Path f)
                                              throws IOException

Overrides:

getQuotaUsage in class DistributedFileSystem

Throws:

IOException

setQuota

public void setQuota(org.apache.hadoop.fs.Path src,
 long namespaceQuota,
 long storagespaceQuota)
              throws IOException

Description copied from class: DistributedFileSystem

Set a directory's quotas

Overrides:

setQuota in class DistributedFileSystem

Throws:

IOException

See Also:

• ClientProtocol.setQuota(String, long, long, StorageType)

setQuotaByStorageType

public void setQuotaByStorageType(org.apache.hadoop.fs.Path src,
 org.apache.hadoop.fs.StorageType type,
 long quota)
                           throws IOException

Description copied from class: DistributedFileSystem

Set the per type storage quota of a directory.

Overrides:

setQuotaByStorageType in class DistributedFileSystem

Parameters:

src - target directory whose quota is to be modified.

type - storage type of the specific storage type quota to be modified.

quota - value of the specific storage type quota to be modified. Maybe HdfsConstants.QUOTA_RESET to clear quota by storage type.

Throws:

IOException

listStatus

public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path p)
                                             throws IOException

Description copied from class: DistributedFileSystem

List all the entries of a directory Note that this operation is not atomic for a large directory. The entries of a directory may be fetched from NameNode multiple times. It only guarantees that each name occurs once if a directory undergoes changes between the calls. If any of the the immediate children of the given path f is a symlink, the returned FileStatus object of that children would be represented as a symlink. It will not be resolved to the target path and will not get the target path FileStatus object. The target path will be available via getSymlink on that children's FileStatus object. Since it represents as symlink, isDirectory on that children's FileStatus will return false. If you want to get the FileStatus of target path for that children, you may want to use GetFileStatus API with that children's symlink path. Please see DistributedFileSystem.getFileStatus(Path f)

Overrides:

listStatus in class DistributedFileSystem

Throws:

IOException

listLocatedStatus

public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listLocatedStatus(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.PathFilter filter)
                                                                                              throws FileNotFoundException,
IOException

Description copied from class: DistributedFileSystem

The BlockLocation of returned LocatedFileStatus will have different formats for replicated and erasure coded file. Please refer to FileSystem.getFileBlockLocations(FileStatus, long, long) for more details.

Overrides:

listLocatedStatus in class DistributedFileSystem

Throws:

FileNotFoundException

IOException

listStatusIterator

public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> listStatusIterator(org.apache.hadoop.fs.Path p)
                                                                                        throws IOException

Description copied from class: DistributedFileSystem

Returns a remote iterator so that followup calls are made on demand while consuming the entries. This reduces memory consumption during listing of a large directory.

Overrides:

listStatusIterator in class DistributedFileSystem

Parameters:

p - target path

Returns:

remote iterator

Throws:

IOException

batchedListStatusIterator

public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.PartialListing<org.apache.hadoop.fs.FileStatus>> batchedListStatusIterator(List<org.apache.hadoop.fs.Path> paths)
                                                                                                                                    throws IOException

Specified by:

batchedListStatusIterator in interface org.apache.hadoop.fs.BatchListingOperations

Overrides:

batchedListStatusIterator in class DistributedFileSystem

Throws:

IOException

batchedListLocatedStatusIterator

public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.PartialListing<org.apache.hadoop.fs.LocatedFileStatus>> batchedListLocatedStatusIterator(List<org.apache.hadoop.fs.Path> paths)
                                                                                                                                                  throws IOException

Specified by:

batchedListLocatedStatusIterator in interface org.apache.hadoop.fs.BatchListingOperations

Overrides:

batchedListLocatedStatusIterator in class DistributedFileSystem

Throws:

IOException

mkdir

public boolean mkdir(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission permission)
              throws IOException

Description copied from class: DistributedFileSystem

Create a directory, only when the parent directories exist. See FsPermission.applyUMask(FsPermission) for details of how the permission is applied.

Overrides:

mkdir in class DistributedFileSystem

Parameters:

f - The path to create

permission - The permission. See FsPermission#applyUMask for details about how this is used to calculate the effective permission.

Throws:

IOException

mkdirs

public boolean mkdirs(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission permission)
               throws IOException

Description copied from class: DistributedFileSystem

Create a directory and its parent directories. See FsPermission.applyUMask(FsPermission) for details of how the permission is applied.

Overrides:

mkdirs in class DistributedFileSystem

Parameters:

f - The path to create

permission - The permission. See FsPermission#applyUMask for details about how this is used to calculate the effective permission.

Throws:

IOException

primitiveMkdir

protected boolean primitiveMkdir(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission absolutePermission)
                          throws IOException

Overrides:

primitiveMkdir in class DistributedFileSystem

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class DistributedFileSystem

Throws:

IOException

getClient

@Private
public DFSClient getClient()

Overrides:

getClient in class DistributedFileSystem

getStatus

public org.apache.hadoop.fs.FsStatus getStatus(org.apache.hadoop.fs.Path p)
                                        throws IOException

Overrides:

getStatus in class DistributedFileSystem

Throws:

IOException

getMissingBlocksCount

public long getMissingBlocksCount()
                           throws IOException

Description copied from class: DistributedFileSystem

Returns count of blocks with no good replicas left. Normally should be zero.

Overrides:

getMissingBlocksCount in class DistributedFileSystem

Throws:

IOException

getPendingDeletionBlocksCount

public long getPendingDeletionBlocksCount()
                                   throws IOException

Description copied from class: DistributedFileSystem

Returns count of blocks pending on deletion.

Overrides:

getPendingDeletionBlocksCount in class DistributedFileSystem

Throws:

IOException

getMissingReplOneBlocksCount

public long getMissingReplOneBlocksCount()
                                  throws IOException

Description copied from class: DistributedFileSystem

Returns count of blocks with replication factor 1 and have lost the only replica.

Overrides:

getMissingReplOneBlocksCount in class DistributedFileSystem

Throws:

IOException

getLowRedundancyBlocksCount

public long getLowRedundancyBlocksCount()
                                 throws IOException

Description copied from class: DistributedFileSystem

Returns aggregated count of blocks with less redundancy.

Overrides:

getLowRedundancyBlocksCount in class DistributedFileSystem

Throws:

IOException

getCorruptBlocksCount

public long getCorruptBlocksCount()
                           throws IOException

Description copied from class: DistributedFileSystem

Returns count of blocks with at least one replica marked corrupt.

Overrides:

getCorruptBlocksCount in class DistributedFileSystem

Throws:

IOException

listCorruptFileBlocks

public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path> listCorruptFileBlocks(org.apache.hadoop.fs.Path path)
                                                                                     throws IOException

Overrides:

listCorruptFileBlocks in class DistributedFileSystem

Throws:

IOException

getDataNodeStats

public DatanodeInfo[] getDataNodeStats()
                                throws IOException

Overrides:

getDataNodeStats in class DistributedFileSystem

Returns:

datanode statistics.

Throws:

IOException

getDataNodeStats

public DatanodeInfo[] getDataNodeStats(HdfsConstants.DatanodeReportType type)
                                throws IOException

Overrides:

getDataNodeStats in class DistributedFileSystem

Returns:

datanode statistics for the given type.

Throws:

IOException

setSafeMode

public boolean setSafeMode(HdfsConstants.SafeModeAction action)
                    throws IOException

Description copied from class: DistributedFileSystem

Enter, leave or get safe mode.

Overrides:

setSafeMode in class DistributedFileSystem

Throws:

IOException

See Also:

• ClientProtocol.setSafeMode(HdfsConstants.SafeModeAction, boolean)

setSafeMode

public boolean setSafeMode(HdfsConstants.SafeModeAction action,
 boolean isChecked)
                    throws IOException

Description copied from class: DistributedFileSystem

Enter, leave or get safe mode.

Overrides:

setSafeMode in class DistributedFileSystem

Parameters:

action - One of SafeModeAction.ENTER, SafeModeAction.LEAVE and SafeModeAction.GET.

isChecked - If true check only for Active NNs status, else check first NN's status.

Throws:

IOException

See Also:

• ClientProtocol.setSafeMode(HdfsConstants.SafeModeAction, boolean)

saveNamespace

public boolean saveNamespace(long timeWindow,
 long txGap)
                      throws IOException

Description copied from class: DistributedFileSystem

Save namespace image.

Overrides:

saveNamespace in class DistributedFileSystem

Parameters:

timeWindow - NameNode can ignore this command if the latest checkpoint was done within the given time period (in seconds).

Returns:

true if a new checkpoint has been made

Throws:

IOException

See Also:

• ClientProtocol.saveNamespace(long, long)

saveNamespace

public void saveNamespace()
                   throws IOException

Description copied from class: DistributedFileSystem

Save namespace image. NameNode always does the checkpoint.

Overrides:

saveNamespace in class DistributedFileSystem

Throws:

IOException

rollEdits

public long rollEdits()
               throws IOException

Description copied from class: DistributedFileSystem

Rolls the edit log on the active NameNode. Requires super-user privileges.

Overrides:

rollEdits in class DistributedFileSystem

Returns:

the transaction ID of the newly created segment

Throws:

IOException

See Also:

• ClientProtocol.rollEdits()

restoreFailedStorage

public boolean restoreFailedStorage(String arg)
                             throws IOException

Description copied from class: DistributedFileSystem

enable/disable/check restoreFaileStorage.

Overrides:

restoreFailedStorage in class DistributedFileSystem

Throws:

IOException

See Also:

• ClientProtocol.restoreFailedStorage(String arg)

refreshNodes

public void refreshNodes()
                  throws IOException

Description copied from class: DistributedFileSystem

Refreshes the list of hosts and excluded hosts from the configured files.

Overrides:

refreshNodes in class DistributedFileSystem

Throws:

IOException

finalizeUpgrade

public void finalizeUpgrade()
                     throws IOException

Description copied from class: DistributedFileSystem

Finalize previously upgraded files system state.

Overrides:

finalizeUpgrade in class DistributedFileSystem

Throws:

IOException

upgradeStatus

public boolean upgradeStatus()
                      throws IOException

Description copied from class: DistributedFileSystem

Get status of upgrade - finalized or not.

Overrides:

upgradeStatus in class DistributedFileSystem

Returns:

true if upgrade is finalized or if no upgrade is in progress and false otherwise.

Throws:

IOException

rollingUpgrade

public RollingUpgradeInfo rollingUpgrade(HdfsConstants.RollingUpgradeAction action)
                                  throws IOException

Description copied from class: DistributedFileSystem

Rolling upgrade: prepare/finalize/query.

Overrides:

rollingUpgrade in class DistributedFileSystem

Throws:

IOException

metaSave

public void metaSave(String pathname)
              throws IOException

Overrides:

metaSave in class DistributedFileSystem

Throws:

IOException

getServerDefaults

public org.apache.hadoop.fs.FsServerDefaults getServerDefaults()
                                                        throws IOException

Overrides:

getServerDefaults in class DistributedFileSystem

Throws:

IOException

getFileStatus

public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path f)
                                              throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Description copied from class: DistributedFileSystem

Returns the stat information about the file. If the given path is a symlink, the path will be resolved to a target path and it will get the resolved path's FileStatus object. It will not be represented as a symlink and isDirectory API returns true if the resolved path is a directory, false otherwise.

Overrides:

getFileStatus in class DistributedFileSystem

Throws:

FileNotFoundException - if the file does not exist.

org.apache.hadoop.security.AccessControlException

IOException

createSymlink

public void createSymlink(org.apache.hadoop.fs.Path target,
 org.apache.hadoop.fs.Path link,
 boolean createParent)
                   throws IOException

Overrides:

createSymlink in class DistributedFileSystem

Throws:

IOException

supportsSymlinks

public boolean supportsSymlinks()

Overrides:

supportsSymlinks in class DistributedFileSystem

getFileLinkStatus

public org.apache.hadoop.fs.FileStatus getFileLinkStatus(org.apache.hadoop.fs.Path f)
                                                  throws IOException

Overrides:

getFileLinkStatus in class DistributedFileSystem

Throws:

IOException

getLinkTarget

public org.apache.hadoop.fs.Path getLinkTarget(org.apache.hadoop.fs.Path path)
                                        throws IOException

Overrides:

getLinkTarget in class DistributedFileSystem

Throws:

IOException

resolveLink

protected org.apache.hadoop.fs.Path resolveLink(org.apache.hadoop.fs.Path f)
                                         throws IOException

Overrides:

resolveLink in class DistributedFileSystem

Throws:

IOException

getFileChecksum

public org.apache.hadoop.fs.FileChecksum getFileChecksum(org.apache.hadoop.fs.Path f)
                                                  throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

getFileChecksum in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

setPermission

public void setPermission(org.apache.hadoop.fs.Path f,
 org.apache.hadoop.fs.permission.FsPermission permission)
                   throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

setPermission in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

setOwner

public void setOwner(org.apache.hadoop.fs.Path f,
 String username,
 String groupname)
              throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

setOwner in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

setTimes

public void setTimes(org.apache.hadoop.fs.Path f,
 long mtime,
 long atime)
              throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

setTimes in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

getDefaultPort

protected int getDefaultPort()

Overrides:

getDefaultPort in class DistributedFileSystem

getDelegationToken

public org.apache.hadoop.security.token.Token<DelegationTokenIdentifier> getDelegationToken(String renewer)
                                                                                     throws IOException

If no mount points configured, it works same as DistributedFileSystem.getDelegationToken(String). If there are mount points configured and if default fs(linkFallback) configured, then it will return default fs delegation token. Otherwise it will return null.

Specified by:

getDelegationToken in interface org.apache.hadoop.security.token.DelegationTokenIssuer

Overrides:

getDelegationToken in class DistributedFileSystem

Throws:

IOException

setBalancerBandwidth

public void setBalancerBandwidth(long bandwidth)
                          throws IOException

Description copied from class: DistributedFileSystem

Requests the namenode to tell all datanodes to use a new, non-persistent bandwidth value for dfs.datanode.balance.bandwidthPerSec. The bandwidth parameter is the max number of bytes per second of network bandwidth to be used by a datanode during balancing.

Overrides:

setBalancerBandwidth in class DistributedFileSystem

Parameters:

bandwidth - Balancer bandwidth in bytes per second for all datanodes.

Throws:

IOException

getCanonicalServiceName

public String getCanonicalServiceName()

Description copied from class: DistributedFileSystem

Get a canonical service name for this file system. If the URI is logical, the hostname part of the URI will be returned.

Specified by:

getCanonicalServiceName in interface org.apache.hadoop.security.token.DelegationTokenIssuer

Overrides:

getCanonicalServiceName in class DistributedFileSystem

Returns:

a service string that uniquely identifies this file system.

canonicalizeUri

protected URI canonicalizeUri(URI uri)

Overrides:

canonicalizeUri in class DistributedFileSystem

isInSafeMode

public boolean isInSafeMode()
                     throws IOException

Description copied from class: DistributedFileSystem

Utility function that returns if the NameNode is in safemode or not. In HA mode, this API will return only ActiveNN's safemode status.

Overrides:

isInSafeMode in class DistributedFileSystem

Returns:

true if NameNode is in safemode, false otherwise.

Throws:

IOException - when there is an issue communicating with the NameNode

allowSnapshot

public void allowSnapshot(org.apache.hadoop.fs.Path path)
                   throws IOException

Overrides:

allowSnapshot in class DistributedFileSystem

Throws:

IOException

See Also:

• HdfsAdmin.allowSnapshot(Path)

disallowSnapshot

public void disallowSnapshot(org.apache.hadoop.fs.Path path)
                      throws IOException

Overrides:

disallowSnapshot in class DistributedFileSystem

Throws:

IOException

See Also:

• HdfsAdmin.disallowSnapshot(Path)

createSnapshot

public org.apache.hadoop.fs.Path createSnapshot(org.apache.hadoop.fs.Path path,
 String snapshotName)
                                         throws IOException

Overrides:

createSnapshot in class DistributedFileSystem

Throws:

IOException

renameSnapshot

public void renameSnapshot(org.apache.hadoop.fs.Path path,
 String snapshotOldName,
 String snapshotNewName)
                    throws IOException

Overrides:

renameSnapshot in class DistributedFileSystem

Throws:

IOException

getSnapshottableDirListing

public SnapshottableDirectoryStatus[] getSnapshottableDirListing()
                                                          throws IOException

Description copied from class: DistributedFileSystem

Get the list of snapshottable directories that are owned by the current user. Return all the snapshottable directories if the current user is a super user.

Overrides:

getSnapshottableDirListing in class DistributedFileSystem

Returns:

The list of all the current snapshottable directories.

Throws:

IOException - If an I/O error occurred.

deleteSnapshot

public void deleteSnapshot(org.apache.hadoop.fs.Path path,
 String snapshotName)
                    throws IOException

Overrides:

deleteSnapshot in class DistributedFileSystem

Throws:

IOException

snapshotDiffReportListingRemoteIterator

public org.apache.hadoop.fs.RemoteIterator<SnapshotDiffReportListing> snapshotDiffReportListingRemoteIterator(org.apache.hadoop.fs.Path snapshotDir,
 String fromSnapshot,
 String toSnapshot)
                                                                                                       throws IOException

Description copied from class: DistributedFileSystem

Returns a remote iterator so that followup calls are made on demand while consuming the SnapshotDiffReportListing entries. This reduces memory consumption overhead in case the snapshotDiffReport is huge.

Overrides:

snapshotDiffReportListingRemoteIterator in class DistributedFileSystem

Parameters:

snapshotDir - full path of the directory where snapshots are taken

fromSnapshot - snapshot name of the from point. Null indicates the current tree

toSnapshot - snapshot name of the to point. Null indicates the current tree.

Returns:

Remote iterator

Throws:

IOException

getSnapshotDiffReport

public SnapshotDiffReport getSnapshotDiffReport(org.apache.hadoop.fs.Path snapshotDir,
 String fromSnapshot,
 String toSnapshot)
                                         throws IOException

Description copied from class: DistributedFileSystem

Get the difference between two snapshots, or between a snapshot and the current tree of a directory.

Overrides:

getSnapshotDiffReport in class DistributedFileSystem

Throws:

IOException

See Also:

• DFSClient.getSnapshotDiffReportListing(java.lang.String, java.lang.String, java.lang.String, byte[], int)

isFileClosed

public boolean isFileClosed(org.apache.hadoop.fs.Path src)
                     throws IOException

Description copied from class: DistributedFileSystem

Get the close status of a file

Specified by:

isFileClosed in interface org.apache.hadoop.fs.LeaseRecoverable

Overrides:

isFileClosed in class DistributedFileSystem

Parameters:

src - The path to the file

Returns:

return true if file is closed

Throws:

FileNotFoundException - if the file does not exist.

IOException - If an I/O error occurred

addCacheDirective

public long addCacheDirective(CacheDirectiveInfo info)
                       throws IOException

Overrides:

addCacheDirective in class DistributedFileSystem

Throws:

IOException

See Also:

• DistributedFileSystem.addCacheDirective(CacheDirectiveInfo, EnumSet)

addCacheDirective

public long addCacheDirective(CacheDirectiveInfo info,
 EnumSet<CacheFlag> flags)
                       throws IOException

Description copied from class: DistributedFileSystem

Add a new CacheDirective.

Overrides:

addCacheDirective in class DistributedFileSystem

Parameters:

info - Information about a directive to add.

flags - CacheFlags to use for this operation.

Returns:

the ID of the directive that was created.

Throws:

IOException - if the directive could not be added

modifyCacheDirective

public void modifyCacheDirective(CacheDirectiveInfo info)
                          throws IOException

Overrides:

modifyCacheDirective in class DistributedFileSystem

Throws:

IOException

See Also:

• DistributedFileSystem.modifyCacheDirective(CacheDirectiveInfo, EnumSet)

modifyCacheDirective

public void modifyCacheDirective(CacheDirectiveInfo info,
 EnumSet<CacheFlag> flags)
                          throws IOException

Description copied from class: DistributedFileSystem

Modify a CacheDirective.

Overrides:

modifyCacheDirective in class DistributedFileSystem

Parameters:

info - Information about the directive to modify. You must set the ID to indicate which CacheDirective you want to modify.

flags - CacheFlags to use for this operation.

Throws:

IOException - if the directive could not be modified

removeCacheDirective

public void removeCacheDirective(long id)
                          throws IOException

Description copied from class: DistributedFileSystem

Remove a CacheDirectiveInfo.

Overrides:

removeCacheDirective in class DistributedFileSystem

Parameters:

id - identifier of the CacheDirectiveInfo to remove

Throws:

IOException - if the directive could not be removed

listCacheDirectives

public org.apache.hadoop.fs.RemoteIterator<CacheDirectiveEntry> listCacheDirectives(CacheDirectiveInfo filter)
                                                                             throws IOException

Description copied from class: DistributedFileSystem

List cache directives. Incrementally fetches results from the server.

Overrides:

listCacheDirectives in class DistributedFileSystem

Parameters:

filter - Filter parameters to use when listing the directives, null to list all directives visible to us.

Returns:

A RemoteIterator which returns CacheDirectiveInfo objects.

Throws:

IOException

addCachePool

public void addCachePool(CachePoolInfo info)
                  throws IOException

Description copied from class: DistributedFileSystem

Add a cache pool.

Overrides:

addCachePool in class DistributedFileSystem

Parameters:

info - The request to add a cache pool.

Throws:

IOException - If the request could not be completed.

modifyCachePool

public void modifyCachePool(CachePoolInfo info)
                     throws IOException

Description copied from class: DistributedFileSystem

Modify an existing cache pool.

Overrides:

modifyCachePool in class DistributedFileSystem

Parameters:

info - The request to modify a cache pool.

Throws:

IOException - If the request could not be completed.

removeCachePool

public void removeCachePool(String poolName)
                     throws IOException

Description copied from class: DistributedFileSystem

Remove a cache pool.

Overrides:

removeCachePool in class DistributedFileSystem

Parameters:

poolName - Name of the cache pool to remove.

Throws:

IOException - if the cache pool did not exist, or could not be removed.

listCachePools

public org.apache.hadoop.fs.RemoteIterator<CachePoolEntry> listCachePools()
                                                                   throws IOException

Description copied from class: DistributedFileSystem

List all cache pools.

Overrides:

listCachePools in class DistributedFileSystem

Returns:

A remote iterator from which you can get CachePoolEntry objects. Requests will be made as needed.

Throws:

IOException - If there was an error listing cache pools.

modifyAclEntries

public void modifyAclEntries(org.apache.hadoop.fs.Path path,
 List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
                      throws IOException

Description copied from class: DistributedFileSystem

Overrides:

modifyAclEntries in class DistributedFileSystem

Throws:

IOException

removeAclEntries

public void removeAclEntries(org.apache.hadoop.fs.Path path,
 List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
                      throws IOException

Description copied from class: DistributedFileSystem

Overrides:

removeAclEntries in class DistributedFileSystem

Throws:

IOException

removeDefaultAcl

public void removeDefaultAcl(org.apache.hadoop.fs.Path path)
                      throws IOException

Description copied from class: DistributedFileSystem

Overrides:

removeDefaultAcl in class DistributedFileSystem

Throws:

IOException

removeAcl

public void removeAcl(org.apache.hadoop.fs.Path path)
               throws IOException

Description copied from class: DistributedFileSystem

Overrides:

removeAcl in class DistributedFileSystem

Throws:

IOException

setAcl

public void setAcl(org.apache.hadoop.fs.Path path,
 List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
            throws IOException

Description copied from class: DistributedFileSystem

Overrides:

setAcl in class DistributedFileSystem

Throws:

IOException

getAclStatus

public org.apache.hadoop.fs.permission.AclStatus getAclStatus(org.apache.hadoop.fs.Path path)
                                                       throws IOException

Description copied from class: DistributedFileSystem

Overrides:

getAclStatus in class DistributedFileSystem

Throws:

IOException

createEncryptionZone

public void createEncryptionZone(org.apache.hadoop.fs.Path path,
 String keyName)
                          throws IOException

Overrides:

createEncryptionZone in class DistributedFileSystem

Throws:

IOException

getEZForPath

public EncryptionZone getEZForPath(org.apache.hadoop.fs.Path path)
                            throws IOException

Overrides:

getEZForPath in class DistributedFileSystem

Throws:

IOException

listEncryptionZones

public org.apache.hadoop.fs.RemoteIterator<EncryptionZone> listEncryptionZones()
                                                                        throws IOException

Returns the results from default DFS (fallback). If you want the results from specific clusters, please invoke them on child fs instance directly.

Overrides:

listEncryptionZones in class DistributedFileSystem

Throws:

IOException

reencryptEncryptionZone

public void reencryptEncryptionZone(org.apache.hadoop.fs.Path zone,
 HdfsConstants.ReencryptAction action)
                             throws IOException

Overrides:

reencryptEncryptionZone in class DistributedFileSystem

Throws:

IOException

listReencryptionStatus

public org.apache.hadoop.fs.RemoteIterator<ZoneReencryptionStatus> listReencryptionStatus()
                                                                                   throws IOException

Returns the results from default DFS (fallback). If you want the results from specific clusters, please invoke them on child fs instance directly.

Overrides:

listReencryptionStatus in class DistributedFileSystem

Throws:

IOException

getFileEncryptionInfo

public org.apache.hadoop.fs.FileEncryptionInfo getFileEncryptionInfo(org.apache.hadoop.fs.Path path)
                                                              throws IOException

Overrides:

getFileEncryptionInfo in class DistributedFileSystem

Throws:

IOException

provisionEZTrash

public void provisionEZTrash(org.apache.hadoop.fs.Path path,
 org.apache.hadoop.fs.permission.FsPermission trashPermission)
                      throws IOException

Overrides:

provisionEZTrash in class DistributedFileSystem

Throws:

IOException

provisionSnapshotTrash

public org.apache.hadoop.fs.Path provisionSnapshotTrash(org.apache.hadoop.fs.Path path,
 org.apache.hadoop.fs.permission.FsPermission trashPermission)
                                                 throws IOException

Description copied from class: DistributedFileSystem

HDFS only. Provision snapshottable directory trash.

Overrides:

provisionSnapshotTrash in class DistributedFileSystem

Parameters:

path - Path to a snapshottable directory.

trashPermission - Expected FsPermission of the trash root.

Returns:

Path of the provisioned trash root

Throws:

IOException

setXAttr

public void setXAttr(org.apache.hadoop.fs.Path path,
 String name,
 byte[] value,
 EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag)
              throws IOException

Overrides:

setXAttr in class DistributedFileSystem

Throws:

IOException

getXAttr

public byte[] getXAttr(org.apache.hadoop.fs.Path path,
 String name)
                throws IOException

Overrides:

getXAttr in class DistributedFileSystem

Throws:

IOException

getXAttrs

public Map<String,byte[]> getXAttrs(org.apache.hadoop.fs.Path path)
                             throws IOException

Overrides:

getXAttrs in class DistributedFileSystem

Throws:

IOException

getXAttrs

public Map<String,byte[]> getXAttrs(org.apache.hadoop.fs.Path path,
 List<String> names)
                             throws IOException

Overrides:

getXAttrs in class DistributedFileSystem

Throws:

IOException

listXAttrs

public List<String> listXAttrs(org.apache.hadoop.fs.Path path)
                        throws IOException

Overrides:

listXAttrs in class DistributedFileSystem

Throws:

IOException

removeXAttr

public void removeXAttr(org.apache.hadoop.fs.Path path,
 String name)
                 throws IOException

Overrides:

removeXAttr in class DistributedFileSystem

Throws:

IOException

access

public void access(org.apache.hadoop.fs.Path path,
 org.apache.hadoop.fs.permission.FsAction mode)
            throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

access in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

getKeyProviderUri

public URI getKeyProviderUri()
                      throws IOException

Specified by:

getKeyProviderUri in interface org.apache.hadoop.crypto.key.KeyProviderTokenIssuer

Overrides:

getKeyProviderUri in class DistributedFileSystem

Throws:

IOException

getKeyProvider

public org.apache.hadoop.crypto.key.KeyProvider getKeyProvider()
                                                        throws IOException

Specified by:

getKeyProvider in interface org.apache.hadoop.crypto.key.KeyProviderTokenIssuer

Overrides:

getKeyProvider in class DistributedFileSystem

Throws:

IOException

getAdditionalTokenIssuers

public org.apache.hadoop.security.token.DelegationTokenIssuer[] getAdditionalTokenIssuers()
                                                                                   throws IOException

Specified by:

getAdditionalTokenIssuers in interface org.apache.hadoop.security.token.DelegationTokenIssuer

Overrides:

getAdditionalTokenIssuers in class DistributedFileSystem

Throws:

IOException

getInotifyEventStream

public DFSInotifyEventInputStream getInotifyEventStream()
                                                 throws IOException

Overrides:

getInotifyEventStream in class DistributedFileSystem

Throws:

IOException

getInotifyEventStream

public DFSInotifyEventInputStream getInotifyEventStream(long lastReadTxid)
                                                 throws IOException

Overrides:

getInotifyEventStream in class DistributedFileSystem

Throws:

IOException

setErasureCodingPolicy

public void setErasureCodingPolicy(org.apache.hadoop.fs.Path path,
 String ecPolicyName)
                            throws IOException

Description copied from class: DistributedFileSystem

Set the source path to the specified erasure coding policy.

Specified by:

setErasureCodingPolicy in interface org.apache.hadoop.fs.WithErasureCoding

Overrides:

setErasureCodingPolicy in class DistributedFileSystem

Parameters:

path - The directory to set the policy

ecPolicyName - The erasure coding policy name.

Throws:

IOException

satisfyStoragePolicy

public void satisfyStoragePolicy(org.apache.hadoop.fs.Path src)
                          throws IOException

Description copied from class: DistributedFileSystem

Set the source path to satisfy storage policy.

Overrides:

satisfyStoragePolicy in class DistributedFileSystem

Parameters:

src - The source path referring to either a directory or a file.

Throws:

IOException

getErasureCodingPolicy

public ErasureCodingPolicy getErasureCodingPolicy(org.apache.hadoop.fs.Path path)
                                           throws IOException

Description copied from class: DistributedFileSystem

Get erasure coding policy information for the specified path.

Overrides:

getErasureCodingPolicy in class DistributedFileSystem

Parameters:

path - The path of the file or directory

Returns:

Returns the policy information if file or directory on the path is erasure coded, null otherwise. Null will be returned if directory or file has REPLICATION policy.

Throws:

IOException

getAllErasureCodingPolicies

public Collection<ErasureCodingPolicyInfo> getAllErasureCodingPolicies()
                                                                throws IOException

Gets all erasure coding policies from all available child file systems.

Overrides:

getAllErasureCodingPolicies in class DistributedFileSystem

Returns:

all erasure coding policies supported by this file system.

Throws:

IOException

getAllErasureCodingCodecs

public Map<String,String> getAllErasureCodingCodecs()
                                             throws IOException

Description copied from class: DistributedFileSystem

Retrieve all the erasure coding codecs and coders supported by this file system.

Overrides:

getAllErasureCodingCodecs in class DistributedFileSystem

Returns:

all erasure coding codecs and coders supported by this file system.

Throws:

IOException

addErasureCodingPolicies

public AddErasureCodingPolicyResponse[] addErasureCodingPolicies(ErasureCodingPolicy[] policies)
                                                          throws IOException

Description copied from class: DistributedFileSystem

Add Erasure coding policies to HDFS. For each policy input, schema and cellSize are musts, name and id are ignored. They will be automatically created and assigned by Namenode once the policy is successfully added, and will be returned in the response; policy states will be set to DISABLED automatically.

Overrides:

addErasureCodingPolicies in class DistributedFileSystem

Parameters:

policies - The user defined ec policy list to add.

Returns:

Return the response list of adding operations.

Throws:

IOException

removeErasureCodingPolicy

public void removeErasureCodingPolicy(String ecPolicyName)
                               throws IOException

Description copied from class: DistributedFileSystem

Remove erasure coding policy.

Overrides:

removeErasureCodingPolicy in class DistributedFileSystem

Parameters:

ecPolicyName - The name of the policy to be removed.

Throws:

IOException

enableErasureCodingPolicy

public void enableErasureCodingPolicy(String ecPolicyName)
                               throws IOException

Description copied from class: DistributedFileSystem

Enable erasure coding policy.

Overrides:

enableErasureCodingPolicy in class DistributedFileSystem

Parameters:

ecPolicyName - The name of the policy to be enabled.

Throws:

IOException

disableErasureCodingPolicy

public void disableErasureCodingPolicy(String ecPolicyName)
                                throws IOException

Description copied from class: DistributedFileSystem

Disable erasure coding policy.

Overrides:

disableErasureCodingPolicy in class DistributedFileSystem

Parameters:

ecPolicyName - The name of the policy to be disabled.

Throws:

IOException

unsetErasureCodingPolicy

public void unsetErasureCodingPolicy(org.apache.hadoop.fs.Path path)
                              throws IOException

Description copied from class: DistributedFileSystem

Unset the erasure coding policy from the source path.

Overrides:

unsetErasureCodingPolicy in class DistributedFileSystem

Parameters:

path - The directory to unset the policy

Throws:

IOException

getECTopologyResultForPolicies

public ECTopologyVerifierResult getECTopologyResultForPolicies(String... policyNames)
                                                        throws IOException

Description copied from class: DistributedFileSystem

Verifies if the given policies are supported in the given cluster setup. If not policy is specified checks for all enabled policies.

Overrides:

getECTopologyResultForPolicies in class DistributedFileSystem

Parameters:

policyNames - name of policies.

Returns:

the result if the given policies are supported in the cluster setup

Throws:

IOException

getTrashRoot

public org.apache.hadoop.fs.Path getTrashRoot(org.apache.hadoop.fs.Path path)

Description copied from class: DistributedFileSystem

Get the root directory of Trash for a path in HDFS. 1. File in encryption zone returns /ez1/.Trash/username 2. File in snapshottable directory returns /snapdir1/.Trash/username if dfs.namenode.snapshot.trashroot.enabled is set to true. 3. In other cases, or encountered exception when checking the encryption zone or when checking snapshot root of the path, returns /users/username/.Trash Caller appends either Current or checkpoint timestamp for trash destination

Overrides:

getTrashRoot in class DistributedFileSystem

Parameters:

path - the trash root of the path to be determined.

Returns:

trash root

getTrashRoots

public Collection<org.apache.hadoop.fs.FileStatus> getTrashRoots(boolean allUsers)

Description copied from class: DistributedFileSystem

Get all the trash roots of HDFS for current user or for all the users. 1. File deleted from encryption zones e.g., ez1 rooted at /ez1 has its trash root at /ez1/.Trash/$USER 2. File deleted from snapshottable directories if dfs.namenode.snapshot.trashroot.enabled is set to true. e.g., snapshottable directory /snapdir1 has its trash root at /snapdir1/.Trash/$USER 3. File deleted from other directories /user/username/.Trash

Overrides:

getTrashRoots in class DistributedFileSystem

Parameters:

allUsers - return trashRoots of all users if true, used by emptier

Returns:

trash roots of HDFS

fixRelativePart

protected org.apache.hadoop.fs.Path fixRelativePart(org.apache.hadoop.fs.Path p)

Overrides:

fixRelativePart in class DistributedFileSystem

createFile

public DistributedFileSystem.HdfsDataOutputStreamBuilder createFile(org.apache.hadoop.fs.Path path)

Description copied from class: DistributedFileSystem

Create a HdfsDataOutputStreamBuilder to create a file on DFS. Similar to FileSystem.create(Path), file is overwritten by default.

Overrides:

createFile in class DistributedFileSystem

Parameters:

path - the path of the file to create.

Returns:

A HdfsDataOutputStreamBuilder for creating a file.

listOpenFiles

@Deprecated
public org.apache.hadoop.fs.RemoteIterator<OpenFileEntry> listOpenFiles()
                                                                 throws IOException

Deprecated.

Description copied from class: DistributedFileSystem

Returns a RemoteIterator which can be used to list all open files currently managed by the NameNode. For large numbers of open files, iterator will fetch the list in batches of configured size.

Since the list is fetched in batches, it does not represent a consistent snapshot of the all open files.

This method can only be called by HDFS superusers.

Overrides:

listOpenFiles in class DistributedFileSystem

Throws:

IOException

listOpenFiles

@Deprecated
public org.apache.hadoop.fs.RemoteIterator<OpenFileEntry> listOpenFiles(EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes)
                                                                 throws IOException

Deprecated.

Overrides:

listOpenFiles in class DistributedFileSystem

Throws:

IOException

listOpenFiles

public org.apache.hadoop.fs.RemoteIterator<OpenFileEntry> listOpenFiles(EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes,
 String path)
                                                                 throws IOException

Overrides:

listOpenFiles in class DistributedFileSystem

Throws:

IOException

appendFile

public DistributedFileSystem.HdfsDataOutputStreamBuilder appendFile(org.apache.hadoop.fs.Path path)

Description copied from class: DistributedFileSystem

Create a DistributedFileSystem.HdfsDataOutputStreamBuilder to append a file on DFS.

Overrides:

appendFile in class DistributedFileSystem

Parameters:

path - file path.

Returns:

A DistributedFileSystem.HdfsDataOutputStreamBuilder for appending a file.

hasPathCapability

public boolean hasPathCapability(org.apache.hadoop.fs.Path path,
 String capability)
                          throws IOException

Description copied from class: DistributedFileSystem

HDFS client capabilities. Uses DfsPathCapabilities to keep WebHdfsFileSystem in sync.

Specified by:

hasPathCapability in interface org.apache.hadoop.fs.PathCapabilities

Overrides:

hasPathCapability in class DistributedFileSystem

Throws:

IOException

resolvePath

public org.apache.hadoop.fs.Path resolvePath(org.apache.hadoop.fs.Path f)
                                      throws IOException

Overrides:

resolvePath in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

delete

public boolean delete(org.apache.hadoop.fs.Path f)
               throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

delete in class org.apache.hadoop.fs.FileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

getFileChecksum

public org.apache.hadoop.fs.FileChecksum getFileChecksum(org.apache.hadoop.fs.Path f,
 long length)
                                                  throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException

Overrides:

getFileChecksum in class DistributedFileSystem

Throws:

org.apache.hadoop.security.AccessControlException

FileNotFoundException

IOException

mkdirs

public boolean mkdirs(org.apache.hadoop.fs.Path dir)
               throws IOException

Overrides:

mkdirs in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getDefaultBlockSize

public long getDefaultBlockSize(org.apache.hadoop.fs.Path f)

Overrides:

getDefaultBlockSize in class org.apache.hadoop.fs.FileSystem

getDefaultReplication

public short getDefaultReplication(org.apache.hadoop.fs.Path f)

Overrides:

getDefaultReplication in class org.apache.hadoop.fs.FileSystem

getServerDefaults

public org.apache.hadoop.fs.FsServerDefaults getServerDefaults(org.apache.hadoop.fs.Path f)
                                                        throws IOException

Overrides:

getServerDefaults in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

setWriteChecksum

public void setWriteChecksum(boolean writeChecksum)

Overrides:

setWriteChecksum in class org.apache.hadoop.fs.FileSystem

getChildFileSystems

public org.apache.hadoop.fs.FileSystem[] getChildFileSystems()

Overrides:

getChildFileSystems in class org.apache.hadoop.fs.FileSystem

getMountPoints

public org.apache.hadoop.fs.viewfs.ViewFileSystem.MountPoint[] getMountPoints()

getStatus

public org.apache.hadoop.fs.FsStatus getStatus()
                                        throws IOException

Overrides:

getStatus in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getUsed

public long getUsed()
             throws IOException

Overrides:

getUsed in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getSlowDatanodeStats

public DatanodeInfo[] getSlowDatanodeStats()
                                    throws IOException

Description copied from class: DistributedFileSystem

Retrieve stats for slow running datanodes.

Overrides:

getSlowDatanodeStats in class DistributedFileSystem

Returns:

An array of slow datanode info.

Throws:

IOException - If an I/O error occurs.