Package org.apache.hadoop.hdfs.web

Class WebHdfsFileSystem

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.fs.FileSystem
org.apache.hadoop.hdfs.web.WebHdfsFileSystem
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.crypto.key.KeyProviderTokenIssuer, org.apache.hadoop.fs.BulkDeleteSource, org.apache.hadoop.fs.DelegationTokenRenewer.Renewable, org.apache.hadoop.fs.PathCapabilities, org.apache.hadoop.security.token.DelegationTokenIssuer
Direct Known Subclasses:
SWebHdfsFileSystem
public class WebHdfsFileSystem extends org.apache.hadoop.fs.FileSystem implements org.apache.hadoop.fs.DelegationTokenRenewer.Renewable, org.apache.hadoop.crypto.key.KeyProviderTokenIssuer
A FileSystem for HDFS over the web.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    protected class 
    WebHdfsFileSystem.ReadRunner
    This class will allow retries to occur for both open and read operations.
    class 
    WebHdfsFileSystem.WebHdfsInputStream
    This class is used for opening, reading, and seeking files while using the WebHdfsFileSystem.

    Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

    org.apache.hadoop.fs.FileSystem.DirectoryEntries, org.apache.hadoop.fs.FileSystem.DirListingIterator<T extends org.apache.hadoop.fs.FileStatus>, org.apache.hadoop.fs.FileSystem.Statistics

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    CANT_FALLBACK_TO_INSECURE_MSG
     
    protected URLConnectionFactory
    connectionFactory
    Default connection factory may be overridden in tests to use smaller timeout values
    static final String
    DFS_HTTP_POLICY_KEY
     
    static final String
    EZ_HEADER
     
    static final String
    FEFINFO_HEADER
     
    static final org.slf4j.Logger
    LOG
     
    static final String
    PATH_PREFIX
    Http URI: http://namenode:port/{PATH_PREFIX}/path/to/file
    protected org.apache.hadoop.io.Text
    tokenServiceName
     
    static final int
    VERSION
    WebHdfs version.

    Fields inherited from class org.apache.hadoop.fs.FileSystem

    DEFAULT_FS, FS_DEFAULT_NAME_KEY, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX

    Fields inherited from interface org.apache.hadoop.security.token.DelegationTokenIssuer

    TOKEN_LOG

  • Constructor Summary

    Constructors
    Constructor
    Description
    WebHdfsFileSystem()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    access(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode)
     
    void
    allowSnapshot(org.apache.hadoop.fs.Path p)
     
    org.apache.hadoop.fs.FSDataOutputStream
    append(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress)
     
    void
    cancelDelegationToken(org.apache.hadoop.security.token.Token<?> token)
     
    protected URI
    canonicalizeUri(URI uri)
     
    void
    close()
     
    void
    concat(org.apache.hadoop.fs.Path trg, org.apache.hadoop.fs.Path[] srcs)
     
    org.apache.hadoop.fs.FSDataOutputStream
    create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
     
    org.apache.hadoop.fs.MultipartUploaderBuilder
    createMultipartUploader(org.apache.hadoop.fs.Path basePath)
     
    org.apache.hadoop.fs.FSDataOutputStream
    createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flag, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
     
    org.apache.hadoop.fs.Path
    createSnapshot(org.apache.hadoop.fs.Path path, String snapshotName)
     
    void
    createSymlink(org.apache.hadoop.fs.Path destination, org.apache.hadoop.fs.Path f, boolean createParent)
    Create a symlink pointing to the destination path.
    boolean
    delete(org.apache.hadoop.fs.Path f, boolean recursive)
     
    void
    deleteSnapshot(org.apache.hadoop.fs.Path path, String snapshotName)
     
    void
    disableECPolicy(String policyName)
     
    void
    disallowSnapshot(org.apache.hadoop.fs.Path p)
     
    void
    enableECPolicy(String policyName)
     
    org.apache.hadoop.fs.permission.AclStatus
    getAclStatus(org.apache.hadoop.fs.Path f)
     
    org.apache.hadoop.security.token.DelegationTokenIssuer[]
    getAdditionalTokenIssuers()
     
    Map<String,String>
    getAllErasureCodingCodecs()
     
    Collection<ErasureCodingPolicyInfo>
    getAllErasureCodingPolicies()
     
    Collection<BlockStoragePolicy>
    getAllStoragePolicies()
     
    String
    getCanonicalServiceName()
     
    URI
    getCanonicalUri()
     
    org.apache.hadoop.fs.ContentSummary
    getContentSummary(org.apache.hadoop.fs.Path p)
     
    long
    getDefaultBlockSize()
     
    protected int
    getDefaultPort()
     
    short
    getDefaultReplication()
     
    protected org.apache.hadoop.security.token.Token<?>
    getDelegationToken()
     
    org.apache.hadoop.security.token.Token<DelegationTokenIdentifier>
    getDelegationToken(String renewer)
     
    ErasureCodingPolicy
    getErasureCodingPolicy(org.apache.hadoop.fs.Path p)
     
    org.apache.hadoop.fs.BlockLocation[]
    getFileBlockLocations(org.apache.hadoop.fs.FileStatus status, long offset, long length)
     
    org.apache.hadoop.fs.BlockLocation[]
    getFileBlockLocations(org.apache.hadoop.fs.Path p, long offset, long length)
     
    org.apache.hadoop.fs.MD5MD5CRC32FileChecksum
    getFileChecksum(org.apache.hadoop.fs.Path p)
     
    org.apache.hadoop.fs.FileStatus
    getFileLinkStatus(org.apache.hadoop.fs.Path f)
     
    org.apache.hadoop.fs.FileStatus
    getFileStatus(org.apache.hadoop.fs.Path f)
     
    org.apache.hadoop.fs.Path
    getHomeDirectory()
     
    static String
    getHomeDirectoryString(org.apache.hadoop.security.UserGroupInformation ugi)
    Deprecated.
    org.apache.hadoop.crypto.key.KeyProvider
    getKeyProvider()
     
    URI
    getKeyProviderUri()
     
    org.apache.hadoop.fs.Path
    getLinkTarget(org.apache.hadoop.fs.Path f)
     
    org.apache.hadoop.fs.QuotaUsage
    getQuotaUsage(org.apache.hadoop.fs.Path p)
     
    org.apache.hadoop.security.token.Token<?>
    getRenewToken()
     
    String
    getScheme()
    Return the protocol scheme for the FileSystem.
    org.apache.hadoop.fs.FsServerDefaults
    getServerDefaults()
     
    SnapshotDiffReport
    getSnapshotDiffReport(org.apache.hadoop.fs.Path snapshotDir, String fromSnapshot, String toSnapshot)
     
    SnapshotDiffReportListing
    getSnapshotDiffReportListing(String snapshotDir, String fromSnapshot, String toSnapshot, byte[] startPath, int index)
     
    SnapshotStatus[]
    getSnapshotListing(org.apache.hadoop.fs.Path snapshotDir)
     
    SnapshottableDirectoryStatus[]
    getSnapshottableDirectoryList()
     
    org.apache.hadoop.fs.FsStatus
    getStatus(org.apache.hadoop.fs.Path path)
     
    BlockStoragePolicy
    getStoragePolicy(org.apache.hadoop.fs.Path src)
     
    protected org.apache.hadoop.io.Text
    getTokenKind()
     
    protected String
    getTransportScheme()
    return the underlying transport protocol (http / https).
    org.apache.hadoop.fs.Path
    getTrashRoot(org.apache.hadoop.fs.Path path)
     
    Collection<org.apache.hadoop.fs.FileStatus>
    getTrashRoots(boolean allUsers)
     
    URI
    getUri()
     
    org.apache.hadoop.fs.Path
    getWorkingDirectory()
     
    byte[]
    getXAttr(org.apache.hadoop.fs.Path p, String name)
     
    Map<String,byte[]>
    getXAttrs(org.apache.hadoop.fs.Path p)
     
    Map<String,byte[]>
    getXAttrs(org.apache.hadoop.fs.Path p, List<String> names)
     
    boolean
    hasPathCapability(org.apache.hadoop.fs.Path path, String capability)
    HDFS client capabilities.
    void
    initialize(URI uri, org.apache.hadoop.conf.Configuration conf)
     
    static Map<?,?>
    jsonParse(HttpURLConnection c, boolean useErrorStream)
     
    org.apache.hadoop.fs.FileStatus[]
    listStatus(org.apache.hadoop.fs.Path f)
    Get FileStatus of files/directories in the given path.
    org.apache.hadoop.fs.FileSystem.DirectoryEntries
    listStatusBatch(org.apache.hadoop.fs.Path f, byte[] token)
    Get DirectoryEntries of the given path.
    List<String>
    listXAttrs(org.apache.hadoop.fs.Path p)
     
    boolean
    mkdirs(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission)
     
    void
    modifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
     
    org.apache.hadoop.fs.FSDataInputStream
    open(org.apache.hadoop.fs.Path f, int bufferSize)
     
    void
    removeAcl(org.apache.hadoop.fs.Path path)
     
    void
    removeAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
     
    void
    removeDefaultAcl(org.apache.hadoop.fs.Path path)
     
    void
    removeXAttr(org.apache.hadoop.fs.Path p, String name)
     
    boolean
    rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)
     
    void
    rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, org.apache.hadoop.fs.Options.Rename... options)
     
    void
    renameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName)
     
    long
    renewDelegationToken(org.apache.hadoop.security.token.Token<?> token)
     
    void
    satisfyStoragePolicy(org.apache.hadoop.fs.Path p)
     
    void
    setAcl(org.apache.hadoop.fs.Path p, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
     
    <T extends org.apache.hadoop.security.token.TokenIdentifier>
    void
    setDelegationToken(org.apache.hadoop.security.token.Token<T> token)
     
    void
    setErasureCodingPolicy(org.apache.hadoop.fs.Path p, String policyName)
     
    void
    setOwner(org.apache.hadoop.fs.Path p, String owner, String group)
     
    void
    setPermission(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.permission.FsPermission permission)
     
    void
    setQuota(org.apache.hadoop.fs.Path p, long namespaceQuota, long storagespaceQuota)
     
    void
    setQuotaByStorageType(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.StorageType type, long quota)
     
    boolean
    setReplication(org.apache.hadoop.fs.Path p, short replication)
     
    void
    setRetryPolicy(org.apache.hadoop.io.retry.RetryPolicy rp)
     
    void
    setStoragePolicy(org.apache.hadoop.fs.Path p, String policyName)
     
    void
    setTestProvider(org.apache.hadoop.crypto.key.KeyProvider kp)
     
    void
    setTimes(org.apache.hadoop.fs.Path p, long mtime, long atime)
     
    void
    setWorkingDirectory(org.apache.hadoop.fs.Path dir)
     
    void
    setXAttr(org.apache.hadoop.fs.Path p, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag)
     
    boolean
    supportsSymlinks()
     
    boolean
    truncate(org.apache.hadoop.fs.Path f, long newLength)
     
    void
    unsetErasureCodingPolicy(org.apache.hadoop.fs.Path p)
     
    void
    unsetStoragePolicy(org.apache.hadoop.fs.Path src)
     

    Methods inherited from class org.apache.hadoop.fs.FileSystem

    append, append, append, append, appendFile, areSymlinksEnabled, cancelDeleteOnExit, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createBulkDelete, createDataInputStreamBuilder, createDataInputStreamBuilder, createDataOutputStreamBuilder, createFile, createNewFile, createNonRecursive, createNonRecursive, createPathHandle, createSnapshot, delete, deleteOnExit, enableSymlinks, exists, fixRelativePart, get, get, get, getAllStatistics, getBlockSize, getChildFileSystems, getDefaultBlockSize, getDefaultReplication, getDefaultUri, getEnclosingRoot, getFileChecksum, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getInitialWorkingDirectory, getLength, getLocal, getName, getNamed, getPathHandle, getReplication, getServerDefaults, getStatistics, getStatistics, getStatus, getStorageStatistics, getUsed, getUsed, globStatus, globStatus, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, listStatusIterator, makeQualified, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, msync, newInstance, newInstance, newInstance, newInstanceLocal, open, open, open, openFile, openFile, openFileWithOptions, openFileWithOptions, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, resolveLink, resolvePath, setDefaultUri, setDefaultUri, setVerifyChecksum, setWriteChecksum, setXAttr, startLocalOutput

    Methods inherited from class org.apache.hadoop.conf.Configured

    getConf, setConf

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.security.token.DelegationTokenIssuer

    addDelegationTokens

  • Field Details

    • LOG

      public static final org.slf4j.Logger LOG

    • VERSION

      public static final int VERSION
      WebHdfs version.
      See Also:

    • PATH_PREFIX

      public static final String PATH_PREFIX
      Http URI: http://namenode:port/{PATH_PREFIX}/path/to/file
      See Also:

    • EZ_HEADER

      public static final String EZ_HEADER
      See Also:

    • FEFINFO_HEADER

      public static final String FEFINFO_HEADER
      See Also:

    • DFS_HTTP_POLICY_KEY

      public static final String DFS_HTTP_POLICY_KEY
      See Also:

    • connectionFactory

      protected URLConnectionFactory connectionFactory
      Default connection factory may be overridden in tests to use smaller timeout values

    • CANT_FALLBACK_TO_INSECURE_MSG

      @VisibleForTesting public static final String CANT_FALLBACK_TO_INSECURE_MSG
      See Also:

    • tokenServiceName

      protected org.apache.hadoop.io.Text tokenServiceName

  • Constructor Details

    • WebHdfsFileSystem

      public WebHdfsFileSystem()

  • Method Details

    • getScheme

      public String getScheme()
      Return the protocol scheme for the FileSystem.
      Overrides:
      getScheme in class org.apache.hadoop.fs.FileSystem
      Returns:
      webhdfs

    • getTransportScheme

      protected String getTransportScheme()
      return the underlying transport protocol (http / https).

    • getTokenKind

      protected org.apache.hadoop.io.Text getTokenKind()

    • initialize

      public void initialize(URI uri, org.apache.hadoop.conf.Configuration conf) throws IOException
      Overrides:
      initialize in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getCanonicalUri

      public URI getCanonicalUri()
      Overrides:
      getCanonicalUri in class org.apache.hadoop.fs.FileSystem

    • getDelegationToken

      protected org.apache.hadoop.security.token.Token<?> getDelegationToken() throws IOException
      Throws:
      IOException

    • getDefaultPort

      protected int getDefaultPort()
      Overrides:
      getDefaultPort in class org.apache.hadoop.fs.FileSystem

    • getUri

      public URI getUri()
      Specified by:
      getUri in class org.apache.hadoop.fs.FileSystem

    • canonicalizeUri

      protected URI canonicalizeUri(URI uri)
      Overrides:
      canonicalizeUri in class org.apache.hadoop.fs.FileSystem

    • getHomeDirectoryString

      @Deprecated public static String getHomeDirectoryString(org.apache.hadoop.security.UserGroupInformation ugi)
      Deprecated.
      Returns:
      the home directory

    • getHomeDirectory

      public org.apache.hadoop.fs.Path getHomeDirectory()
      Overrides:
      getHomeDirectory in class org.apache.hadoop.fs.FileSystem

    • getWorkingDirectory

      public org.apache.hadoop.fs.Path getWorkingDirectory()
      Specified by:
      getWorkingDirectory in class org.apache.hadoop.fs.FileSystem

    • setWorkingDirectory

      public void setWorkingDirectory(org.apache.hadoop.fs.Path dir)
      Specified by:
      setWorkingDirectory in class org.apache.hadoop.fs.FileSystem

    • jsonParse

      @VisibleForTesting public static Map<?,?> jsonParse(HttpURLConnection c, boolean useErrorStream) throws IOException
      Throws:
      IOException

    • getFileStatus

      public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path f) throws IOException
      Specified by:
      getFileStatus in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getAclStatus

      public org.apache.hadoop.fs.permission.AclStatus getAclStatus(org.apache.hadoop.fs.Path f) throws IOException
      Overrides:
      getAclStatus in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • mkdirs

      public boolean mkdirs(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission) throws IOException
      Specified by:
      mkdirs in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • supportsSymlinks

      public boolean supportsSymlinks()
      Overrides:
      supportsSymlinks in class org.apache.hadoop.fs.FileSystem

    • createSymlink

      public void createSymlink(org.apache.hadoop.fs.Path destination, org.apache.hadoop.fs.Path f, boolean createParent) throws IOException
      Create a symlink pointing to the destination path.
      Overrides:
      createSymlink in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • rename

      public boolean rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException
      Specified by:
      rename in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • rename

      public void rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, org.apache.hadoop.fs.Options.Rename... options) throws IOException
      Overrides:
      rename in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • setXAttr

      public void setXAttr(org.apache.hadoop.fs.Path p, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag) throws IOException
      Overrides:
      setXAttr in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getXAttr

      public byte[] getXAttr(org.apache.hadoop.fs.Path p, String name) throws IOException
      Overrides:
      getXAttr in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getXAttrs

      public Map<String,byte[]> getXAttrs(org.apache.hadoop.fs.Path p) throws IOException
      Overrides:
      getXAttrs in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getXAttrs

      public Map<String,byte[]> getXAttrs(org.apache.hadoop.fs.Path p, List<String> names) throws IOException
      Overrides:
      getXAttrs in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • listXAttrs

      public List<String> listXAttrs(org.apache.hadoop.fs.Path p) throws IOException
      Overrides:
      listXAttrs in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • removeXAttr

      public void removeXAttr(org.apache.hadoop.fs.Path p, String name) throws IOException
      Overrides:
      removeXAttr in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • setOwner

      public void setOwner(org.apache.hadoop.fs.Path p, String owner, String group) throws IOException
      Overrides:
      setOwner in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • setPermission

      public void setPermission(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.permission.FsPermission permission) throws IOException
      Overrides:
      setPermission in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • modifyAclEntries

      public void modifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException
      Overrides:
      modifyAclEntries in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • removeAclEntries

      public void removeAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException
      Overrides:
      removeAclEntries in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • removeDefaultAcl

      public void removeDefaultAcl(org.apache.hadoop.fs.Path path) throws IOException
      Overrides:
      removeDefaultAcl in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • removeAcl

      public void removeAcl(org.apache.hadoop.fs.Path path) throws IOException
      Overrides:
      removeAcl in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • setAcl

      public void setAcl(org.apache.hadoop.fs.Path p, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException
      Overrides:
      setAcl in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • allowSnapshot

      public void allowSnapshot(org.apache.hadoop.fs.Path p) throws IOException
      Throws:
      IOException

    • satisfyStoragePolicy

      public void satisfyStoragePolicy(org.apache.hadoop.fs.Path p) throws IOException
      Overrides:
      satisfyStoragePolicy in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • enableECPolicy

      public void enableECPolicy(String policyName) throws IOException
      Throws:
      IOException

    • disableECPolicy

      public void disableECPolicy(String policyName) throws IOException
      Throws:
      IOException

    • setErasureCodingPolicy

      public void setErasureCodingPolicy(org.apache.hadoop.fs.Path p, String policyName) throws IOException
      Throws:
      IOException

    • unsetErasureCodingPolicy

      public void unsetErasureCodingPolicy(org.apache.hadoop.fs.Path p) throws IOException
      Throws:
      IOException

    • getErasureCodingPolicy

      public ErasureCodingPolicy getErasureCodingPolicy(org.apache.hadoop.fs.Path p) throws IOException
      Throws:
      IOException

    • createSnapshot

      public org.apache.hadoop.fs.Path createSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) throws IOException
      Overrides:
      createSnapshot in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • disallowSnapshot

      public void disallowSnapshot(org.apache.hadoop.fs.Path p) throws IOException
      Throws:
      IOException

    • deleteSnapshot

      public void deleteSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) throws IOException
      Overrides:
      deleteSnapshot in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • renameSnapshot

      public void renameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName) throws IOException
      Overrides:
      renameSnapshot in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getSnapshotDiffReportListing

      @VisibleForTesting public SnapshotDiffReportListing getSnapshotDiffReportListing(String snapshotDir, String fromSnapshot, String toSnapshot, byte[] startPath, int index) throws IOException
      Throws:
      IOException

    • getSnapshotDiffReport

      public SnapshotDiffReport getSnapshotDiffReport(org.apache.hadoop.fs.Path snapshotDir, String fromSnapshot, String toSnapshot) throws IOException
      Throws:
      IOException

    • getSnapshottableDirectoryList

      public SnapshottableDirectoryStatus[] getSnapshottableDirectoryList() throws IOException
      Throws:
      IOException

    • getSnapshotListing

      public SnapshotStatus[] getSnapshotListing(org.apache.hadoop.fs.Path snapshotDir) throws IOException
      Throws:
      IOException

    • setReplication

      public boolean setReplication(org.apache.hadoop.fs.Path p, short replication) throws IOException
      Overrides:
      setReplication in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • setTimes

      public void setTimes(org.apache.hadoop.fs.Path p, long mtime, long atime) throws IOException
      Overrides:
      setTimes in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getDefaultBlockSize

      public long getDefaultBlockSize()
      Overrides:
      getDefaultBlockSize in class org.apache.hadoop.fs.FileSystem

    • getDefaultReplication

      public short getDefaultReplication()
      Overrides:
      getDefaultReplication in class org.apache.hadoop.fs.FileSystem

    • concat

      public void concat(org.apache.hadoop.fs.Path trg, org.apache.hadoop.fs.Path[] srcs) throws IOException
      Overrides:
      concat in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • create

      public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException
      Specified by:
      create in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • createNonRecursive

      public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flag, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException
      Overrides:
      createNonRecursive in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • append

      public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress) throws IOException
      Specified by:
      append in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • truncate

      public boolean truncate(org.apache.hadoop.fs.Path f, long newLength) throws IOException
      Overrides:
      truncate in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • delete

      public boolean delete(org.apache.hadoop.fs.Path f, boolean recursive) throws IOException
      Specified by:
      delete in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • open

      public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path f, int bufferSize) throws IOException
      Specified by:
      open in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • listStatus

      public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f) throws IOException
      Get FileStatus of files/directories in the given path. If path corresponds to a file then FileStatus of that file is returned. Else if path represents a directory then FileStatus of all files/directories inside given path is returned.
      Specified by:
      listStatus in class org.apache.hadoop.fs.FileSystem
      Parameters:
      f - given path
      Returns:
      the statuses of the files/directories in the given path
      Throws:
      IOException

    • listStatusBatch

      public org.apache.hadoop.fs.FileSystem.DirectoryEntries listStatusBatch(org.apache.hadoop.fs.Path f, byte[] token) throws FileNotFoundException, IOException
      Get DirectoryEntries of the given path. DirectoryEntries contains an array of FileStatus, as well as iteration information.
      Overrides:
      listStatusBatch in class org.apache.hadoop.fs.FileSystem
      Parameters:
      f - given path
      Returns:
      DirectoryEntries for given path
      Throws:
      FileNotFoundException
      IOException

    • getDelegationToken

      public org.apache.hadoop.security.token.Token<DelegationTokenIdentifier> getDelegationToken(String renewer) throws IOException
      Specified by:
      getDelegationToken in interface org.apache.hadoop.security.token.DelegationTokenIssuer
      Overrides:
      getDelegationToken in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getAdditionalTokenIssuers

      public org.apache.hadoop.security.token.DelegationTokenIssuer[] getAdditionalTokenIssuers() throws IOException
      Specified by:
      getAdditionalTokenIssuers in interface org.apache.hadoop.security.token.DelegationTokenIssuer
      Overrides:
      getAdditionalTokenIssuers in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getRenewToken

      public org.apache.hadoop.security.token.Token<?> getRenewToken()
      Specified by:
      getRenewToken in interface org.apache.hadoop.fs.DelegationTokenRenewer.Renewable

    • setDelegationToken

      public <T extends org.apache.hadoop.security.token.TokenIdentifier> void setDelegationToken(org.apache.hadoop.security.token.Token<T> token)
      Specified by:
      setDelegationToken in interface org.apache.hadoop.fs.DelegationTokenRenewer.Renewable

    • renewDelegationToken

      public long renewDelegationToken(org.apache.hadoop.security.token.Token<?> token) throws IOException
      Throws:
      IOException

    • cancelDelegationToken

      public void cancelDelegationToken(org.apache.hadoop.security.token.Token<?> token) throws IOException
      Throws:
      IOException

    • getFileBlockLocations

      public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.FileStatus status, long offset, long length) throws IOException
      Overrides:
      getFileBlockLocations in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getFileBlockLocations

      public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.Path p, long offset, long length) throws IOException
      Overrides:
      getFileBlockLocations in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getTrashRoot

      public org.apache.hadoop.fs.Path getTrashRoot(org.apache.hadoop.fs.Path path)
      Overrides:
      getTrashRoot in class org.apache.hadoop.fs.FileSystem

    • getTrashRoots

      public Collection<org.apache.hadoop.fs.FileStatus> getTrashRoots(boolean allUsers)
      Overrides:
      getTrashRoots in class org.apache.hadoop.fs.FileSystem

    • access

      public void access(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode) throws IOException
      Overrides:
      access in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getContentSummary

      public org.apache.hadoop.fs.ContentSummary getContentSummary(org.apache.hadoop.fs.Path p) throws IOException
      Overrides:
      getContentSummary in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getQuotaUsage

      public org.apache.hadoop.fs.QuotaUsage getQuotaUsage(org.apache.hadoop.fs.Path p) throws IOException
      Overrides:
      getQuotaUsage in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • setQuota

      public void setQuota(org.apache.hadoop.fs.Path p, long namespaceQuota, long storagespaceQuota) throws IOException
      Overrides:
      setQuota in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • setQuotaByStorageType

      public void setQuotaByStorageType(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.StorageType type, long quota) throws IOException
      Overrides:
      setQuotaByStorageType in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getFileChecksum

      public org.apache.hadoop.fs.MD5MD5CRC32FileChecksum getFileChecksum(org.apache.hadoop.fs.Path p) throws IOException
      Overrides:
      getFileChecksum in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getCanonicalServiceName

      public String getCanonicalServiceName()
      Specified by:
      getCanonicalServiceName in interface org.apache.hadoop.security.token.DelegationTokenIssuer
      Overrides:
      getCanonicalServiceName in class org.apache.hadoop.fs.FileSystem

    • setStoragePolicy

      public void setStoragePolicy(org.apache.hadoop.fs.Path p, String policyName) throws IOException
      Overrides:
      setStoragePolicy in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getAllStoragePolicies

      public Collection<BlockStoragePolicy> getAllStoragePolicies() throws IOException
      Overrides:
      getAllStoragePolicies in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getStoragePolicy

      public BlockStoragePolicy getStoragePolicy(org.apache.hadoop.fs.Path src) throws IOException
      Overrides:
      getStoragePolicy in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • unsetStoragePolicy

      public void unsetStoragePolicy(org.apache.hadoop.fs.Path src) throws IOException
      Overrides:
      unsetStoragePolicy in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getServerDefaults

      public org.apache.hadoop.fs.FsServerDefaults getServerDefaults() throws IOException
      Overrides:
      getServerDefaults in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getLinkTarget

      public org.apache.hadoop.fs.Path getLinkTarget(org.apache.hadoop.fs.Path f) throws IOException
      Overrides:
      getLinkTarget in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getFileLinkStatus

      public org.apache.hadoop.fs.FileStatus getFileLinkStatus(org.apache.hadoop.fs.Path f) throws IOException
      Overrides:
      getFileLinkStatus in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getStatus

      public org.apache.hadoop.fs.FsStatus getStatus(org.apache.hadoop.fs.Path path) throws IOException
      Overrides:
      getStatus in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • getAllErasureCodingPolicies

      public Collection<ErasureCodingPolicyInfo> getAllErasureCodingPolicies() throws IOException
      Throws:
      IOException

    • getAllErasureCodingCodecs

      public Map<String,String> getAllErasureCodingCodecs() throws IOException
      Throws:
      IOException

    • setRetryPolicy

      @VisibleForTesting public void setRetryPolicy(org.apache.hadoop.io.retry.RetryPolicy rp)

    • getKeyProviderUri

      public URI getKeyProviderUri() throws IOException
      Specified by:
      getKeyProviderUri in interface org.apache.hadoop.crypto.key.KeyProviderTokenIssuer
      Throws:
      IOException

    • getKeyProvider

      public org.apache.hadoop.crypto.key.KeyProvider getKeyProvider() throws IOException
      Specified by:
      getKeyProvider in interface org.apache.hadoop.crypto.key.KeyProviderTokenIssuer
      Throws:
      IOException

    • setTestProvider

      @VisibleForTesting public void setTestProvider(org.apache.hadoop.crypto.key.KeyProvider kp)

    • hasPathCapability

      public boolean hasPathCapability(org.apache.hadoop.fs.Path path, String capability) throws IOException
      HDFS client capabilities. Uses DfsPathCapabilities to keep in sync with HDFS.
      Specified by:
      hasPathCapability in interface org.apache.hadoop.fs.PathCapabilities
      Overrides:
      hasPathCapability in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException

    • createMultipartUploader

      public org.apache.hadoop.fs.MultipartUploaderBuilder createMultipartUploader(org.apache.hadoop.fs.Path basePath) throws IOException
      Overrides:
      createMultipartUploader in class org.apache.hadoop.fs.FileSystem
      Throws:
      IOException