Package org.apache.hadoop.hdfs.security.token.block

Class BlockTokenSecretManager

java.lang.Object
org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
org.apache.hadoop.hdfs.security.token.block.BlockTokenSecretManager
@Private public class BlockTokenSecretManager extends org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
BlockTokenSecretManager can be instantiated in 2 modes, master mode and worker mode. Master can generate new block keys and export block keys to workers, while workers can only import and use block keys received from master. Both master and worker can generate and verify block tokens. Typically, master mode is used by NN and worker mode is used by DN.

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.hadoop.security.token.SecretManager

    org.apache.hadoop.security.token.SecretManager.InvalidToken

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
    DUMMY_TOKEN
     
    static final org.slf4j.Logger
    LOG
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, int nnIndex, int numNNs, String blockPoolId, String encryptionAlgorithm, boolean useProto)
     
    BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, int nnIndex, int numNNs, String blockPoolId, String encryptionAlgorithm, boolean useProto, boolean shouldWrapQOP)
     
    BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, String blockPoolId, String encryptionAlgorithm, boolean useProto)
    Constructor for workers.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    addKeys(ExportedBlockKeys exportedKeys)
     
    void
    addKeys(ExportedBlockKeys exportedKeys, boolean updateCurrentKey)
    Set block keys, only to be used in worker mode
    void
    checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)
     
    void
    checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes)
    Check if access should be allowed. userID is not checked if null.
    void
    checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds)
    Check if access should be allowed. userID is not checked if null.
    void
    checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)
    Check if access should be allowed. userID is not checked if null
    void
    checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds)
    Check if access should be allowed. userID is not checked if null
    static <T> void
    checkAccess(T[] candidates, T[] requested, String msg)
    Check if the requested values can be satisfied with the values in the BlockToken.
    void
    clearAllKeysForTesting()
     
    org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier
    createIdentifier()
    Create an empty block token identifier
    protected byte[]
    createPassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier)
    Create a new password/secret for the given block token identifier.
    ExportedBlockKeys
    exportKeys()
    Export block keys, only to be used in master mode
    org.apache.hadoop.hdfs.security.token.block.DataEncryptionKey
    generateDataEncryptionKey()
    Generate a data encryption key for this block pool, using the current BlockKey.
    org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
    generateToken(String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds)
    Generate a block token for a specified user
    org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
    generateToken(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds)
    Generate an block token for current user
    BlockKey
    getCurrentKey()
     
    int
    getSerialNoForTesting()
     
    boolean
    hasKey(int keyId)
     
    byte[]
    retrieveDataEncryptionKey(int keyId, byte[] nonce)
    Recreate an encryption key based on the given key id and nonce.
    byte[]
    retrievePassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier)
    Look up the token password/secret for the given block token identifier.
    void
    setBlockPoolId(String blockPoolId)
     
    void
    setKeyUpdateIntervalForTesting(long millis)
     
    void
    setSerialNo(int nextNo)
     
    void
    setTokenLifetime(long tokenLifetime)
    set token lifetime.
    boolean
    updateKeys(long updateTime)
    Update block keys if update time > update interval.

    Methods inherited from class org.apache.hadoop.security.token.SecretManager

    checkAvailableForRead, createPassword, createSecretKey, generateSecret, retriableRetrievePassword, update, validateSecretKeyLength

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • LOG

      public static final org.slf4j.Logger LOG

    • DUMMY_TOKEN

      public static final org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> DUMMY_TOKEN

  • Constructor Details

    • BlockTokenSecretManager

      public BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, String blockPoolId, String encryptionAlgorithm, boolean useProto)
      Constructor for workers.
      Parameters:
      keyUpdateInterval - how often a new key will be generated
      tokenLifetime - how long an individual token is valid
      useProto - should we use new protobuf style tokens

    • BlockTokenSecretManager

      public BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, int nnIndex, int numNNs, String blockPoolId, String encryptionAlgorithm, boolean useProto)

    • BlockTokenSecretManager

      public BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, int nnIndex, int numNNs, String blockPoolId, String encryptionAlgorithm, boolean useProto, boolean shouldWrapQOP)

  • Method Details

    • setSerialNo

      @VisibleForTesting public void setSerialNo(int nextNo)

    • setBlockPoolId

      public void setBlockPoolId(String blockPoolId)

    • exportKeys

      public ExportedBlockKeys exportKeys()
      Export block keys, only to be used in master mode

    • addKeys

      public void addKeys(ExportedBlockKeys exportedKeys) throws IOException
      Throws:
      IOException

    • addKeys

      public void addKeys(ExportedBlockKeys exportedKeys, boolean updateCurrentKey) throws IOException
      Set block keys, only to be used in worker mode
      Throws:
      IOException

    • updateKeys

      public boolean updateKeys(long updateTime) throws IOException
      Update block keys if update time > update interval.
      Returns:
      true if the keys are updated.
      Throws:
      IOException

    • generateToken

      public org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> generateToken(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds) throws IOException
      Generate an block token for current user
      Throws:
      IOException

    • generateToken

      public org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> generateToken(String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds)
      Generate a block token for a specified user

    • checkAccess

      public void checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Check if access should be allowed. userID is not checked if null. This method doesn't check if token password is correct. It should be used only when token password has already been verified (e.g., in the RPC layer). Some places need to check the access using StorageTypes and for other places the StorageTypes is not relevant.
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken

    • checkAccess

      public void checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Check if access should be allowed. userID is not checked if null. This method doesn't check if token password is correct. It should be used only when token password has already been verified (e.g., in the RPC layer). Some places need to check the access using StorageTypes and for other places the StorageTypes is not relevant.
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken

    • checkAccess

      public void checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken

    • checkAccess

      public static <T> void checkAccess(T[] candidates, T[] requested, String msg) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Check if the requested values can be satisfied with the values in the BlockToken. This is intended for use with StorageTypes and StorageIDs. The current node can only verify that one of the storage [Type|ID] is available. The rest will be on different nodes.
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken

    • checkAccess

      public void checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Check if access should be allowed. userID is not checked if null
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken

    • checkAccess

      public void checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token, String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Check if access should be allowed. userID is not checked if null
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken

    • setTokenLifetime

      public void setTokenLifetime(long tokenLifetime)
      set token lifetime.

    • createIdentifier

      public org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier createIdentifier()
      Create an empty block token identifier
      Specified by:
      createIdentifier in class org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
      Returns:
      a newly created empty block token identifier

    • createPassword

      protected byte[] createPassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier)
      Create a new password/secret for the given block token identifier.
      Specified by:
      createPassword in class org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
      Parameters:
      identifier - the block token identifier
      Returns:
      token password/secret

    • retrievePassword

      public byte[] retrievePassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Look up the token password/secret for the given block token identifier.
      Specified by:
      retrievePassword in class org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>
      Parameters:
      identifier - the block token identifier to look up
      Returns:
      token password/secret as byte[]
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken

    • generateDataEncryptionKey

      public org.apache.hadoop.hdfs.security.token.block.DataEncryptionKey generateDataEncryptionKey()
      Generate a data encryption key for this block pool, using the current BlockKey.
      Returns:
      a data encryption key which may be used to encrypt traffic over the DataTransferProtocol

    • retrieveDataEncryptionKey

      public byte[] retrieveDataEncryptionKey(int keyId, byte[] nonce) throws org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException
      Recreate an encryption key based on the given key id and nonce.
      Parameters:
      keyId - identifier of the secret key used to generate the encryption key.
      nonce - random value used to create the encryption key
      Returns:
      the encryption key which corresponds to this (keyId, blockPoolId, nonce)
      Throws:
      org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException

    • getCurrentKey

      public BlockKey getCurrentKey()

    • setKeyUpdateIntervalForTesting

      @VisibleForTesting public void setKeyUpdateIntervalForTesting(long millis)

    • clearAllKeysForTesting

      @VisibleForTesting public void clearAllKeysForTesting()

    • hasKey

      @VisibleForTesting public boolean hasKey(int keyId)

    • getSerialNoForTesting

      @VisibleForTesting public int getSerialNoForTesting()